Application list


The Application list holds the Application and Application Groups with their associated gestures and actions:

​Some applications can be hidden (not listed) as described here. This is especially true if the All Applications entry is the only one listed.

Starting Commander for the first time you will be presented with the default list of pre-defined applications and group configurations. The list will also include any applications defined in the Preset list and found to be installed on the system.

The standard 'All Applications' entry defines the default set of touch events and associated actions.

The 'All Applications' entry cannot be deleted but the associated touch events and actions can be adjusted as required.

There are three separate 'All Applications' configurations to cater for different system wide configuration. You can select the appropriate configuration from the preset selection.

Drag and drop a selected item allows you to re-position an entry in the list.

Application list configuration

The application list can be configured in a number of ways:

  • Add applications that reside on the system
  • Add / replace applications or application groups from a preset list
  • Add an application to an application group
  • Delete applications or application Group entries
  • Rename application or group listings
  • Adjust the application or application group settings
Function bar

The application list function bar is used to configure the application list as follows: 

Control Description 
  Add a predefined or new Application, create/add to a Group or add an Application/Group UI element

  Remove an item from the list
  Application setting dialog used to add / remove applications within a group and Disable UI element processing for an application or group.
The application can also be hidden if deemed unnecessary to list in the applications list.


Options menu 

In addition to the function controls shown above, 'Ctrl + click' or 'right click' in the application list area invokes the list options menu and allows you to Add a preset or new application, Create /add to group, Add UI element, Remove or Edit the entry:

Applications

These can be added, removed and configured as required.

The application icon and name is extracted from the application when added and cannot be changed in the application list.

Invoke the list options menu to add an application.

It is only necessary to add an application if it requires different gestures or gesture actions to those inherited from the 'All Application' level.

Application group

An Application Group defines a group of applications that share the same touch event definitions and actions.

These can be added, deleted and configured as required.

A group is indicated by the display of multiple application icons:

Double clicking on a group entry will allow the entry to be renamed:

Invoke the Add options menu to add an application to the group:

Use the Settings option to edit the group:

Hidden application groups

Applications and groups that, once configured and working as expected and therefore unlikely to change, can be hidden from the Application List if required. Some installations come with the default set of preconfigured groups hidden such that only the All Applications entry is shown in the Applications list.

To list 'hidden' applications select the 'Show hidden Application Groups' option in the in the File Menu, View dropdown. They will appear in the list slightly greyed out. This allows for them to be viewed / updated if required:

To hide/show an Application or Group set the 'Hide application group' checkbox as appropriate in the application setting dialog.

Locked application groups

An application group can be locked to prevent any setting changes as per the UPDD Test application:

A locked application indicates that you should not change the gesture application settings as they are set to allow intended functioning of the application.

To show the extended Application setting dialog with the Locked checkbox enabled hold down the Option (MacOS) / Alt (Windows) key when selecting the setting control button 

Presets

By default the Application List has a number of predefined applications and groups and also ships with a number of 'preset' applications that can be added as required.

The preset items exist to allow you to:

  • reset a predefined application that you may have modified and wish to resent back to its original settings
  • to switch the functionality of a number of the predefined applications.
  • to add an application with custom configurations

The presets are held in separate lists:

List Description 
 Standard These are the standard, default, list of Applications and Application groups
 Optional List the optional, alternate configurations for some of the standard entries
 Custom Custom configurations for known** applications that require tailored settings.

There are the All Application entries to allow you to switch between standard gesture mode, simple single touch mode and one where only the TUIO interface is required.

There are two Browser entries that allow you to switch between iOS emulation and TUIO mode to support multi-touch browser applications.

There are various custom entries to cater for applications that have known configuration requirements.

Hovering over any of the defined applications will describe the configuration change as in the 'Digital Audio Workstation'  Studio One application:

**Please inform us if you find a common application that works better with custom configuration changes and will we consider it for inclusion in this list so that other users can benefit.

​UI Elements

These specify specific controls (UI elements) used in the Application's dialog (User Interface - UI) that may require a different set of touch events and associated actions to that defined at the higher application level and are listed as a sub branch to the application name:

Users have reported in some cases gesture processing within an application can be very slow. When a gesture is being performed on top of an application dialog (UI) Commander uses accessibility function calls to perform UI queries. With some applications these calls never respond and eventually time out, delaying the processing of the gesture. This has been observed in Dashboard and some 3rd party applications. In this case you need to disable UI element processing to avoid the delay.  This will prevent the application from being able to have different gestures depending on the UI element, but will fix delays in apps that respond poorly the UI element queries, such as DashBoard, grandMA3 and Little DJ Touch (Reloop).

The bottom line is that to utilise UI elements and separate gestures associated with them the applications must conform to certain UI constructs and work as expected with Apple's accessibility API.

We have identified a few common UI elements that likely require different settings when in use, set as follows:

Table

Table elements are not seen as 'clickable' by MacOS so are not considered when searching for the nearest UI element to click if this feature is enabled. When selecting the top item in a table this may be close to the table headers, which are seen as clickable, so the headers would be clicked.

This UI entry modifies one finger tap to set its action to click with  "click nearest clickable UI element" advanced option set to always be off.

Scrollable areas

This element is a scrollable container within the dialog and as standard would typically be used to display information that is larger than the visible area and thus needs to be scrolled to view the displayed data.

For MacOS we have set a number of single finger touch events to work within the scroll UI, allowing these areas to work like iOS whereby you can use a single finger to scroll.

Menu items with submenus

This element is used to display menu selections where the menu item invokes a sub menu. For this UI element we have set the One Finger tap touch event to move the mouse cursor when a tap occurs such that the sub menu is displayed. A click action on the menu item would likely close the displayed sub menu such that you could never select a sub menu item.

This specific UI element only matches menu items that contain a submenu. For example, it would match the "Recent Items" menu in the Apple menu.

UI element configuration

To add an additional UI Element to the selected Application or Group invoke the list options menu and select 'Add UI Element'.

In this example the 'Preview' application has been selected in the Application list. The add UI dialog is shown thus:

You can now add either a 'Type of UI element' or a 'Specific UI element' to an application or group.

A Type of UI element covers all UI elements of the same type within the application and a Specific UI element relates to that one only.

To add the selected UI element select the 'Select UI Element' button.

The UI element selector is now enabled. Move your mouse over the application dialog and each individual UI element will be highlighted with a 'dashed boarder'.

To see all UI elements at a given position, highlight the UI element and use the up / down arrow keys to move around the list of UI elements at that position.

This example video is purely used to show the UI Element selection process. Note the 'dashed border' switches between the subsection of the dialog (blue dashes) to the whole dialog (green dashes) as the arrow keys on the keyboard are used to highlight the UI elements in that area of the dialog. Remember you only need to add in a UI element if different gesture actions are needed within the UI element, such as the ones we already define. In this example, adding a UI element to All Applications would then be inherited by all applications and apply to any 'Group' UI element used within the applications dialogs unless it was deleted from the specific application:  

 When you have the target UI element highlighted use the Return key to select it. This will then be added to the active application or group as per this example:

Once a UI element has been added you can rename and define the Touch Events and actions as required.

There is also a separate settings for inheriting gestures from All Applications or a matching UI element under All Applications and the owner application:

UI element naming

The UI element name can be changed as required. When adding a UI element the name returned from the selection process will be used.

UI element display names can be changed as required. Double click the name of a UI element to change its name to more meaningful name.

UI element advance options

When defining a UI element there are a number of advanced options that can be set:

Option  Description
Offset boundary This feature is used to specify a portion of a UI element that should be used when detecting gestures.
This is useful in situations where an application does not define individual UI elements so the UI selection process may choose a UI element that is bigger than the area required.
This has been introduced to cater for the fact that Firefox does not report its UI elements to other applications. The only UI element that can be selected is the entire Firefox window and then offset the top boundary to start at the top of the content window:

 

Touch behavior is customized to the portion of the window that contains web content and not the text field, buttons, or tabs. This allows the drag gesture to scroll the web area but still perform click-and-drags for the tabs, allowing them to be moved around.

Must contain child UI elements Indicates that the related Touch Events and Actions will only apply to the UI element(s) if it contains child UI elements.

A good example of this is the 'Menu item' UI elements defined by default. If a menu item displays a sub-menu for additional menu selections then tapping it should move the cursor to the point of tap to show the sub-menu and not generate a click which will likely close the displayed sub-menu, which would make it impossible to select any of the items in the sub-menu.

 Application list   Touch events and related actions 
   

 Application list   Touch events and related actions 
   

 Application list   Touch events and related actions 
   

Search