Application list

The Application list holds the Application and Application Groups with their associated gestures and actions:

​Some applications can be hidden (not listed) as described here. This is especially true if the Default Gestures entry is the only one listed.

Note: 'Default Gestures' was previous listed as 'All Applications' and some screen shots still refer to the old listing! 

Starting Commander for the first time you will be presented with the default list of pre-defined applications and group configurations. The list will also include any applications defined in the Preset list and found to be installed on the system.

The standard 'Default Gestures' entry defines the default set of touch events and associated actions.

This entry cannot be deleted but the associated touch events and actions can be adjusted as required.

There are three separate 'Default Gestures' configurations to cater for different system wide configuration.

You can select the appropriate configuration from the preset selection.

Standard
	Default set of system wide gestures and actions including two finger definitions to invoke Smart Magnifier
Optional
	Full... Full system wide without two finger Smart Magnifier definitions Simple... One finger drag and press, no multi-touch gestures defined TUIO... TUIO interface only to pass all touches to TUIO clients

Drag and drop a selected item allows you to re-position an entry in the list.

Application list configuration

The application list can be configured in a number of ways:

• Add applications that reside on the system
• Add / replace applications or application groups from a preset list
• Add an application to an application group
• Delete applications or application Group entries
• Rename application or group listings
• Adjust the application or application group settings

Function bar

The application list function bar is used to configure the application list as follows: 

Control	Description
	Add a predefined or new Application, create/add to a Group or add an Application/Group UI element
	Remove an item from the list
	Application setting dialog used to add / remove applications within a group and Disable UI element processing for an application or group. When disabled, Commander won't look for which UI element a gesture is occurring within for the given application, such as a scroll area, a menu, etc. In some applications, the UI element processing can cause a delay and introduce a touch latency. The application can also be hidden if deemed unnecessary to list in the applications list. UI Elements, referrenced above, are described here.

Options menu 

In addition to the function controls shown above, 'Ctrl + click' or 'right click' in the application list area invokes the list options menu and allows you to Add a preset or new application, Create /add to group, Add UI element, Remove or Edit the entry:

Applications

These can be added, removed and configured as required.

The application icon and name is extracted from the application when added but can be renamed if required.

Invoke the list options menu as seen above, to add an application.

It is only necessary to add an application if it requires different gestures or gesture actions to those inherited from the 'All Application' level.
Important: Added Applications should redefine any gestures and actions that are important to its functioning, even if it's currently the same gesture and action inherited from All Applications, since All Application gesture and actions could be changed.

Application group

An Application Group defines a group of applications that share the same touch event definitions and actions.

These can be added, deleted and configured as required.

A group is indicated by the display of multiple application icons:

Double clicking on a group entry will allow the entry to be renamed:

Invoke the Add options menu to add an application to the group:

Use the Settings option to edit the group:

Hidden application groups

Applications and groups that, once configured and working as expected and therefore unlikely to change, can be hidden from the Application List if required. Some installations come with the default set of preconfigured groups hidden such that only the All Applications entry is shown in the Applications list.

To list 'hidden' applications select the 'Show hidden Application Groups' option in the in the File Menu, View dropdown. They will appear in the list slightly greyed out. This allows for them to be viewed / updated if required:

To hide/show an Application or Group set the 'Hide application group' checkbox as appropriate in the application setting dialog.

Locked application groups

An application group can be locked to prevent any setting changes as per the UPDD Test application:

A locked application indicates that you should not change the gesture application settings as they are set to allow intended functioning of the application.

To show the extended Application setting dialog with the Locked checkbox enabled hold down the Option (MacOS) / Alt (Windows) key when selecting the setting control button 

Presets

By default the Application List has a number of predefined applications and groups and also ships with a number of 'preset' applications that can be added as required.

The preset items exist to allow you to:

• reset a predefined application that you may have modified and wish to resent back to its original settings
• to switch the functionality of a number of the predefined applications.
• to add an application with custom configurations

The presets are held in separate lists:

There are the All Application entries to allow you to switch between standard gesture mode, simple single touch mode and one where only the TUIO interface is required.

There are two Browser entries that allow you to switch between iOS emulation and TUIO mode to support multi-touch browser applications.

There are various custom entries to cater for applications that have known configuration requirements.

Hovering over any of the defined applications will describe the configuration change as in the 'Digital Audio Workstation'  Studio One application:

**Please inform us if you find a common application that works better with custom configuration changes and will we consider it for inclusion in this list so that other users can benefit.

​UI Elements

These specify specific controls (UI elements) used in the Application's dialog (User Interface - UI) that may require a different set of touch events and associated actions to that defined at the higher application level and are listed as a sub branch to the application name:

Users have reported in some cases gesture processing within an application can be very slow. When a gesture is being performed on top of an application dialog (UI) Commander uses accessibility function calls to perform UI queries. With some applications these calls never respond and eventually time out, delaying the processing of the gesture. This has been observed in Dashboard and some 3rd party applications, such as grandMA3. In this case you need to disable UI element processing to avoid the delay.  This will prevent the application from being able to have different gestures depending on the UI element, but will fix delays in apps that respond poorly the UI element queries, such as DashBoard, grandMA3 and Little DJ Touch (Reloop).

The bottom line is that to utilise UI elements and separate gestures associated with them the applications must conform to certain UI constructs and work as expected with Apple's accessibility API.

Since version 1.6.4, UPDD Commander now notifies when an application takes too long to respond using the following warning dialog:

You can then choose to automatically disable UI element detection for the app, handle the app manually or ignore this warning.

Standard UI elements

We have identified a few common UI elements that likely require different settings when in use, set as follows:

Table

Table elements are not seen as 'clickable' by MacOS so are not considered when searching for the nearest UI element to click if this feature is enabled. When selecting the top item in a table this may be close to the table headers, which are seen as clickable, so the headers would be clicked.

This UI entry modifies one finger tap to set its action to click with  "click nearest clickable UI element" advanced option set to always be off.

Scrollable areas

This element is a scrollable container within the dialog and as standard would typically be used to display information that is larger than the visible area and thus needs to be scrolled to view the displayed data.

For MacOS we have set a number of single finger touch events to work within the scroll UI, allowing these areas to work like iOS whereby you can use a single finger to scroll.

Menu items with submenus

This element is used to display menu selections where the menu item invokes a sub menu. For this UI element we have set the One Finger tap touch event to move the mouse cursor when a tap occurs such that the sub menu is displayed. A click action on the menu item would likely close the displayed sub menu such that you could never select a sub menu item.

This specific UI element only matches menu items that contain a submenu. For example, it would match the "Recent Items" menu in the Apple menu.

UI element configuration

To add an additional UI Element to the selected Application or Group invoke the list options menu and select 'Add UI Element'.

In this example the 'Preview' application has been selected in the Application list. The add UI dialog is shown thus:

You can now add either a 'Type of UI element' or a 'Specific UI element' to an application or group.

A Type of UI element covers all UI elements of the same type within the application and a Specific UI element relates to that one only.

To add the selected UI element select the 'Select UI Element' button.

The UI element selector is now enabled. Move your mouse over the application dialog and each individual UI element will be highlighted with a 'dashed boarder'.

To see all UI elements at a given position, highlight the UI element and use the up / down arrow keys to move around the list of UI elements at that position.

This example video is purely used to show the UI Element selection process. Note the 'dashed border' switches between the subsection of the dialog (blue dashes) to the whole dialog (green dashes) as the arrow keys on the keyboard are used to highlight the UI elements in that area of the dialog. Remember you only need to add in a UI element if different gesture actions are needed within the UI element, such as the ones we already define. In this example, adding a UI element to All Applications would then be inherited by all applications and apply to any 'Group' UI element used within the applications dialogs unless it was deleted from the specific application:  

 When you have the target UI element highlighted use the Return key to select it. This will then be added to the active application or group as per this example:

Once a UI element has been added you can rename and define the Touch Events and actions as required.

Disabling UI element processing

There are two reason you may need to disable UI element processing:

1. If there are different gesture sets that apply to listed application UI Elements, say Scrollable Area, but you want the the same gesture set at the Application level to apply to all elements then you can just disable UI processing so that it won't try to determine the UI element under the gesture. Typically at the app level One Finger Drag is set to Click and Drag, but within a Scrollable UI element it is set to Scroll action.
   
   
2. UI Element processing introduces gesture latency
   
   Where applications conform to certain Apple development guidelines, then the Accessibility API can be used to get a list of application UI elements, their attributes and locations within the dialog.​
   
   In some applications, the accessibility API will stall if the app does not respond with a list of UI elements and times out after 250 ms. This will cause latency issues with touches within the application. Click UI element processing will have to be disabled in these cases within the application or group settings:
   
   You can disable for listed or unlisted applications
   
   The difference between "Disable UI elements for unlisted applications" and "Disable UI elements for [app]" is in the name. The former is available in the settings for "All Applications" and affects apps that are not listed in applications list, and therefore the settings and gestures from "All Applications" will apply to them. The latter only applies to a specific app.

Inherited gestures

Applications and their elements inherit matching gesture setting from the higher level All Application gesture definition.

This inheritence is not always required and can be disabled as required.

There are separate settings for inheriting gestures from All Applications or a matching UI element under All Applications and the owner application:

Application level options	UI element options

UI element naming

The UI element name can be changed as required. When adding a UI element the name returned from the selection process will be used.

UI element display names can be changed as required. Double click the name of a UI element to change its name to more meaningful name.

UI element advance options

When defining a UI element there are a number of advanced options that can be set:

Application list	Touch events and related actions

Application list	Touch events and related actions

Application list	Touch events and related actions