Modifiers

When a gesture entry is defined, it is associated with the gesture performed in its basic form, that is unassociated with a modifier trigger.

However, a gesture can contain any number of additional definitions that are related to a modifier, such that if the modifier is active at the time the gesture is performed, then the associated modifier gesture entry is selected.

Multiple different actions associated with a single gesture allow for the creation of very powerful gesture sets that can be used to expand the way that touch gestures can be used to interact and control and an application's functions.

To add a modifier entry, you simply select the modifier add option in the gesture definition:

The four modifier types are as follows:

Single gesture definitions can have as many unique modifier definitions as required.

A modifier definition is added into the gesture relative to the add control button selected(+).

In this example, a single keyboard modifier definition has been added to the One Finger Press gesture:

You can delete a modifier definition using the (-) control.

Modifiers are processed in the order they are defined, so if more than one modifier trigger is active, the top most one is selected.

You can reorder modifiers within the gesture by dragging the modifier by its title bar.

A detailed description of each modifier type follows.

Keyboard

For each gesture, you can add up to four keyboard modifiers, as selected from the keyboard selection dialog:

This will create a keyboard modifier entry listing the associated keyboard modifier.

Set the Action required when the keyboard modifier is selected on the keyboard and the gesture is performed.

This example shows two keyboard modifiers defined:

UI Element

UI elements are areas and controls that make up an application's visual on screen dialog window.

Depending on an application's UI element layout, you may wish to change the actions invoked when certain gestures are performed.

When you add a UI element modifier, you are requested to select the modifier using the UI element selection dialog:

When you select the 'Select UI Element' the system will highlight the UI element under the system pointer with a dotted line around the edge of the area covered by the UI Element. You can use the mouse as normal at this point, so it can be used to load up the appropriate application/ dialog as normal, should the desired UI element not be on the screen. Once the desired UI Element is highlighted under the system pointer with the dotted highlighter, use the return key on the keyboard to select the UI element

Here you can see the New Message UI element highlighted in the Mail system:

In this example, a UI modifier has been set up to generate a 'Space'  when Spotify is focused to start / stop play by tapping on the touch screen:

Important note:  The UI element processing uses the accessibility API to determine the UI elements within an application. There are a number of issues that can arise when defining UI elements:

1. Some applications can have 1000's of UI elements, such as Mail with a very large mailbox, and the UI selection can potentially take a long time to complete. To avoid this potential problem, the UI element selector uses a simple (fast) technique to extract the UI element details. In rare cases where this does not work it uses the more robust, but slower technique that could appear to have stalled / crashed the program (beachball) but it will return eventually!
2. Some applications will only return 1 or 2 UI elements, such as the entire dialog window and/or the top toolbar.
3. Some applications do not confirm to Apple application standards and the accessibility API will not find any UI elements to select!
4. Some applications choke on the UI element request and do not respond and the function times out.
5. Degenerate UI elements are ones that can be identified during the identification process (located under the cursor during the selection process) but cannot be found during application UI element processing when searching if a UI element exists, as the application will not always report UI elements that are contained within another. It is for this reason the UI selection process prevents the use of degenerate UI elements.

Toolbar

Utilising toolbars requires UPDD driver version 7.1.40 and above. This version of the driver only supports 10.14 and above

A Toolbar is an area of the touchscreen that can be used to mask off touches. Toolbars are given unique names and can be located anywhere on the touchscreen. A toolbar can have any number of cells (rows and columns). They are created and maintained in the driver's Console program, toolbar section.

When you add a toolbar modifier, you are requested to select a toolbar name from the list of toolbars created on the system:

In the above example, all the toolbars shown have a single cell (1x1). If a toolbar contains more than one row or column, it would show the cell layout to select the appropriate toolbar cell.

When a toolbar is touched, the touch is ignored and no cursor movement occurs, however UPDD Commander is aware if a toolbar is active.

There are two different toolbar active states:

In this example, a toolbar modifier is used to invoke the UPDD Annotate utility:

Setting

The touch driver allows for the creation of settings via its Command Line Interface utility program. This program allows for ad hoc setting to be created that can be referenced by UPDD Commander.

This allows for the manual or automated use of settings to be used as modifier triggers.

You could, for example, have a general modifier setting called UCM (UPDD Commander Modifiers) used to control modifiers or a very specific name, such as LogicPro. to define all modifier associated with Logic Pro DAW.

To use the settings option, you first need to create the global (no device) driver setting using the CLI utility: upddutils nodevice set [name] [value]

In this example, the setting UCM has been created and set to 1 and the 'get' parameter has been used to retrieve the setting value (to see that it has been set)

When you add a setting modifier, you are asked to specify the setting and trigger value:

 Once the name and value are entered, the setting is then referenced in the gesture, thus:

Thereafter, this setting can be externally set using the command line interface to whatever value is required to invoke the appropriate action.

The setting modifier option is predominantly used by external processes to control the gesture actions performed.

You could, for example, use a gesture, say One Finger Swipe from Top/bottom/left/right edge to set three settings values 1,2, and 3, and one to set an unused value 0.

Modifier examples

Utilising modifiers allows you to enhance your interaction with the system and application purely via touch gestures.

You will need to determine what is the best modifier trigger to use. Keyboard and toolbar triggers are selected manually, UI elements are selected automatically based on an application's display, and settings are normally set indirectly by am external process

Here are just a few examples.

Enhance single touch monitors

There are still a lot of old single touch monitor in use, and they can only be set up to handle one finger gestures, unlike multitouch monitors that can utilise one to five-finger gestures.

However, using modifiers, you can invoke a much higher number of gesture actions associated with the various One Finger gestures.   

Program manipulation

In some situations, you may be controlling an application's functionality purely via an interactive display or whiteboard such that touch toolbars can be used to switch gesture/action functionality to suit.

For example, you may be working in a browser content window and switching between 'One Finger drag / Click and drag' to ink within a drawing application, to' One Finger Drag / Scroll' to Scroll the contents.

Switching macro automation sets

Actions can call Automator scripts, Apple scripts, Execute shell scripts, macOS Shortcuts or Flexi-deck automation macros.

Using modifiers, you can switch between different sets of these control functions.

For example, Flexi-deck has 1000's of macros to control Logic Pro functions, and these are normally invoked by onscreen menu button sets that are displayed appropriate to the Logic Pro functions on display.

Using the UI trigger you can create menu sets that can directly invoke the flexi-deck macros, such as loading your popular plugins or invoking panning:

In this example, which shows how a FD macro is selected from the active Flexi-deck project, keyboard modifiers are used to either toggle on / off the Waves plugin selection menu or used to load some popular plugins:

Command + One Finger Tap will load the main Waves plugin launcher selection page

Modifier considerations

When modifiers are defined, it is generally expected that only one modifier will be active at a time but it is always possible that the will be more than one modifier trigger active at any one time.

When more than one trigger is active for a given gesture then the top most definition is selected at the time the gesture is performed.

The sequence of modifier definitions can be viewed in the Commander settings dialog.

When the settings window is on display, any active modifier triggers are shown with a green dot.

Further, if you select 'Select next performed gesture' to analyse gesture selection it will also select the gesture being triggered, based on the current active modifer triggers.

In this example, a number of modifiers have been defined against the One Finger Tap gesture and at the time of the screen shot, two were active. The UCM setting was set to 1 and the Command key depressed. A Tap gesture performed at this time would invoke the Flexi-deck action.