Applications will inherit the basic user interfaces supported by the development tools used to create an application, which as an absolute minimum will be mouse clicks, clicks and drags etc.
Clicks and clicks and drags are posted into the operating system by the drivers that support hardware capable of generating 'mouse' clicks, such as mice, trackpads, touch screens etc.
The standard Windows HID driver supports touchscreens and supports positional touches and gestures that are performed on the touch screen and multi-touch trackpads. Under macOS, the magic mouse and multi-touch trackpad also support gestures using a propriety Apple driver. The generic macOS HID driver supports keyboards and mice but not touch screens, as the macOS desktop was not designed for touch interaction, whereas iOS was!
When writing applications with basic user interface requirements that may be used with touch, it's likely they will just rely on mouse emulation. On Windows this is natively supported by the Windows HID driver and on macOS you can use UPDD touch software in basic mouse emulation mode.
Applications that respond to standard gestures can utilise Windows gesture touch support and macOS trackpad gesture support, or use UPDD Commander software to process touch gestures in both Windows and macOS.
However, difficulties arise when you want to implement custom multi-touch support in cross-platform development to try and achieve the same functionality in both Windows and macOS, as the touch interface implemented within the development framework may work fine on Windows, with its native support for touch screens, but not work in macOS.
This means that Windows users of a specific application may have a better touch experience than macOS users.
This document is a brief introduction to help developers consider ways to improve cross-platform support within multi-touch aware applications.
One multi-touch issue that is often raised by end users, and serves as a very good example of the issues that arise, is the use of sliders on Digital Audio Workstation mixer desks and lighting applications.
These make extensive use of slider controls that best work when the user can control two or more sliders at the same time and under Windows works fine, but with many apps only work with one slider under macOS.
This is typically because the development framework used to create the sliders utilises a touch API that maps to Windows touch events, but that do not have the equivalent mapping under macOS, and likely default to mouse mapping.
There are things to consider when developing controls for cross-platform multi-touch usage that will allow touch functions to work across multiple OS, as discussed below:
| Option |
Description
|
| UPDD API |
The UPDD touch device has a very comprehensive API that allows an application to receive touch data directly into the application, removing the reliance on external mouse and gesture processing |
| TUIO protocol |
An industry standard for receiving touch data into an application via TUIO. The UPDD software implements its own TUIO server to service TUIO client applications. Other touch TUIO servers are available. |
| Qt |
This is a widely used cross-platform development tool that supports touch interaction via the QTouchEvent API. By default this API works in Windows, by not in macOS, and reverts to mouse. We have created a Qt plugin to satisfy Qt touch events on macOS. |
| Native |
We have created a macOS framework that simplifies the use of touch within a native macOS application. This also caters for applications made using the Swift UI framework. |
| JUCE |
JUCE is a popular cross-platform framework for creating applications and plug-ins for DAWs. We provide a modification to its source code that implements multi-touch support for macOS applications.
|
| Others |
Depending on requirement, we could offer a simple to use 'Touch library' that simplifies the UPDD API interface that is quite technical in its implementation. In addition, we could create a Javascript plugin to perform a similar function to the Qt Plugin, to satisfy JavaScript touch events within macOS |
Please contact us if you are using a cross-platform development platform that implements touch support and works well on Windows but not on macOS to see if there is anything we can do to help improve this situation!
Source code example
To aid the development of multi-touch controls we have created a slider application that utilises the various methods of touch interfaces we have created to simplify touch intergration.
Qt
We have created two flavours of an example Qt multi-touch program, one that uses the TUIO protocol and one that uses the Qt plugin (compatible with UPDD V7) to satisfy the touches.
When invoked, two copies of the dialog are loaded that should be placed on the touch monitor. This allows you to see how two instances of the same application can interface with touch.
With the TUIO version, you will see an option to set the port on which the TUIO server is delivering the touch data to the application. The TUIO server (part of UPDD Commander) and the Qt Plugin are receiving touch data from the UPDD driver via the API.
The TUIO version provides a more or less minimal example of how to use TUIO to control sliders in Qt, and might be applicable to other frameworks as well. In order for the demo to work, it has to be configured using the "TUIO Settings..." button, where each touch display must be associated with a TUIO server by port number.
The source code of this application is available at these links, one that runs as a TUIO Client, and one the utiliises the Qt Plugin.
Apple Native
The UPDDMultitouch framework allows adding multi-touch support to macOS AppKit and SwiftUI applications. It attempts to make the process as simple as possible by allowing any NSView or NSView-derived class that implements the UPDDMultitouchView protocol to receive multi-touch events in a similar manner as iOS.
Touch program considerations
If an application, or parts of an application UI, is implementing custom touch controls, it is important to consider that an external generic touch mouse emulation software is not active on the custom control at the same time as the application is also handling the touch. When using UPDD software, the UPDD Commander function can be used to disable touches being processed on top of specific applications or UI elements within the application such that external touches do not interfere with custom touches within the application.
Commercial examples
The Studio One DAW uses TUIO as its touch interface.
The developer of Mixing Station was able to utilise the UPDD API and impement direct touch support (within one day). The application now receives touches directly from the driver and processes them using the same code that processes touches received from the Windows touch Event API.
The Vista lighting application uses the Qt plugin to allow multi-touch features, such as multiple sliders, to work under macOS.
Vista release 4.1, is the most recent version to implement touch support as earlier versions had a number of unresolved issues with touch implementation. This version should be used with UPDD 7.1.126 and above.
The QtTouchEventAPI now receives touches in both Windows and macOS environments.
