TUIO is an open and cross-platform protocol for delivering touch and object data from touch screens and surfaces to touch enabled applications. This is of particular importance on macOS as it does not have any built-in representation of touch events for applications to use, unlike Windows or mobile operating systems. TUIO has the advantage of being an open protocol with several libraries and reference implementations that can be used by both open and commercial applications, and is already supported by many devices and applications as listed on the official TUIO web page here: https://www.tuio.org/?software
When building a touch enabled application, though, there are some challenges with TUIO. Because it's a general protocol that represents touch data using normalized coordinates that are abstracted from any hardware, that means it's up to touch-enabled client applications to determine how touches should be interpreted and related to on-screen elements, and this generally requires configuration from the user. Minimally, users will need to specify which of their displays should be controlled by TUIO.
TUIO is based on the OSC protocol, which transmits data from a TUIO server to TUIO client using UDP packets. In order for a TUIO client application to receive touches, then it's necessary to at minimum to configure the following:
- A UDP port number on which the application will receive TUIO data
- Which display on their system that will be controlled by the touches
The user will also need to configure any TUIO server software so that it will transmit touch data to the IP address and port number of the system that's running the client application. (How this is done varies depending on the TUIO server software.) Further, the client application will need to determine how to correlate the normalized coordinates of TUIO touches with the screen coordinates on the host operating system. The most common case is where the normalized coordinates will correspond to the edges of a touch screen display, with the coordinate (0,0) corresponding to the upper-left corner of the display, and (1,1) corresponding to the lower right corner. In that case, it should be enough to have the user simply specify a port number and corresponding display.
In the case of there being more than one touch display, generally the best way to set that up is to have multiple TUIO servers transmitting data on different ports, with each server transmitting the touches of one display. The TUIO touch enabled application should then allow the user to specify multiple port number and display combinations for each of their touch displays.
It is possible though for a TUIO server to handle the touches of multiple touch displays by mapping each display into a specific range of coordinates within the normalized coordinate space. It will then be up to the client application to allow the user to configure exactly how normalized coordinates will be mapped to system pixel coordinates. Because this is generally more complicated to both implement for the developer and configure for the user, we generally don't recommend using this approach unless there's a clear need for it.
UPDD Commander can be configured to run as a TUIO server that converts UPDD touches to TUIO. It can use different port numbers for different touch devices, making it relatively simple to configure it to send TUIO touches to any TUIO client application that works as described above, where one TUIO server corresponds to a single touch display. It can also send represent the touches of multiple touch displays on a single TUIO server by dividing the X axis evenly between each touch device.
The
TUIO slider example project demonstrates how to receive TUIO touches in a Qt application, along with a basic settings window that allows the user to specify one or more port numbers and corresponding displays. This same approach can be used in any application, regardless of which programming language and UI framework is used to implement it.