We have undertaken some experimental work with a browser extension to offer multi-touch within a browser.
This required modifying UPDD TUIO to offer an option to dispatch TUIO events using a WebSocket interface which is now on by default. This change is in UPDD TUIO Server 2.2.0 if using the old stand alone UPDD TUIO server. This uses the same port numbers that the regular TUIO server(s) are configured to use.
The new UPDD Commander (a replacement utility for both UPDD Gestures and UPDD TUIO server) also has an option to dispatch TUIO events using WebSocket protocol which is also on by default. When using this bridge with UPDD Commander it is important to switch the Browsers configuration in Commander to utilise the TUIO configuration via the Presets option.
The extension is available for Firefox and Chrome.
The Firefox extension was rejected from inclusion in their add-on site on the grounds that it was too specialized. The good news is that you can still install Firefox add-ons from downloaded files so the add-on is available here. Clicking the link will cause Firefox to announce that it prevented the site from asking you to install software, but you can select the "Allow" button and proceed with the installation.
When requesting this extension in Chrome you may see a warning that "this extension can read and change all your data on the websites that you visit". Chrome shows this warning because the extension has permission to modify individual web pages (i.e. by injecting a script that allows touches to work). If you have concerns you can view the source code to the extension and confirm it's not malicious. We hope that it also helps that there's a legitimate company behind this extension rather than an individual.
The extension can be configured to select which server and port number to connect to.
By default it uses 'localhost' and port 3333 which matches the default settings for UPDD TUIO which means that both UPDD TUIO and the extension should work together without any additional configuration.
In the unlikely event that you need to adjust these settings...
Chrome - First open the Extensions tab by picking More Tools > Extensions in Chrome's main menu. Click "Details" under Touch-Base TUIO Bridge. Then finally click "Extension Options".
Firefox - Open the Extensions page for Firefox (available from its main menu) and select the Preferences button listed next to our extension.
Once installed you can test with an appropriate touch-enabled website. During testing we tested with https://www.paulirish.com/demo/multi
You can touch the screen and have it paint colored trails that follow your touches as seen in this dual touch demo:
When using these extensions with UPDD Gestures it's likely you won't want UPDD Gestures to invoke any actions from touches in the web browser since the page will be processing touches directly.
In that case it's probably better to run UPDD TUIO without Gestures running, or remove the browser from the list in UPDD Gestures and then manually disable all of the actions for gestures that will interfere with the browser.
UPDD Commander TUIO interface is configured at an application level and automatically disables gestures at the TUIO client application level.
The extension currently has a few limitations:
- It can't distinguish between which display a touch comes from, so all touches are assumed to come from the display containing the browser window.
- It can only receive touches from one source, meaning it's not possible to have multiple browser windows receiving touches from different touch devices simultaneously.
- There's no option to set touches to be scaled -- all touches are processed as unscaled -- though such an option could be added fairly easily if required.