Audiotool OSC Integration for Live Performance
This project was created in 24 hours for the London Music Technology Hackathon, and ended up receiving 1st Prize in it's respective category!
Inspiration
The inspiration for this project comes from a resurgence of audiovisual tool-building and performance in the creative technology scene. While Audiotool already facilitates real-time collaboration, there was no way to extend it’s use past being a standalone DAW. We wanted to incorporate Audiotool into the ever-growing live audiovisual performance toolkit.
What it does
This project creates a bidirectional OSC (Open Sound Control) bridge between Audiotool and external applications such as TouchDesigner, Max/MSP, and TouchOSC. This bidirectionality enables:
- Audiotool as an OSC Sender: Changes to mixer, effect, project, and device parameters in Audiotool get sent as OSC messages that external applications can receive. For example, changing the pattern in a Tonematrix changes which visual appears on screen.
- Audiotool as an OSC Receiver: External applications can send OSC messages that modify these parameters once Audiotool receives them. For example, hand gestures modify reverb and delay in Audiotool.
With OSC capability, Audiotool transforms from a standalone DAW into a tool capable of being used for live performance.
How we built it
The architecture consists of two main components:
-
Node.js OSC Bridge: Implemented via the osc-js library, this bridge handles bidirectional communication between our Node app and OSC-enabled software.
-
OSC Message Handling: Mappings were implemented with reference to Audiotool’s Nexus SDK to send and receive OSC messages.
Challenges we ran into
- Real-time Performance: Unfamiliarity with Audiotool’s event transactions led to increased latency, dropped commands, and other unexpected issues when sending and receiving OSC messages, especially when modifying multiple parameters at once.
What’s next
- Extended OSC mappings: Map additional Audiotool devices and parameters, including synthesizer controls, automation curves, and track-level effects
- Improving Real-time Performance: Implement message batching and better transaction handling to mitigate any real-time hiccups.