Real-time AI for artists
Pose and object detection, depth from a plain camera, real-time image generation, language and vision models, and machine learning trained by example — running on your own computer, on the same timeline as sound, video and lights. Free and open-source.
Free and open-source (GPLv3) · Windows, macOS, Linux, Raspberry Pi and web browser
ossia score runs AI models as ordinary objects in a patch: a camera goes in, positions, images, text or sound come out, and they drive the rest of the piece. Models run locally, on your computer, not in a cloud service.
What you can do
- See people and objects: multi-person pose detection with smoothing, object detection and segmentation, emotion detection, body meshes, and depth from an ordinary RGB camera (pose detection, AI recognition).
- Generate images in real time: StreamDiffusion and librediffusion with SD and SDXL Turbo, Flux.2 Klein, ControlNet, IP-Adapter and LoRAs, from a camera, a video or generated content (StreamDiffusion). Requires Windows or Linux with an NVIDIA graphics card.
- Language and vision-language models: Qwen3 and Gemma 3, and models that describe a video stream in words; compose and morph prompts over time (Qwen LLM, prompt composer).
- Sound: stem separation and neural vocoders.
- Machine learning by example: train a classifier or a regressor from a few examples of gestures or sensor data, and interpolate between states (classifier, regressor).
- Your own models: a generic ONNX object runs other models, on CUDA, TensorRT, DirectML or Core ML; some models also run in the browser version through WebNN.
- From AI to everything else: poses become MIDI, OSC, 3D geometry or light cues (tracking to MIDI, tracking to geometry).
Reflet, Noukii (Anouk Ethier) with Isabella Salas and Allison Moore, at SAT: show logic and real-time imagery from custom generative models in ossia score. © Agustina Isidori
AI needs, and how ossia score answers them
| What you need | How ossia score does it |
|---|---|
| Body tracking without special sensors | Pose detection and depth estimation from an ordinary webcam. |
| AI imagery that reacts to the audience or the music | Real-time image generation driven by cameras, sensors and audio analysis, with prompts that change over time. |
| Keeping data on site | Models run on your computer; nothing is sent to a cloud service. |
| Interaction designed by example, without coding | Classifiers and regressors trained from a few recorded gestures. |
| A show that plays the same every night | AI objects are part of the timeline, with cues, scenes and conditions. |
Free and open-source (GPLv3) · Windows, macOS, Linux, Raspberry Pi and web browser