Link Search Menu Expand Document

ossia

This page lists all the systems, file formats, etc… that score is able to inter-operate with.

Operating systems

score works on Linux, macOS, Windows, and partially on the web platform.

Its development mainly happens on an ArchLinux system. As score is built with Qt, it should be portable to any system where Qt runs.

Network protocols

  • OSC (Open Sound Control): the standard intermedia protocol. It is implemented through a heavily modified version of Ross Bencina’s oscpack library.
  • OSCQuery:
  • Minuit.
  • HTTP.
  • WebSockets.
  • CoAP.
  • MQTT.
  • LSL (Lab Streaming Layer).

Lighting protocols

  • Art-Net / DMX: the standard for lighting fixtures. Support is implemented through libartnet, which has been integrated inside libossia. score is able to load fixtures definitions in the open-fixture-library format.
  • LED Strip Support: Comprehensive support for LED strips including NeoPixel (WS2812), individual LED control, and strip/pane/volume layouts through Art-Net. Supports RGB, RGBW, and specialized diode configurations (Warm/Cold White, Amber, UV).
  • s.ACN / E1.31 is supported.
  • ENTTEC DMX USB Pro devices are supported (Mk1 and Mk2).
  • DMX input and output are both supported.

Hardware protocols

  • Serial port: score can read/write directly through serial ports, either directly or through Bluetooth. Support is currently based on the Qt SerialPort library but is being ported to ASIO to allow it to run in environments that cannot use Qt.
  • Game pads: they are supported through the SDL2 gamepad library. Most gamepads and joysticks should work without issue.
  • Wiimotes: they are supported through the WiiUse library.
  • LeapMotion / UltraLeap: they are supported through the UltraLeap SDK (Gemini / Hyperion) which must be installed on the target computer.
  • GPIOs, ADCs, PWMs etc.: they are supported through the SimpleIO library which is a simple wrapper over the raw kernel access. This feature is only available on Linux with the relevant hardware, for instance on Raspberry Pi.
  • BLE: Bluetooth Low Energy devices are supported through the SimpleBLE. Both reading advertisments / beacons / manufacturer data and GATT services is supported.
  • GPS: score can connect to a gpsd server and expose the GPS data.
  • Phidgets: they are supported through an implementation in libossia. Note that score must be built from source with the Phidgets API for the Phidgets protocol to be enabled.

Audio systems

  • JACK: support is implemented in libossia.
  • PulseAudio: experimental support is implemented in libossia.
  • PipeWire: implemented in libossia and in score.
  • ALSA, the native Linux backend, supported through PortAudio. A direct implementation is also provided for instance for working with as low latency as possible on embedded devices, but it only supports output, not duplex / input.
  • CoreAudio: the native macOS backend, supported through PortAudio.
  • MME, WASAPI, WDMKS: the native Windows backends, supported through PortAudio.
  • ASIO: the low-latency pro-audio Windows backend developed by Steinberg, supported through PortAudio.
  • SDL: support is implemented in libossia. It is mainly used to provide audio for the WebAssembly build of score.

Video protocols

  • Spout is supported on Windows.
  • Syphon is supported on macOS.
  • Shmdata is supported on Linux and macOS.
  • Sh4lt: Advanced shared memory video/audio streaming protocol supporting any data type (multichannel audio, video frames, 3D models, messages). Provides zero-copy, server-less communication between applications with dynamic buffer resizing and hot connection/disconnection. Available for input and output.
  • NDI is supported on Windows, Linux and macOS.
  • NDI PTZ is supported for controlling cameras with PTZ feature.

Transport synchronisation

  • JACK transport: score can act as a master or a slave.

MIDI

All the MIDI support in score comes from the libremidi library:

For real-time communication, the following implementations are provided:

  • ALSA, through either the raw or sequencer API.
  • JACK.
  • PipeWire.
  • The native operating systems MIDI API: MME for Windows, CoreMIDI for macOS.
  • WebMIDI.

In addition, score is able to load Standard MIDI files (SMF).

See the MIDI documentation for more information.

Audio file formats

score uses FFMPEG for its audio needs.

It should support most codecs and formats listed at this page. Every standard format (WAV, W64, AIFF, MP3, OGG Vorbis, FLAC, etc.) are supported without issues.

score handles WAV files in a specific way, through the dr_wav library, to allow for memory-mapping the data for large files.

score is able to read ACID tags to devise for instance BPM info from sound files.

See the sound file process documentation for more information.

Video file formats

score uses FFMPEG for its video needs.

It should support most codecs and formats listed at this page. In particular, H.264, H.265, Apple ProRes, AVCHD, etc. should be supported.

HAP codecs are handled in a different way: for maximum performance, decoding is done by score (which allows doing it on the graphics card, while FFMPEG’s HAP decoding happens on the CPU which defeats the point of the codec).

See the video process documentation for more information.

Hardware decoding

score can use OS video decoding APIs through FFMPEG. This allows for instance to decode H264 video on a Raspberry Pi 4 and apply live effects on it with only 15% CPU usage.

Supported APIs are:

  • DXVA2 / D3D11 on Window.
  • VideoToolbox on macOS.
  • V4L2-M2M on Raspberry Pi and embedded.

If one builds score from source with a custom FFMPEG version, or through a Linux distribution, it is also possible to have CUDA and QuickSync support.

Hardware rendering

score will try to render some common video texture formats on the GPU with shaders instead of converting them to RGB on the CPU, for maximum performance. Make sure that your video frames are in one of these pixel formats: if so, decoding won’t take CPU time.

  • RGB, RGBA, ARGB, ABGR and any variation thereof, planar or packed, 8 bits (int) or 32 bits (float)
  • YUV420P
  • YUV420P10
  • YUV420P12
  • YUV422P
  • YUV422P10
  • YUV422P12
  • NV12
  • YUYV422, UYVY422
  • HAP, HAP-Q, HAP-M

Supported color spaces for source videos are BT.601, BT.709 and BT.2020.

Image file formats

score uses Qt’s QImage for decoding images. The supported formats are PNG, GIF, JPEG.

See the image process documentation for more information.

3D file formats

score can load .obj and .ply files through the Object Loader process. In addition, the JS object allows to leverage Qt3D and QtQuick3D which both support loading many more formats (most importantly glTF / GLB).

Graphics APIs

score uses Qt RHI as graphics abstraction for the video pipeline. It is able to use OpenGL ES 2.0, Vulkan, Metal, Direct3D 11 and Direct3D 12 in a very efficient way.

score shaders are written with the Interactive Shader Format specification.

See the shader process documentation for more information on how to write score shaders. See the general video documentation for general information on the score graphics rendering pipeline.

Audio plug-ins

score supports the following audio plug-in systems:

  • AirWindows on all platforms.
  • CLAP on all platforms.
  • Steinberg VST3 on all platforms.
  • LV2 on Linux. Note that currently this requires building score on your own computer or use a Linux distro package.
  • JSFX is embedded in score through ysfx.
  • Faust, the Faust programming language developed by GRAME. score embeds the Faust compiler and libraries.
  • Pure Data is embedded in score through libpd.
  • It is possible to write simple audio instruments and effects with the various math-expression processes.
  • It is possible to write simple audio instruments and effects with the JavaScript process.
  • It is possible to write more advanced instruments and effects in C++ with the C++ JIT process.