Inputs

Controllers, video, sensors, networks and people, read into the graph as live data.

Each live source is a node you drop in. While any live input is present, that node and everything downstream of it cook every frame.

Every source emits its data from the node's output pins, to be wired anywhere. There is no separate mapping layer to configure; see The parameter bridge for what that layer does and does not do today.

Deleting the last node that uses a device releases it, so the webcam light goes off and the lighting socket closes. Adding a node reconnects.

Control and messaging

These carry numbers, strings and messages.

• MIDI: one CC or note per node, with a channel and a number, remapped into a range you set. MIDI Input

• OSC: one address per node, with value, vec2, vec3 and color readings of whatever arrived. OSC Input

• DMX / Art-Net / sACN: a universe and a channel, giving the raw byte, a normalised value, and the whole 512-slot universe as an array. DMX Input

• Serial: microcontrollers and custom hardware over USB serial. Bidirectional: it parses incoming lines, CSV or JSON, and its send parameter writes back to the port. Serial

• WebSocket: two-way messaging with web pages and services. WebSocket Input

• MQTT: subscribes to one topic on a broker and outputs the latest payload.

• Ableton Link: shared tempo, beat and phase with Ableton Live and Link-enabled apps on the same network. Ableton Link

Video and screen

Live picture arriving as raster, ready to composite, warp or sample. Any of these can run through an image or compositing node, or be sampled as a Field to drive other effects.

• Camera: webcams and capture cards, at a device index and a requested resolution. It is the one live source that also works in a web export, where the page asks the browser for the webcam and hands the frame to the graph as an ordinary image. Camera Input

• Network Camera: IP cameras and RTSP streams, with optional credentials.

• Screen Capture: a display or a single window, macOS only. Screen Capture

• Syphon: GPU texture sharing with other macOS apps, with no copy to disk.

• NDI: video-over-IP across the network, with a low-bandwidth mode. NDI Input

• Browser Source: renders a live web page or a block of HTML and reads it back as an image, with CSS and JavaScript injection.

The Video node is a file player rather than a live source, so it does not put the graph into continuous cooking. See Video.

People and devices

• Gamepad: sticks, triggers, D-pad and buttons, with a deadzone. Gamepad Input

• Keyboard: key press and release state. Keyboard

• Pointer: position, buttons, scroll, pressure and tilt, plus what the pointer is over in the scene: object index, name, hit point and hit normal. Pointer

• Body Tracking: 133 keypoints across body, hands, face and feet, output as a Collection to scatter with or drive geometry from. Body Tracking

• Vision Pose / Vision Segment: on-device body pose and person cut-out, macOS only. See Vision & ML.

Audio

Audio Input takes a microphone or an audio interface and already gives you a waveform, a spectrum, an overall volume and bass, mid and treble bands. Feed its spectrum to Audio Analysis for what it does not carry: kick, snare and hi-hat energy, a beat pulse, an estimated BPM, and spectral centroid, rolloff, flatness, flux and crest. See Audio Input, Audio Analysis and Numbers, signals & audio. What to do with the sound once it is in the graph is covered in Audio.

How it reaches the graph

A background service per device kind is polled only while a matching input node is present in the graph. Values leave the node as ordinary data: video as raster, tracking as a Collection, signals as numbers, ready to wire downstream.

See also

• Outputs

• Vision & ML

• The parameter bridge

• Recording live input