A TouchDesigner CPlusPlus TOP plugin that streams live point-cloud data from any RoboSense LiDAR into a GPU texture, ready for downstream processing such as blob detection and zone-based attention scoring.
Built on the rs_driver kernel (vendored, header-only, patched for macOS). Supports 20 device types spanning mechanical spinning LiDARs and MEMS / solid-state sensors.
The sensor broadcasts UDP packets at 10–20 Hz. The plugin receives them via rs_driver
on a background thread, projects each complete scan into an RGBA32Float spherical
range image whose dimensions match the selected device, and uploads it to TouchDesigner
every cook.
RoboSense sensor RoboSenseLidarTOP plugin TouchDesigner
port MSOP ─────────▶ rs_driver packet thread
port DIFOP ─────────▶ │
▼
processLoop thread
project points → TOP_Buffer
swap pointer (mutex)
│
▼
execute() (TD cook)
uploadBuffer ──────────────▶ RGBA32Float texture
Each pixel encodes one (azimuth column, ring row) cell of the scan:
| Channel | Content | Unit |
|---|---|---|
| R | Radial distance | m |
| G | Return intensity | 0 – 1 |
| B | X world coordinate | m |
| A | Z world coordinate | m |
Empty cells (no echo) are zero in all four channels. Filter on R > 0 in GLSL to exclude
them.
Texture dimensions depend on the selected device type — see Supported devices.
The Lidar Type parameter selects the device. The plugin allocates a texture whose dimensions exactly match that sensor's native scan geometry.
All standard RS/RSP models fire at ~0.2° per block (1800 columns at 600 RPM). RSAIRY and RSFAIRY fire wider blocks due to 96-channel interleaving.
| Lidar Type | Channels | az. res. | tex width | tex height | Notes |
|---|---|---|---|---|---|
| RSHELIOS | 32 | 0.2° | 1800 | 32 | Helios-5515 default |
| RSHELIOS_16P | 16 | 0.2° | 1800 | 16 | Helios 16-channel variant |
| RS16 | 16 | 0.2° | 1800 | 16 | |
| RS32 | 32 | 0.2° | 1800 | 32 | |
| RSBP | 32 | 0.2° | 1800 | 32 | Bird's Eye Point |
| RS48 | 48 | 0.2° | 1800 | 48 | |
| RS80 | 80 | 0.2° | 1800 | 80 | |
| RS128 | 128 | 0.2° | 1800 | 128 | |
| RSAIRY | 96 | 0.4° | 900 | 96 | 96-ch interleaved |
| RSFAIRY | 96 | 0.25° | 1440 | 96 | 96-ch interleaved |
| RSP48 | 48 | 0.2° | 1800 | 48 | Precision series |
| RSP80 | 80 | 0.2° | 1800 | 80 | Precision series |
| RSP128 | 128 | 0.2° | 1800 | 128 | Precision series |
These sensors steer beams electronically. They are projected into the same azimuth-to-column mapping as mechanical sensors; columns outside the sensor's FoV stay zero.
| Lidar Type | Channels | Frame rate | tex width | tex height | Notes |
|---|---|---|---|---|---|
| RSM1 | 5 | 10 Hz | 1800 | 5 | Resonant MEMS |
| RSM1_JUMBO | 5 | 10 Hz | 1800 | 5 | RSM1 via jumbo UDP |
| RSM2 | 5 | 10 Hz | 1800 | 5 | MEMS v2 |
| RSM3 | 28 | 10 Hz | 1800 | 28 | Multi-beam MEMS |
| RSE1 | 1 | 10 Hz | 1800 | 1 | Single-line sweep |
| RSMX | 2 | 10 Hz | 1800 | 2 | 2-beam MEMS |
| RSEMX | 192 | 10 Hz | 1800 | 192 | 2-D focal-plane array |
The full timing table (block durations, blks/frame) is in docs/rs_driver_devices.md.
| Parameter | Type | Default | Description |
|---|---|---|---|
| MSOP Port | Int | 6699 | UDP port for measurement data packets |
| DIFOP Port | Int | 7788 | UDP port for device info / calibration packets |
| Lidar Type | Menu | RSHELIOS | Selects the device; determines texture dimensions |
| Start | Pulse | — | Start (or resume) the driver |
| Stop | Pulse | — | Stop the driver |
| Restart | Pulse | — | Stop then start (useful after network interruption) |
The driver is started automatically on the first cook. Port and type are read once at startup; change them by reloading the plugin (delete and re-place the node).
The node exposes live device telemetry as an Info CHOP:
| Channel | Content |
|---|---|
temperature |
Sensor temperature (°C) |
voltage |
Supply voltage (V) |
connected |
1 while DIFOP packets arrive, 0 otherwise |
qx qy qz qw |
Orientation quaternion (from DIFOP) |
x y z |
Position offset (m, from DIFOP) |
The node exposes firmware and identity info as an Info DAT (4 rows × 2 columns):
| Row | Key | Example value |
|---|---|---|
| 0 | serial |
A1B2C3D4E5F6 |
| 1 | mac |
AA:BB:CC:DD:EE:FF |
| 2 | top_firmware |
1.2.3.4.5 |
| 3 | bottom_firmware |
1.2.3.4.5 |
RoboSenseLidarTOP/
├── CMakeLists.txt root build file
├── Makefile convenience wrapper (make / make clean)
├── plugin/
│ ├── src/
│ │ ├── RoboSenseLidarTOP.hpp class declaration + DeviceSpec lookup table
│ │ └── RoboSenseLidarTOP.cpp implementation + TD plugin exports
│ └── td_sdk/ TD C++ SDK headers (not committed; see Step 1)
│ ├── TOP_CPlusPlusBase.h
│ └── CPlusPlus_Common.h
├── rs_driver/ RoboSense driver (submodule, header-only)
├── docs/
│ ├── rs_driver_devices.md per-device specs and texture dimension table
│ ├── architecture.md integration strategy notes
│ └── lidar_handoff.md hardware & network reference (German)
├── flake.nix Nix dev-shell (macOS)
└── flake.lock
| Tool | Minimum version |
|---|---|
| CMake | 3.16 |
| C++ | C++14 |
| Tool | Notes |
|---|---|
| Xcode Command Line Tools | xcode-select --install |
| TouchDesigner | 2025.32460 tested |
| Tool | Notes |
|---|---|
| Visual Studio 2022 | C++ workload required |
| TouchDesigner | 2022 build 25220 tested |
The two SDK headers ship with every TouchDesigner installation but are not open source and
are not committed to this repo (the private submodule is only used to build the plugin in CI/CD). Copy them into plugin/td_sdk/ before building.
macOS
TD_SDK="/Applications/TouchDesigner.app/Contents/Resources/tfs/Samples/CPlusPlus/CPUMemoryTOP"
cp "$TD_SDK/TOP_CPlusPlusBase.h" plugin/td_sdk/
cp "$TD_SDK/CPlusPlus_Common.h" plugin/td_sdk/Windows (PowerShell)
$TD = "C:\Program Files\Derivative\TouchDesigner\Samples\CPlusPlus\CPUMemoryTOP"
Copy-Item "$TD\TOP_CPlusPlusBase.h" plugin\td_sdk\
Copy-Item "$TD\CPlusPlus_Common.h" plugin\td_sdk\If the path above doesn't exist, search for
TOP_CPlusPlusBase.hunder the TD installation directory.
make # debug build → build/RoboSenseLidarTOP.plugin
make release # release build → build-release/RoboSenseLidarTOP.plugin# Debug (default — symbols, no optimisations)
cmake -B build -DCMAKE_BUILD_TYPE=Debug
cmake --build build
# Release (optimised)
cmake -B build-release -DCMAKE_BUILD_TYPE=Release
cmake --build build-releaseWith a custom SDK path:
cmake -B build -DTD_SDK_PATH=/path/to/sdk/headers
cmake --build buildOpen a Visual Studio 2022 x64 Developer Command Prompt:
cmake -B build -G "Visual Studio 17 2022" -A x64
cmake --build build --config ReleaseOutput: build\Release\RoboSenseLidarTOP.dll
With a custom SDK path:
cmake -B build -G "Visual Studio 17 2022" -A x64 -DTD_SDK_PATH="C:\path\to\sdk"
cmake --build build --config Release| Variable | Default | Description |
|---|---|---|
TD_SDK_PATH |
plugin/td_sdk |
Directory containing the two SDK headers |
The sensor and the host PC must be on the same subnet. Tested configuration:
| Device | IP | Subnet |
|---|---|---|
| RS-Helios-5515 | 192.168.2.200 | 255.255.255.0 |
| PC (plugin host) | 192.168.2.102 | 255.255.255.0 |
The plugin opens two UDP ports (receive only):
| Port | Direction | Purpose |
|---|---|---|
| 6699 | Sensor → PC | MSOP — measurement data packets |
| 7788 | Sensor → PC | DIFOP — device info / calibration packets |
Both ports are configurable via the MSOP Port and DIFOP Port parameters.
Windows firewall — add inbound rules if packets are blocked:
netsh advfirewall firewall add rule name="LIDAR MSOP" dir=in action=allow protocol=UDP localport=6699
netsh advfirewall firewall add rule name="LIDAR DIFOP" dir=in action=allow protocol=UDP localport=7788Port conflict — only one process can hold a UDP port at a time. If a UDP In DAT in TouchDesigner is already bound to port 6699, close it before loading the plugin.
Multi-homing — if the PC has multiple network interfaces, ensure the LiDAR's subnet is routed to the correct NIC. On Windows, having a VPN client (e.g. Tailscale) active can intercept packets; stop it if connectivity issues arise.
- Drop a CPlusPlus TOP node into your network.
- Set its Plugin Path parameter to the built file:
- macOS debug:
build/RoboSenseLidarTOP.plugin - macOS release:
build-release/RoboSenseLidarTOP.plugin - Windows:
build\Release\RoboSenseLidarTOP.dll
- macOS debug:
- On the Lidar parameter page, choose your Lidar Type and confirm the ports.
- The node outputs an RGBA32Float texture at the sensor's scan rate (~10 Hz).
The texture dimensions are set automatically by the selected device type. For example, an RSHELIOS outputs 1800 × 32, an RS128 outputs 1800 × 128, and an RSEMX outputs 1800 × 192.
RoboSenseLidarTOP (e.g. 1800×32)
└─ Resolution TOP (scale height ×10, Nearest filter — preserves float values)
└─ GLSL TOP (threshold R channel: dist > 0 && dist < max_range)
└─ Blob Track TOP → CHOP (centroids + blob count per zone)
For motion detection, subtract consecutive frames via a Feedback TOP + Composite TOP (Difference mode) on the R channel before thresholding — this isolates only points that moved between scans.
Middle-click the CPlusPlus TOP to open its info popup. The error string from rs_driver
appears there (e.g. socket bind failure, wrong port).
- Confirm the sensor is reachable:
ping 192.168.2.200 - Verify nothing else holds the MSOP port:
- macOS:
lsof -i UDP:6699 - Windows:
netstat -an | findstr 6699
- macOS:
- The plugin logs
rs_driverwarnings to stdout — run TouchDesigner from a terminal to see them. - After changing Lidar Type, delete and re-place the node — the driver initialises once on the first cook.
xattr -d com.apple.quarantine build/RoboSenseLidarTOP.pluginCMake prints TouchDesigner SDK headers not found. Re-run the copy step from
Step 1.
The SDK API version is encoded in TOPCPlusPlusAPIVersion inside TOP_CPlusPlusBase.h.
If a TD update breaks the plugin, re-copy both headers from the new installation and
rebuild.
This plugin was developed for a kinetic sound installation: a tensegrity sculpture in a museum space whose sound parameters and motor control respond in real time to how many visitors are near the sculpture and how close they are. The LiDAR provides an anonymised, privacy-safe presence signal — no image data, only distances.
The RS-Helios-5515 is the primary hardware target. Full hardware reference, network diagnostics, and the earlier Python prototype are documented (in German) in docs/lidar_handoff.md.
- RoboSense rs_driver — the vendored driver kernel
- rs_driver device specs — tex dimensions for all 20 supported types
- TouchDesigner CPlusPlus TOP API
- RS-Helios-5515 User Guide v3.0.1 — consult RoboSense / Reichelt for the PDF