Manipulation
ActivePhysical AI on Jetson
Robot manipulation program spanning measured Jetson inference and probes, measured Isaac Sim evaluation, and an implemented but unmeasured Orin ROS 2 host.
Measured results
- Inference on AGX Thor101.6 ms TensorRT · 9.8 HzMeasured 2026-08-20
- Thermals42.0 °C peakAGX Thor, 2026-08-20
- Isaac Sim evaluation35 of 36 sim turns · 19.2 mm p50RTX 5090, 2026-08-20
- Learning from demonstrations0 → 16 of 50 cup placementsIsaac Sim, 2026-10-19 to 2026-10-22
Sim to real
Simulation · Isaac SimReal world · Physical robotSame arm, same Ludo board. The Isaac Sim digital twin on the left, the physical robot on the right.
- SimulationManufacturer URDF
- SimulationOpenUSD scene in Isaac Sim
- SimulationACT policy training
- Edge computeInference on Jetson
- PhysicalROS 2 control
- PhysicalSynria arm
The problem
Manipulation policies are usually shown in simulation or in a short demo clip. The question that matters for deployment is different: does the policy still work when it runs on embedded compute, against a real arm with its own kinematics, under real latency?
Approach
- Build the robot and its workcell in Isaac Sim from the manufacturer’s URDF, described in OpenUSD, so the digital twin matches the physical bench.
- Evaluate the policy in simulation first, where every run can be repeated.
- Run inference on Jetson with TensorRT and measure it, including thermals, instead of assuming desktop GPU numbers carry over.
- Integrate with ROS 2 so the same interfaces drive the simulated and the physical arm.
Where it stands
Jetson inference and the Isaac Sim evaluation are measured. The Orin ROS 2 host is implemented but not yet measured.
In October the work moved to how the arm learns. On the same 50 cup placements in Isaac Sim, a policy trained on 131 flawless scripted demonstrations placed the cup on its mark 0 times; one trained on 122 demonstrations that kept deliberate mistakes and their corrections placed it 16 times. The same recipe at 291 demonstrations placed it 20 times, inside the noise, so a second placement seed comes before adopting it. MoveIt 2 planning for the arm planned 50 of 50 seeded goals in simulation. No learned policy has run on the physical arm.