All projects
Manipulation · Active
Physical AI on Jetson
A robot manipulation program that carries a learned policy from Isaac Sim to a real arm, with Jetson inference measured at every step.
SimHardwareView on GitHub
The problem
Manipulation policies are usually shown in simulation or in a short demo clip. The question that matters for deployment is different: does the policy still work when it runs on embedded compute, against a real arm with its own kinematics, under real latency?
Approach
- Build the robot and scene in Isaac Sim from the manufacturer’s URDF, described in OpenUSD.
- Train an ACT policy for a pick-and-drop task in simulation.
- Run inference on Jetson and measure it, instead of assuming desktop GPU numbers carry over.
- Integrate with ROS 2 so the same interfaces drive the simulated and physical arm.
Status
Policy training in Isaac Sim is in progress. Hardware transfer follows once the policy completes clean episodes in simulation.