Experiment ·
One driving decision on the Thor, measured against the bench
NVIDIA's open Alpamayo 1 reasoning model answered in 0.89 s on an RTX 5090 and 4.38 s on a Jetson AGX Thor. The Thor spent less energy per decision.
We ran NVIDIA’s open Alpamayo 1 model, a 10 billion parameter vision-language model for autonomous driving that reads four cameras and writes a reasoning trace before it proposes a trajectory, on the two machines in the lab that can hold it: an RTX 5090 workstation and a Jetson AGX Thor. Same eight driving clips from the model’s own dataset, one warm-up decision excluded, three decisions per clip, the upstream sampling defaults and a fixed seed. Every number below comes from a committed JSON record.
| RTX 5090 | Jetson AGX Thor | |
|---|---|---|
| Latency per decision, median | 0.89 s | 4.38 s |
| Latency per decision, p95 | 1.16 s | 5.88 s |
| Time to first token, median | 0.38 s | 1.84 s |
| Decode rate, median | 60.0 tokens/s | 9.7 tokens/s |
| GPU memory after load | 20.6 GiB | 20.6 GiB |
| Power during a decision, median | 377 W (GPU rail) | 64 W (whole board) |
| Energy per decision, median | 316 J | 290 J |
The two power rows are not like for like: the 5090 figure is the GPU rail from the driver, the Thor figure is the whole board’s input power, which includes the CPU, memory and carrier. That makes the energy comparison conservative in the Thor’s favour by an unknown margin on the workstation side, not the other way round.
What it means
The Thor is about five times slower per decision and it uses less energy per decision. Both machines needed the same 20.6 GiB of memory, which is why the 8 GB Orin NX in the rover does not take part: the model does not fit there, and we say so rather than quote a number from a smaller model.
Where the Thor’s time goes is also measured. Under the profiler, the reasoning and trajectory stage dominates (about 4.9 of the 5.9 s of one profiled decision), and the preprocessing of the camera frames is a small slice. We wrote a CUDA kernel for that preprocessing anyway, because it was the one part of the pipeline we could change: a separable resize and normalise pass that runs in 0.65 ms on the 5090 and 10.7 ms on the Thor, against 1.23 ms and 15.6 ms for the PyTorch operators it replaces, verified to within one 8-bit step against the reference and bit-exact between the Python extension and a standalone C++ driver.
What most teams get wrong
The bench number becomes the plan. A decision rate measured on a workstation GPU is quoted as if it carries to the embedded computer, and the embedded computer is then blamed for being slow. It is slow; it is also the only one of the two that fits the power budget of a vehicle, and on this model it does the work for less energy per answer.
What we do about it
We measure on the hardware that will do the job, under the budget it will have, and publish the two side by side with the protocol. The next step is a quantized variant of the model on the Thor, measured the same way, with the trajectory error against the dataset’s own future reported next to the speed, so a faster number cannot hide a worse answer.
The runs, the profiler records, the kernel and its walkthrough are in bench2field, case study 02.