Skip to content
See the work
All work
Jetson Edge AI Security: cover image

Edge inference

Active

Jetson Edge AI Security

Defensive edge telemetry runtime with measured Jetson AGX Thor inference and thread-pool power, plus operator-reviewed alerts.

Read the case study View the code on GitHub

Measured results

  • Detector latency0.0237 ms p95AGX Thor, 2026-09-08
  • Board power54.1 W to 24.3 WDefault vs single-threaded runtime, 2026-09-08
  • Junction temperatureAbout 15 °C coolerSingle-threaded, three sessions
How the system fits together
  1. PhysicalLive network traffic
  2. Edge computeSuricata on AF_PACKET
  3. Edge computeFixed CSV schema
  4. Edge computeONNX inference on Thor
  5. PhysicalOperator-reviewed alerts

The problem

Robots and edge devices sit on networks too. Sending all of their traffic to the cloud for analysis is slow and expensive, so the detection has to run on the device, within its power and compute budget.

Approach

  • edge-traffic-sensor, a companion capture pipeline, uses Suricata over AF_PACKET and writes a fixed CSV schema that the inference side consumes.
  • Development runs on an x86 RTX 5090 host. The production target is Jetson AGX Thor (ARM64).
  • Models run through ONNX Runtime, and inference cost, board power, and temperature are measured on the device.
  • Alerts go to an operator for review rather than triggering automatic action.

What the measurements showed

Runtime threading configuration, not the model, drove most of the power draw. Running the detector single-threaded did the same work at far lower board power and a cooler junction, and the gap held across three sessions. The worst single-event latency is still unresolved and is called out as open.

TensorRT was tested on the Thor on 2026-10-29 rather than assumed. Neither shipped model could become a TensorRT engine, because both are tree and linear ONNX-ML models that TensorRT cannot read, and the harness fell back to the CPU as designed. So the honest description of the edge inference is ONNX Runtime on the Thor’s CPU, where the 106 KB detector answers in tens of microseconds. TensorRT becomes relevant with a neural detector, the next model on the roadmap.

Design principles

One concern per change. A defensive boundary. The data contract matters more than convenience.

Evidence

The charts behind the results

Chart: 31 W more power. Same work.
31 W more power. Same work.
Chart: The gap held in all three sessions.
The gap held in all three sessions.
Chart: Where the 31 W goes.
Where the 31 W goes.
Chart: About 15 °C cooler at the junction.
About 15 °C cooler at the junction.
Chart: Lower power, lower p95 latency.
Lower power, lower p95 latency.
Chart: Background load hit the default config hardest.
Background load hit the default config hardest.
Chart: The cost depends on the workload.
The cost depends on the workload.
Chart: The worst single event is unresolved.
The worst single event is unresolved.