A bunch of sensors and a dream
For many years working in self-driving tech, I've been fascinated by the possibilities our sensor rigs can open in robotics. Cameras are good at capturing the semantics and visual fidelity of the world. Lidar scans provide physically grounded geometry which helps build an extremely accurate 3D map. But the coolest and most challenging thing is to combine them. If we do, we can reconstruct the surface of objects and add texture to it. The result is a polygon mesh and it is a good way to capture an accurate spatial memory of the world. With some additional work, these memories can be turned into realistic training environments for robots, thanks to NVIDIA's Omniverse stack.
In this project, Kadr, I'd like to explore how to make these virtual environments and how useful they can be for humans and robots.
Why now?
In 2017 these capture capabilities were only available to projects with multimillion budgets. Lidars were notoriously expensive, not very reliable, and clearly not something you could carry. In an autonomous car, the compute unit itself was a server-grade unit with a high-end CPU and GPU. The whole system drew hundreds of watts of power. NVIDIA Jetson devices had already been around for a while but still kinda struggled, at least until they introduced the Xavier generation.
Fast forward to 2026. I see several trends converging:
- Lidars got cheaper, more compact, more reliable. The Livox Mid-360 is now a sub-$1000 sensor, and the Mid-360S is even cheaper. Lidars are clearly not obsolete. A lot of 3D scanners utilize all sorts of lidars, and the demand will only grow.
- Compute made a giant leap forward. The current Jetson AGX Orin is a beast. It is even comparable with GTX 1080 Ti in terms of performance: 2048 CUDA cores vs 3584. TX2 (the Jetson of 2017) had only 256 cores. Feature-wise, this is the tip of the iceberg: AGX Orin is just way more advanced.
- The PyTorch and NVIDIA software ecosystems matured. They provide better tools for model optimization, deployment, inference. They have mature tools for simulation: plenty of robotics projects now run on NVIDIA Isaac.
- AI.
AI
I hope AI can really boost this project. It used to take whole specialized teams to do the CAD, assemble the sensor rig, write the firmware, build the infrastructure, business logic and make it work reliably. I'm sure those times are over. One person who knows how everything fits together, plus several specialist AI agents, can get far in 2026. The trick is to manage the scope of the project. Maybe I'm completely wrong and too optimistic, but come on, the whole point of the experiment is to see how far I can go.
Sensors and a dream
Here are the main characters of this project:
My priority is to build the Kadr-1 rig, use as much battle-tested software as possible (ROS2, TF2, FAST-LIVO2, a lot of PyTorch everywhere), and design an architecture that produces a valid polygon mesh from the space I scan. The mesh should be importable into NVIDIA Isaac Sim and renderable for the web. For viewing online, I may have to convert it to gaussian splats, but that's a secondary problem. Of course, there are plenty of known and unknown unknowns, but that's the point. I'm thinking on the fly here.
The rig should keep the sensors fixed in place and leave no room for any drift. This is the #1 hardware challenge which determines the future of the project. My next short term goal is to design and print a rigid case for sensors, install it, calibrate the sensors and make sure I leave as little room for parallax as possible. But that's a topic for some future notes.