Centimeter-accurate camera tracking for large-scale 3D capture (Gaussian splatting)

Overview

This customer project comes from Aryonix, founded by Alexandre Cavaleri. Aryonix develops a real-time rendering engine capable of creating cinematic, photorealistic and immersive 3D environments from photographs. Aryonix collaborated with Downhill Wings, a project dedicated to exploring, mapping and filming downhill routes for inline skates, longboards and other gravity sports. The project uses a technique called Gaussian splatting.

What is Gaussian splatting? In simple terms, Gaussian splatting turns hundreds or thousands of photographs into a detailed, interactive 3D model of a real-world scene. Unlike traditional 3D representation built from polygons or meshes, it represents the scene as millions of tiny colored 3D points ("Gaussians"), allowing highly realistic rendering from almost any viewpoint.

For downhill sports, it is important that a digital reconstruction matches the real world. The route must have the correct position, scale and slope so riders can accurately evaluate its difficulty, safety and overall riding experience.

The main challenge with reconstructing such long outdoor routes is computing the camera poses: tools like COLMAP drift and accumulate errors over the distance, struggle with naturally occurring repetitive patterns and featureless views (vine rows, road surface, sky), and without a GPS prior the result is not tied to real-world coordinates, scale or slope.

To overcome these limitations, Aryonix used GNSS Compass – RTK Starter Kit to provide centimeter-accurate camera positions and heading during the recording.

The rig

To solve this we used a GNSS compass starter kit to provide centimeter-accurate camera positions and heading as camera priors to COLMAP.

GNSS Compass – RTK Starter Kit: GNSS/RTK kit with UM982 based board and two high quality helical antennas
Blackmagic URSA Mini Pro 4.6K G2 cinema camera, shooting RAW
Hosa MIT-129 XLR adapter, converts the board’s PPS signal to audio levels compatible with the camera, and protects the board from the camera’s phantom power
Phone running SW Maps, RTK logging + powering the board

Synchronizing Video with RTK

Knowing the camera position is only useful if every video frame can be matched to the correct RTK measurement.

The GNSS receiver outputs a PPS (Pulse Per Second) signal. This is a highly accurate electrical pulse generated exactly once every GPS second. Instead of relying on the camera’s internal clock, this PPS signal was recorded directly into one audio channel of the camera through its XLR input.

Because the audio is sampled at 48 kHz, each PPS pulse can be detected with sub-millisecond accuracy. In comparison, a single video frame recorded at 24 fps lasts about 42 milliseconds. This allows every frame to be timestamped much more accurately than using the camera’s own timing information.

Recording a PPS pulse every second also makes it possible to detect and continuously correct any clock drift in the camera throughout the entire recording. The PPS signal marks the beginning of each GPS second, while the RTK log recorded by the phone provides the corresponding GNSS timestamps. Starting both recordings at approximately the same time is enough to match the PPS pulses with the RTK data automatically.

If this sounds complex, check our post Understanding GPS Timepulse or PPS.

Capturing the Scene

The team walked an 800-meter route through the Lavaux vineyards in Switzerland while continuously recording video.

Instead of simply walking in a straight line, the camera was moved in a spiral pattern to observe the scene from multiple viewing angles. This improves the quality of the 3D reconstruction while avoiding shadows cast by the operator.

Occasionally, short side paths were captured before returning to the main route, adding additional viewpoints that improve reconstruction accuracy.

Moving objects such as pedestrians and vehicles were handled directly during capture by briefly pointing the camera away, reducing the amount of cleanup required later.

Results

After synchronizing the RTK measurements with the video frames, the camera positions were supplied to the reconstruction pipeline as camera priors. Instead of estimating every camera position from images alone, the software starts from centimeter-accurate RTK measurements.

The resulting 3D representation is rendered in real time using the Aryonix Engine while preserving the correct real-world position, orientation, scale and slope of the route.

Although demonstrated here with Gaussian splatting, the same workflow applies to any reconstruction method that relies on 2D images and camera poses: the synchronized camera priors can be fed to tools like COLMAP to train Gaussian splats, NeRFs, or traditional photogrammetry reconstructions.

By providing accurate camera positions throughout the recording, RTK GNSS transforms a workflow that would normally suffer from accumulated drift into a reliable method for reconstructing long outdoor environments. The result is a photorealistic 3D scene that can also be used for accurate route measurements, including slopes that are often more detailed than those available from existing digital elevation models.