Corvus ISR Publishes Synthetic Benchmark Showing Tracker Improvements

Corvus ISR’s latest public tracker benchmark provides a transparent comparison between two models on a fixed-seed synthetic scene. This synthetic setup guarantees perfect ground truth, enabling precise measurement of tracker performance without real-world noise or variability. The scene runs with seed 1337, including a 20-second warm-up followed by 120 seconds of measured data, ensuring consistency across tests. The sensor model, detection generation, and metric definitions are identical for both models, isolating the tracker algorithms as the only variable.

Age 18–24?Offer from Amazon

Prime made for students and young adults

• Fast, free delivery for dorm and study essentials
• Prime Video and Amazon Music included
• Member-only deals

Try Prime for Young Adults →Free trial for eligible 18–24 year olds

As an affiliate, we earn on qualifying purchases.Hide

The two models under comparison are the baseline v1 “greedy nearest-neighbour” and the newer v2 “confirmed-track auction”. The v1 model uses a simple two-pass greedy association, constant-velocity prediction, and fixed 2-second coasting — a deliberately basic approach representing the published floor. In contrast, v2 incorporates a three-tier auction for track confirmation, velocity-consistency gating, noise-scaled reservation prices, and confidence-decayed coasting, reflecting a more sophisticated tracking strategy. These differences are reflected in the benchmark results, which show significant performance gains.

In the published results, the v2 model reduces ID switches per minute by approximately 42% across several scenarios. For example, with 150 movers at 2 fps, ID switches decrease from 2,042 to 1,183. Dense scenes with 400 movers see a drop from 14,032 to 8,040. Additional stressors like frame starvation, occlusion, and jitter also demonstrate reductions of around 18%, emphasizing the robustness of the improved algorithm. Notably, detection rate remains identical, as it is a sensor property, confirming that the improvements stem solely from the tracker.

Why publish these failure numbers publicly? Because both models still commit thousands of identity errors per minute under stress, and synthetic scenes provide perfect ground truth to measure these errors precisely. Every future tracker must be benchmarked with a public matrix against the same seed. This approach promotes transparency; “Vendors who show only successes ask for faith; a published failure matrix asks for measurement.” By openly sharing these results, Corvus ISR encourages honest performance assessment rather than marketing hype.

From an engineering perspective, v2 maintains real-time processing, averaging approximately 1.2 milliseconds per sensor tick at a density of 400 objects. The worst-case scenario reaches about 5 milliseconds, still well within a 10-millisecond budget, making it suitable for live deployment. The entire benchmarking process is accessible via the live demo, where anyone can reproduce the results. Simply press “Run benchmark” without needing signup or NDA, ensuring transparency and reproducibility.

It’s important to recognize that all data in these tests are fully synthetic—no real persons, vehicles, or locations are involved; every pixel is generated. This synthetic environment with perfect ground truth allows for methodical performance analysis free from real-world uncertainties. Publishing failure metrics in such a controlled setting underscores the importance of measurement over marketing, providing a reliable benchmark for future developments.

For science-minded readers, understanding the methodology behind synthetic benchmarking reveals how perfect ground truth enables precise evaluation. It also highlights why synthetic scenes are invaluable in pushing the boundaries of AI-based tracking systems. With the ability to run the same benchmark yourself, you can verify the models’ capabilities and limitations firsthand, fostering a more transparent, scientific approach to advancing AI technology in complex tracking tasks.

Powered by Thorsten Meyer AI