DreamVu HandTrack Outperforms HaMeR and WiLoR in Hand Tracking Evaluation on Ego-Exo4D
Finds 98% of hands in first-person video it was never trained on, with about 60% fewer missed hands than two leading open models.
PHILADELPHIA — October 20, 2026 — DreamVu, a physical AI data infrastructure company, today released results showing that its hand pose model, DreamVu HandTrack, outperformed HaMeR and WiLoR, two widely used open hand pose models, on all six measures in an evaluation on Ego-Exo4D data. HandTrack also runs faster than real time.
Robots that learn manipulation from human demonstrations need accurate hand tracking. Every missed hand or misplaced joint becomes an error in the training data. DreamVu built HandTrack for real-world first-person video. It follows the camera wearer's two hands through a whole recording and keeps left and right apart. Given a camera's calibration, it also adapts to wide-angle lenses that produce a circular image.
DreamVu tested HandTrack on the hand pose validation set of Ego-Exo4D, a large research dataset of skilled human activity such as cooking, bike repair and playing music, recorded with camera glasses. HandTrack was trained on DreamVu's own footage from a different type of camera. No Ego-Exo4D data was used to train or tune it.
All three models processed the same 65 clips and were scored in 2D against the same manual labels: 4,144 frames and 6,477 hands. HaMeR and WiLoR were run with their official pretrained weights and a shared hand detector. Both open models work frame by frame; HandTrack uses the whole video, as it is designed to.
| Metric | HaMeR | WiLoR | DreamVu HandTrack |
|---|---|---|---|
| Joint accuracy (PCK@0.10) | 82.93% | 74.33% | 84.32% |
| Hand coverage (recall) | 95.12% | 94.94% | 98.02% |
| Left/right accuracy | 99.16% | 99.22% | 99.80% |
| Reference-joint coverage | 95.63% | 95.47% | 98.20% |
| Detection precision | 99.47% | 99.36% | 100.00% |
| Detection F1 | 98.13% | 98.02% | 99.49% |
The largest gain is in hand coverage. HandTrack found 98% of hands in the test footage, compared with about 95% for the other models. It made about four times fewer left/right mistakes and produced no false detections. Fewer gaps and fewer mix-ups mean cleaner robot training data.
In DreamVu's tests, HandTrack processes 1080p video at 36.4 frames per second, faster than the video plays. It also has a camera setting for glasses and fisheye cameras.
"Robot learning depends on the quality of the human data behind it. DreamVu HandTrack tracks hands more completely and accurately than the leading open models, on a dataset it was never trained on. And it now runs faster than real time."
— Sashi Reddi, Co-Founder and CEO of DreamVu
Availability
Robotics teams interested in using DreamVu HandTrack can contact DreamVu at sales@dreamvu.ai. A side-by-side video comparison is available at dreamvu.ai/handtrack.
About DreamVu
DreamVu designs and builds patented 360° 3D cameras for robots, and the software that runs on them. The company is headquartered in Philadelphia, PA, with offices in Palo Alto, CA and R&D in Hyderabad, India.
Contact
sales@dreamvu.ai