Teaching Machines Dexterous Manual Judgement
Language models scaled on billions of internet tokens. Robots have no web of physical demonstrations. We build sim-to-real precision agriculture pruning policies and crowdsourced mobile skill challenges that generate dense bimanual reward models from camera feeds.
Grapevine Pruning: Judgement Over Repetition
Unlike repetitive pick-and-place, winter grapevine cane pruning requires complex spatial reasoning: deciding which dormant canes to keep, counting viable buds, and executing clean shear cuts under severe visual occlusion.
Interactive Cane Pruning Decision Model
Click on a candidate node (A, B, C, D) to evaluate bud retention and model reward score.
The Mobile "Skill Challenge" Data Engine
Niantic used mobile player scans to assemble geospatial models. We adapt this crowdsourcing paradigm to domestic robotic manipulation: mobile skill challenges where users attempt physical dexterity tasks and on-device computer vision grades each attempt.
The Breakthrough: The Rater is the Reward Model
A neural vision model that scores "did they fold the shirt cleanly / stack the tumblers stably / tie the bowline correctly" is fundamentally a success detector—identical to an RL reward function. Every attempt yields dense, labelled trajectory data that passive video scrapers can never provide.
Structured Domestic Tasks over Random Motion
Instead of front-camera fitness jumping jacks, our challenge specifications feature propped-phone or headband egocentric angles capturing close-up hand-object physics: deformable dough, cloth, micro-beans, and fluid meniscus alignment.
10 Standardized Camera-Rated Manipulation Challenge Concepts
Scored primarily from end-state verification plus temporal hand-pose trajectories. Phase 1 targets (Steady Stack, Dal Sort, Fold Off) form the foundation for undergraduate student vision raters.
Steady Stack
Stack steel tumblers, brass cups, or coins as high as possible against a 30s timer.
Dal Sort
Separate mixed rajma, chana, and moong beans into three distinct bowls.
Fold Off
Fold a cotton T-shirt or towel into a compact rectangle against time.
Roti Round
Roll dough into a uniform circle using a traditional rolling pin (belan).
Knot Master
Tie a specified knot (shoelace, reef knot, or bowline) shown on screen.
Pour Perfect
Pour water from a container into a glass precisely to a marked indicator line.
Privacy Architecture Designed Around India's DPDP Act
Collecting visual data from homes requires ethical safeguards and privacy preservation. YuktYantra specifies on-device sanitization before any video chunk ever leaves the participant's device.
Local Face & Identity Masking
A lightweight edge model detects and blurs human faces, personal photos, and background identity markers directly in device RAM before video encoding.
Strict 18+ Verifiable Opt-In
Explicit digital consent flow designed around DPDP requirements. Zero collection from minors, with encrypted, revocable contributor consent logs.
Selective Wi-Fi Sync
Only scored challenge episodes are preserved. High-score attempts and informative failure trajectories are queued locally and synced over Wi-Fi.