Splat2Act preprint reports 63.9% robot pickup success after phone scan

While many expect robotics to reach 'zero-shot' deployment, needing no site-specific adaptation, new evidence points away from that future. A generalist vision-language-action policy achieved 0% pickup success in new environments without adaptation. Instead of abstracting away the site, the cost center for robotics is shifting: less physical demonstration labor, more investment in synthetic digital-twin compute.

The finding matters because the same arXiv preprint , not peer-reviewed, says π₀.₅ reached 0% pickup success in captured deployment scenes without adaptation, versus 62.5% aggregate pickup success for its strongest personalized policy.

The scan replaces a demonstrator

Splat2Act is not presented as a better robot hand, a cheaper arm or a new generalist policy. The preprint describes an automated real-to-sim-to-real pipeline that personalizes a pretrained vision-language-action policy to a specific scene and object set using only a handheld phone scan and images of the user’s objects.

Its core claim is labor substitution: less real-world demonstration collection in every user environment, more synthetic demonstration generation inside a reconstructed scene. No one in the reported packet is on the record, and the evidence comes only from the paper’s summary rather than independent deployment accounts.

The paper reports that Splat2Act builds a photorealistic digital twin with 3D Gaussian Splatting and collision geometry, discovers tabletop and non-tabletop receptacles, constructs feasible pickup tasks and generates successful demonstrations with a privileged simulation oracle. That is a narrower claim than zero-shot manipulation.

The preprint’s own baseline makes the point: a strong generalist, π₀.₅, achieved 0% pickup success in captured deployment scenes without adaptation. If that number holds outside the paper, the cost center in indoor robotics shifts from collecting demonstrations by hand to making site-specific synthetic data reliable enough to buy.

The benchmark is site-bound

The paper’s headline numbers are substantial but bounded. It says the pipeline was applied to 101 scanned campus environments with a median navigable area of 104.87 m² and 7 receptacles. Across those scans, the pipeline yielded 852 receptacles and over 68,000 pickup demonstrations.

The strongest personalized policy reached 62.5% aggregate pickup success in the target scene. On a real Stretch 3, the same policy achieved 63.9% success on a 12-task subset, close to its 66.7% simulated success on those tasks. Those figures measure adaptation inside captured deployment scenes, not broad autonomy across unseen layouts.

The missing comparison is economic. The preprint says collecting real demonstrations in every user’s environment is prohibitively expensive, but it does not report the cost of phone capture, 3D Gaussian Splatting reconstruction, collision geometry creation, simulation-oracle generation, fine-tuning or failed deployment retries.

It also does not establish whether the 12-task subset on Stretch 3 is representative of the harder long tail in buildings. A procurement team would still need to ask what baseline is being beaten, on what hardware, with what operator time, and how often the simulated policy fails when objects, lighting or receptacles do not match the scan.

Zero-shot is the wrong buyer story

The dominant read is that generalist vision-language-action models will eventually remove site-specific robotics setup. Splat2Act points the other way. In the preprint’s setup, the generalist policy does not travel cleanly into captured deployment scenes; it needs local adaptation.

The commercial implication is not that every office, lab or indoor facility can skip setup. It is that the setup may become a scan-and-train workflow rather than a teleoperation campaign. That changes who owns the rollout: less field demonstrator labor at each site, more responsibility for teams that can maintain digital twins, simulation assets and fine-tuning runs.

The counter-read is straightforward: this may be a campus-scene result rather than an enterprise deployment result. The source does not report outside customers, independent replication, injury-sensitive tasks, long-duration operation or messy object categories beyond the constructed pickup tasks.

It also relies on a privileged simulation oracle, which is not the same thing as a robot learning safely from ordinary work. For robotics buyers, the falsifiable question is whether vendors can keep the real Stretch 3 number close to the simulated result when the site is not curated for a paper and the objects are not convenient for reconstruction.

Labor moves upstream

If the result is reproducible, the underpriced labor change is not mass replacement at the robot’s end effector. It is a restructuring of deployment labor before the robot arrives. Today’s costly unit is the site-specific demonstration: humans showing a robot how to handle shelves, counters, tables and user objects.

In the Splat2Act framing, the human provides a phone scan and object images; the system then constructs the scene, finds receptacles and produces demonstrations in simulation. The work does not vanish. It moves from physical demonstration and reset labor into spatial capture quality control, simulation validation and exception handling.

That leaves an under-noticed middle exposed: integrators whose advantage is collecting and managing demonstrations in each customer environment, and buyers who assume a generalist manipulation policy is a reusable asset once purchased. The preprint instead suggests the durable asset may be the site model and the personalized policy trained on it.

The signals to watch are concrete: whether follow-up papers report more than 101 environments, whether real-robot testing expands beyond a 12-task subset, whether the simulated 66.7% success continues to track real 63.9% success, and whether future versions disclose the cost of generating over 68,000 demonstrations.