NVIDIA surgical simulator brings AI to robot training labs

NVIDIA surgical simulator Cosmos-H-Dreams turns live robot commands into surgical video, giving researchers a faster test bed for AI policies.

NVIDIA said the model can produce operating-field video at about 160 frames per second on one RTX PRO 6000 GPU. That compares with roughly 10 frames per second for the slower system used to create it, putting the tool closer to the speed needed for interactive testing.

The company made the model weights and serving code available on July 23, 2026, and followed with a technical explanation four days later. The release matters because surgical robotics research often depends on simulators, but hand-built tissue physics can be costly, narrow and slow to adapt.

Real-time video replaces hand physics

Cosmos-H-Dreams does not use a conventional physics engine. NVIDIA said it learns from paired surgical video and robot movement data, then generates the camera view that should follow a given set of commands.

The system starts from one image, reads a live sequence of robot-arm actions and creates the next 12 video frames before taking in another action batch. NVIDIA said it can be steered from a browser keyboard, mapped through Meta Quest controller motion or connected to a trained surgical policy.

The released version is limited to tabletop suturing on the da Vinci Research Kit, a research platform used in academic labs. It produces video at 288 by 512 pixels and receives commands at an effective 10 Hz, which makes it a bench simulator rather than a clinical operating-room product.

Distillation bought the frame rate

NVIDIA said the faster model was distilled from Cosmos-H-Surgical-Simulator, a larger offline teacher built on its 2-billion-parameter Cosmos-Predict2.5 video model. The teacher can evaluate a recorded trajectory after completion, but it was not designed for a user or autonomous policy to guide the scene while it is being produced.

The student model was trained to continue from its own generated history, a method NVIDIA calls self-forcing distillation. It also creates each latent video frame in two to four denoising steps, while the FlashDreams inference library keeps a rolling cache of previous frames and reuses repeated GPU work.

NVIDIA also kept flawed examples in the training mix, including dropped needles, missed throws and knots that failed. The company’s stated logic is direct: a simulator used to judge robot policies must show what poor control choices cause, not only what successful demonstrations look like.

CMR connects Versius to Cosmos

NVIDIA said it worked with CMR Surgical and Cambridge Consultants, the Capgemini-owned engineering firm, to link the model with the surgeon controller used for CMR’s Versius system. CMR showed the setup to a surgical audience at a robotic surgery conference in Florida in late July 2026.

CMR’s role is larger than a demonstration. NVIDIA credits the company with nearly 500 hours of anonymized Versius procedure data for the Open-H Embodiment dataset, and CMR describes itself as the dataset’s largest contributor.

The companies are also drawing a sharp regulatory boundary. CMR said the simulation is "for research and demonstration purposes only," is not included in the cleared Versius Plus system and is not intended for patient care or clinical decisions.

Versius Plus is cleared in the US only for adult gallbladder removal, while a gynecology use remains pending, according to CMR’s release. NVIDIA also described Cosmos-H-Dreams as a research and development platform, not a diagnostic system and not software for controlling a physical surgical robot.

Three paths for adoption

If the model stays accurate outside tabletop suturing, NVIDIA gains a stronger position in robotics infrastructure: its GPUs, inference libraries and world models could become part of how labs test surgical AI. That path would help the wider surgical-robotics sector cut experiment time without immediately changing hospital purchasing or clinical rules.

If accuracy breaks when procedures, anatomy or instruments change, the macro effect is narrower: investment may still flow into world models, but clinical translation would slow. NVIDIA would then have a useful research demo rather than a broadly reusable simulator, and robotics companies would remain dependent on task-specific validation.

If regulators and manufacturers later accept video-generated environments as one layer of evidence, the sector could move toward larger preclinical testing pipelines. The open questions are whether simulated failures match real failures, whether anonymized surgical data can scale safely and whether procedure-by-procedure clearance will keep adoption confined to narrow use cases.