System prompts can be reconstructed from LLM outputs

Researchers said they have created a technique that can rebuild an original prompt from a large language model’s output with high accuracy, even when only a single response is available.

The approach, called Previous-Token Prediction , is designed to work without access to a target model’s internal weights or its architecture. Researchers said the method relies on training an inverse model using synthetic data to infer what input likely produced a given output.

Previous-Token Prediction and how the method works

According to the researchers According to the researchers, the inverse model can recover both exact prompts and also generate semantically similar versions that closely match the intent of the original instructions. They said this reconstruction can be performed from one response, rather than requiring multiple outputs or interactive probing. The researchers also said the technique can remain effective even if the attacker does not know which underlying model produced the text. That claim, if borne out across deployments, broadens the range of systems that could be exposed through the text they publish. Security and intellectual property exposure for organizations Officials and companies that embed LLMs into workflows often rely on system prompts to define behavior, constraints, and operational rules. The researchers said proprietary system prompts can include trade secrets, internal moderation logic, or specialized instructions that shape how a model behaves in production settings.

Token Prediction

The ability to reverse-engineer those instructions from ordinary outputs raises both security and intellectual property concerns. Organizations could face risks if sensitive operational guidance becomes extractable from responses produced in public-facing products or services.

Implications for AI-integrated workflows and mitigations

The findings point to a potential change in the threat environment for AI deployments, the researchers said. In particular, they warned that prompt leakage may occur through routine output exposure rather than through direct compromise of a model provider’s infrastructure.

As a result, organizations using LLMs for sensitive tasks may need to reassess what is embedded in prompts and what is emitted in responses. The researchers said future security approaches may require new mitigation strategies to reduce the chance that proprietary instructions can be reconstructed by unauthorized parties.

Key unknowns include how reliably the method performs across different operational settings and how well defenses can limit reconstruction while preserving output quality. The researchers’ results nonetheless underline that output text itself can carry recoverable traces of the instructions that generated it.