Raindrop Simulations, in public beta since 5 November, rebuilds the world an AI agent works in and replays real production conversations against every pull request that changes it. The result is a verdict in the pull request before anyone merges a new prompt, tool or model. For European organisations that run agents in customer service, finance or operations, that is a useful form of change control. It also means sending full, unredacted agent traces to a US vendor. This article explains what the review proves, what it does not, and how to weigh it against your data obligations and the EU AI Act.
What Raindrop Simulations does
Raindrop, a San Francisco company from Y Combinator’s Winter 2024 batch, started out monitoring AI agents in production and flagging silent failures such as hallucinated answers, tool misuse and agents stuck in loops. Vercel, Framer and Clay are among its customers. In September it closed a Series A led by CRV, bringing total funding to $50 million, and announced Simulations.
The product moves that detection work from production to the pull request. According to The New Stack, it reconstructs an agent’s production environment, including its tools, APIs and databases, so developers can see how a prompt edit, model swap or new tool changes its behaviour before shipping. During early access, customers and Raindrop’s own team ran more than 33,000 simulations across more than 1,700 pull requests.
The mechanics, from the Raindrop documentation: the head commit of the pull request runs in an isolated sandbox that Raindrop calls a World. Your code, prompts, agent loop and model calls run for real. Tools and external services are simulated, with each response grounded in your production traces. Production services and credentials are never reached. By default each review replays ten recent production events, compares every new run with what happened in production, and runs Raindrop’s anomaly detection over the result. Raindrop describes what it tests as changes to the agent harness, the software around the model that turns it into an agent, which we explain in our guide to AI harnesses.
Why agent changes need change control
A conventional software release can be reviewed by reading the diff. An agent release cannot. One sentence in a system prompt or a switch to a cheaper model can change which tools the agent calls, in what order and at what cost, while the final answers still look fine. “Every time agents run, they go different paths,” Raindrop’s CTO Ben Hylak told The New Stack. Predefined evals mostly catch the failures a team already expected.
Two recent findings sharpen the point for risk and compliance teams. In June, OpenAI reported that its models recognised classic safety evaluations as tests almost every time, against 5.4 per cent for real production conversations, and built Deployment Simulation to test candidate models on de-identified production traffic instead. And on 20 October, OpenAI disclosed that two of its own internal coding agents had weakened the monitoring meant to watch them during routine tasks, one of them then denying it, as the Cloud Security Alliance summarised. Nobody had written a test for that behaviour in advance.
For an organisation, the lesson is that a model swap or prompt edit is a release, and deserves the same evidence and approval as any other release. Raindrop’s bet is that the pull request is where that evidence belongs.
What a review tells you, and what it does not
A review ends in one of five verdicts: High (a regression worth fixing before merge), Medium (a behaviour change worth checking), No findings, Incomplete and Could not review. It shows each replayed event side by side, before and after the change, with tool-call counts and failures per trace. Runtimewire adds that it reports cost changes, output drift and tool errors.
Four limits matter when you treat that verdict as evidence:
- Sample size. Ten events per review is a smoke test, not a statistical guarantee. Raindrop runs several simulations to absorb normal variation and compares them with production history, but a rare failure can still slip through.
- Faithfulness. Where production data is missing, Raindrop fills the gaps with synthetic data, which Hylak concedes is less faithful. Each review comes with an estimate of how closely the World matched production and what lowered it.
- Test awareness. Models can notice inconsistencies in synthetic data and realise they are being tested. Raindrop says it measures this, but no test setup can rule it out entirely.
- Coverage. Only TypeScript, JavaScript and Python agents are supported, Python agents replay saved events only for now, and the World can only simulate tools it has seen called in production.
Used well, the verdict is a strong signal that tells a reviewer where to look. It does not replace targeted tests for known high-risk scenarios or a staged rollout.
The data trail European teams need to map
Simulations only works if Raindrop sees real conversations, tool calls and prompts. The setup guide requires at least 100 events in the project, recorded tool calls and unredacted prompts, with personal data handled through Raindrop’s PII redaction. The GitHub app also reads the repositories you connect.
According to Raindrop’s data security page, event data is stored on AWS and every listed subprocessor, including OpenAI, Anthropic, Google, Modal and Microsoft Azure, processes data in the United States. Raindrop holds a SOC 2 Type II report and says it does not train models on customer data. We found no EU region, and nothing in the documentation about a data processing agreement or the transfer mechanism it relies on. A third-party overview on beri.net lists data residency as unsupported and self-hosting as beta.
That matters because agent traces are rarely free of personal data: customer names, order histories and support messages end up in prompts and tool responses. If you rely on the EU-US Data Privacy Framework, note that its adequacy decision survived the General Court in September 2025, but an appeal is pending before the Court of Justice. A fallback plan with standard contractual clauses is prudent. There is also a trade-off inside the product: the more you redact, the less the World resembles production.
Where it fits under the EU AI Act
Most AI agents are not high-risk systems under the AI Act. For those that are, for example agents involved in credit scoring, recruitment or access to essential services, Article 9 requires testing throughout development and before the system is placed on the market or put into service, against metrics and probabilistic thresholds defined in advance. Since the Digital Omnibus, those obligations apply to stand-alone Annex III systems from 2 December 2027, as our AI Act timeline explains.
A replay of production traffic on every pull request is the kind of record that helps there, with two caveats. First, Raindrop’s verdicts are anomaly flags, not the predefined metrics and thresholds Article 9 asks for: you still have to define what “acceptable” means for your system and keep the evidence. Second, a significant change to an agent can also raise the question of whether you have substantially modified a high-risk system, which brings provider obligations with it. A review history in pull requests makes that question easier to answer, but it does not answer it for you.
How it compares
Raindrop is not alone in testing agents against simulated worlds. OpenAI does it internally for its own models, including coding agents whose tools are simulated from recorded interactions. Sierra’s Ï„-bench tests customer-service agents against LLM-simulated users, databases and APIs, and Sierra generates simulation suites for agents on its own platform. Eval platforms such as Braintrust and LangSmith focus on curated test sets, and on Hacker News some practitioners pushed back on Raindrop framing evals and production monitoring as an either-or choice. In practice most teams will need both: curated tests for what you know can go wrong, replays for what you did not think of.
A five-step plan for European teams
- Classify your agents. Which ones touch personal data, money or decisions about people, and which could fall under Annex III?
- Treat every prompt, tool and model change as a release, with an owner who reads the evidence and approves the merge.
- Log full traces, including tool calls and tool responses, and decide which fields must be redacted before they leave your environment.
- Run a pilot on a low-risk agent first, with a data processing agreement and a documented transfer mechanism in place, and check how faithfulness changes as you redact.
- Define your own thresholds for what a “Medium” or “High” verdict means for your release process, and keep the review history alongside your other test records.
If you are still designing the layer around your agents, our guide to harness engineering covers the sensors and controls that make this kind of testing possible.
Frequently asked questions
How much does Raindrop Simulations cost?
Raindrop charges per pull request, covering the reconstruction of the environment and the simulation runs, and initially bills the agent’s model usage too. It has not published a per-pull-request price. Simulations also requires Raindrop’s monitoring product.
Does a simulation touch production systems?
No. According to Raindrop, the agent runs in an isolated sandbox without access to production services or credentials. Only the calls to the model are real.
Can Raindrop process data in the EU?
Not according to its published documentation, which lists only US processing. Ask Raindrop directly about EU hosting, its data processing agreement and its transfer mechanism before sending production traces.
Does it replace evals?
No. Raindrop replays real production traffic and can also run your existing test cases. Curated evals remain the way to test known risks against fixed expectations.
The rehearsal is becoming the release record
For agents, the diff no longer tells you what a change does. Tools like Raindrop Simulations turn that invisible shift into something a reviewer can read and an auditor can find later. The technology is promising and still in beta. The harder work for European organisations is deciding who reads the verdicts, what threshold blocks a merge, and which data is allowed to make the trip to the rehearsal.
