The announcement, posted on October 1, describes a view of playtime, payer conversion rate and average revenue per user that updates every five minutes. Creators can find the metrics in Creator Hub under Experiments. The early analysis runs during the first 24 hours, or until the first daily results arrive. These numbers are intended to help identify a serious regression before the experiment affects players for an entire day.
A red warning requires more than a negative headline number. Roblox says the entire confidence interval must fall below the relevant harm threshold: a 10% decline for playtime, or a 20% decline for either average revenue per user or payer conversion rate. The system still displays estimated lift with fewer than 10,000 enrolled players across the variant and control, but it does not make a harm decision at that sample size.
When the system detects critical harm, it notifies both the game owner and experiment owner through email and the Creator Hub notification tray. A webhook is optional for teams that want notifications in their own tools. Receiving an alert leaves the decision with the creator: the announced workflow is to inspect the metrics and decide whether to stop the experiment early.
The update also checks sample ratio mismatch every five minutes during the first day, then daily afterwards. This condition means the enrollment proportions differ from those expected for the experiment and can make its results unreliable. Roblox recommends stopping and restarting an experiment when this warning appears. It is a separate signal from an unfavorable change in a performance metric.
For a team planning a test, a useful preparation is to agree who will inspect an alert and what questions they will ask before the test begins. Keeping the proposed change, intended audience and stopping rationale together can make that discussion easier. Those are practical planning suggestions, rather than additional platform requirements or a guarantee that a particular experiment will be safe.
Early warnings should also be read in the context of the test they describe. A quiet notification tray does not establish that a new design is better, and a provisional improvement does not answer every question about player experience. Roblox explicitly reserves rollout decisions for daily results and the full experiment duration. Outside critical harm or sample ratio mismatch, its guidance is to let the experiment finish instead of reacting to early novelty effects.