Roll out an optimization safely

The range (30 days, 90 days, 180 days or 1 year) and the filter bar (Provider, Model, Team, API key and so on) narrow every figure on the Overview tab.

Columns are Mechanism, Mode, Applied, Would apply, Saved, Would save and Conf p50. The Mode badge names the current mode. Would apply and Would save read — here for every mechanism except Prompt Optimization; the per-window observe counts are in Optimization settings.

The Settings tab of Optimization settings counts applied and would-apply requests per mechanism in its Last 7 days column, described in the next station.

Switch its Breakdown by to Model and write down Success and $/completed request per model. After promotion you want Success unchanged and $/completed requestlower. Gateway health's Request outcomes and Retries & fallbacks give the same baseline over time.

Recommendations on Optimizations names what a change could still save, with an estimated monthly effect that is never added to savings. The Output budget, last 30 days (UTC) line, under the mechanism table in Optimization settings, shows Caps applied, Observed safety rate, Rollbacks · cost and Added latency p50 / p95.

The mechanism table has columns Mechanism, Mode, Min confidence, Last 7 days, with one row per mechanism.

It reads {applied} applied · {would_apply} would; hover it for saved {fmtUsd} · would save {fmtUsd}. A row with 0 applied · 96 would has enough evidence to act on; a cell that reads — has no rows yet, so keep observing.

No badge means the control is live. Observe only means you can set auto and save, but no request changes because of it yet. Not wired means the control is disabled and the value is stored but ignored. Hover either badge for the backend line behind it.

A number input that accepts 0.5 to 1 in steps of 0.05, present only for Semantic response cache, Output budget and Routing adjustments (the others show —). In auto, a decision below this bar behaves as observe and records the reason low_confidence, so start at the default and lower it only if almost every request reads skipped · low confidence.

Leave every other row as it is. For In-flight dedup or Output budget the banner Auto dedup and output budgets change what clients receive. appears at once, with the consequence: Coalesced followers stream nothing until the leader settles; a capped call that truncates bills one rollback attempt.

Because a mechanism moves to auto, the dialog Start applying these mechanisms automatically? opens. Its body names the promoted mechanism and warns that clients may receive cached or coalesced answers, a smaller output budget, or a different model.

The button reads Working…, then the table shows the saved mode and the Mode badge in By mechanism on Optimizations reads auto. Cancel keeps your draft unsaved. A failed save shows the server message or Save failed.

Open it at the foot of the tab, paste an OpenAI-style body in the textarea and click Check decision. The plan grid shows Reference as {reference.model_key} ({reference.source}) and one cell per mechanism as {mode} · {decision} · {reason}. It runs the decision node only: no policy, no provider call, nothing stored.

Narrow with the API key chip if you promoted for one team's traffic. The Request drawer opens.

Its summary counts modes in the plan, for example 1 auto · 4 observe. After your save the auto count should have risen by one.

Seven rows, one per mechanism. Each shows the mode badge (auto green, observe amber, off grey), the decision word when it is not skip, a state badge, and the confidence as a percentage when above zero.

Use the table below. A row that says observe did not change the request; it only recorded what it would have done.

An executed request shows Reference cost (est.), − Routing (est.), − Compression (est.), = Cost. A served request (cache hit or dedup follower) shows a single Served · saved (est.) line. The footer reads engine added {n} ms. View full request opens the request page, whose Savings receipt splits the saving by mechanism.

Apply the same filter you used in Observability, for example the API key chip set to one key, so the figures match the traffic you promoted.

Routing, Compression and Cache + dedup step the Reference down to Cost. The Provider cache (est.) step that follows is never Cloud Score savings.

Applied climbs, and Saved fills where the mechanism records a dollar estimate. Prompt optimization has its own tab, Prompt Optimization.

A promotion that works removes the recommendation it addresses. Their monthly figures stay estimates and are never added to savings.

Choose off only if you also want to stop recording; observe keeps the would-apply counts growing so you can try again with a higher Min confidence. Leave Min confidence and the advanced fields as they are; the mode alone decides whether anything is applied.

No dialog: only a promotion to auto or a cache default going on asks first. The button reads Saving…, then the table shows observe and theMode badge in By mechanism on Optimizations reads observe.

Entries already stored stay in the store. To clear them, use the Purge row of the Response cache section on the same tab with One API key or One model and confirm Purge cache entries?. A caller can also opt any single request out with constraints.cache = "no-store" or the X-Dex-Cache-Control header, no admin needed.