Provider key migration: an afternoon per service

Getting provider keys out of your services

Companion to "Getting provider keys out of your services". Engineering guidance, not security advice for a specific environment.

The migration is small. The project around it is not, and almost all of that project is inventory and revocation. Plan accordingly.

Phase 0: find out what you actually have

Do this before touching any service. It usually takes longer than every code change combined.

Provider dashboards: list every key, its creation date, and its last-used date. Last-used is the field that matters. Repositories, including internal ones. Internal repositories are more likely to contain a plaintext secret than public ones, because nobody is scanning them with the same urgency. CI/CD variables and runner environments. On compromised machines studied in 2025, most exposure sat on build infrastructure rather than laptops. Notebooks, shared drives, ticket comments, chat history. Container images and build artefacts. Anything a contractor or an agency was given, ever.

Expect to find keys with no owner. Record them anyway; they are the ones revocation will surface.

Phase 1: pick the pilot service deliberately

Not the simplest service. A representative one.

Uses streaming, if any of your services do. Uses tool or function calling, if any of your services do. Sends a provider-specific parameter, if any of your services do. Has enough traffic that a response comparison means something. Has an owner who will still be around next week.

Phase 2: the two-line change

Base URL points at your endpoint. Key is the application key, from the vault, not a provider key. Model name is logical, so routing can change without a deploy. No SDK change. If you are swapping client libraries, something has gone wrong with the plan.

Then verify the things that do not always survive:

Streaming event sequence and usage accounting. Tool call shapes, including parallel calls and strict schemas if you use them. Any provider-specific request fields, including reasoning, thinking and safety parameters. Error shapes, status codes and retry behaviour under failure. Timeouts. The path is one hop longer than it was.

Phase 3: compare before you cut over

Run both paths for a period that covers your unusual routes, not just your busy ones. Compare responses on a sample, not just status codes. Compare latency at p50 and p99, and know what the added hop costs you. Confirm the audit record contains what you will need later: project, policy version, model, decision.

Phase 4: roll out

One service at a time, each with a named owner. Application keys issued per project, not one shared key across the estate. Per-project rate and budget limits set at rollout, not "later". This is the control that caps the damage from a leaked application key, and later never arrives. Alerting on unusual volume per project.

Phase 5: revoke, which is the actual migration

Everything before this point is additive and reversible. This step is neither, which is why it works.

Pick a window during working hours with people watching. Disable rather than delete first, if the provider supports it, so rollback is a click. Revoke provider keys in order of last-used date, oldest first. Keep a rollback path for a short, fixed period, then close it. Log every breakage. This list is the inventory you could not build in Phase 0. Rotate anything that ever touched a repository, even if it appears unused. Cleaning git history without revoking does nothing. Confirm at the provider that no key outside the vault remains active.

Phase 6: keep it that way

Secret scanning in CI, with push protection where available. A rule that provider credentials exist in exactly one place, and a periodic check that it is still true. New services get an application key by default, so the ungoverned path is never the easy path. The gateway itself is treated as production infrastructure: pinned versions, reviewed updates, restricted egress, vaulted upstream credentials.

What to tell your security reviewer

Three sentences, and they are all inventory claims rather than architecture claims:

1. Provider credentials exist in one vault, held by one layer, and no application holds one.
2. Every call is attributable to a project, and every project has a spend ceiling.
3. Revocation of any application key is a single action with a known blast radius.

If any of those is not true yet, the migration is not finished, whatever the code says.