Open the project. There's a .env file. It has the production database password, the Stripe live key, the AWS credentials, and a JWT signing secret that hasn't changed since the first deploy. The same file is on three laptops, in a Slack thread from onboarding, and — you'll want to sit down for this — in your git history from that commit eight months ago where someone added it by accident and "removed" it the next day.
A removed file is not a removed secret. Git remembers everything. So does anyone who cloned the repo.
What this actually costs
A single leaked cloud credential is enough to spin up a fleet of crypto-mining instances on your account before you notice. The bills from those incidents routinely run into five and six figures before the provider's fraud detection or your own alerting catches it. That's the cheap outcome. The expensive outcome is a leaked production database credential that lets someone read your entire customer table — and now you're not paying an AWS bill, you're paying for breach notification, legal, and the enterprise contracts that walk.
Secrets are the highest-value target in your system. One credential is often the whole game. And they leak constantly: scanners crawl public GitHub continuously, and a committed AWS key is typically found and used within minutes of being pushed. Not days. Minutes.
The good news: this is one of the cheapest serious problems to fix. The architecture is well-understood and the tooling is mature. It mostly requires deciding to do it.
A secret store, not a config file
The foundational move is to stop treating secrets as configuration. Configuration is non-sensitive: feature flags, log levels, the public API base URL. Secrets are the things that grant access: database passwords, API keys, signing secrets, tokens. They need different handling.
Secrets belong in a dedicated secret store — AWS Secrets Manager, HashiCorp Vault, GCP Secret Manager, or your platform's equivalent. The store gives you four things a .env file never will: encryption at rest, access control over who and what can read each secret, an audit log of every access, and an API to fetch secrets at runtime instead of baking them into the deployment.
The pattern at runtime:
app boots
└─ requests `DATABASE_URL` from the secret store
└─ store checks: is this service identity authorized?
└─ yes → returns secret, logs the access
└─ no → denies, alerts
└─ secret lives in memory, never on disk, never in the image
The secret is never written to a file, never committed, never present in the container image. It's fetched at boot by an authenticated service identity and held in memory. If an attacker pulls your image or reads your repo, they get nothing.
Least privilege, enforced
A secret store solves storage. It doesn't solve scope. The second principle is least privilege: every identity gets access to exactly the secrets it needs and nothing more.
The common failure is one set of god-credentials shared across every service. The payments service can read the analytics database password. The marketing site can read the production signing key. When any one service is compromised, everything is compromised, because everything shared one keyring.
Least privilege means the payments service can read payment secrets and nothing else. The web app can read the web app's secrets and nothing else. Each service has its own identity, and the secret store's access policy maps identities to the minimal secret set. Now a compromise of one service is contained to that service's blast radius instead of the whole company.
This also makes rotation safe, which is the next problem.
Rotation that actually happens
Your JWT signing secret hasn't changed since launch. Your database password is the one a contractor set up two years ago. The "rotation policy" is a sentence in a security questionnaire response, not a thing that occurs.
Rotation matters because secrets leak slowly and silently. A credential might be compromised for months before it's used. Rotation bounds that exposure window: if every secret rotates every 90 days, a leaked secret is useless after at most 90 days, whether or not you ever knew it leaked.
Manual rotation doesn't happen because it's tedious and risky — change the password, and pray every service that uses it picks up the new value without downtime. So you automate it. A modern secret store rotates supported secrets on a schedule, coordinating the change so consumers always have a valid credential. For database credentials specifically, the store can issue short-lived, dynamically-generated credentials per service — credentials that expire on their own and never need manual rotation at all.
The rule: if a secret can't be rotated without a coordinated downtime window, that's an architecture problem to fix, not a reason to never rotate.
Secrets in the pipeline
CI/CD is where good secret discipline goes to die. Your build needs the deploy credentials. Your tests need a database password. Your pipeline needs a registry token. So secrets end up pasted into pipeline configuration, printed in build logs, or stored in plaintext CI variables.
Handled correctly: the CI system pulls secrets from the same store at job time using a short-lived, scoped identity — not from a pile of permanent plaintext variables. Deploy credentials are scoped to deploying and nothing else. Build logs are scrubbed; a secret echoed into a log is a leaked secret, so the pipeline masks them and you never echo a credential to debug. And no secret ever flows into the built artifact — secrets are injected at runtime in the target environment, not at build time into the image.
The test environment gets its own throwaway secrets, never production values. There is no reason your CI test database should share a password with anything real.
Detecting what already leaked
Here's the uncomfortable part. You can fix every pattern above starting today, and you'll still have secrets sitting in your git history from before you knew better. Fixing the going-forward problem doesn't address the existing exposure.
Two moves. First, scan. Run a secret scanner — there are mature, well-known tools for this — across your entire git history, not just the current tree. It'll find the AWS key from commit 200, the API token in the old config, the password in the test fixture. Wire the same scanner into CI as a pre-merge gate so a new secret fails the build instead of shipping.
Second, and this is the part teams skip: every secret that has ever been committed is compromised and must be rotated, full stop. You cannot un-leak it by deleting the file or rewriting history. Someone may have it. Rotate every credential the scanner finds. Then turn on push protection so the next accidental commit is blocked at the source.
found a committed secret?
└─ rotate it (deleting the commit does NOT make it safe)
└─ then prevent the next one (pre-commit + CI scan + push protection)
What fixed looks like
No .env file with production secrets exists. Every secret lives in a managed store, encrypted at rest, with access scoped per service identity and every access logged. Applications fetch secrets at runtime; nothing sensitive is in the repo, the image, or a developer's laptop. Database credentials are short-lived and dynamically issued. Other secrets rotate automatically on a schedule, and a rotation can happen without downtime because the architecture was built for it.
CI pulls secrets at job time through a scoped, short-lived identity, masks them in logs, and never bakes one into an artifact. A secret scanner runs on every commit and across full history, and push protection blocks accidental commits before they land. Everything that ever leaked has been rotated. When a security questionnaire asks about your secret rotation policy, you describe a system, not an intention.
This is the secrets layer of the broader security posture we assemble in the Compliance-Ready SaaS engagement.
This is for you if
You're a funded team running real production workloads with real customer data, and you know your secret handling grew up by accident — .env files, shared keys, a rotation policy that's aspirational. You want it fixed before it becomes a five-figure cloud bill or a breach disclosure.
This work is part of security hardening engagements starting around $50k, or a designed-in component of larger build engagements ($100k+).
It's not for hobby projects or pre-revenue prototypes with nothing worth stealing — adopt the secret-store pattern early because it's cheap, but you don't need a formal engagement to put a key in Vault. And it's not for teams that want the scanner installed but won't rotate the secrets it finds. Detection without rotation is a list of your own open doors.