Netlify Deploy Traps

A Netlify deploy tells you it published. That statement is true and it is not the question. The question is which files went out, which rules went with them, and whether the thing you changed is the thing now serving. Those can all be wrong while the deploy log is green.

Every incident below is from a network of static sites and functions in production. None of them threw. Each was found by asking the platform what it is actually serving, rather than by reading the command that was typed.

Start here

Three probes answer almost everything in this file. Reach for them before theorizing.

Ask what files are live. listSiteFiles on the deploy returns the manifest. A healthy static site is rooted at /index.html. If the manifest shows /scripts/..., /data/... or /exports/..., you published a repository instead of a site.

Ask what a URL actually answers. curl -sI https://<domain>/ and read the status, the Location header and the Server header together. Server: Netlify with a self-referential Location and a body around 42 bytes of text/plain is Netlify's redirect engine firing. A rule is running. Go find the rule and stop checking DNS.

Ask the function to speak for itself. Netlify function logs are sampled. Print your own line on every invocation and read that, or plant a row in a database that the function must consume and watch the row.

The deploy summary is not one of these probes. It reads like "4 new files uploaded, 73 functions deployed" because Netlify only uploads changed assets. A small upload count says nothing at all about what the manifest declares.

The --dir flag overrides netlify.toml, silently

netlify deploy --dir <path> ignores the publish value in netlify.toml. No warning, no diff, no confirmation of which one won.

The danger is that the correct value differs per site and the flag does not care. A site whose repo root is its web root genuinely publishes .. A site whose publish directory is build output must never receive that value. Nothing in the CLI knows which one you are pointed at, so the same command is right on one site and catastrophic on the next. Measured cost of getting it the wrong way round once: 44 minutes down, and the entire working tree served.

Two failures land at once, and the first hides the second:

1. Every page 404s, homepage included, because index.html is now at

/public/index.html. That path serving 200 is the tell.

1. The whole working tree becomes downloadable. Exported CSVs of contact

records, JSON data files and deploy scripts all served 200.

The second one is the one that matters and the first one is the one that gets reported. Fix in that order anyway, because the rollback closes both.

--dir deploys from disk, not from git. Files that are gitignored ship. The CLI does skip dotfiles, so .git, .env and similar stayed private, but that is a property of the CLI and not a guard you designed. Do not build a threat model on it.

Build from git for any site whose publish directory is build output. A local public/ goes stale within days, so --dir public --no-build puts an old site up and reports success.

Rolling back and rebuilding are different moves

netlify api restoreSiteDeploy to the last good deploy is instant and it closes an exposure now. It changes only what is served. It never touches files on disk, so uncommitted work is not at risk.

Rebuilding from the main branch is the other move, and for a site with a real build step that ingest can run for minutes. Going straight to a rebuild leaves the bad files served for the whole of it.

Restore first, then rebuild to pick up any commits the restore stepped back past.

Afterward, be honest about what you cannot know. A static site with analytics disabled and no log drain keeps no access logs for static assets, and analytics cannot be enabled retroactively. An exposure window with no logs is unknown, not empty. Do not write "nobody downloaded it" in the incident note. Write that it cannot be determined, and turn analytics on before the next one.

The same flag moves files and leaves the rules behind

One repository, two brands, two Netlify sites. netlify.toml can name only one publish directory, and it named the first brand. The second brand built into its own directory and was deployed with --dir.

--dir moved the files. It did not move the redirect rules.

The first brand's _redirects held a catch-all forward:

/*  https://otherbrand.example/:splat  301!

That rule shipped attached to the second brand's pages. Every URL on the site answered 301 to itself, homepage included. curl -L burned four hops and never landed. It ran about 20 minutes.

What made it expensive to find:

• listSiteFiles was clean. 205 files, a real /index.html at 31,927

bytes, no _redirects in the manifest at all. The files were right. Only the rules were wrong, and the manifest does not describe rules.

• --no-build did not help. Redeploying the same correct --dir did not help.
• The TLS certificate was issued, the DNS zone was on the right account, and

the records pointed at the right site. Every usual suspect checked out clean, which is the part that costs the hours.

• The deploy preview URL also redirected to the apex domain, which is normal

primary-domain behavior and reads exactly like a second symptom.

The fix that proved it: move the other brand's _redirects out of the way, redeploy, site returns 200. Nothing else changed.

If one repo builds more than one site, do not deploy it by hand. Write a deploy script that rewrites the publish line for the brand being deployed, prints the _redirects that will actually ship, deploys, restores the line, then checks the live status code against what that brand expects. A brand whose correct answer is 200 and a brand whose correct answer is 301 cannot share one unchecked command.

A blob read can return what was there before your own write

Netlify Blobs reads are eventually consistent by default. getStore(...).get(key) is served from a cache and can hand back the value from before your own write.

This broke a publisher that used a claim, pause, read back lock. It wrote its own id, read back a stale value, concluded that another process had won the claim, and returned a 409 saying nothing had been double posted. Nothing had been posted at all. Three approved items sat in the queue displaying as published. It reproduced twice in a row.

Any read whose answer decides something must pass { consistency: 'strong' }. That means every lock or claim readback, and every "have we already done this" index check.

list() is always eventually consistent and takes no consistency option at all. A listing can name a key that was deleted minutes ago. The way around it is to re-read every key from the listing with a strong read and drop the ones that come back empty.

The lock pattern in this codebase was correct in one file and carried a comment explaining why. The bug arrived when a second feature copied the pattern and not the flag. A correctness flag with a comment next to it is still going to be dropped by the next copy. Put it in the store constructor where it cannot be separated from the calls.

A conditional write that did not happen returns normally

@netlify/blobs v11 gives you a real compare and swap with no database. setJSON(key, data, opts) accepts onlyIfMatch: etag and onlyIfNew: true, and getWithMetadata(key, { type: 'json' }) hands back { data, etag }.

const db = getStore({ name: 'bookings', consistency: 'strong' })
for (let i = 0; i < 6; i++) {
  const cur = await db.getWithMetadata(key, { type: 'json' })
  if (clashes(cur?.data)) return { ok: false, reason: 'taken' }
  const res = await db.setJSON(key, next,
    cur ? { onlyIfMatch: cur.etag } : { onlyIfNew: true })
  if (res.modified) return { ok: true }
  // somebody wrote between our read and our write, go around again
}

Proved live: three simultaneous requests for the same resource and the same dates gave one success and two rejections, every time, and the losers were handed the next open slot.

The trap is res.modified. A conditional write that was refused does not throw. It returns an object with modified: false, and code that ignores the return value reports success and double books. This is the same shape as a PostgREST update that matches zero rows and returns 200. The call came back fine and nothing happened.

Put the contested state in one blob per contested thing, never one blob per record, or two callers are not competing for the same key and the compare and swap protects nothing.

No log line is not the same as no run

Netlify function logs are sampled. The platform's own Duration: report line is not emitted for every invocation.

So "no log entry" and "did not run" are indistinguishable from outside. Half an hour went into a wrong conclusion here once, reporting an entire cohort of scheduled functions as dead when they were merely under-logged.

Make the function print its own line. A line your code emits is carried reliably in a way the platform's report line is not. One tick log with something identifying in it is enough.

Or watch a side effect instead. Park a row the function must consume and watch the row. The database is not sampled.

A five minute schedule is not every five minutes

Measured on a busy site: a row planted at 15:06:57 was not picked up until 15:35:09. Twenty nine minutes, about six missed tick boundaries, by a function scheduled */5 * * * *. It then behaved perfectly. This is scheduling latency, not a code fault.

Runs actually observed in one 18 minute window:

The likely cause was contention. One function scheduled * * * * * was firing about three times a minute at 15 to 20 seconds a run, which is roughly 48 seconds of execution per minute from a single function before anything else is counted. Under that load the */5 cohort was delayed.

Two things follow from this. First, anything with a promise attached to it should not be on a schedule at all. A message that has to reach a customer quickly belongs in the request that created it, not in a sweep. Second, a watchdog on a */5 schedule checks far less often than it reads like it does, which means the alarm for a dead pipeline is itself slower than the page implies.

Before blaming the schedule, count the runs. A function firing three times a minute on a one minute cron is its own bug and it starves everything else.

Form detection defaults to off

A Netlify site can default to ignore_html_forms: true. Every form on it can carry data-netlify="true" and the site will register no form and store no submission.

An audit of 21 sites found that every site with a Netlify form had detection off and had recorded zero submissions, ever. Submitting fails as a 404, so visitors saw an error rather than a false thank you, but the lead is gone either way.

The reason it survived a QA gate is worth more than the fix. The gate said "form submits and the lead notification arrives", which somebody ticked by looking at the page.

Fixing one, in order:

1. PATCH /sites/{id} with

{"processing_settings":{"ignore_html_forms":false,"html":{"pretty_urls":true}}}. A top-level ignore_html_forms is silently ignored on git-linked sites.

1. POST /sites/{id}/builds with a JSON content type header, because detection

runs only at build time. An empty body without the header returns no deploy id.

1. listSiteForms to get the form id.
2. POST /hooks?site_id=... for a submission_created notification, with the

fields at top level. Nesting them returns a 422 complaining about a URL.

1. POST the live form and confirm a 200 and a row. Steps 1 to 4 are reading.

This step is the audit.

netlify link --id wants the site UUID, not the subdomain name.

A secret environment variable is masked into your own shell

A Netlify env var marked secret is masked in env:get, which is expected, and it is also injected masked into dev:exec, which is not.

Measured: the child process received a 20 character string, 16 of them asterisks. A request built from it returned 401.

There is no local route to a secret's value. Only the deployed function sees the real one.

What makes this easy to miss is that non-secret variables come through dev:exec intact. The probe looks like it worked, because most of the variables did.

If a local script needs such a value, it has to arrive out of band into a gitignored file. Check what .gitignore already covers before choosing the filename, especially on a site whose repository is also its web root, where a stray file is a published file.

The verification moves, collected

• After any hand deploy, read the manifest. listSiteFiles rooted anywhere

but /index.html is a wrong publish directory.

• **After any deploy that changes routing, curl the apex and read three

headers.** Status, Location, Server. A self-referential Location is a rule, not DNS.

• **If one repo builds two sites, deploy through a script that checks the live

status code**, and give each brand its expected code.

• Restore the last good deploy before rebuilding. One is instant, the other

is as slow as your build.

• Set consistency: 'strong' on the store, not on the call site.
• Check res.modified on every conditional write and treat false as a

failed attempt, not as success.

• Make every scheduled function print its own line, and reason from that

line or from a database side effect, never from the absence of a platform log.

• Count actual runs before trusting a cron expression.
• Submit every live form once and confirm the row. A form that renders is

not a form that captures.

• Print the length of a secret you loaded locally before using it. Twenty

characters of mostly asterisks is the answer.

Every one of these asks the platform what is true right now. The deploy command you typed, the config file you read, and the schedule you wrote are all statements of intent. None of them is evidence.

---

Written from production incidents across a network of Netlify sites and functions, by Clickflame. Companion skills: supabase-rls-audit and postgrest-silent-failures, for the same class of quiet failure in the database and in the client.