HTTP Date Headers: 21 of 25 Homepages Send No Last-Modified

Direct answer: Reading only response headers from 25 homepages: 21 send no Last-Modified at all, 4 send one and all 4 parse; among those, 1 has a Last-Modified identical to Date (E04, generated with the response). Only 4 sites send an ETag (3 strong, 1 weak), 17 send Cache-Control (10 with max-age=0), and none send Age. The only site comparable with a sitemap lastmod, E16, matches the year but not the day.

Studies 22, 24 and 25 looked at fields inside the page: copyright, lastmod, page dates, JSON-LD. This one takes an independent line — the date the server puts in the response headers. It is not template-driven, but it does not necessarily track content updates either.

Method: robots first, headers only

Using the 25 homepage URLs from study 4, each site's robots.txt is fetched first and evaluated for LingyiResearchBot. A single normal GET then records status and response headers only, without keeping the body. Up to three retries on non-200, five seconds between requests, 20-second timeout, no JavaScript, no login, no form submission. Dates are parsed as HTTP-date; asctime without a zone is treated as UTC.

Result: fewer date headers than page fields

Four dated pages, four different stories

Times are raw header values (GMT); the gap is computed against the Date of the same response.

The four cases invite four different readings: E04's header is generated with the response and cannot be a content date; E27 is 27 minutes older than the response, which may be a cache or a scheduled job; E30 stopped in June while the footer says 2002, so neither field stands alone as update evidence; E16 agrees with its sitemap on the year but not the day, showing the two sources are not aligned.

Why homepages rarely send Last-Modified

• Dynamic rendering: a homepage assembled at request time has no natural file modification time.
• CDN / reverse proxy: headers may be rewritten or stripped before they reach the client.
• Cache policy: 17 sites send Cache-Control and 10 use max-age=0, caring about “do not cache” rather than “when it changed”.
• Nothing requires it: Last-Modified is optional.

So a missing Last-Modified says neither “the site never updates” nor “the server is unmaintained”; it says this response did not provide the field.

What this means for acceptance

1. Treat response headers as a separate evidence source. Record them apart from page fields; they do not substitute for one another.
2. Check whether Last-Modified equals Date. When equal, it is response time, not content time.
3. Compare dates with the same convention. Headers use GMT, sitemaps use ISO; a different day is not automatically a conflict — check the time zone and format first.
4. A missing header is not a defect verdict. Dynamic homepages often omit it; write “field absent”, not “not updated”.
5. Read cache headers separately. max-age=0 describes caching policy, not content freshness.

Limits

• Single request: CDN node, cache hit and origin fetch all affect headers.
• Homepages only: article pages and static assets may differ.
• Headers can be rewritten: client-side headers cannot prove origin configuration.
• Very few comparable samples: only one site has both Last-Modified and a sitemap lastmod.
• 25-site pilot: no industry claim and no quality judgement.

Data and reproduction

Anonymised data: equipment-http-date-headers-2026.csv (25 rows: sample_id, status, Last-Modified state and value, Date, generated-with-response flag, ETag strength, cache headers, Age, comparisons with sitemap and footer). The repository keeps docs/research/2026-11-http-date-headers.csv.

Parser: scripts/http_date_probe.py; unit tests: scripts/test_http_date_probe.py (14 tests, no network). Run:

python -m unittest scripts.test_http_date_probe

Related: the update-date fields pilot (JSON-LD, meta, time, visible text), the freshness-signal pilot (copyright, lastmod, page dates), the one-day recheck (field stability), and the five-source matrix (25-site year comparison).

Sources and definitions

1. RFC 9110: Last-Modified, Date, ETag and conditional requests: rfc-editor.org.
2. Cache-Control directives: RFC 9111.
3. Sampling: 2026-11-02 14:41–14:46 Asia/Shanghai, UA LingyiResearchBot/1.0, five seconds between requests.