Direct answer: Requesting one nonexistent .html path on each of 25 homepages, 22 return a real 404 (88%), one returns a 200 error page (E27, final URL /404.html), and two return 200 for the unknown path (E12, E18; script-driven pages with no 404 wording). No site redirects to the homepage and no 403/500 appears.
The 22 hard-404 pages vary widely in size: seven are under 1 KB (smallest 13 bytes), four exceed 100 KB, and the median is about 10.5 KB. Twenty of the 22 mention 404/not found/error in the title or body; two do not.
Method: one random path per site
The study reuses the 25 homepages from study 4 and takes the host recorded in study 48, appending /__ly_check_<8 hex>.html. Requests run serially on 2026-11-03 between 14:41 and 14:42, at least 0.8 s apart, following redirects and recording status, final URL, title, body and response size.
| Class | Rule | Boundary |
|---|---|---|
| Hard 404 | Final status 404 or 410 | One random path only |
| Soft 404 | 200 with a 404 marker in title/body, or a /404 path | Fixed list; unseen wording may be misclassified |
| Redirect home | 200 with final path / or /index.* | None in this run |
| Other 200 | 200 without marker and not home | May be an SPA shell |
| Other status | 403, 500, 502 and similar | A WAF block is not the site's error page |
Result: status codes mostly right, error pages uneven
| Outcome | Sites | Share of 25 |
|---|---|---|
| Hard 404 / 410 | 22 | 88.0% |
| Soft 404 (200 error page) | 1 | 4.0% |
| Unknown path returns 200 | 2 | 8.0% |
| Redirect home | 0 | 0% |
| 403 / 500 / other | 0 | 0% |
E27 ends on /404.html with title “404” and body “您访问的页面不存在”; the status is 200 while the content is an error page. E12 and E18 return 200 on the random path and render with scripts without any 404 wording, a different soft-404 risk.
Size of the 22 hard-404 pages
| Body size | Sites | Note |
|---|---|---|
| < 1 KB | 7 | 13–860 bytes, close to a server default page |
| 1–10 KB | 5 | — |
| 10–100 KB | 6 | — |
| > 100 KB | 4 | Largest about 248 KB |
| Total | 22 | Median about 10.5 KB |
Three of the seven sub-1 KB pages return a single “404 Not Found” line and one is only 13 bytes. The four over 100 KB load the full template. Neither is automatically wrong, but the cost and usefulness of the error page differ sharply.
How many error pages explain what happened
| Item | Count | Note |
|---|---|---|
| Hard 404 with 404/not found/error wording | 20 / 22 | Title or body |
| Hard 404 without any marker | 2 | E17 body is template CSS; E20 is only 101 bytes |
| 200 response with a 404 marker | 1 | E27 |
| Error page with navigation or a way back | Not counted | Only text and size recorded here |
The study does not count links back to the homepage, search or recommendations; that is a design layer that needs a browser pass rather than status codes and text.
What this means for acceptance
- Check the status first. 22 of 25 return 404/410, so the basic behaviour is right.
- Fix 200 error pages. E27 has the right content with status 200; a search engine will treat the invalid URL as a valid page.
- Test unknown paths on SPAs. E12 and E18 return 200 without 404 wording; the router must provide an error state or content.
- Tiny error pages lack an exit. Seven are under 1 KB, leaving the reader to go back or edit the URL.
- Huge error pages cost too. Four exceed 100 KB; an error page should not carry the full homepage payload.
Limits
- One random path: only a
.htmlsuffix is tested, not directories, parameters, old article URLs or case variants. - WAF/CDN: a random path may trigger a block page; no 403 appeared this time, but another run may differ.
- Fixed markers: unseen custom 404 wording may be classified as “other 200” and needs a manual check.
- Design not judged: return links, search, recommendations and branding are not counted.
- 25-site pilot: no industry claim.
Data and reproduction
Anonymised data: equipment-soft-404-2026.csv (25 rows × 9 columns: status, class, final path, marker, body size, title, requested and final URL). The repository keeps docs/research/2026-11-soft-404.csv.
Parser: scripts/soft_404_probe.py; unit tests: scripts/test_soft_404_probe.py (11 tests, no network). Run:
python -m unittest scripts.test_soft_404_probe
Related: the mobile viewport pilot, the homepage security header pilot, and the protocol and hostname pilot.
Sources and definitions
- Site sample: the 25 homepages from study 4; hosts from the final URLs recorded in study 48.
- Requests: 2026-11-03 14:41–14:42 Asia/Shanghai, one random .html path per site, serial, ≥0.8 s apart.
- Classification: status code plus fixed title/body markers and final path.
- Body size is the actual number of bytes received.