Direct answer: Merging the three published anonymised datasets onto 24 shared sites by sample_id: all five AI bots allowed 16 (66.7%), usable sitemap 13 (54.2%), contact entry 20 (83.3%), recognisable inquiry form 9 (37.5%). Only 4 sites (16.7%) pass all three core conditions — five bots allowed, sitemap usable, inquiry form present — and 4 pass none of them. This is a pilot, not a benchmark.
The sitemap page used 24 sites, the contact pilot 25 and the robots scan originally 30. Those denominators differ, so stacking their percentages side by side distorts the picture. This page adds one new thing: the merge onto a common base, with a recomputable matrix.
Method: merge only, no new requests
No requests were made for this page. Inputs are the anonymised CSVs already in the repository: the robots scan, the contact-path pilot and the sitemap scan. Intersecting on sample_id leaves 24 sites; six boolean flags per site produce 2026-11-four-layer-matrix.csv.
| Layer | Pass rule | Original denominator |
|---|---|---|
| robots | GPTBot, OAI-SearchBot, PerplexityBot, Bytespider and ClaudeBot all allowed | 30 companies → 25 judgeable |
| sitemap | HTTP 200 with a urlset or sitemapindex root | 24 testable sites |
| Contact entry | A same-origin contact/inquiry link in the homepage's static HTML | 25 analysable homepages |
| Inquiry form | email/tel/textarea plus a submit or text field on the contact page | 20 fetched contact pages |
| Product page (subset) | Exactly one H1; JSON-LD optional | 15 product pages mapped to 15 sites |
After merging, everything is counted on the 24-site base. Product pages cover only 15 of those sites, so they are reported separately and kept out of the three-layer headline. The definitions in each original page are unchanged.
Result 1: pass rate per layer
| Layer | Pass | Share of 24 |
|---|---|---|
| All five bots allowed | 16 | 66.7% |
| All five bots blocked | 1 | 4.2% |
| No declaration (absent) | 7 | 29.2% |
| Usable sitemap | 13 | 54.2% |
| Contact entry | 20 | 83.3% |
| Inquiry form | 9 | 37.5% |
| Exactly one H1 on the product page (15-site subset) | 6 | 40.0% |
Contact entry and inquiry form are the widest gap: 20 sites offer an entry, only 9 end in a form. The entry is a navigation problem, the form is a page implementation problem; treating them as one acceptance item hides 11 sites where the click leads nowhere to type.
Result 2: three-layer combinations
| Combination | Sites | Meaning |
|---|---|---|
| Bots allowed + sitemap usable + inquiry form | 4 | All three layers pass |
| Bots allowed + sitemap usable (ignoring forms) | 11 | Crawlable with a URL list, form not guaranteed |
| None of the three passes (bots not all allowed) | 4 | Policy, discovery and lead capture all miss |
| Contact entry but no inquiry form | 11 | The click exists; the static form does not |
| No usable sitemap but a contact entry | 9 | Discovery missing, lead path still present |
| No robots declaration (absent) | 7 | 1 still serves a usable sitemap, 6 do not |
| All five bots blocked but sitemap usable | 1 | Policy blocks the bots while the file stays public |
The blocked-but-published case is worth a second look: two layers configured by different people at different times, with nothing checking them against each other. Similarly, 9 sites have no usable sitemap while their contact entry works fine — "usable" and "findable" are not the same property.
Result 3: the product-page subset
Fifteen product pages map to 15 sites: 6 have exactly one H1 and 1 carries JSON-LD. Put those 15 into the three-layer rule and only 3 pass all three (the sites behind E02, E06 and E11); among them, exactly one product page has a single H1. So even where crawling, discovery and lead capture hold up, product-page structure can still miss.
No causal claim here: the product sample is 15 pages, one per site and not randomly chosen.
Five conclusions
- Align denominators first. Mixing 30, 25, 24 and 15 inflates or deflates gaps; this page uses 24.
- Entry is not a form. 20 have an entry, 9 have a recognisable form; split them at acceptance.
- robots and sitemap drift apart. One site blocks all five bots yet publishes a sitemap; 6 of the 7 absent sites also lack a usable sitemap.
- Passing all three is rare. 4 of 24 pass crawling, discovery and lead capture; 4 fail all three.
- Product pages are their own layer. Among three-layer passes, the product H1 can still fail; do not infer it from the homepage.
What this page cannot answer
- Not the industry: one list, one city, list-order selection; 24 sites is a pilot.
- Not an outcome: no traffic, ranking, AI citation or inquiry conversion is linked; pass/fail describes static state only.
- Not the whole site: one homepage, one contact entry, one sitemap and one product link per site.
- No motive: absent, blocked and 404 can be omission, migration or policy; no attribution is made.
- Single snapshot: data comes from different hours of 2026-11-01 and sites may have changed.
Data and recalculation
Cross-tab: equipment-four-layer-matrix-2026.csv (24 rows, 7 fields: sample_id plus six flags). The repository keeps docs/research/2026-11-four-layer-matrix.csv. Every count above can be recomputed from it and the three source datasets by sample_id.
- robots policy: 30-site AI crawler policy scan · anonymised CSV
- sitemap chain: 13 usable files in 24 sites · anonymised CSV
- contact path: 25 homepages, 20 contact pages · anonymised CSV
- product entry pages: 15-page pilot · anonymised CSV
Sources and definitions
- Rules and definitions for each source study live in its own page; this page only merges and counts by sample_id.
- robots verdicts: RFC 9309: rfc-editor.org.
- sitemap verdicts: sitemaps.org protocol: sitemaps.org.
- Merge date: 2026-11-01; source snapshots were captured at different hours between 15:00 and 21:00 that day.