A structured benchmark of 268 publicly identifiable Singapore business websites (commercial operating entities) across 9 industry groups, measuring which AI search crawlers can — and cannot — access them.
Of 174 domains whose crawler policy could be directly observed, 58 — 33.3% — block at least one crawler used by AI search systems to discover and cite content.
We separate observable AI-crawler policy from infrastructure non-response. A block declared in robots.txt is a policy decision; a 403, timeout or unscannable response is an access outcome, not evidence of crawler policy. These are reported separately below.
Of the blocked sites, 86.2% are broad access restrictions catching AI crawlers incidentally rather than AI-specific decisions.
Policy-layer figures are based on 174 domains whose robots.txt was successfully retrieved and parsed. A further 94 domains returned no observable policy (13 access-denied, 81 unscannable) and are reported separately in the Infrastructure layer section. They are not counted as open or blocked.
The central finding: Most blocked businesses are not actively choosing to exclude AI search. They have broad access restrictions set years ago that are now inadvertently catching AI crawlers. This is a configuration problem, not a strategic decision.
We separate observable AI-crawler policy from infrastructure non-response. A block in robots.txt is a policy decision. A 403, timeout or unscannable response is an access outcome, not evidence of crawler policy. These 94 domains are reported here and excluded from every policy-layer figure.
Why this matters: a domain that denies the crawler at the infrastructure layer has not expressed an AI-crawler policy — it has prevented one from being read. Treating such a response as “open” would overstate access; treating it as “blocked” would overstate restriction. Reporting it separately keeps the policy-layer figures based only on directly observed robots.txt behaviour.
The Singapore signature: non-response, not denial. Like Great Britain, Singapore restricts access mostly by not responding rather than by actively denying. Of the 94 domains that returned no observable policy, 81 (30.2% of those approached, the highest unscannable rate in the series) were unscannable through connection failure or timeout, against just 13 active access denials. This is passive non-response, the opposite of the United States pattern of managed-WAF edge denial. Singapore also has the lowest policy-layer block rate in the series at 33.3%, consistent with a market whose largest businesses lean toward global visibility.
Of the 58 sites blocking AI retrieval crawlers, the source of the block was classified into three categories.
The infrastructure-imposed subset is the most commercially significant finding: these site owners may be blocking AI search discovery without ever having made that decision. The indeterminate category of 17 Cloudflare-hosted sites most likely represents explicit blocks, but the configuration path cannot be confirmed by automated means alone.
Block rates vary across sectors. Healthcare, Retail and Hospitality are highest; Building & Trades lowest. Rates are computed on policy-observed domains per sector (readable robots.txt only). As a concentrated city-state market, several Singapore sectors are small; rates rest on modest bases and should be read accordingly.
Healthcare at 50.0%, Retail at 47.6% and Hospitality at 47.4% are the highest-blocking Singapore sectors, though each rests on a small base. Building & Trades at 11.1% is by far the most open, the lowest single sector rate in the entire series. Singapore is a concentrated market and several of these sectors reflect close to the full population of citable named businesses rather than a sample of a larger pool.
Group A (retrieval/citation crawlers) drives the headline finding. Group B (training crawlers) is reported separately, because blocking training crawlers is often a deliberate and legitimate content-protection decision.
The Googlebot parity finding holds in Singapore. Googlebot is blocked at 28.7%, right alongside the AI retrieval crawlers (GPTBot 32.2%, ClaudeBot 32.2%). Most Singapore AI-crawler blocks are broad restrictions, not targeted AI decisions. The 14 retrieval crawlers cluster tightly (28.7%–32.2%), indicating that where AI is blocked, it is typically blocked uniformly across operators rather than selectively.
Block rates by content management system among policy-observed domains. WordPress is the only platform with a usable base in the Singapore sample; the others rest on very small samples.
Most sites return no identifiable CMS signature, so platform-level rates are based on the minority that do. WordPress at 28.3% (n=53) sits a little below the overall Singapore sample average and is the only platform with enough domains for a meaningful rate. Drupal (n=6), Shopify (n=5), Webflow (n=3), Wix (n=3) and Squarespace (n=1) have too few domains in this sample to report reliably.
Roles. This study is published by the Periodic Table of Digital Authority (PTODA), the methodology owner. It was conducted using the PTODA C01 Crawler v1.2, a deterministic robots.txt reference instrument, under PTODA C01 Crawler Methodology v1.2. AUTHORITY44 provided technical infrastructure and execution support as commercial operator. Douglas Lord is the founder of both PTODA and AUTHORITY44; this relationship is disclosed in full. The sample was constructed from named public directories with no reference to commercial relationships. The methodology is fully documented and reproducible. This study publishes aggregate, anonymised findings only. No named individual site results are published.
Attribution chain: Douglas Lord (researcher, author) · Periodic Table of Digital Authority (publisher & methodology owner) · PTODA C01 Crawler v1.2 (research instrument) · AUTHORITY44™ (commercial operator) · Digital Dominator Pty Ltd ABN 28 616 931 116 (operating entity).
Intellectual property notice: This study, its methodology, findings, data, and all associated content are the original work of Douglas Lord and the property of Digital Dominator Pty Ltd (ABN 28 616 931 116). The Periodic Table of Digital Authority™ is a coined framework and trade mark pending (TM 2644497). AUTHORITY44™ is a trade mark pending (TM 2643932). All rights reserved.
You may cite findings from this study with appropriate attribution identifying the author (Douglas Lord), the publisher (Periodic Table of Digital Authority — periodictableofdigitalauthority.com), and the research instrument (PTODA C01 Crawler v1.2). You may not reproduce this study in full, present these findings as your own research, or use the framework name or trade marks without prior written consent. Use of this research is subject to the Terms of Use.