Angeo AEO Audit

One CLI command that tells you exactly why ChatGPT, Gemini, Claude, and Perplexity aren't recommending your store — and how to fix it.

• 🏠 Project home: angeo.dev
• 📦 Source: github.com/angeo-dev/module-aeo-audit
• 🐛 Issues: github.com/angeo-dev/module-aeo-audit/issues
• 📖 Full guide: Magento 2 AEO Guide 2026

---

Compatibility

Component	Version
Magento Open Source	2.4.6, 2.4.7, 2.4.8
Adobe Commerce	2.4.6, 2.4.7, 2.4.8
Adobe Commerce Cloud	All current versions
PHP	8.1 – 8.5
Themes	Luma, Hyvä

Tested with: Magento Open Source 2.4.7-p3 + PHP 8.3 + Hyvä 1.3.

---

What's new in v4.0.0

The evidence layer. Signals 1–15 audit configuration; v4 adds two signals that audit reality: waf_reality (does the edge actually let the bots in that robots.txt invites?) and ai_crawler_activity (which AI crawlers demonstrably visited, from GDPR-safe evidence sources). Plus: purpose-classified bot grading (blocking GPTBot is a licensing choice, blocking OAI-SearchBot is an AEO failure), GitHub Actions CI with a Mage-OS installability job, i18n dictionary, configurable cron schedule, vendor-neutral & switchable fix hints, and a security fix in the CrUX checker (TLS verification re-enabled, API key moved from URL to header, encrypted key now actually decrypted). Full details in CHANGELOG.md.

What's new in v3.0.0

Major release — see CHANGELOG.md for the breaking-change migration guide if you have custom checkers.

20 signals (up from 9), reflecting the actual AEO landscape of 2026: AI shopping integrations, merchant policies, agentic commerce, and structured-data quality.

6 new checkers:

• merchant_policies — MerchantReturnPolicy + OfferShippingDetails — required by Google AI Mode and ChatGPT Shopping since Jan 2026
• organization_schema — brand entity in AI knowledge graphs
• ucp_profile — Universal Commerce Protocol (/.well-known/ucp), with built-in security check that detects leaked JWK private keys
• jsonld_quality — three-page schema breadth audit (homepage / category / product), WebSite+SearchAction, BreadcrumbList, ItemList
• well_known — discovery matrix for /.well-known/{ucp,ai-plugin.json,security.txt,mcp}
• core_web_vitals — LCP / INP / CLS via Google CrUX API (free, opt-in with API key)

Refactored architecture (this is the BC-break):

• Shared Service\HttpCache — eliminates duplicate fetches across checkers (hundreds of redundant HTTP requests on multi-store audits before, dozens now)
• Service\StoreUrlSampler — single source of truth for product / category / CMS URL sampling
• New --category and --fail-on-severity CLI flags for CI workflows
• Per-checker exception isolation — slow or failing checkers no longer halt the audit run

Note on access-log monitoring: an ai_bot_traffic checker was prototyped during v3 development and excluded from the release after security review — it encouraged broad read access on /var/log/nginx/, didn't work on Cloud/containerised hosting, and was dominated by false positives behind edge caches. AI-bot traffic is better measured at the edge (Fastly/Cloudflare Analytics) or via APM (New Relic, Datadog) than inside a PHP module. See CHANGELOG.md "Considered and rejected" for the full rationale. The live_signal category remains in CheckerInterface for third-party modules with secure live-signal sources — notably angeo/module-aeo-brand-visibility.

---

What it checks — 18 signals

Two layers since v4.0.0: configuration signals verify the store is set up for AI engines; the evidence layer verifies AI engines actually reach it.

#	Signal	Code	Weight	Category	What it validates
1	robots.txt — AI bots	robots_txt	1.0	technical	12 AI bots, syntax errors, versioned UAs, conflicting rules
2	llms.txt — content map	llms_txt	1.0	technical	llmstxt.org v2 structure, store-locale + currency match, cross-host links, empty blockquote, headings before the first H2, HTML links where mirrors exist
3	llms.jsonl — catalog	llms_jsonl	0.75	technical	JSON Lines validity, required fields, eCommerce fields
4	sitemap.xml	sitemap	0.8	technical	XML, lastmod, .gz, catalog disproportion
5	Product schema	product_schema	1.0	technical	JSON-LD on real product, offers, Hyvä detection
6	Merchant policies ★ NEW	merchant_policies	0.9	technical	hasMerchantReturnPolicy, OfferShippingDetails, priceValidUntil, itemCondition
7	Organization schema ★ NEW	organization_schema	0.8	technical	Organization / OnlineStore on homepage, sameAs, logo
8	UCP profile ★ NEW	ucp_profile	0.9	technical	/.well-known/ucp, signing keys, leaked-private-key detection
9	AI product feed	ai_product_feed	1.0	feed	Feed file, /.well-known/ai-plugin.json, REST endpoint
10	JSON-LD quality ★ NEW	jsonld_quality	0.7	technical	Breadcrumb, ItemList, WebSite+SearchAction, duplicate schemas
11	Canonical + hreflang	canonical	0.7	technical	Canonical agrees with og:url + JSON-LD url; hreflang on multi-store
12	Open Graph	open_graph	0.7	technical	All 5 OG tags, description length
13	FAQ schema	faq_schema	0.5	technical	FAQPage JSON-LD on homepage or sampled CMS page
14	Well-known matrix ★ NEW	well_known	0.5	technical	ucp / ai-plugin.json / security.txt / mcp inventory
15	Core Web Vitals ★	core_web_vitals	0.5	external_api	LCP / INP / CLS via Google CrUX (API key required)
16	WAF reality check ★★	waf_reality	0.9	technical	Probes the edge with real AI crawler UAs — flags bots robots.txt allows but the WAF/CDN blocks (challenge pages detected even behind HTTP 200)
17	AI crawler activity ★★	ai_crawler_activity	0.5	live_signal	Which AI crawlers actually visited, by class (search / training / fetch), from GDPR-safe evidence sources
18	A2A Agent Card ★★★	agent_card	0.6	technical	/.well-known/agent-card.json — required when the UCP profile declares an a2a transport, informational otherwise; flags cards stranded at the pre-0.3 /.well-known/agent.json path
19	llms.txt v2 link relations ★ NEW	link_relations	0.7	technical	rel="alternate" type="text/markdown" + rel="describedby" in <head> or Link: header; follows both and fails on a declared-but-broken target; Link: header on the mirror; both page.html.md and page.md forms
20	agents.md ★ NEW	agents_md	0.7	technical	Delivery / returns / privacy actually reachable, linked policies, cross-reference to llms.txt, text/html shadowing, /sitemap_agentic_discovery.xml

★ = added in v3.0.0 · ★★ = added in v4.0.0 · ★★★ = added in v4.1.0.

The agent card is conditional on purpose

Almost no Magento store has opted into A2A, and scoring them all down for a protocol they never adopted would make the number less useful, not more. So agent_card asks the UCP profile first. If the store advertises an a2a transport, the card is part of the contract and its absence is a FAIL. If it does not, the signal passes with applicable: false in the details.

One case is flagged regardless: a card served only at /.well-known/agent.json, the path A2A used before 0.3. A spec-compliant 1.0.0 client never looks there, so the operator believes they have published a card that, to the protocol, does not exist.

v4 grading philosophy: bots are judged by purpose

The bot catalog (BotRegistry) classifies every AI agent as training (collects pages to train models), search (indexes for answer engines — this traffic earns citations), or fetcher (real-time page access for a user asking right now):

• Blocking a search crawler (OAI-SearchBot, PerplexityBot, Claude-SearchBot) fails the audit — the store disappears from AI answers.
• Blocking a training crawler (GPTBot, ClaudeBot, CCBot, Bytespider) is a respected licensing choice — reported, never punished. Opting out of model training does not remove a store from AI search.
• Opt-out tokens (Google-Extended, Applebot-Extended) never crawl anything themselves and are reported as conscious-choice notices.

The evidence layer

waf_reality and ai_crawler_activity corroborate each other:

waf_reality	ai_crawler_activity	Diagnosis
edge blocks probe	zero hits	Real block — allow-list AI crawlers in your WAF/CDN bot management
edge blocks probe	healthy hits	Verified-bot program correctly rejecting UA spoofers — no action needed
consistent	zero hits	Store likely not yet discovered — work on feeds, llms.txt, citations
consistent	search hits	Fully AI-visible ✔

Evidence sources for ai_crawler_activity (pluggable via Angeo\AeoAudit\Api\BotHitSourceInterface):

1. Built-in instrumentation (default, zero-config): a frontend plugin counts AI bot requests at the PHP layer on every hosting model. Limitation reported honestly: requests served entirely from full-page cache never reach PHP and are not counted.

2. Webserver access log (opt-in): set an absolute path under Stores → Configuration → Angeo AEO → Live Signal. Only the trailing 8 MB is read; combined and JSON-lines formats are auto-detected. Never chmod the live log — grant targeted read access instead:

   # point-grant read to the PHP user (survives permission audits):
   setfacl -m u:www-data:r /var/log/nginx/access.log
   
   # …or copy bot-filtered lines on rotation (logrotate postrotate):
   grep -Ei 'GPTBot|OAI-SearchBot|ClaudeBot|Claude-SearchBot|PerplexityBot|Amazonbot|Applebot|meta-externalagent|CCBot|Bytespider' \
     /var/log/nginx/access.log.1 > /var/log/ai-bots/access.log
   

   Behind a CDN the origin log under-counts (cache hits never reach the origin) — the report says so explicitly.

3. Your own adapter (CDN analytics APIs are the highest-fidelity source — they see cache hits too). Implement BotHitSourceInterface, register via di.xml on the checker's sources argument.

Privacy: only aggregates — bot code, class, store, date, count — are ever persisted (angeo_aeo_bot_hit). No IPs, no URLs, no raw user agents, no log lines. Retention (default 90 days) is enforced by cron. Absence of any source is an informational note, never a score penalty.

---

Installation

composer require angeo/module-aeo-audit
bin/magento setup:upgrade
bin/magento cache:flush

For full coverage, install the companion modules:

composer require \
  angeo/module-llms-txt \
  angeo/module-rich-data \
  angeo/module-openai-product-feed \
  angeo/module-openai-product-feed-api \
  angeo/module-ucp \
  angeo/module-aeo-brand-visibility

---

CLI usage

# Audit all stores
bin/magento angeo:aeo:audit

# Specific store
bin/magento angeo:aeo:audit --store=en_us

# JSON output (for dashboards / CI)
bin/magento angeo:aeo:audit --format=json

# Markdown report to file
bin/magento angeo:aeo:audit --format=markdown --output=/var/www/html/aeo-report.md

# Fast technical-only checks (skip external APIs)
bin/magento angeo:aeo:audit --category=technical

# Run only external-API checks (Core Web Vitals + any third-party live signals)
bin/magento angeo:aeo:audit --category=external_api,live_signal

# Fail build if score below threshold
bin/magento angeo:aeo:audit --fail-on=80

# Fail build if any critical-severity check fails
bin/magento angeo:aeo:audit --fail-on-severity=critical

# Run without saving to DB (CI / read-only environments)
bin/magento angeo:aeo:audit --no-save

Sample output:

  AEO Score: [████████████████░░░░] 81% — Good
  ✓ Pass: 12  ⚠ Warn: 3  ✗ Fail: 1

  Critical fixes needed:
  → Install angeo/module-openai-product-feed and register at chatgpt.com/merchants

  💡 Fix with angeo modules:
     composer require angeo/module-openai-product-feed angeo/module-openai-product-feed-api
     composer require angeo/module-ucp

---

Configuration

Some checkers need configuration. All are accessed via: Stores → Configuration → Angeo AEO.

Setting	Purpose
CrUX API Key	Required by core_web_vitals checker. Free key from console.cloud.google.com — enable the Chrome UX Report API. Stored encrypted.

---

Admin UI

• Marketing → Angeo AEO → AEO Audit Results — full history grid
• Marketing → Angeo AEO → Score Trend — line chart of AEO score over time
• ▶ Run Audit Now button (on the Audit Results grid, Score Trend, and result view pages) — triggers an on-demand audit. Since 3.1.0 this is a POST action protected by the admin form key and the dedicated Angeo_AeoAudit::run_audit ACL permission; the old GET menu entry was removed.

---

Score interpretation

Score	Label	Typical situation
0–25%	Critical	Default Magento install. AI crawlers blocked. No schema.
26–50%	Needs Improvement	Some fixes applied. Feed or merchant policies missing.
51–75%	Needs Improvement	Core signals in place. UCP, ai-plugin.json, or hreflang missing.
76–90%	Good	Strong foundation. Minor gaps in well-known or CWV.
91–100%	Excellent	Full 2026 AEO compliance.

---

Cron

Scheduled audit — default every Monday at 03:00 server time, configurable under Stores → Configuration → Angeo AEO → Scheduled Audit since v4.0.0. Results saved to DB (last 50 per store); expired bot-hit counters pruned per the retention setting.

bin/magento cron:run --group=default

For fast daily checks (without external APIs or log scans), schedule an additional cron job calling the audit with --category=technical.

---

Extending with custom checks

Implement Angeo\AeoAudit\Api\CheckerInterface (or extend Angeo\AeoAudit\Model\Checker\AbstractChecker, which provides HTTP cache, URL sampling and JSON-LD parsing), and register via di.xml:

<type name="Angeo\AeoAudit\Model\AuditRunner">
    <arguments>
        <argument name="checkers" xsi:type="array">
            <item name="my_check" xsi:type="object">Vendor\Module\Model\Checker\MyChecker</item>
        </argument>
    </arguments>
</type>

v3 interface:

public function getName(): string;       // "My Custom Check"
public function getCode(): string;       // "my_check"
public function getWeight(): float;      // 0.0–1.0
public function getCategory(): string;   // CheckerInterface::CATEGORY_*
public function getSeverity(): string;   // CheckerInterface::SEVERITY_*
public function getFixCommand(): string; // "composer require vendor/fix-module" or ""
public function check(\Magento\Store\Api\Data\StoreInterface $store): CheckResult;

Migrating from v2? See CHANGELOG.md for the migration guide.

---

Running tests

vendor/bin/phpunit -c app/code/Angeo/AeoAudit/phpunit.xml

v3 ships with unit tests covering all 15 checkers, both services (HttpCache, StoreUrlSampler), the AuditRunner, and the report value objects.

---

Code quality

# Magento Coding Standard
vendor/bin/phpcs --standard=Magento2 \
    --extensions=php,phtml --severity=10 \
    app/code/Angeo/AeoAudit/

# PHPStan static analysis
vendor/bin/phpstan analyse -l 5 app/code/Angeo/AeoAudit/

---

The Angeo AI Visibility Suite

Module	Signal	Purpose
angeo/module-aeo-audit	—	This module — audit all 18 signals
angeo/module-robots-txt-aeo	#1	Inject AI bot rules into robots.txt
angeo/module-llms-txt	#2, #3	Generate llms.txt and llms.jsonl
angeo/module-rich-data	#5, #6, #7, #13	Product, Organization, FAQ JSON-LD + merchant policies
angeo/module-openai-product-feed	#9	ACP product feed for ChatGPT Shopping
angeo/module-openai-product-feed-api	#9	REST API — 6 ACP endpoints
angeo/module-openai-instant-checkout	—	Agentic Commerce Protocol — instant checkout from ChatGPT
angeo/module-ucp	#8	Universal Commerce Protocol — /.well-known/ucp
angeo/module-aeo-brand-visibility	(extends)	Live AI visibility across ChatGPT, Claude, Perplexity, Gemini, Groq

---

Contributing

Issues and PRs welcome at github.com/angeo-dev/module-aeo-audit.

Before opening a PR:

1. Run vendor/bin/phpunit -c phpunit.xml — all tests must pass
2. Run vendor/bin/phpcs --standard=Magento2 — no MCS violations
3. Add tests for any new checker

---

License

MIT — see LICENSE

---

Made with care by Ievgenii Gryshkun — open-source contributions to the Magento + AI commerce ecosystem.