Why would AI miss my content if Google sees it?#

Google renders pages in a full browser, so it eventually sees content that JavaScript adds. Many AI crawlers do not. One analysis of more than 500 million fetches by OpenAI's GPTBot found no evidence that it executes JavaScript, and crawlers from Anthropic and Perplexity are widely reported to work the same way. A page can therefore rank on Google and still look blank to the tools your customers increasingly ask.

Which websites are most at risk?#

  • Sites built as single-page apps, where the page arrives empty and scripts fill it in
  • Website builders and themes that load key sections (reviews, menus, room lists, prices) through widgets
  • Pages where important text sits inside sliders, tabs, or pop-ups that load after the page
  • Sites that inject structured data with JavaScript instead of including it in the HTML

How can I check my own site in five minutes?#

  1. Open an important page, right-click, and choose View Page Source (not Inspect).
  2. Search the source for a sentence from the middle of the page. If you cannot find it, crawlers that skip JavaScript cannot either.
  3. Search the source for application/ld+json. That is your structured data; it should be there.
  4. Turn off JavaScript in your browser settings and reload the page. What you see is roughly what many AI crawlers see.
  5. Run the page through Google PageSpeed Insights on mobile and note whether it passes Core Web Vitals.

How do I fix it?#

  • Pre-render or server-render your pages so the full HTML, including text, links, and structured data, arrives in the first response.
  • Move critical content out of widgets. Rooms, treatments, prices, and FAQs should be plain text on the page.
  • Put structured data in the HTML, not in a tag manager.
  • Keep it fast. Speed affects rankings and conversions: pages that load in 1 second convert about three times better than pages that take 6 seconds (Portent). Only 38.4% of WordPress sites pass Google's mobile Core Web Vitals (HTTP Archive and Chrome UX Report, June 2026).
  • Let AI crawlers in. Check that your robots.txt and security settings do not block crawlers such as GPTBot, OAI-SearchBot, ClaudeBot, and PerplexityBot, unless you have chosen to.

Does my website platform matter?#

It matters less than how the site is built, but defaults differ. Static and pre-rendered sites send complete HTML by design. Builder-based and plugin-heavy sites can do it too, with care. Whatever you use, run the five-minute check above on your most important pages.