Can AI Crawlers See JavaScript Content?

In short, your page may look perfect in a browser while a bot that reads only the first HTML response sees an empty shell. This guide shows you how to diagnose the problem, which rendering approach works, and which facts to move into HTML first. For how Google handles scripts in detail, see our JavaScript SEO guide.

What is the difference between raw HTML and the rendered page?

Raw HTML is the text your server sends in its first response. When a browser or a bot requests a page, that is the first thing it receives. The rendered page, by contrast, is what the browser builds after it reads that HTML and runs the scripts. The two can match, but in single page applications (SPAs) they often differ a lot.

For example, the first response for a product page might hold only a title, a menu, and an empty container. The product name, description, and specs then arrive through a separate data request. Visitors never notice, because the browser fills everything in within a moment.

A bot that does not run scripts, however, sees only that first response. So from its point of view, the page has no product. An AI assistant cannot summarize, cite, or recommend what its crawler never saw.

How does Google process JavaScript pages?

Google's JavaScript SEO basics documentation describes three phases: crawling, rendering, and indexing. First, Googlebot fetches the URL if robots.txt allows it, reads the HTML, and extracts links. Next, the page enters a render queue, where a headless Chromium runs the scripts.

According to the same page, a URL may wait in that queue for seconds or longer. Finally, Google indexes the page using the rendered HTML. In other words, Google usually sees JavaScript content, but with a delay and an extra step.

The documentation still recommends server-side rendering or pre-rendering, even though Google can run JavaScript. It points to speed for both users and crawlers as the reason. It also states plainly that not all bots can run JavaScript. That one sentence is the starting point for any plan that involves AI crawlers.

Do AI crawlers run JavaScript?

The honest answer: we cannot say for every crawler. Many providers do not state in their official docs whether their bots execute scripts. For instance, OpenAI's crawler documentation describes OAI-SearchBot, GPTBot, and ChatGPT-User and what each one does. It does not, however, describe rendering behavior.

Treat that gap as a design rule, not a mystery. Given the uncertainty, the safe assumption is that a crawler may not run JavaScript. Therefore, never let critical facts depend on scripts alone. Google's warning that not all bots can run JavaScript points in the same direction.

Behavior can also change over time. A crawler that skips scripts today might render them later, or the reverse. So the right strategy is not to guess bot behavior but to keep content readable under any condition. We list the main AI crawlers in our AI crawlers guide.

How can you tell whether your site has this problem?

The simplest test is to view the page source. Right-click the page in your browser and choose the option to view source. That view shows the server's first response. It differs from the elements panel in developer tools, which shows the rendered state.

1. Open the page source.
2. Search for a few words from your main heading and first paragraph.
3. Look for critical facts one by one: product name, pricing explanation, address, service description.
4. Write down every fact you cannot find.

Then repeat the test with JavaScript turned off in the browser. If the page loads empty or half-empty, a crawler that skips scripts gets the same view. Also run the test on both desktop and mobile layouts, since some sites load different components on mobile. This two-minute check often beats a full technical audit for speed.

Running these checks across every template while you also handle content and product work can be hard to sustain. That is why our AI SEO services start with a raw HTML inventory per template. If you run the work yourself, the steps below follow the same order.

Which tools give you a deeper check?

Viewing source is fast, but it covers one page at a time. For a wider picture, combine several methods:

• Search Console URL Inspection: shows how Google rendered the page, which is ideal for the Google side.
• A plain command line request: downloads the page without running scripts, so you can save the raw response and search it.
• A crawl with rendering off: crawl the whole site with JavaScript rendering disabled, then compare against a crawl with rendering on.
• Server logs: show which URLs AI crawlers request and which status codes they receive.

For example, templates whose word counts differ sharply between the two crawls form your priority list. That way you see where the problem clusters without opening pages one by one.

Which facts must live in raw HTML for AI crawlers?

You do not have to move everything at once. Start with the information that helps most in AI answers. We suggest this order:

1. The page title, main heading, and opening paragraph.
2. The name of the product or service, what it does, and who it serves.
3. Business name, address, phone, and hours.
4. Answers to frequently asked questions.
5. Internal links and main navigation links.
6. Structured data, as long as it matches the visible text.

Contact details deserve special care. If your business moved, for instance, the new address should read the same in your Business Profile and in your site's raw HTML; our guide to updating a Business Profile address after a move covers the profile side. Also make sure text hidden behind tabs or "show more" buttons sits in the first response.

One more trap: sometimes the content does exist in the source, but only inside an embedded data block rather than as readable text. Here too, plain text inside heading and paragraph tags is the safer choice.

What separates server-side, static, and client-side rendering?

web.dev's Rendering on the Web article defines the core terms. Server-side rendering (SSR) runs the app on the server and sends ready HTML to the browser. Static rendering produces HTML at build time as a separate file for each URL. Client-side rendering (CSR), on the other hand, builds the page in the browser with scripts.

Hydration means client scripts add interactivity to HTML that the server already sent. So the content sits there from the start, and scripts only bring buttons and forms to life.

Which rendering method should you choose?

The choice depends on page type and your current stack. There is no single right answer, but a general decision order helps:

• If content rarely changes, static rendering is the simplest option.
• If content changes often and is public, server-side rendering fits.
• If you already run an SPA and a rewrite would take long, consider pre-rendering as a bridge.
• For dashboards behind a login, client-side rendering is fine, since crawlers have no business seeing that content.

For example, a framework like Nuxt makes SSR easier for Vue sites, as we explain in our Nuxt and Vue SSR guide. Still, switching frameworks is a big decision. First measure which templates actually fail, then focus on those alone.

Also document the rule you pick. When someone adds a new template or redesigns a page, the same rule should apply: critical content ships in the first response. Otherwise, a small change a year later can quietly undo the work.

Is dynamic rendering still an option?

Dynamic rendering serves pre-rendered HTML to bots and the normal JavaScript app to users. The web.dev article mentions it as worth considering for JavaScript-heavy sites. That said, the approach needs care.

Bots and users must get essentially the same content. If you show bots different text, different prices, or hidden keywords, that counts as cloaking and conflicts with search engine spam policies. Thus, dynamic rendering should only ever mean delivering the same content through a different technique.

It also forces you to maintain two systems. When the cache goes stale, bots may see old content. For that reason, moving to server-side or static rendering is usually the cleaner long-term path. Think of dynamic rendering as a bridge for a transition period.

For instance, a large store might first fix category and product templates with pre-rendering. Then, once the new stack is ready, it can move those templates to SSR for good.

Could JavaScript be why AI answers do not cite you?

Yes, it could, but it is rarely the only reason. If your page is empty for a crawler that skips scripts, the assistant behind that crawler cannot cite it. On the other hand, weak content, a robots.txt block, slow responses, or clearer answers elsewhere can produce the same result.

So diagnose in order. First, check whether the content exists in raw HTML. Second, check whether robots.txt blocks the relevant crawler. Third, confirm in server logs that the crawler actually requests the page. Finally, look at how clearly your page answers the question.

We collected the other likely causes in our diagnostic checklist for sites AI does not cite. That way you can separate the JavaScript issue from other causes and spend effort in the right place.

What do Google's AI features require?

Google's AI features documentation says AI Overviews and AI Mode have no additional technical requirements. A page must be indexed and eligible to show with a snippet. The same page also recommends keeping important content available as text and allowing crawling.

In other words, Google treats JavaScript content in its AI features the same way it does in regular search. If rendering succeeds, the content enters consideration. If rendering fails, or if key text only appears after an interaction, the same problem carries over.

Moreover, Google is not your only target. Other assistants have not made comparable public commitments about their crawlers. As a result, raw HTML is the common denominator across every surface. Put simply, what works for Google is also the safest baseline for the rest.

How do lazy loading and infinite scroll affect AI crawlers?

Lazy loading fetches content as the user scrolls. It helps with images. However, if text or product lists depend on scrolling, a crawler that never scrolls never reaches them.

Infinite scroll carries a similar risk. The first ten products on a category page may sit in the HTML, while the rest only load on scroll. In that case, take these steps:

• Offer paginated links next to infinite scroll, each with its own URL.
• Do not tie text content to lazy loading, and keep it for images.
• Keep text inside tabs and accordions in the first HTML, and let scripts only toggle visibility.
• Use real link elements for important links instead of buttons wired to click events.

For example, if FAQ answers only load from the server after a click, the crawler sees the questions but not the answers. That means losing the very content type AI answers quote most. The same applies to reviews pulled in from a third-party widget, so consider placing at least a summary of them in the first HTML.

What happens when structured data loads through JavaScript?

Some sites inject structured data later through a tag manager or a script. Google can read it during rendering. A crawler that skips scripts, however, never sees structured data added after the first response.

So, where possible, output structured data on the server inside the first HTML response. It should also match the visible text, and Google's AI features documentation lists that match among its recommendations. We take a realistic look at what markup does for AI visibility in our article on schema markup for GEO.

Remember that structured data does not replace visible content. If the page has markup but no text, both users and bots get an incomplete picture. Move the text into HTML first, then add the markup.

Where do performance and crawl efficiency fit in?

Heavy scripts do not only hide content. They also slow the page down. Google's documentation cites speed when it recommends server-side or pre-rendering. A slow page may look incomplete to a crawler that times out.

In addition, if every page's content depends on a separate data request, a crawler has to make several requests per page. On large sites, that slows crawling. Therefore, preparing HTML on the server makes life easier for crawlers and users alike.

We cover how JavaScript affects speed, and how to optimize it, in our JavaScript performance guide. In short, fast, ready HTML solves two problems at once. A page that ships content in its first response also shows visitors something meaningful sooner, which may help conversions too, though you should treat that as a possible side benefit, not a promise.

How can robots.txt and resource blocks break rendering?

According to Google's documentation, Google does not render JavaScript on pages that robots.txt blocks. Likewise, if you block script files or data endpoints, even a crawler that renders cannot finish the page. People overlook this mistake often.

1. Check that robots.txt does not block folders that hold scripts and styles.
2. Make sure the data endpoint that delivers page content stays reachable.
3. Confirm that CDN or firewall rules do not block legitimate crawlers.
4. Manage the bots you want to exclude on purpose, by name.

For instance, if a security rule returns an empty response to unknown user agents, a crawler sees nothing even when your raw HTML is perfect. So look beyond page code to server and CDN settings as well. These settings often belong to different people, so bring everyone into the same diagnosis call.

Which site types hit this problem most often?

Not every site faces the same level of risk. In our field experience, some site types are more exposed than others:

• Company sites built as SPAs: every page opens with the same empty shell and content arrives later.
• Page builder sites with heavy add-ons: some blocks print text only through scripts.
• Headless storefronts: product data comes from a separate service, and a wrong render setting leaves the first response empty.
• Booking and appointment sites: service descriptions often load inside a widget.
• Store locator pages: the address list only appears through map interaction.

That said, belonging to one of these groups does not mean you have the problem. Many modern frameworks offer SSR by default. So test instead of generalizing. If the raw source holds the content, you have nothing to worry about. If not, fixing the one component responsible is often enough.

What should you tell your developers?

Marketing and development teams often speak different languages. "AI cannot see us" gives a developer no clear task. Instead, write a concrete, testable request.

1. List the affected templates with sample URLs.
2. Name the fields each template must include in the first HTML response: heading, description, pricing explanation, address, FAQ answers.
3. State the acceptance test: these fields stay readable with JavaScript off.
4. Make clear that design and interactivity stay, and only the delivery of content changes.
5. Agree in advance on which test you will run together after the fix.

That turns a vague "SEO fix" into a testable development task. It also lets the team pick the solution that fits the stack. One team may choose SSR, while another moves a template to static generation. What matters is that the page passes the acceptance test. Also add a priority order, so developers spend their time on the templates with the most traffic and the most questions first.

How do you verify the fix?

After the change, rerun the same tests. Check that critical facts appear in the raw source, that the page stays readable with JavaScript off, and that a crawl with rendering off now shows higher word counts. Then use URL Inspection in Search Console to see what Google renders.

On the AI side, patience matters. Crawlers need time to revisit, and assistant answers need time to update. Meanwhile, track crawler visits in your server logs and ask your chosen questions at regular intervals.

Log the results in a simple sheet: date, URL, fields found in the raw source, and crawler visits. For a quick first look, try our AI visibility checker. Still, no tool can guarantee that a page will appear in an AI answer. The technical fix opens the door; content quality and source diversity decide whether you walk through it.

Is being readable enough to get recommended?

No. Fixing JavaScript content lets AI crawlers read you, but it does not make them recommend you. Recommendations also need independent mentions, real reviews, and consistent brand facts.

Think of the two jobs as links in a chain. The first link is technical access: content in HTML, crawler not blocked, page fast. The second link is trust: other sources say the same thing about you. If either link is missing, the chain breaks.

We cover the second link in our guide to getting into AI best-of recommendations. Once this article gives you the technical base, you can move on to those steps. Put simply, become readable first, then earn trust, because content work on a page no crawler can see stays invisible.

What should you check this week?

The checklist below breaks the fix into small steps:

1. Pick your five most important templates: home, service, product, category, blog.
2. Open the raw source of each one and search for critical facts.
3. Open each page with JavaScript off and note what is missing.
4. Run one crawl with rendering off and one with rendering on, then compare word counts.
5. Review robots.txt, CDN, and firewall rules.
6. Plan which missing facts move into raw HTML, in priority order.
7. Rerun the same tests after the fix and record the results.

Consider a sample scenario. A service business finds that its service descriptions load only through scripts. It moves the service template to SSR, places FAQ answers in the HTML, and starts tracking crawler visits in its logs. Later checks then show what the change did.

Who should run this work with you?

On simple sites, you can run the tests above yourself, and you need no paid tool for the first checks. Changing the rendering method, reworking templates, and tracking results over time, however, take both developer and SEO time. Bridging the two teams is often the hardest part.

The generative engine optimization services from Talha Aslan and team bring the raw HTML inventory, rendering advice, crawler access checks, and measurement into one plan. We do not guarantee rankings or citations. Instead, we explain what we change, why, and which metric we watch. Whichever path you choose, keeping critical facts in HTML is a solid start for every AI surface.