304 Not Modified and Last-Modified: How to Make Googlebot Crawling More Efficient

We at Talha Aslan and team see this topic come up often in technical SEO audits. Many site owners, however, assume a 304 Not Modified response is an error. In fact, it works the other way around when you set it up correctly. Googlebot does not download the full HTML for unchanged pages, so it can spend that effort on new or updated URLs. However, a broken setup is risky too, because Google may keep an outdated version of a page you just edited.

This guide walks through conditional requests, the Last-Modified and ETag headers, how Google uses them according to its official documentation, and what to check in your CDN and caching layer. We avoid code blocks on purpose. The goal is to help you speak the same language as your developer or hosting team.

How does a conditional request actually work?

A conditional request is a request in which the client says, "If my stored copy is still valid, do not send the content again." On the first visit, the server sends the page with 200 OK and adds a validator. That validator is either a date (Last-Modified) or a content-specific tag (ETag).

On the next visit, the client sends this validator back. The server compares it and picks one of two outcomes:

1. If the content has not changed, it returns 304 Not Modified with no body.
2. If the content has changed, it returns 200 OK with the new content and a new validator.

This mechanism does not belong to Googlebot alone. Browsers also use the same method. For example, the HTTP cache guide on web.dev points out that a 304 response transfers very little data. As a result, a well built conditional request flow helps both user experience and crawl efficiency.

The key point here is simple: the server makes the decision. In other words, the client only says, "I have this version." Knowing whether the page really changed is entirely the job of your system.

What do the Last-Modified and If-Modified-Since headers do?

Last-Modified is a response header. The server sends it with the page, and it states the date when the page last changed. If-Modified-Since is a request header. The client sends it on the next request. In other words, the first one travels from the server, and the second one travels back to the server.

Here is the flow. When Googlebot first crawls a page, it sees the Last-Modified value in the response. Google's crawling errors documentation explains that when its crawlers send If-Modified-Since, the value is the date and time Google last crawled the content. If nothing changed after that date, the server can answer with 304 Not Modified.

The date format also matters. Google's overview of Google crawlers says the Last-Modified date must follow the HTTP standard. Therefore, dates in a local time zone, with translated month names, or with missing parts cause trouble. The standard format includes the weekday, day, short month name, year, time, and the word GMT.

Also, the date should reflect the moment the page changed in a meaningful way. A system that prints the current time on every request makes the header useless. That is because the page looks new every time, so a 304 never happens.

Why does Google prefer ETag and If-None-Match?

An ETag is a unique tag the server assigns to a specific version of a resource. In practice, it works like a fingerprint (hash) of the content. On the next request, the client sends the tag back in the If-None-Match header. If the tag matches, the server returns 304 Not Modified. If not, it sends the new content.

Google states a clear preference here. According to the same crawler overview, when both ETag and Last-Modified appear in a response, Google's crawlers use the ETag value. Moreover, Google recommends ETag over Last-Modified to indicate caching preference. The reason is practical: with ETag, you do not have to worry about date formatting issues.

So is Last-Modified pointless? Not at all. That is because some clients and intermediate layers only look at dates. In practice, sending both headers together and keeping them consistent is the most robust approach. In our audits, we first check whether the ETag stays stable. Then we check whether the Last-Modified date matches real changes.

The same document also mentions the max-age field of the Cache-Control header. It is optional, yet it can help crawlers decide when to recrawl a URL. On the other hand, Google notes that it does not support other HTTP caching directives.

How does Google interpret a 304 Not Modified response?

Google's HTTP status code documentation answers this directly. When its crawlers get a 304, they signal to the next processing system that the content matches the last crawl. On the indexing side, the document says the pipeline may recalculate signals for the URL, but otherwise the status code has no effect on indexing.

In practice, that sentence has real weight. A 304 Not Modified response:

• Does not remove the page from the index.
• Does not raise or lower rankings on its own.
• Lets Google reuse the copy it stored earlier.
• Reduces the cost of generating and sending the page body.

In short, 304 is an efficiency mechanism, not a ranking signal. Nobody can honestly promise that a 304 setup will push you to the top, and we do not promise that either. Still, if you have many pages to crawl, cutting the effort spent on unchanged URLs can indirectly help important pages get crawled more regularly.

The reverse also holds. If you return 304 after the content changed, Google keeps using the old copy. That is why a correct 304 setup matters more than simply having one.

Does Googlebot send conditional headers on every request?

No, it does not. Google's troubleshooting page explains that its crawlers do not include these headers on every crawl attempt. Instead, it depends on the purpose of the request. The page also notes that AdsBot is more likely to send them.

The same page adds an interesting detail. A server may send a 304 with no body for any Googlebot request if the content has not changed since Googlebot's last visit, even when the request carries no conditional headers. However, that requires your server to know when Googlebot last visited, which most sites cannot track without extra logic.

In addition, not every Google crawler handles caching the same way. Google's documentation says Googlebot supports caching when it recrawls URLs for Google Search. Storebot-Google, by contrast, supports it only in certain conditions. So if your logs show some Google requests always getting 200, that alone does not mean something broke.

Put simply, your goal should not be "every Googlebot hit gets a 304." Your goal is that the server makes the right call whenever a conditional request arrives.

Why does 304 Not Modified matter for crawl efficiency?

Google's crawl budget management guide lists HTTP caching and support for the 304 status code among its efficiency recommendations. According to the guide, if a page has not changed since Google last crawled it, a 304 tells Google to reuse the cached version.

The same guide links crawl capacity to server health. When response times stay stable or improve, the limit can rise. Therefore, fast, bodiless 304 responses can help indirectly because they reduce server load. For the server speed side, see our guide to TTFB and server response time.

We will not repeat the crawl budget concept here. Our crawl budget guide covers which sites really need to care about it. If your crawl rate dropped suddenly, we also collected the likely causes in why Googlebot crawls less.

It is hard to confirm with a one-time check that settings like these work on your site. Through our SEO consulting service, we monitor server responses, logs, and Search Console data together, then fix the issue on your site and keep tracking it.

Which sites should prioritize 304 Not Modified support?

First, the urgency is not the same for every site. Google's crawl budget guide describes its audience roughly as follows: large sites with many unique pages that change moderately often, medium or larger sites with rapidly changing content, and sites with a large share of URLs showing "Discovered, currently not indexed" in Search Console.

We have a separate article on that last status: what Discovered currently not indexed means. Based on our field experience, these site types tend to benefit most from 304 support:

• E-commerce stores with thousands of product and category pages.
• News and content sites with large archives that rarely change.
• Listing and catalog sites that produce filter and listing pages.
• Mid-size sites running on shared or limited server resources.

On the other hand, a corporate site with a few dozen pages does not need to treat 304 as a priority. For that kind of site, fixing content, internal linking, and indexing issues first brings a better return. That said, if your platform already supports it by default, there is no reason to turn it off.

How do 200 OK, 304 Not Modified, and other responses compare?

The table below summarizes the explanations in Google's HTTP status code documentation along with their practical crawl impact. However, it is not a ranking guide. It shows what each code tells Google.

The most important distinction is this: a 304 behaves like a success response but carries no content. A 5xx, in contrast, slows crawling down. So if a failure in your caching layer starts returning 5xx instead of 304, crawl efficiency flips in the wrong direction overnight. The same Google document covers server errors in more detail.

How do you test whether your site returns 304 Not Modified?

Before you start, keep one thing in mind: you want to see how templates behave, not just a single page. Pick at least one URL from each template, such as the homepage, a category, a product or post, and a static file. Then follow these steps:

1. Open the Network tab in your browser's developer tools and make sure the option that disables the cache is off.
2. Load the page for the first time and note whether the document response includes ETag and Last-Modified.
3. Reload the page normally and check whether the status column shows 304 for the document request.
4. Make a small change to the content and reload again. This time you should see 200 and a new ETag.
5. Repeat the test with a command line HTTP client and put the ETag value into an If-None-Match header.

In practice, step four is the one people skip most often. Yet it catches the real danger. A system that returns 304 after the content changed tells Google to keep using the old version. As an example scenario, if you change the price on example.com/product-a and still get a 304, the problem sits in your caching layer.

Where do you find 304 responses in the Search Console Crawl Stats report?

In Search Console, you will find this data in the Crawl stats report under Settings. According to Google's help page, the report groups responses by type and shows each one as a percentage of all crawl responses. The "Not modified (304)" row means the page has not changed since the last crawl request.

We also suggest checking these areas in the report:

• Crawl purpose: Discovery covers URLs Google never crawled before, while Refresh covers recrawls of known pages.
• Average response time: the average for all resources fetched from your site in the selected period.
• Host status: whether Google ran into availability issues while crawling your site.
• File type and Googlebot type breakdowns: these help you separate which resources Google crawls and how often.

Still, be careful with interpretation. If a site has lots of refresh crawls and the 304 share sits near zero, conditional request support may not work at all. On the other hand, Google publishes no target value for this share. So we do not give you a threshold to hit. The useful move is to compare the periods before and after a change in the same report.

How do you read 304 Not Modified lines in server logs?

Search Console gives you a summarized view. Server logs, however, show every single request. That makes logs the most reliable way to verify a 304 setup. We covered the general method in our guide to server log file analysis for SEO, so here we only focus on points specific to 304.

Ask these questions of your logs:

1. Do the requests that claim to be Googlebot really come from Google? Do not skip the reverse DNS check Google recommends.
2. Which templates never get a 304? For instance, if posts get 304 while product pages always get 200, the templates differ.
3. Do URLs with a 304 actually show a response size close to zero?
4. On the day you updated a page, did the first Googlebot request for that URL get a 200?

However, scanning large log files by hand takes time. Instead, you can run a first pass with our log file analyzer and see the distribution by status code. After that, review the suspicious templates one by one. This way, you replace the vague belief that "the site supports 304" with evidence of where it actually works.

What problems does a wrong Last-Modified value cause?

There are two typical ways to get Last-Modified wrong, and each one does different damage. The first is a date that always equals "now." In that case, the page looks new on every request. Conditional requests never end in a 304, and the efficiency gain disappears entirely.

The second is sneakier: a date that never changes. For example, a template might print the page's original creation date as Last-Modified. Then, even after you update the content, the server may say "unchanged" and return 304. As a result, Google might miss the new price, the new section, or the correction you made.

Next, a third issue involves component changes. The main text of a product page might stay the same while stock status, price, or review count changes. If Last-Modified only tracks the main text, the system misses changes that users actually see.

Our recommendation follows from this. Last-Modified should reflect the latest change in any meaningful data that affects the HTML output. Minor parts, such as the copyright year or a random recommendation box, should not trigger it.

How do you generate a correct ETag on dynamic pages?

The most robust way to think about an ETag is as a fingerprint of the page's meaningful content. For example: same content, same fingerprint. Changed content, different fingerprint. That way, the tag reflects the real state of the page. On dynamic sites, though, this simple idea breaks in a few places.

These are the failure points we see most often:

• A part that changes on every response, such as a session token, timestamp, or random ID, feeds into the fingerprint, so the ETag differs every time.
• Behind several servers, each one produces a different ETag for the same file. The client sees a different tag depending on which server answers.
• The compression layer gives the same strong ETag to compressed and uncompressed versions, or it strips the tag completely.
• Personalized content goes out to logged in and logged out users with the same tag.

The HTTP standard distinguishes weak and strong ETags. A weak tag treats semantically equivalent versions as the same. A strong tag expects byte-for-byte identity. Your choice depends on your stack. What matters is that the same content gets the same tag on every server and every request.

How does a CDN or caching layer affect the 304 Not Modified flow?

If you use a CDN or a reverse proxy, Googlebot often talks to that middle layer rather than your origin server. Consequently, you need to know who makes the 304 decision. Sometimes the CDN checks its own copy and returns 304. Other times, it forwards the request to the origin.

In practice, three situations can occur. First, the CDN passes the origin's ETag and Last-Modified values through unchanged, which is the ideal case. Second, the CDN strips these headers or replaces them with its own values, which wastes your work on the origin. Third, the CDN holds a stale copy for too long and returns 304 even though the content changed.

Above all, the third case carries the most risk. Here is an example scenario: you update a campaign page on example.com but forget to purge the CDN cache. Googlebot sends a conditional request, the CDN finds a match with the old ETag, and it returns 304. Google keeps using the outdated copy.

If you plan to set up a reverse proxy cache, our Varnish Cache guide covers the basic configuration logic. Whatever tool you use, the rule stays the same. Purging the related cache entry whenever content changes must become a fixed step in your publishing process.

What should a CDN and cache configuration checklist include?

Use the list below with your team when you set up a new CDN or switch caching plugins. We wrote each item as a test question. Any item you cannot answer "yes" to is a place to investigate.

1. Do HTML responses include an ETag or Last-Modified header?
2. Next, does the CDN pass these headers through from the origin unchanged?
3. Does the same URL return the same ETag from different edge locations?
4. When content changes, does the related cache entry clear automatically?
5. After an update, does the first conditional request get 200 and a new ETag?
6. Do personalized or session based pages stay out of the shared cache?
7. Do error pages (404, 5xx) stay out of the cache as fake 200 or 304 responses?
8. Did you choose Cache-Control values on purpose for each page type?
9. Do you version static files such as CSS, JavaScript, and images?
10. Do you watch the response breakdown in the Crawl stats report after each change?

Moreover, filling out this list once is not enough. We recommend running it again whenever a theme, plugin, or CDN setting changes. That is because 304 behavior often breaks silently, as a side effect of some other change.

What should you watch for on WordPress and hosted platforms?

On WordPress, hosted e-commerce platforms, and site builders, your web server, caching plugin, and CDN usually decide 304 behavior together. The web.dev guide also notes that some web servers set these headers by default, while others leave them out unless you configure them.

On WordPress, we often see the following pattern. A page caching plugin produces static HTML, and the server adds headers to those files automatically. However, when the plugin stops serving a page, dynamic PHP responses usually lack an ETag. Also, some security or performance plugins rewrite headers.

On hosted platforms, however, your access to server settings may be limited. In that case, the best you can do is measure current behavior with the test steps above. Then, if needed, contact the platform's support team with a concrete example. Instead of writing "304 does not work," share the URL, the headers, and the behavior you expect.

Note: we are not a hosting company, and we do not run servers. As part of SEO consulting, though, we prepare clear technical findings for your hosting team or developer and then verify the result with you after implementation.

How do you keep sitemap lastmod and the Last-Modified header in sync?

The lastmod tag in your sitemap and the HTTP Last-Modified header live in different places, yet they answer the same question: when did this page last change in a meaningful way? Google's sitemap documentation says it uses the lastmod value if it is consistently and verifiably accurate. As an example, it mentions comparing the value to the last modification of the page.

What happens if the two values disagree? Suppose the sitemap says a page changed yesterday while the server's Last-Modified header shows a date from months ago. As a result, the signals conflict. That conflict can lead Google to stop trusting your lastmod values.

Our advice is simple: generate both values from a single source, namely the "last meaningful change" field of your content. We explain correct lastmod usage in our sitemap lastmod guide, and we cover the basics in what an XML sitemap is and how to create one. That way, Google sees the same story in both the sitemap and the HTTP response.

Which mistakes make a 304 setup dangerous?

When you add 304 Not Modified support, the most harmful mistakes are the ones that damage content accuracy, not efficiency. These are the risky cases we see most often in audits:

• Returning 304 after the content changed, so Google keeps using the old version.
• Returning a bodiless 304 during maintenance or an error, which hides the problem.
• Returning 304 with an old ETag on redirected URLs, so Google misses the new target.
• Keeping old headers in cache for pages where you changed noindex or canonical tags.
• Giving the same tag to mobile and desktop versions on sites that serve different HTML.

Specifically, all of these share one trait. The system says "unchanged" when something did change. Therefore, when you test your 304 setup, do not only ask "do I get a 304?" Also ask "do I get a 200 after a change?" The second question matters more than the first.

Before a large template change, a domain migration, or a new theme launch, purge the cache on purpose and watch the first crawls in your logs. It is a safe habit that prevents surprises.

When should you hand this work to an SEO team?

At first glance, conditional requests look simple on paper. On real sites, however, the origin server, caching plugin, CDN, load balancer, and CMS all act at once. Finding the layer where the problem sits usually means reading logs, headers, and Search Console data together.

We suggest getting help if any of these apply to you:

• You have thousands of pages, and Googlebot reaches important ones late.
• Google keeps showing old versions of content you updated weeks ago.
• Your crawl stats show server errors or high response times.
• You plan to move to a new CDN or caching layer.

We find these technical crawling issues on your site, give your developers clear implementation steps, and track the outcome on a regular basis. We do not guarantee rankings. Instead, we build a measurable improvement process based on Google's official documentation. To get a quote tailored to your site, reach us through our SEO consulting page.

Core guide: What Is Digital Marketing? Its types, examples and how it is done, on one page with an interactive film that travels through the brain.