Most sites never think about sitemap size. You generate the file once, drop it in the root of your site, submit it, and move on. Then the site grows, an e-commerce catalog gets imported, or a forum archive gets added, and one day the sitemap quietly crosses a line it was never allowed to cross.
Nothing dramatic happens when it does. There is no error page and no email. The file just stops being a reliable list of your site, and the part that got cut off simply goes unread. This guide explains where that line is, what a sitemap index file does about it, and how to split a large site in a way that actually helps you later.
The Two Limits Every Sitemap Has
The sitemap protocol sets two hard ceilings on a single file. A sitemap can hold at most 50,000 URLs, and it can be at most 50 megabytes uncompressed. Whichever limit you hit first is the one that matters, and both are defined in the Sitemaps.org protocol that Google, Bing, and most other crawlers follow.
Fifty thousand sounds like a lot until you count what a sitemap tends to include. A store with 8,000 products and a handful of filter variants per product can reach the cap without anyone noticing. A news archive with a decade of daily posts gets there too.
The size limit is the quieter one. If you add image or video extensions, or include long URLs with query strings, a sitemap with only 30,000 entries can still weigh in over 50 MB. People tend to check the URL count and forget the file size entirely.
What Actually Happens Past the Limit
Search engines do not promise a graceful fallback. A file that exceeds the limits may be rejected outright, or it may be read only up to the cutoff point, depending on the crawler and the day. Neither outcome is something you want to find out about by accident.
The worse case is the partial read, because it looks like success. Search Console reports that the sitemap was fetched, and everything seems fine, but the URLs at the bottom of the file never get discovered through that route. Those are often your newest pages, since most generators list content oldest to newest.
If you ever see a gap between the number of URLs you submitted and the number Search Console says it discovered, check the file size and entry count first. It is a five minute check and it rules out the most boring explanation before you go hunting for exotic ones.
Photo by Paul Groom Photography Bristol on Pexels
What a Sitemap Index File Is
A sitemap index is a small XML file that lists other sitemap files instead of listing pages. Think of it as a table of contents. You submit the index once, and the crawler follows each entry to fetch the individual sitemaps underneath it.
The structure is simple. Where a normal sitemap wraps its entries in a urlset element, an index wraps them in a sitemapindex element, and each child is a sitemap entry with a loc for the file's address and an optional lastmod date. That is the whole format.
The index has its own ceiling, which is generous: it can list up to 50,000 sitemap files, each holding up to 50,000 URLs. In practice that means the protocol can describe two and a half billion URLs before anything needs to change, which is more than almost any site will ever need.
Splitting by Size Versus Splitting by Meaning
The laziest way to split a sitemap is to cut it every 50,000 URLs and number the pieces: sitemap-1.xml, sitemap-2.xml, and so on. That satisfies the protocol, and it is what most automatic tools do by default. It also tells you almost nothing when something goes wrong.
The better habit is to split by meaning wherever you can. Put products in one sitemap, blog posts in another, category pages in a third, and static pages in a fourth. If a section grows past the limit, split that section further, and keep the name readable, like sitemap-products-2.xml.
This matters most when you look at indexing reports. Search Console lets you filter coverage data by the sitemap a URL came from, so a meaningful split turns a vague question like "why are fewer pages indexed this month?" into a specific one, such as "why did indexing drop on the product sitemap but not the blog one?"
Keeping lastmod Honest
The lastmod field is the one piece of optional metadata that crawlers actually pay attention to, and only when it has proven trustworthy. If every URL in your sitemap reports the same lastmod date, or the date changes every time the file regenerates regardless of content changes, crawlers learn to ignore it.
On an index file, each child sitemap carries its own lastmod, and this is where the split pays off. If only your blog sitemap changed today, only that entry should show today's date. A crawler can then skip the product sitemap entirely and spend its time where something is new.
Set lastmod to the date the content last meaningfully changed, not the date the file was built. A typo fix is borderline, but a rebuilt template that touches every page is definitely not a reason to update every date. Honest dates earn you the benefit of the doubt, and that benefit is the whole point of including them.
Photo by Cameron Basson on Pexels
Where the Files Have to Live
There is one placement rule that trips people up. A sitemap can only describe URLs at or below the directory where the sitemap itself is located. A file at /blog/sitemap.xml can list pages under /blog/, but it cannot vouch for pages in /shop/.
The simplest way to avoid the problem is to put the index and every child sitemap in the root of the site. That way the location rule never comes up, and nothing needs to be explained to a future maintainer. If you do keep children in subfolders, make sure each one only lists URLs from its own folder or deeper.
Everything also has to be on the same host. A sitemap for www.example.com should not list URLs from shop.example.com, and an index should not point to child files on a different domain unless you have verified ownership of both in Search Console. When in doubt, keep it all on one host and one protocol.
Compression, Encoding, and the Small Print
Child sitemaps can be gzip compressed to save bandwidth, and the file will carry a .xml.gz extension. The 50 MB limit applies to the uncompressed size, so compressing a file does not buy you extra room. It only makes the download faster, which can help on very large sites where the crawler fetches many files in a row. The GNU gzip project documents the format if you want to script it yourself.
Every sitemap must be UTF-8 encoded, and any special characters in URLs have to be properly escaped. An ampersand in a query string becomes &, and apostrophes, quotes, and angle brackets have their own entity forms. A single unescaped ampersand makes the whole file invalid XML, and a strict parser will stop reading at that point.
This is the kind of detail that is easy to get right when a tool handles it and easy to get wrong when someone edits the file by hand. If you have ever opened a sitemap in a text editor to fix one URL and then lost an hour to a parse error, you already know why.
"Splitting a sitemap is never the hard part. The hard part is deciding what the pieces mean, because six months from now somebody will need to debug indexing on this site, and numbered files tell them nothing." - Dennis Traina, founder of 137Foundry
Photo by Brett Sayles on Pexels
Submitting the Index and Reading the Results
Once the index exists, you submit that one URL and nothing else. In Search Console, the Sitemaps report lists the index and expands to show each child with its own status, discovered URL count, and last read date. Google's Search Central documentation explains how to read those columns and what each error means.
You can also advertise the index in your robots.txt file with a single Sitemap: line pointing at the full URL. That lets any crawler find it without a manual submission, including search engines you have never registered with. If you also use Bing Webmaster Tools, submit the same index there, since it reads the identical format.
After submitting, give it a few days before judging the numbers. Discovery counts lag, and a freshly split sitemap often shows a dip before it settles. If a child sitemap shows an error, fix it, resubmit the index, and check the individual file in a browser to confirm it is valid and loads quickly.
A Short Checklist for Large Sites
Before you call a split sitemap finished, run through a few quick checks. They take minutes and catch nearly every common mistake.
- Every child sitemap is under 50,000 URLs and under 50 MB uncompressed.
- The index and the children are all reachable at their stated URLs with a 200 status.
- Each child only lists URLs it is allowed to list, on the right host and protocol.
- Lastmod values reflect real content changes, at both the page and the sitemap level.
- The index URL is submitted in Search Console and listed in robots.txt.
For background on how the format came about and how different engines adopted it, the Wikipedia article on sitemaps gives a concise history. It is also a useful reminder that this is a shared, open convention rather than something owned by any single search engine.
Photo by Larry Hyler on Pexels
Letting a Tool Handle the Boring Parts
None of this is conceptually difficult, but it is fiddly, and fiddly work is where small errors creep in. Counting entries, watching file sizes, escaping characters, and naming children consistently are all jobs a script does better than a person with a text editor.
The XML Sitemap Generator from EvvyTools takes a list of URLs, checks syntax, removes duplicates, and splits the output into an index plus child files automatically when you cross the limits. You still decide how the pieces should be grouped, but the formatting and escaping stop being your problem.
If you want to keep going on the technical side of search, the EvvyTools blog has more walkthroughs in the same practical spirit, and the tools directory lists every free utility on the site. The short version of everything above is this: stay under the limits, split by meaning, keep your dates honest, and let the index do the job it was designed for.
