Build My Site for Free

Matt Cutts: We always try to maximize the relevancy and accuracy of our index. And you’d want to make a distinction between crawling and indexing because sitemap submission does not guarantee that we will crawl the URLs on that list.

It is helpful to help us discover new URLs or to make canonicalization decisions, but we don’t guarantee you that if you submit a sitemap we’ll go and crawl it.

There have been some people who did some experiments where they saw that happened, but I’m not going to confirm or deny that, and policy can always change on exactly how we do use sitemaps submissions.

But crawling and indexing is different, so if you do a ping a lot of the times Google will come and crawl you but often it’s Google blog search because if you’re doing those Word Press or Web logs or feed burner pings, those pings are often the sort of things that are blogs, and so a blog search might come and crawl you five minutes later.

But then if you show up, you might show up in the blog search core post, not in our main web index core post. So just because you get crawled, it doesn’t mean that you are getting some type of index boost or anything like that.

We do sort of try to rationally decide what’s the best quality of data, how do we get that; sometimes it’s crawling stuff immediately. Like with blog search you have a very fast, very real time sort of results. And sometimes it’s taking sitemaps and then that might result in crawling at a different pace or you might not give any boost at all. But we do use that information in lots of ways to try to help us improve canonicalization and help us try to improve the quality of our index.

So, you know I wouldn’t say, “Oh ping, that’s the way to automatically get crawled,” or anything like that. If you make great content, you get to be well-known, we will probably crawl you relatively frequently and see updated content anytime you make a good change.