Matt Cutts: I believe the answer is yes. I believe we have support to crawl FTP but really that’s pretty far off the beaten path. So if at all possible I would try to make sure that your pages are HTTP or HTTPS.
We did build a crawler way back in the day with the support for multiple protocols so the ability to crawl everything from WML and WAP and stuff
Continue Reading »
Matt Cutts: There is a team of people who are working on being able to crawl and index AJAX and index JavaScript and parse and execute JavaScript as well as other types of rich content. The trend in 2011 is going to be the same as it was in 2010, which is improving our ability to understand JavaScript, improving our ability to index AJAX, improving our ability to index rich
Continue Reading »
Matt Cutts: We recently published a paper in VLDB, which I believe stands for Very Large Data Bases, that talks about our criteria all the way. We try to do it safely so that people don’t want their forms to be crawled, we won’t crawl them.
And so there are some very simple things you can do. Rather than having text that has to be filled out like a zip code,
Continue Reading »