Search Engine Forums, SEO Forum

All major search engines use spider programs to scour the web, collect documents, give each a unique reference, scan their text, and hand them off to an indexing program. Where the scan picks up hyperlinks to other documents, those documents are then fetched in their turn. Google’s spider is called Googlebot and you can see it hitting your site if you look at your web logs. A typical Googlebot entry (in the browser section of your logs) might look like this: Mozilla/5.0 (compatible; Googlebot/2.1; http://www.google.com/bot.html)

google index page or site through crawler and you wanna check with websmaster tool

they use google bots where they crawl websites over the web.

And there are many kinds of bots that do different things, only the best thing we can do is to represent the contents of our site clearly to these bots..

leftybogs wrote:logs are one way to tell if your site crawled by them.. its a computer program that scan sites then pass it trough their database that collects the results and show it on their SERP.

The exact process of crawling a web page didn't exactly presented by Google itself. If we know their method then it will be a lot easier for us to know what do to when we want to get index easily but those are just a small yet incomplete algorithms that we observe.

Right,it's the common way to check if google bots have visited your pages.

That's why we need to update our website regularly or perform the SEO in a right process. And Google would love to crawl our site.

It is the common way for webmasters to check web LOG to find the feet of lovely googlebot.

Check Google webmaster tools to see it also you can check your Awstatus to see when and where Google bot has visited your pages

Ya Google webmaster tool show the recent crawling of the site...