Ever wondered why Google returned search results that lead to sites that require a registration? How did Google index the site without a registration? Many sites want their site indexed in Google to receive more hits, so they allow Googlebots in. Because of this reason we can take advantage of this. All we have to do is disguise ourself as a Googlebot and many sites will let us in without registration.
To do this we have to change the User Agent. The user agent defines the browser and what version you are using. For example if you go to this site it will tell you what browser you are using, and it finds this info out by the user agent.
Changing the user agent is easy, to do this in Opera you have to uses these settings:
User Agent: Googlebot/2.1 Compatible: http://www.googlebot.com/bot.html
Firefox users can use this extension and import this file (right click and save) that has a huge list of user agent settings including Googlebot/2.1 and many other spiders/bots.
Thursday, October 2, 2008
Subscribe to:
Post Comments (Atom)
Posts
0 comments:
Post a Comment