Download Learn Programming with

CHAPTER 5
The Web Bot
As anyone who has spent any time at all online can tell you, there is a lot of information available on the Internet.
According to Google’s indexes, as of 2013 there are 4.04 billion web pages in existence. (See Figure 5-1.) Sure, a lot
of those pages are probably cat pictures and pornography, but there are also hundreds of millions of pages with
information on them. Useful information. It has been said that every piece of information that has been digitized
exists somewhere on the Internet. It just has to be found—not an easy task when the Internet looks something
like Figure 5-1.
Figure 5-1. Visual map of the Internet (© 2013 http://internet-map.net, Ruslan Enikeev)
67