SEO History

Webmasters and content providers first began optimizing sites for search engines in the mid-1990s. At the time, all webmasters needed to do was submit the address of a page or URL to the various engines, which would then send a "spider" to "crawl" that page, downloading and storing it on the search engine's own server. There, a second program known as an indexer would extract the information on the page, including the words it contains, the location of these words, the weight of specific words, and all links to other pages. This information is placed into a scheduler for crawling at a later date.

Early versions of search algorithms relied on webmaster-provided information such as the keyword meta tag, which provides a guide to each page's content. However, using meta data to index pages was found to be less than reliable because the webmaster's choice of keywords could be an inaccurate representation of the site's actual content. Incomplete data in meta tags could cause pages to rank for irrelevant searches. Web content providers could also manipulate a number of attributes within the HTML source of a page in order to rank well in search engines.

By relying so much on factors such as keyword density, which were within a webmaster's exclusive control, early search engines suffered from abuse and ranking manipulation. Since the popularity of a search engine is determined by its ability to produce the most relevant search results, search engines had to adapt by developing more complex ranking algorithms, taking into account additional factors that were more difficult to manipulate.

By 2004, leading search engines like Google and Yahoo had incorporated a wide range of undisclosed factors in their ranking algorithms to reduce the impact of link manipulation.

Real-time search was introduced in late 2009 in an attempt to make search results more timely and relevant. Historically, site administrators spend months or even years optimizing a website to increase search rankings. With the growth of social media sites and blogs, the leading engines had to make changes to their algorithms to allow fresh content to rank quickly within the search results.