January 9, 2001 - A severe underestimation of expected traffic levels crippled a highly anticipated British government Web site only hours after it had officially launched.
On January 2, Britain's Public Record Office (PRO) unveiled a Web site (census.pro.gov.uk) featuring the complete results of the government's 1901 census, only to have the site immediately come under heavy pressure due to an overwhelming amount of traffic.
After numerous failed attempts to handle the traffic load, the site was taken offline until it could be equipped with the hardware and connectivity required to cope with traffic demands.
Britain first decided to publish the results of its 1901 census online in 1998 - a task that would require the government to digitize 32 million names from about 1.5 million pages of data. Realizing it was a far too difficult task to handle alone, the PRO awarded a 10 year, £7m contract to QinetiQ, a government-owned IT firm, in 1998 to help digitize the census and develop a corresponding Web site. The resulting site allowed users to search the census by name, address, institution or place and view the digitized document for a small fee. Results were pulled from an Oracle database that required more than 100GB of disk space and that worked in tandem with an image library featuring more than 400GB of pictures.
For the site's hosting and connectivity requirements, QinetiQ turned to British Telecom. Designed to handle up to 1.2 million hits per day, the site reportedly attracted 1.2 million hits an hour on a regular basis while it was online. The site received 30 million hit requests on opening day alone as a steady barrage of traffic flooded the site. When visitor traffic from one geographic region began to subside, traffic from another area would increase, resulting in a steady and overwhelming stream of visitors.
BT currently operates a global network featuring 54,000 route kilometers of fiber that connect 275 cities, 20 countries and 200 access nodes.
When problems initially arose, the PRO tried various measures to keep the site up and running, including adding servers, optimizing images to reduce bandwidth consumption, and limiting the number of users allowed on the site at one time. The site was finally taken down earlier this week.
An updated statement on the census Web site says its "widely reported" difficulties were due to "overwhelming public interest and demand", which "exceeded our expectations by a factor of 20."
The site also called for assistance from visitors to get the site up and running again. "Continual attempts to access the site by using the browser 'Refresh' button only compounds our problems and we would be grateful if you could avoid doing this," the site reads. "We are grateful for the patience you have already shown, as we know that many of you have felt frustrated." In the meantime, users were encouraged to continue their research by using records available at libraries across the country.
The site is expected to re-launch early next week. While demand for the site's information will subside eventually, the site will likely be re-launched with significantly more servers and network capacity in place.
