December 5, 2002

Snowday and lessons on not caching and using static pages

It is a personal snowday as the U.S. Federal Government is open (so far), but on unscheduled leave. Oddly enough the OPM (Office of Personnel Management) status site does not have a no-cache setting in the header. This means you browsers, even your modern browser, will cache the page and require you the user to manually refresh the page. These pages may even be cached locally by ISPs and connectivity providers, meaning the information is not up to date. The page is rather light, with few graphics, but it is not a static page, meaning each page is generated dynamically, which requires more horsepower to serve the page. One days like today every Washington, DC area federal employee and contractor is hitting this page, as well as many DC area private sector employees (most DC companies are open if the Federal Government is open).

I thought most IT folks learned their lesson with the need for "dynamic" pages a year or two ago, but I guess not. Many previously dynamic pages are not dynamic on the back end with content management systems doing the same work they always have done, but building static pages. If the content on the pages is not changing often (this is a subjective term and can roughly defined as content changing ever few hours with heavy site loads are usually candidates for static pages) or are not serving parsable datapoints (tables of data that can have subsets of data selected for viewing). Much of the "dynamic page" hype was generated by marketing folks to non-technical types who made decisions on cool or manly mindsets. Many folks started actually thinking about the need for dynamic pages a couple years ago. Those that decided they did not need dynamic pages for all or even most of their site began to realize they could save on the heavier hardware needed to churn out everypage. These static output folks also found they could withstand much higher hit rates with ease. I have been in meetings with folks that were asking about stress testing site that were static running on boxes that were formerly running dynamic sites. Those of us that understood the processor savings knew this was a foreign concept and they did not understand the server would be able to handle twice to eight times the load previously handled.



Posted Comments

I like the idea of the server caching page data in memory and killing the cache periodically. This is an option in PHP no?

Caching dynamic content is common with many of the scripting languages and for many databases also. PHP caches pages easily, but the benefit is not too broad (breadth of pages that can be cached for large site is only a small portion). It was the caching of popular pages that started folks thinking about creating static pages for much of their content and archiving the rarely accessed information in dymanically retrievable systems. All the information is stored in the content system anyway and there are many ways to skin this cat. Given the resources needed the static pages offer a wider breadth pages and using fewer resources (usually RAM).

There are many variables to consider besides how often content updates prior to moving to the world of static page builds. The size of the overal site is usually one to consider, as the larger the site the more that needs to globally be updated. Having dynamic elements on the pages, like links to new inforation (which can be brought it with a server side include). The list can be long, but it can be a good exercise to help think through the information structure of a site, which at times can grow ad hoc.

Web Mentions

This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License.