# WASD Hypertext Services - Technical Overview

## 17 - Cache

> 17.1 - Cache Suitability Considerations
> 17.2 - Cache Content Validation
> 17.3 - Cache Configuration
> 17.4 - Cache Control
> 17.5 - Circumventing The Cache

[next] [previous] [contents] [full-page]

WASD HTTPd provides an optional, configurable, monitorable file data and revision time cache. File data, so that requests for documents can be fulfilled without reference to the underlying file system, potentially reducing request latency and more importantly improving overall server performance and system impact, and file revision time, so that requests specifying an "If-Modified-Since:" header can also benefit from the above.

Note that it is a file-system cache. Only documents generated from the file system are cached, not from any potentially dynamic sources, such as scripts, directory listings, SSI documents, etc. The reason should be obvious. Only the file system provides a reliable mechansim for ensuring the validity of the cached data (i.e. has the original changed in some way since loaded?)

Files are cached according to mapped path (not necessarily the same path supplied with the request) and not by the file name represented by any path. This is a design decision targeted at avoiding any access to RMS before searching the cache. For example, the ambiguous reference to the directory

      /ht_root/

may result in the following file being accessed (due to home page resolution)

      HT_ROOT:[000000]HOME.HTML

and the contents returned to the client and consequently cached. Each time the path "/ht_root/" is subsequently requested it will be path hit and serviced from the cache entry without any recourse to RMS.

Of course the same file may be requested with the unambigious path

      /ht_root/home.html

which is completely different to the first instance, although ultimately accessing the same file. Hence one file may be cached multiple times against distinct paths. Although isolated instances of this are possible, the likelihood of significant resources being consumed in practice should be quite low.

###
Why Implement Caching?

Caching, in concept, attempts to improve performance by keeping data in storage that is faster to access than it's usual location. The performance improvement can be assessed in three basic ways; reduction of

- response when accessing the data (latency and transfer time)
- processing involved (CPU cycles)
- impact on the usual storage location (file system I/O)

This cache is provided to address all three. Where networks are particularly responsive a reduction in request latency can often be noticable. It is also suggested a cache "hit" may consume less CPU cycles than the equivalent access to the (notoriously expensive) VMS file system. Where servers are particularly busy or where disk subsystems particularly loaded a reduction in the need to access the file system can significantly improve performance while simultaneously reducing the impact of the server on other system activities. The author's feeling is though, that for most VMS sites high levels of hits are not a great concern, and for these caching can easily be left disabled, particularly if the file system's virtual I/O cache is enabled.

A comparison between cached and non-cached performance is provided in 18 - Server Performance.

Why take so long to implement caching? (introduced in version 4.5) Well, WASD's intranet services are not particularly busy sites. This coupled with powerful hardware has meant performance has not been an overriding concern. However, this cache module came about because I felt like creating it and it was an obvious lack of functionality within the server, not because WASD (the organisation) needed it.

###
Terminology

This is what is meant when used.

- "hit" refers to a request path being found in cache. If the data is still valid the request can be supplied from cache.
- "load"ing the cache refers to reading the contents of a file into cache memory.
- "valid" means that the file from which the cached data was originally read has not had it's revision date changed (the implication being is the file contents have not changed.

###
17.1 - Cache Suitability Considerations

A cache is not always of benefit! It's cost may outweigh it's return.

Any cache's efficiencies can only occur where subsets of data are consistently being demanded. Although these subsets may change slowly over time a consistent and rapidly changing aggregate of requests lose the benefit of more readily accessable data to the overhead of cache management, due to the constant and continuous flushing and reloading of cache data. This server's cache is no different, it will only improve performance if the site experiences some consistency in the files requested. For sites that have only a small percentage of files being repeatedly requested it is probably better that the cache be disabled. The other major consideration is available system memory. On a system where memory demand is high there is little value in having cache memory sitting in page space, trading disk I/O and latency for paging I/O and latency. On memory-challenged systems cache is probably best disabled.

To help assessment of the cache's efficiency for any given site monitor the administration menu's cache report.

Two sets of data provide complementary information, cache activity and file request profile.

- Activity information summarizes the cache search behaviour, in particular that of the hash table.

  The "searched" item, indicates the number of times the cache has been searched. Most importantly, this may include paths that can never be cached because they represent non-file requests (e.g. directory listings). Requests involving scripts, and some others, never attempt a cache search.

  The "hit" item, indicates the number of times the hash table directly provided a cached path. This is very efficient.

  The "miss" item, indicates the number of times the hash table directly indicated a path was not cached. This is decisive and is also very efficient.

  The "collision" item, indicates the number of times multiple paths resolved to the same hash table entry. Collisions require further processing and are far less efficient. The sub-items, "collision hits" and "collision misses" indicate the number of times that further processing resulted in a found or not-found cache item.

  A large number of cache misses compared to searches may only indicate a large number of non-cacheable requests and so depending on that further datum is not of great concern. A large proportion of collisions (say greater than 12.5%) is however, indicating either the hash table size needs increasing (1024 should be considered a minimum) or the hashing algorithm in the software need reviewing :^)

- Files information summarizes the site's file request profile.

  With the "loads not hit" item, the count represents the cumulative number of files loaded but never subsequently hit. If this percentage is high it means most files loaded are never hit, indicating the site's request profile is possibly unsuitable for caching.

  The item "hits" respresents the cumulative, total number of hits against the cumulative, total number of loads. The percentage here can range from zero to many thousands of percent :^) with less than 100% indicating poor cache performance and from 200% upwards better and good performance. The items "1-9", "10-99" and "100+" show the count and percentage of total hits that occured when a given entry had experienced hits within that range (e.g. if an entry has had 8 previous hits, the ninth increments the "1-9" item whereas the tenth and eleventh increments the "10-99" item, etc.)

  Other considerations also apply when assessing the benefit of having a cache. For example, a high number and percentage of hits can be generated while the percentage of "loads not hit" could be in the also be very high. The explanation for this would be one or two frequently requested files being hit while most others are loaded, never hit, and flushed as other files request cache space. In situations such as this it is difficult to judge whether cache processing is improving performance or just adding overhead.

###
Recommendation

Monitor the site's cache behaviour and adjust parameters from an assessment based on the guidelines above.

Again, the author's suggestion is, that for most VMS sites, high levels of access are not a great concern, and for these caching can easily be left disabled.

###
17.2 - Cache Content Validation

The cache will automatically revalidate the file data after a specified number of seconds ([CacheValidateSeconds] configuration parameter), by comparing the original file revision time to the current revision time. If different the file contents have changed and the cache contents declared invalid. If found invalid the file transfer then continues outside of the cache with the new contents being concurrently reloaded into the cache.

Cache validation is also always performed if the request uses "Pragma: no-cache" (i.e. as with the Netscape Navigator Reload function). Hence there is no need for any explicit flushing of the cache under normal operation. If a document does not immediately reflect any changes made to it (i.e. validation time has not been reached) validation (and consequent reload) can be "forced" with a browser reload.

If a site's contents are relatively static the validation seconds could be set to an extended period (say 3600 seconds, one hour) and then rely on an explicit "reload" to force validation of a changed file.

The entire cache may be purged of cached data either from the server administration menu or using command line server control, as in the following example

      $ HTTPD /DO=CACHE=PURGE

###
17.3 - Cache Configuration

The cache is controlled using HTTPD$CONFIG file configuration directives. A number of parameters control the cache's behaviour. See the example configuration file HT_ROOT:[EXAMPLE]HTTPD$CONFIG.CONF.

- Cache enables and disables caching.

- CacheChunkKBytes sets the granularity of memory blocks allocated to file data (in kilobytes). To reduce the number of, and possible blocks sizes of memory, being requested (potentially improving memory allocation performance) data allocated to cache is done in multiples of this chunk size.

- CacheEntriesMax and CacheTotalKBytesMax provide growth limits to cache expansion. Maximum entries limits the number of files loaded into the cache before entries begin to be reused (flushing the original contents). Maximum total kilobytes allocated to the cache provides a ceiling on the memory consumed. These parameters operate to limit each other (i.e. if one reaches it's limit before the other, the other will not grow further either).

- CacheFileKBytesMax provides a limit on file size (in kilobytes). Files larger than the specified limit will never be cached.

- CacheFrequentHits and CacheFrequentSeconds attempt to reduce unproductive reuse of cache entries by providing the cache with some indication of what constitutes a frequently hit entry. If it is frequently hit then it should not be immediately reused when there is a demand for cache space. The first parameter sets the number of hits an entry must sustain before being a candidate for "CacheFrequentSeconds" assessment. If a file has been hit at least "CacheFrequentHits" times in total and the last hit was within the number of seconds set by "CacheFrequentSeconds" it will not be flushed and reused. If it has not been hit within the specified period it will be reused.

- CacheHashTableEntries sets the size of the hash table, used to rapidly index into the cached paths. Generally bigger is better (each entry consumes four bytes, hence 1024 entries consume 4096 bytes). Due to the hashing algorithm employed the size should be in magnitudes of two (e.g. 1024, 2048, etc.), if not, the specified amount is rounded down to the nearest (e.g. 1000 to 512).

- CacheValidateSeconds The interval after which a cache entry's original, content revision time is revalidated against the file's current revision time. If not the same the contents are declared invalid and reloaded. Setting this to a greater period reduces disk I/O but revised files may not be obvious within an acceptable timer unless a re-validation is forced with a reload.

###
17.4 - Cache Control

The cache may be enabled, disabled and purged from the server administration menu (see 15 - Server Administration). In addition the same control may be exercised from the command line (see 5.3.2 - Server Command Line Control) using

      $ HTTPD /DO=CACHE=ON
      $ HTTPD /DO=CACHE=OFF
      $ HTTPD /DO=CACHE=PURGE

If cache parameters are altered in the configuration file the server must be restarted to put these into effect. Disabling the cache on an ad hoc basis (from menu or command line) does not alter the contents in any way so it can merely be reenabled with use of the cache's previous contents resuming. In this way comparisions between the two environments may more easily be made.

###
17.5 - Circumventing The Cache

There are often good reasons for bypassing or avoiding the cache. For instance, where a document is being refreshed within the cache revalidation period specified by [CacheValidateSeconds] (see 17.2 - Cache Content Validation). There are two mechanisms available for bypassing or invalidating the file cache.

1. SET the path to the file or files as NOCACHE (see 10.3.5 - SET Rule). This directs the server to always get the file from the file-system.

         SET /path/not/to/cache/* NOCACHE

2. Specify a version component when requesting the file. WASD never caches a file if the request contains a version component. It does not need to be a full version number, a semi-colon is sufficient. For example:

         /ht_root/robots.txt;

[next] [previous] [contents] [full-page]