<COMMENT>(--------------------------------------------------------------------)

<CHAPTER>(Server Performance\chapter_server_perf)

<P> The server has a single-process, multi-threaded, asynchronous I/O design.
On a single-processor system this is the most efficient approach.  On a
multi-processor system it is limited by the single process context
(with scripts executing within their own context).  For I/O constrained
processing (the most common in general Web environments) the AST-driven
approach is quite efficient.

<P> The server has been tested with up to 60 concurrent requests originating
from 6 different systems and continues to provide an even distribution of data
flow to each client (albeit more slowly :^)  The author has also observed sites
where traffic levels frequently reach 50 to 100 concurrent requests both for
standard and proxy serving.

<P> The test system was a lightly-loaded AlphaServer 2100 4/275, VMS v7.2 and
DEC TCP/IP 5.0A. No <EMPHASIS>(Keep-Alive:) functionality was employed so each
request required a complete TCP/IP  connection and disposal.  DNS (name
resolution) and access logging were disabled.  The server and test-bench utility
were located on separate systems with 100 Mbps FDDI network interconnection.

<P> As of v7.1 the performance data is collected using the <QUOTE>(ApacheBench)
utility (see <REFERENCE>(head_apache_bench)).  Previous comparisons were
performed using the WASD WWWRKOUT utility but it was felt the use of a more
common tool would be the prefered option.  DCL procedures with sets of
ApacheBench calls are used to benchmark requests.  These procedures and the
generated output from benchmark runs (made available via
$@<EMPHASIS>(procedure)/OUTPUT=<EMPHASIS>(filename)) are available in the

<COMMENT>(HTML=
<A HREF="/ht_root/exercise/perf_*ab*.*?httpd=index&readme=no">
HT_ROOT:[EXERCISE]</A> directory.
)

<COMMENT>(HTML/OFF)
HT_ROOT:[EXERCISE] directory.
<COMMENT>(HTML/ON)
)

<NOTE>(These results are indicative only!)
On a clustered, multi-user system too many things vary slightly all the time. 
Hence the batching of accesses, interleaved between servers, attempting provide
a representative result.
<ENDNOTE>

<HEAD>(OSU/Apache Comparison\head_osu_comparison)

<P> Until v5.3 a direct comparison of performance between OSU and WASd had not
been made (even to satisfy the author's own occasional curiosity).  After a
number of users with experience in both environments commented <HELLIPSIS>
WASD <EMPHASIS>(seemed) faster, was it? <HELLIPSIS> it was decided to make and
provide comparisons using the same metrics used on WASD for some time.

<P> Every endeavour has been made to ensure the comparison is as equitable as
possible (e.g. each server executes at the same process priority, has a
suitable cache enabled, runs on the same machine in the same relatively
quiescent environmmnt.  Each test run was interleaved between
each server to try and distribute any environment variations. Tests showing a
port 7080 were to WASD, port 7777 to the OSU server, and port 8888 to Apache. 
All servers were configured <QUOTE>(out-of-the-box), minimal changes (generally
just path mappings), WASD executing via the FREEWARE_DEMO.COM procedure.

<P> Of course performance is just one of a number of considerations in any
software environment (otherwise we wouldn't be using VMS now would we? ;^)  No 
specific conclusions are promoted by the author.  Readers may draw their own
from the results recorded below.

<P> For this document the results were derived using the WASD v7.1.1, CSWS
V1.0-1 (VMS Apache 3.1.12), and OSU 3.9 servers.

<HEAD1>(Simple File Request Turn-Around\head_perf_file_turn)

<P> A series of tests using batches of accesses. The first test returned an
empty file measuring response and file access time, without any actual
transfer. The second requested a file of 64K characters, testing performance
with a more realistic load. All were done using one and ten concurrent
requests.  Note that the Apache measurement is <QUOTE>(out-of-the-box) - the
author could find no hint of a file cache, let-alone how to enable/disable one.

<P><CENTER_LINE>(<EMPHASIS>(Cache Disabled - Requests/Second\BOLD))

<TABLE>
<TABLE_SETUP>(5\10\10\10\10)
<TABLE_HEADS>(Response\Concurrent\WASD\OSU\Apache)
<COMMENT>(HTML=<TR></TR>)
<TABLE_ROW>(0K\1\86\31\18)
<TABLE_ROW>(0K\10\107\28\37)
<COMMENT>(HTML=<TR></TR>)
<TABLE_ROW>(64K\1\45\10\12)
<TABLE_ROW>(64K\10\50\14\36)
<ENDTABLE>

<P><CENTER_LINE>(<EMPHASIS>(Cache Enabled - Requests/Second\BOLD))

<TABLE>
<TABLE_SETUP>(5\10\10\10\10)
<TABLE_HEADS>(Response\Concurrent\WASD\OSU\Apache)
<COMMENT>(HTML=<TR></TR>)
<TABLE_ROW>(0K\1\285\44\15)
<TABLE_ROW>(0K\10\387\49\29)
<COMMENT>(HTML=<TR></TR>)
<TABLE_ROW>(64K\1\74\17\12)
<TABLE_ROW>(64K\10\103\26\39)
<ENDTABLE>

<P> Result files:

<LIST>(UNNUMBERED)

<LE>

<COMMENT>(HTML=
<A HREF="/ht_root/exercise/perf_files_nocache_ab_v711.txt">
HT_ROOT:[EXERCISE]PERF_FILES_NOCACHE_AB_V711.TXT</A>
)

<COMMENT>(HTML/OFF)
HT_ROOT:[EXERCISE]PERF_FILES_NOCACHE_AB_V711.TXT
<COMMENT>(HTML/ON)

<LE>

<COMMENT>(HTML=
<A HREF="/ht_root/exercise/perf_files_ab_v711.txt">
HT_ROOT:[EXERCISE]PERF_FILES_AB_V711.TXT</A>
)

<COMMENT>(HTML/OFF)
HT_ROOT:[EXERCISE]PERF_FILES_AB_V711.TXT
<COMMENT>(HTML/ON)

<ENDLIST>

<P> In comparison with v6.1 results the additional layer introduced into v7.0/1
for handling ODS-2 and ODS-5 file systems does not appear to have noticably
affected file access rates at all.  With both WASD cached and non-cached
<EMPHASIS>(throughput actually improves\BOLD) at ten concurrent requests
(probably due to the latency of the serial TCP/IP connection/disconnection in
one-by-one, compared to several happening concurrently).

<P> Note that the response and transfer benefits decline noticably with file
size (transfer time).  The difference between cached and non-cached with the
zero file size (no actual data transfer involved) gives some indication of the
raw difference in response latency, some 200-300% improvement. This is a fairly
crude analysis, but does give some indication of cache efficiencies.

<P> Just one other indicative metric of the two servers, CPU time consumed
during the file measurement runs.  The value for Apache was not measured as it
would be distributed over an indeterminate number of child subprocessess.

<P><CENTER_LINE>(<EMPHASIS>(CPU Time Consumed (Seconds)\BOLD))

<TABLE>
<TABLE_SETUP>(4\10\10\10)
<TABLE_HEADS>(Cache\WASD\OSU\Apache)
<COMMENT>(HTML=<TR></TR>)
<TABLE_ROW>(Disabled\22\136\-)
<TABLE_ROW>(Enabled\11\62\-)
<ENDTABLE>

<HEAD>(File Transfer Rate\head_perf_file_rate)

<P> Under similar conditions results indicate a potential transfer rate
<EMPHASIS>(well in excess of 1 Mbyte per second\BOLD).  This serves to
demonstrate that server architecture should not be the limiting factor in file
throughput.

<P><CENTER_LINE>(<EMPHASIS>(Transfer Rate - MBytes/Second\BOLD))

<TABLE>
<TABLE_SETUP>(5\20\10\10\10)
<TABLE_HEADS>(Response\Concurrent\WASD\OSU\Apache) 
<COMMENT>(HTML=<TR></TR>)
<TABLE_ROW>(4.0MB (7900 blocks)\1\1.5\0.5\1.5)
<TABLE_ROW>(4.0MB (7900 blocks)\10\1.6\0.4\2.2)
<ENDTABLE>

<P> Result files:

<LIST>(UNNUMBERED)

<LE>

<COMMENT>(HTML=
<A HREF="/ht_root/exercise/perf_xfer_ab_v71.txt">
HT_ROOT:[EXERCISE]PERF_XFER_AB_V711.TXT</A>
)

<COMMENT>(HTML/OFF)
HT_ROOT:[EXERCISE]PERF_XFER_AB_V711.TXT
<COMMENT>(HTML/ON)

<ENDLIST>

<P> Significantly, there were no dramatic drops in transfer rate between
one and ten concurrent requests!  In fact an small increase in throughput!  The
results for Apache indicate one occasion where a collection of subprocesses
performs very well (with assistance from generous VCC_<HELLIPSIS> cache
settings). 

<HEAD>(File Record Format\head_perf_record_format)

<P> The server can handle STREAM, STREAM_LF, STREAM_CR, FIXED and UNDEFINED
record formats very much more efficiently than VARIABLE or VFC files.

<P> With STREAM, FIXED and UNDEFINED files the assumption is that HTTP
carriage-control is within the file itself (i.e. at least the newline (LF),
all that is required required by browsers), and does not require additional
processing.  With VARIABLE record files the carriage-control is implied and
therefore each record requires additional processing by the server to supply
it.  Even with variable record files having multiple records buffered by the
HTTPd before writing them collectively to the network improving efficiency,
stream and binary file reads are by Virtual Block and are written to the
network immediately making the transfer of these very efficient indeed!

<P> So significant is this efficiency improvement a module exists to
automatically convert VARIABLE record files to STREAM-LF when detected by the
file transfer module.  This is disabled by default but the user is strongly
encouraged to enable it and to ensure that stream format files are provided
to the server by other hypertext generating and processing utilitites.

<HEAD1>(Scripting\head_perf_scripting)

<P> Persistant-subprocesses are probably the most efficient solution for
child-process scripting under VMS. See <QUOTE>(Scripting Environment) document. 
The I/O still needs to be on-served to the client by the server.

<P> A simple performance evaluation shows the relative merits of the four
WASD scripting environments available, plus a comparison with OSU and Apache.

<COMMENT>(HTML=
<A HREF="/ht_root/src/cgiplus/cgiplustest.c">
HT_ROOT:[SRC.CGIPLUS]CGIPLUSTEST.C</A>,
)

<COMMENT>(HTML/OFF)
HT_ROOT:[SRC.CGIPLUS]CGIPLUSTEST.C,
<COMMENT>(HTML/ON)

which executes in both standard CGI and CGIplus environments, and an ISAPI
example DLL,

<COMMENT>(HTML=
<A HREF="/ht_root/src/cgiplus/isapiexample.c">
HT_ROOT:[SRC.CGIPLUS]ISAPIEXAMPLE.C</A>,
)

<COMMENT>(HTML/OFF)
HT_ROOT:[SRC.CGIPLUS]ISAPIEXAMPLE.C,
<COMMENT>(HTML/ON)

which provides equivalent output.

A series of accesses were made. The first test
returned only the HTTP header, evaluating raw request turn-around time. The
second test requested a body of 64K characters, again testing performance with
a more realistic load.

<P>
<EMPHASIS>(DECnet-based scripting\BOLD)
was tested using essentially the same environment as subprocess-based CGI,
assessing the performance of the same script being executed using DECnet to
manage the processes.  Three separate environments have been evaluated,
WASD-DECnet-CGI, WASD-OSU-emulation and OSU. The OSU script used the
WASD CGISYM.C utility to generate the required CGI symbols (also see
<REFERENCE>(head_osu_face2face)).  DECnet Phase-IV was in use.

<P><CENTER_LINE>(<EMPHASIS>(CGI Scripting - Requests/Second\BOLD))

<TABLE>
<TABLE_SETUP>(9\7\7\7\7\7\7\7\7)
<TABLE_HEADS>(Response\Concurrent\CGI\CGIplus\ISAPI\DECnet-CGI\OSU-emul\OSU\Apache)
<COMMENT>(HTML=<TR></TR>)
<TABLE_ROW>(0KB\1\12.3\68.7\67.8\7.7\4.4\4.3\0.8)
<TABLE_ROW>(0KB\10\19.2\80.4\71.6\11.5\7.8\7.7\0.9)
<COMMENT>(HTML=<TR></TR>)
<TABLE_ROW>(64KB\1\7.8\31.8\32.1\6.0\3.5\2.9\0.7)
<TABLE_ROW>(64KB\10\13.7\34.6\33.8\8.9\6.4\5.9\0.9)
<ENDTABLE>

<P> Result files:

<LIST>(UNNUMBERED)

<LE>

<COMMENT>(HTML=
<A HREF="/ht_root/exercise/perf_scripts_ab_v711.txt">
HT_ROOT:[EXERCISE]PERF_SCRIPTS_AB_V711.TXT</A>
)

<COMMENT>(HTML/OFF)
HT_ROOT:[EXERCISE]PERF_SCRIPTS_AB_V711.TXT
<COMMENT>(HTML/ON)

<ENDLIST>

<HEAD>(Scripting Observations\head_perf_observe) 

<P> Although these results are indicative only, they do show CGIplus and ISAPI
to have a potential for improvement over standard CGI of up to 400%, a not
inconsiderable improvement. Of course this test generates the output stream
very simply and efficiently and so excludes any actual processing time that may
be required by a <QUOTE>(real) application.  <EMPHASIS>(If the
script/application has a large activation time the reduction in response
latency could be even more significant\BOLD) (e.g. Perl scripts and RDMS access
languages).

<HEAD>(DECnet Observations\head_perf_decnet_observe) 

<P> This section comments on non-persistant scripts (i.e. those that must
run-up and run-down with each request - general CGI behaviour). Although not
shown here measurements of connection reuse show significant benefits in
reduced response times, consistency of response times and overall throughput, 
showing a difference of some 200% over non-reuse (similar improvements were
reported with the OSU 3.3a server).

<P> With ten simultaneous and back-to-back scripts and no connection reuse many
more network processes are generated than just ten. This is due to the
NETSERVER maintenance tasks such as log creation and purging, activating and
deactivating the task, etc., adding latency into this script environment. The
throughput was generally still lower than with subprocess-based scripting.

<P> While earlier versions cautioned on the use of DECnet-based scripting this
has been relaxed somewhat through connection reuse.

<HEAD>(WASD/OSU Comparison\head_osu_face2face) 

<P> A direct comparison of CGI performance between WASD and OSU scripting is
biased in favour of WASD, as OSU scripting is based on it's own protocol with
CGI behaviour layered-in above scripts that require it. Therefore a non-CGI
comparison was devised.  The script,

<COMMENT>(HTML=
<A HREF="/ht_root/script/face2face.com">
HT_ROOT:[SCRIPT]FACE2FACE.COM</A>,
)

<COMMENT>(HTML/OFF)
HT_ROOT:[SCRIPT]FACE2FACE.COM,
<COMMENT>(HTML/ON)

is designed to favour neither environment, merely return the plain-text string
<QUOTE>(Hello!) as quickly as possible.

<CODE_EXAMPLE><LITERAL>
  $! OSU and WASD scripting face-to-face in a script that favours neither unduly
  $ if f$type(WWWEXEC_RUNDOWN_STRING) .nes. ""
  $ then
  $    write net_link "<DNETTEXT>"
  $    write net_link "200 Success"
  $    write net_link "Hello!"
  $    write net_link "</DNETTEXT>"
  $ else
  $    write sys$output "Content-Type: text/plain"
  $    write sys$output ""
  $    write sys$output "Hello!"
  $ endif
<ENDLITERAL><ENDCODE_EXAMPLE>

<P><CENTER_LINE>(<EMPHASIS>(Face-to-Face - Requests/Second\BOLD))

<TABLE>
<TABLE_SETUP>(9\7\7\7\7\7\7\7\7)
<TABLE_HEADS>(\Concurrent\CGI\CGIplus\ISAPI\DECnet-CGI\OSU-emul\OSU\Apache)
<COMMENT>(HTML=<TR></TR>)
<TABLE_ROW>(<QUOTE>(Hello!)\1\17.5\n/a\n/a\n/a\n/a\12.6\n/a)
<TABLE_ROW>(<QUOTE>(Hello!)\10\39.5\n/a\n/a\n/a\n/a\19.9\n/a)
<ENDTABLE>

<P> Result files are the same as for scripting metrics above.

<HEAD1>(SSL\head_perf_ssl)

<P> At this time there are no definitive measurements of SSL performance (see
<REFERENCE>(chapter_ssl)).  One might expect that because of the
CPU-intensive cryptography employed in SSL requests that performance,
particularly where concurrent requests are in progress, would be significantly
lower.  In practice SSL seems to provide more-than-acceptable responsiveness.

<HEAD1>(Suggestions\head_perf_suggestions)

<P> Here are some suggestions for improving the performance of the server,
listed in approximate order of significance. Note that these will have
proportionally less impact on an otherwise heavily loaded system.

<LIST>(NUMBERED)

<LE> Disable host name resolution (configuration parameter [DNSLookup]).
<EMPHASIS>(DNS latency can slow request processing significantly!\BOLD) 
Most log analysis tools can convert numeric addresses so DNS resolution is
often an unnecessary burden.

<COMMENT>(HTML=<P>)

<LE> Ensure served files are not VARIABLE record format (see above). Enable
STREAM-LF conversion using a value such as 250 (configuration parameter
[StreamLF], and SET against required paths using mapping rules).

<COMMENT>(HTML=<P>)

<LE> Use persistant-subprocess DCL/scripting (configuration parameter
[ZombieLifeTime])

<COMMENT>(HTML=<P>)

<LE> Use CGIplus-capable or ISAPI scripts whenever possible.

<COMMENT>(HTML=<P>)

<LE> Enable caching (configuration parameter [Cache]).

<COMMENT>(HTML=<P>)

<LE> Ensure the server account's WSQUO and WSEXTENT quotas are adequate.  A
constantly paging server is a slow server!

<COMMENT>(HTML=<P>)

<LE> Tune the network and DCL output buffer size to the Maximum Transfer Unit
(MTU) of the server's network interface.  Using Digital TCP/IP Services (a.k.a.
UCX) display the MTU.

<CODE_EXAMPLE>
  TCPIP> SHOW INTERFACE
                                                             Packets
  Interface   IP_Addr         Network mask          Receive          Send     MTU

   SE0        203.127.158.3   255.255.255.0          376960        704345    1500
   LO0        127.0.0.1       255.0.0.0                 306           306       0
<ENDCODE_EXAMPLE>

<P> In this example the MTU of the ethernet interface is 1500 (bytes).  Set the
[BufferSizeNetWrite] configuration directive to be some multiple of this.  In
the case of 1500, say 3000, 4500 or 6000.  Also set the [BufferSizeDclOutput]
to the same value.  Rationale: always use completely filled network packets
when transmitting data.

<COMMENT>(HTML=<P>)

<LE> Disable logging (configuration parameter [Logging]).

<COMMENT>(HTML=<P>)

<LE> Set the HTTP server process priority higher, say to 6 (use startup
qualifier /PRIORITY=).  Do this after due consideration.  It will only improve
response time if the system is also used for other, lower priority purposes. 
It will not help if Web-serving is the sole acitivity of the system.

<COMMENT>(HTML=<P>)

<LE> Reduce to as few as possible the number of mapping and authorization
rules, particularly those that have conditions that require additional
evaluation.  Also see <REFERENCE>(chapter_mapping_rule).

<COMMENT>(HTML=<P>)

<LE> Use a pre-defined log format (e.g. <QUOTE>(common), configuration
parameter [LogFormat]). User-specified formats require more processing for each
enrty.

<COMMENT>(HTML=<P>)

<LE> Disable request history (configuration parameter [RequestHistory]).

<COMMENT>(HTML=<P>)

<LE> Disable activity statistics (configuration parameter [ActivityDays]).

<ENDLIST>

<COMMENT>(--------------------------------------------------------------------)
