<COMMENT>(--------------------------------------------------------------------)

<FRONT_MATTER>
<TITLE_PAGE>

<TITLE>(HFRD Hypertext Services\- Technical Overview)

<LINE>

<HEAD>(1st December 1995\head_date)

<P><EMPHASIS>((with minor revisions for freeware release))

<P> Supercedes: 9th August 1995, 1st May 1995

<HEAD>(Abstract\head_abstract)

<P> This document is a technical overview of the implementation of the HFRD 
VMS HyperText Transport Daemon.  It contains information on server 
configuration and CGI scripting, as well as brief descriptions of the major 
code modules of the server. 

<P> It covers the environment supported by the version 3 release of the HFRD 
VMS Hypertext Services HTTP server (December 1995).

<P> Also see <QUOTE>(HFRD Hypertext Environment) for information on using the 
HFRD VMS Hypertext Services.

<COMMENT>(HTML/OFF)
<P> It is strongly suggested those using printed versions of this document 
also access the Hypertext version.  It provides online access to some 
examples, etc.
<COMMENT>(HTML/ON) 

<HEAD>(Author\head_author)

<P> Mark G. Daniel
<LINE> Senior Information Technologist

<P> Mark.Daniel@dsto.defence.gov.au

<P> +61 (8) 2596031 (bus) 
<LINE> +61 (8) 2596673 (fax)

<P> Mark G. Daniel
<LINE> High Frequency Radar Division
<LINE> Defence Science and Technology Organisation
<LINE> PO Box 1500
<LINE> Salisbury
<LINE> South Australia 5108

<HEAD>(Printed Copy\head_printed_copy)

<COMMENT>(....................................................................)

<COMMENT>(HTML=
<P> For Hypertext browsers a PostScript version of this document is available
 <A HREF="/ht_root/doc/htd/htd.ps">viewing</A> 
 and for
 <A HREF="/print/ht_root/doc/htd/htd.ps">printing</A>.
)

<COMMENT>(HTML/OFF)
<P> This book is available for printing to a PostScript printer.  Use a 
hypertext browser to access a print menu in this same location.
<COMMENT>(HTML/ON) 

<P> Some of the online demonstrations may not work due to the local 
organisation of the hypertext environment differing from HFRD where it was 
originally written. 

<COMMENT>(....................................................................)

<ENDTITLE_PAGE>

<COMMENT>(--------------------------------------------------------------------)

<COPYRIGHT_PAGE>

<P> <EMPHASIS>(HFRD VMS Hypertext Services, Copyright (C) 1996 Mark G. 
Daniel.\BOLD) 

<P> This package is free software; you can redistribute it and/or modify it 
under the terms of the GNU General Public License as published by the Free 
Software Foundation; version 2 of the License, or any later version. 

<P> This package is distributed in the hope that it will be useful, but 
WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or 
FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more 
details. 

<P> You should have received a copy of the GNU General Public License along 
with this package; if not, write to the Free Software Foundation, Inc., 675 
Mass Ave, Cambridge, MA 02139, USA. 

<ENDCOPYRIGHT_PAGE>

<CONTENTS_FILE>

<ENDFRONT_MATTER>

<COMMENT>(--------------------------------------------------------------------)

<CHAPTER>(Introduction\chapter_introduction)

<P> This document provides an technical overview of the HFRD VMS HyperText 
Transport Daemon environment.  It is primarily written for use internal to 
HFRD and assumes that perpsective without apology.  Any additional usage is 
subordinate to its role within HFRD.  The software has been written expressly 
for supporting an intra-organisational hypertext environment on a VMS 
platform.  It too, is unreservedly tailored to this purpose and the 
requirements of HFRD.  All programs were designed only to specifically comply 
with the requirements of VAX C and DEC C, within a DEC TCP/IP Services for VMS 
environment. 

<P> The document <EMPHASIS>(assumes\BOLD) a basic understanding of the 
hypertext technologies and uses terms without explaining them (e.g. HTTP, 
HTML, URL, CGI, etc.)   The reader is refered to documents specifically on 
these topics (these are often best consulted on-line, on the Internet WWW). 

<P> Also see <QUOTE>(HFRD Hypertext Environment) for information on using the 
HFRD VMS hypertext facilities.

<COMMENT>(HTML/OFF)
<P> It is strongly suggested those using printed versions of this document 
also access the Hypertext version.  It provides online demonstrations of some 
concepts.
<COMMENT>(HTML/ON) 

<HEAD>(Reasons For a Local HTTPd\head_local_httpd)

<P> Reasons for developing a local HTTPd server are few but compelling:

<LIST>(UNNUMBERED)

<LE> It was prefered to support the hypertext environment on a VMS platform.  
This is currently the most widely used and accessable environment within HFRD. 

<LE> Existing servers (and there are quite a few variations) are largely Unix 
based, although it is being supported (to a greater or lesses extent) across a 
wide range of platforms.  Ports to VMS, if they exist, are often in progress 
or half-baked, employing <EMPHASIS>(Unix)isms that don't translate elegantly 
to the VMS environment. 

<LE> The VMS version of the CERN server (3.0-6) was evaluated during mid-1994: 

<LIST>(UNNUMBERED\-)

<LE> It is not multi-threaded under VMS (i.e. cannot support concurrent 
clients).  For example, a lengthy search may delay other clients for 
unacceptable periods. 

<LE> Its performance was good with document transfers, but became poor when 
running a <EMPHASIS>(script). 

<LE> It is acknowleged in the release notes that it cannot handle a client 
cancelling a data transfer (a not-uncommon action).  This was confirmed 
experimentally.

<ENDLIST>

<CP> Future versions of this server should be evaluated periodically as it is 
receiving a good deal of support and attention. 

<LE> HyperText Transport Protocol, in its current form, is relatively simple 
to implement to the level required to support intra-Divisional requirements. 

<LE> <EMPHASIS>(As of December 1995\BOLD) the HFRD HTTPd has worked extremely 
well and has a number of facilities tailored for the VMS environment.  It can 
continue to be utilized until there are overwhelming reasons for implementing 
something else.

<ENDLIST>

<COMMENT>(--------------------------------------------------------------------)

<CHAPTER>(HyperText Transport Protocol Daemon - Overview\chapter_httpd)

<P> The most fundamental component of the HFRD VMS Hypertext Services 
environment is the HTTPd, or HyperText Protocol Transport Daemon, or HTTP 
server.  It provides full multi-threaded support.  VMS ASTs (Asynchronous 
System Traps) are used to construct an I/O event-driven server. 

<P> It provides a complete implementation of a basic HTTP server, including: 

<LIST>(UNNUMBERED)

<LE> concurrent, multi-threaded client support

<LE> <QUOTE>(GET), <QUOTE>(HEAD), and <QUOTE>(POST) (albeit, limited by the
lack of authentication) HTTP method support

<LE> versatile directory listing <EMPHASIS>((generic and VMS-style))

<LE> CGI-compliant scripting
<EMPHASIS>((with configurable, automatic, MIME content-type initiated 
activation))

<LE> HTML pre-processing <EMPHASIS>((server-side includes))

<LE> clickable-image support <EMPHASIS>((NCSA and CERN formats))

<LE> "If-Modified-Since:"/"304 Not Modified" functionality 
<EMPHASIS>((document is only sent if modified since time specified by client)) 

<LE> Web-standard, <QUOTE>(common)-format access log (allowing processing by 
most log-analysis tools)

<LE> host-level access controlled on per-host or per-domain acceptance or 
rejection 

<ENDLIST> 

<P> It executes permanently on the server host, listening for client 
connection requests on TCP/IP port 80 (by default).  It provides concurrent 
services for a (technically) unlimitted number of clients (constrained only by 
the server resources).  When a client connects HTTPd performs the following 
functions: 

<LIST>(NUMBERED)

<LE> creates a <EMPHASIS>(thread) for this request

<LE> reads and analyzes the HTTP request sent,
<LINE> depending on the nature of the request <HELLIPSIS>

<LIST>(UNNUMBERED)

<LE> initiates I/O-driven transfer of the requested file

<LE> initiates I/O-driven interpretation of a menu file

<LE> initiates I/O-driven processing of a pre-processable HTML file

<LE> initiates I/O-driven processing of a clickable-image mapping file

<LE> initiates I/O-driven directory listing

<LE> spawns a subprocess to execute a script (DCL procedure) with:

<LIST>(UNNUMBERED\-)

<LE> <EMPHASIS>(SYS$INPUT\SMALLCAPS) and <EMPHASIS>(SYS$OUTPUT\SMALLCAPS) 
assigned to intermediate <EMPHASIS>(mailboxes) (essentially <EMPHASIS>(pipes)) 

<LE> <EMPHASIS>(HTTP$INPUT\SMALLCAPS) logical name providing a 
<EMPHASIS>(mailbox) allowing the script to read the raw HTTP data stream from 
the client 

<LE> CGI-compliant symbols representing the important CGI variables of the 
request

<LE> for the life of the subprocess HTTPd:

<LIST>(UNNUMBERED)

<LE> controls the essential behaviour of the subprocess via its 
<EMPHASIS>(SYS$INPUT\SMALLCAPS) mailbox

<LE> receives data written by the subprocess to its 
<EMPHASIS>(SYS$OUTPUT\SMALLCAPS) via the associated mailbox, writing this to 
the client

<LE> receives data sent by the client, writing this to the mailbox associated 
with the subprocess' <EMPHASIS>(HTTP$INPUT\SMALLCAPS)

<ENDLIST> 

<ENDLIST> 

<ENDLIST> 

<LE> closes the connection to the client and disposes of the thread data 
structures 

<ENDLIST>

<P> For I/O intensive activities like file transfer and directory listing, the 
AST-driven code provides an efficient, multi-threaded environment for the 
concurrent serving of multiple clients. 

<P> For scripts, the technique of using multi-threaded, concurrent, spawned 
subprocesses, attached to standard input/output streams, provides a versatile, 
extensible, powerful scripting environment.  Any DCL procedure or image 
executing within the subprocess can behave as an HTTP server.  This capability 
is employed to easily extend the basic services provided by the core daemon 
code.  An HTTP script/server for this environment does not need to concern 
itself with network activities, it merely reads and writes from the standard 
I/O streams. 

<COMMENT>(--------------------------------------------------------------------)

<CHAPTER>(HTTPd Server Account and Environment\chapter_server_account)

<P> The HTTPd server account should be a standard account, preferably in a
group of its own (definitely at least a non-system, non-user group), with 
sufficient quotas to handle the expected traffic. 

<HEAD>(VMS Account\head_server_account)

<P> The following provides a guide to the account:

<CODE_EXAMPLE>
  Username: HTTP$SERVER                      Owner:  HyperText Daemon
  Account:  HTTPD                            UIC:    [377,377] ([HTTPD,HTTP$SERVER])
  CLI:      DCL                              Tables: DCLTABLES
  Default:  HT_ROOT:[HTTP$SERVER]
  LGICMD:   LOGIN
  Flags:  Restricted DisNewMail
  Primary days:   Mon Tue Wed Thu Fri
  Secondary days:                     Sat Sun
  Primary   000000000011111111112222  Secondary 000000000011111111112222
  Day Hours 012345678901234567890123  Day Hours 012345678901234567890123
  Network:  ##### Full access ######            ##### Full access ######
  Batch:    ##### Full access ######            ##### Full access ######
  Local:    -----  No access  ------            -----  No access  ------
  Dialup:   -----  No access  ------            -----  No access  ------
  Remote:   -----  No access  ------            -----  No access  ------
  Expiration:            (none)    Pwdminimum:  6   Login Fails:     0
  Pwdlifetime:         90 00:00    Pwdchange:      (pre-expired)
  Last Login:            (none) (interactive), 11-MAY-1995 08:44 (non-interactive)
  Maxjobs:         0  Fillm:       300  Bytlm:       300000
  Maxacctjobs:     0  Shrfillm:      0  Pbytlm:           0
  Maxdetach:       0  BIOlm:       512  JTquota:       1024
  Prclm:         100  DIOlm:       512  WSdef:         1000
  Prio:            4  ASTlm:       600  WSquo:         2000
  Queprio:         0  TQElm:       100  WSextent:     20000
  CPU:        (none)  Enqlm:       256  Pgflquo:     200000
  Authorized Privileges:
    NETMBX    TMPMBX
  Default Privileges:
    NETMBX    TMPMBX
<ENDCODE_EXAMPLE>

<HEAD>(Account LOGIN.COM\head_server_login_com)

<P> The following is suggested as the <EMPHASIS>(LOGIN.COM\SMALLCAPS) for the
server account.  It provides a secure DCL environment for the server image to
execute within.

<CODE_EXAMPLE>
  $ SET NOCONTROL=Y
  $ SET NOON
  $ IF F$MODE() .EQS. "OTHER"
  $ THEN
  $    IF F$TRNLNM("MULTINET") .EQS. ""
  $       THEN HTTPD = "$HT_EXE:HTTPD"
  $       ELSE HTTPD = "$HT_EXE:HTTPD_MULTINET"
  $    ENDIF   
  $    HTTPD_LOOP:
  $       HTTPD /LOG=HT_LOGS:'F$GETSYI("NODENAME")'.LOG
  $!      (non-error exit, must be a restart, loop immediately)
  $       IF $STATUS THEN GOTO HTTPD_LOOP
  $!      (error exit, wait, then try to start the server again)
  $       WAIT 00:01:00
  $       GOTO HTTPD_LOOP
  $!   END_HTTPD_LOOP:
  $    STOP/ID=0
  $ ENDIF
  $ IF F$MODE() .EQS. "BATCH"
  $ THEN
  $    PURGE /KEEP=3 HT_SERVER_LOGS:'F$GETSYI("NODENAME")'80.LOG
  $    RUN SYS$SYSTEM:LOGINOUT -
           /DETACHED /AUTHORIZE -
           /INPUT=NL: /OUTPUT=HT_SERVER_LOGS:'F$GETSYI("NODENAME")'_HTTPD.LOG
  $    STOP/ID=0
  $ ENDIF
  $!(interactive and network modes stop here!)
  $ STOP/ID=0
<ENDCODE_EXAMPLE>

<HEAD>(Privileged Image\head_server_install)

<P> As this image is to be installed with privileges unauthorized use should 
be prevented by applying an ACL similar to the following against the 
executable image: 

<CODE_EXAMPLE>
  $ SET SECURITY HT_EXE:HTTPD.EXE -
    /ACL=((IDENT=HTTP$SERVER,ACCESS=R+E),(IDENT=*,ACCESS=NONE))
<ENDCODE_EXAMPLE>

<P> This can be done once, at installation, or for peace-of-mind 
<EMPHASIS>((a.k.a. VMS-ish paranoia)) at each server startup. 

<P> As the HTTP$SERVER account should be completely unprivileged, and the 
HTTPd image requires NETMBX, TMPMBX, PRMMBX, PSWAPM, SYSNAM and SYSPRV
privileges, it must be installed using a command similar to the following: 

<CODE_EXAMPLE>
  $ INSTALL = "$SYS$SYSTEM:INSTALL"
  $ INSTALL ADD HT_EXE:HTTPD.EXE /PRIVILEGE=(PRMMBX,PSWAPM,SYSPRV,SYSNAM) 
<ENDCODE_EXAMPLE> 

<HEAD>(Logical Names\head_server_logical_names)

<P> The following logical names are essential for the operation of the HTTPd 
server and must be defined before startup:

<LIST>(UNNUMBERED)

<LE> <EMPHASIS>(HTTPD$CONFIG\BOLD) - location of the configuration file 
(defined system-wide, or in the job table if server-specific)

<LE> <EMPHASIS>(HTTPD$MAP\BOLD) - location of the mapping rule file 
(defined system-wide, or in the job table if server-specific)

<LE> <EMPHASIS>(HTTPD$GMT\BOLD) - offset from GMT (e.g. <QUOTE>(+10:30), 
<QUOTE>(-01:15))

<LE> <EMPHASIS>(HTTPD$LOG\BOLD) - if logging is enabled and no log file name
specified on the command line, this logical must be defined to locate the file

<LE> <EMPHASIS>(HT_EXE\BOLD) - directory containing the executable images

<LE> <EMPHASIS>(HT_LOGS\BOLD) - optional definition, for convenient log 
file specification

<LE> <EMPHASIS>(HT_SERVER_LOGS\BOLD) - optional definition, for convenient 
detached server process log file specification 

<ENDLIST>

<P> The following logical name is created by the executing HTTPd server and 
defines the name of the control mailbox:

<LIST>(UNNUMBERED)
<LE> <EMPHASIS>(HTTPD<EMPHASIS>(port)$CONTROL\BOLD)
<ENDLIST>

<P> The following logical names are created by the executing HTTPd server if 
the HTTPd monitor utility is enabled:

<LIST>(UNNUMBERED)
<LE> <EMPHASIS>(HTTPD<EMPHASIS>(port)$COUNT\BOLD)
<LE> <EMPHASIS>(HTTPD<EMPHASIS>(port)$PID\BOLD)
<LE> <EMPHASIS>(HTTPD<EMPHASIS>(port)$REQUEST\BOLD)
<ENDLIST>

<HEAD>(Server Process Logging Directory\head_logging_directory)

<P> The server process log directory (output for the detached HTTPd server 
processes) may require explicit access controls for the HTTPd account.  This 
can be done by applying an ACL similar to the following: 

<CODE_EXAMPLE>
  $ SET SECURITY HT_ROOT:[LOG]SERVER.DIR -
    /ACL=((IDENT=HTTP$SERVER,ACCESS=R+W+E, OPTIONS=DEFAULT), -
          (IDENT=HTTP$SERVER,ACCESS=R+W+E), -
          (IDENT=*,ACCESS=NONE, OPTIONS=DEFAULT), -
          (IDENT=*,ACCESS=NONE))
<ENDCODE_EXAMPLE>

<P> As with the ACL on the server executable this can be done once, at 
installation (or, if right over the top, at each server startup).  Appropriate 
disk quotas may also need to be applied.

<HEAD>(Startup\head_server_startup)

<P> Putting all this together the HTTPd server startup procedure becomes 
something similar to the following: 

<CODE_EXAMPLE>
  $ DEFINE /SYSTEM /TRANSLATION=CONCEALED HT_ROOT DSA811:[HT_ROOT.]
  $!
  $ SET SECURITY HT_ROOT:[LOG]SERVER.DIR -
    /ACL=((IDENT=HTTP$SERVER,ACCESS=R+W+E, OPTIONS=DEFAULT), -
          (IDENT=HTTP$SERVER,ACCESS=R+W+E), -
          (IDENT=*,ACCESS=NONE, OPTIONS=DEFAULT), -
          (IDENT=*,ACCESS=NONE))
  $!
  $ IF F$GETSYI("ARCH_NAME") .EQS. "VAX"
  $    THEN DEFINE /SYSTEM HT_EXE HT_ROOT:[VAX]
  $    ELSE DEFINE /SYSTEM HT_EXE HT_ROOT:[AXP]
  $ ENDIF
  $!
  $ DEFINE /SYSTEM HT_LOGS HT_ROOT:[LOG]
  $ DEFINE /SYSTEM HT_SERVER_LOGS HT_ROOT:[LOG.SERVER]
  $ DEFINE /SYSTEM HTTPD$CONFIG HT_ROOT:[000000]HTTPD$CONFIG.CONF
  $ DEFINE /SYSTEM HTTPD$MAP HT_ROOT:[000000]HTTPD$MAP.CONF
  $ DEFINE /SYSTEM HTTPD$GMT "+10:30"
  $!
  $ INSTALL = "$SYS$SYSTEM:INSTALL"
  $ IF F$TRNLNM("MULTINET") .NES. ""
  $ THEN
  $    SET SECURITY HT_EXE:HTTPD.EXE -
       /ACL=((IDENT=HTTP$SERVER,ACCESS=R+E),(IDENT=*,ACCESS=NONE))
  $    INSTALL ADD HT_EXE:HTTPD.EXE -
                   /PRIVILEGE=(SYSPRV,SYSNAM,PRMMBX,PSWAPM)
  $ ELSE
  $    SET SECURITY HT_EXE:HTTPD_MUTLINET.EXE -
       /ACL=((IDENT=HTTP$SERVER,ACCESS=R+E),(IDENT=*,ACCESS=NONE))
  $    INSTALL ADD HT_EXE:HTTPD_MULTINET.EXE -
                   /PRIVILEGE=(SYSPRV,SYSNAM,PRMMBX,PSWAPM)
  $ ENDIF
  $ SUBMIT /QUEUE=SYS$BATCH /USER=HTTP$SERVER /NOLOG /NOPRINT -
           HT_ROOT:[HTTP$SERVER]LOGIN.COM 
<ENDCODE_EXAMPLE>

<COMMENT>(--------------------------------------------------------------------)

<HEAD1>(HTTPd Command Line\head_httpd_command_line)

<HEAD2>(Server Startup\head_server_command_startup)

<P> When starting up the server several characteristics of the may be 
specified using qualifiers on the command line.  If not specified appropriate 
defaults are employed.

<LIST>(UNNUMBERED)

<LE> <EMPHASIS>(/CGI_PREFIX=\BOLD) the prefix to the CGI symbol names created 
for a script (defaults to <QUOTE>(WWW_), similar to the CERN VMS HTTPd, see 
<REFERENCE>(chapter_scripting)) 

<LE> <EMPHASIS>(/FILBUF=\BOLD) number of bytes in the read buffer for files 
open for processing (i.e. menu files, image mapping configuration files,
pre-processed HTML files, etc., not direct file transfers) 

<LE> <EMPHASIS>(/[NO]LOG[=name]\BOLD) either disables logging (overrides 
configuration directive), or enables logging and optionally specifies the log 
file name (also see section <REFERENCE>(head_server_logical_names), logging 
is disabled by default)

<LE> <EMPHASIS>(/NETBUF=\BOLD) minimum number of bytes in the network read 
buffer

<LE> <EMPHASIS>(/OUTBUF=\BOLD) number of bytes in the output buffer 
(for direct file transfers, buffered output from menu interpretation,
HTML-preprocessing, etc.) 

<LE> <EMPHASIS>(/PRIORITY=\BOLD) server process priority (default is 4)

<LE> <EMPHASIS>(/SUBBUF=\BOLD) number bytes in a subprocess' SYS$OUTPUT 
buffer

<LE> <EMPHASIS>(/[NO]SWAP=\BOLD) controls whether the server process may be 
swapped out of the balance set (default is swapping disabled)

<ENDLIST>

<P> <EMPHASIS>(Note: \BOLD) buffer sizes apply on a per-request (thread) 
basis, and may be tailored for specific environments at server startup. 

<HEAD2>(Server Control\head_httpd_control)

<P> A foreign command for the HTTPD control functionality will need to be 
assigned in the adminstration users' LOGIN.COM, for example: 

<CODE_EXAMPLE>
  HTTPD == "$HT_EXE:HTTPD"
<ENDCODE_EXAMPLE>

<P> The control functionality (via the <EMPHASIS>(/DO=\SMALLCAPS) qualifier) 
is available to the privileged user.  If a non-default server port, or 
multiple servers on the one system are being used, then it will be necessary to 
append a <EMPHASIS>(/PORT=\SMALLCAPS) qualifier to the any command, and issue 
it multiple times.

<HEAD3>(Server Shutdown\head_server_shutdown)

<P> The running server may be elegantly shut down, without loss of existing 
client requests, using the following command on the server system: 

<CODE_EXAMPLE>
  $ HTTPD /DO=EXIT
<ENDCODE_EXAMPLE>

<P> The running server may be immediately and unconditionally shut down using 
the following command on the server system: 

<CODE_EXAMPLE>
  $ HTTPD /DO=ABORT
<ENDCODE_EXAMPLE>

<P> The running server may be elegantly restarted, without loss of existing 
client requests, using the following command on the server system: 

<CODE_EXAMPLE>
  $ HTTPD /DO=RESTART
<ENDCODE_EXAMPLE>

<HEAD3>(Mapping Reload\head_config_reload)

<P> The <EMPHASIS>(mapping rules) may be reloaded into the running server using
the following command on the server system:

<CODE_EXAMPLE>
  $ HTTPD /DO=MAP
<ENDCODE_EXAMPLE>

<HEAD3>(Counter Reset\head_server_counter_zeroed)

<P> The server counter values are carried over when a server (re)starts 
(provided the system has stayed up).  To reset these counters use the 
following command on the server system: 

<CODE_EXAMPLE>
  $ HTTPD /DO=ZERO
<ENDCODE_EXAMPLE>

<P> Note that the count of the number of time the counters have been zeroed is 
not reset by this command.

<HEAD3>(Logging\head_server_logging)

<P> The HTTPd server request log may be <EMPHASIS>(opened) (enabled) or
<EMPHASIS>(closed) (disabled) using the following commands:

<CODE_EXAMPLE>
  $ HTTPD /DO=LOG=OPEN
<ENDCODE_EXAMPLE>

<CP>and

<CODE_EXAMPLE>
  $ HTTPD /DO=LOG=CLOSE
<ENDCODE_EXAMPLE>

<P> Any records still in the RMS buffers may be flushed using:

<CODE_EXAMPLE>
  $ HTTPD /DO=LOG=FLUSH
<ENDCODE_EXAMPLE>

<HEAD3>(Authentication\head_server_authentication)

<P> Also see <REFERENCE>(head_config_auth).

<P> The HTTPd's authentication facility tracks failures and after a given
threshold consistently rejects authentication requests for the given username
without actually validating the username/password.  This is done to prevent
password attacks via the HTTPd server.

<P> The authentication database may be displayed using:

<CODE_EXAMPLE>
  $ HTTPD /DO=AUTH=ALL
<ENDCODE_EXAMPLE>

<P> Authentication database failure records only may be displayed using:

<CODE_EXAMPLE>
  $ HTTPD /DO=AUTH=FAIL
<ENDCODE_EXAMPLE>

<P> Any failure records may have their count reset to zero using:

<CODE_EXAMPLE>
  $ HTTPD /DO=AUTH=RESET=<EMPHASIS>(realm):<EMPHASIS>(username)
<ENDCODE_EXAMPLE>

<COMMENT>(--------------------------------------------------------------------)

<CHAPTER>(HTTPd Configuration\chapter_config)

<P> HFRD's HTTPd configuration is a subset based on the CERN HTTPd.  By 
default, the system-table logical name <EMPHASIS>(HTTPD$CONFIG\SMALLCAPS) 
locates a common configuration file, unless an individual configuration file 
is specified using a job-table logical name.  Simple editing of this file 
changes the configuration.  Comment lines may be included by prefixing them 
with the hash <QUOTE>(#) character.  Configuration file directives are not 
case-sensitive.  Any changes to the configuration file can only be enabled by
restarting the HTTPd process using the following command on the server system: 

<CODE_EXAMPLE>
  $ HTTPD /DO=RESTART
<ENDCODE_EXAMPLE>

<P> The reader is also refered to the CERN HTTPd documentation for background 
information on HTTP daemon configuration.  Also see 
<REFERENCE>(chapter_mapping_rule) for mapping rule configuration. 

<HEAD1>(Authentication\head_config_auth)

<P> The HFRD VMS HTTPd provides little in the way of authorization and path 
access control.  It does have a scheme providing user name authentication from 
the host's VMS authorization database (SYSUAF.DAT).  <EMPHASIS>(This is not 
recommended for Internet, WAN or insecure LAN use.\BOLD)  Passwords are 
transmitted encoded but unencrypted from client to server, and so on anything 
but a carefully secured LAN poses a <EMPHASIS>(distinct and real security 
threat\BOLD).  Even on an internal LAN <EMPHASIS>(a privileged account should 
never, ever be authenticated in this fashion.)  Hence, by default, this 
facility is disabled.  You have be warned! 

<P> The author acknowleges this to be a significant deficiency and would be 
willing to remedy it if appropriate demand exists.

<HEAD1>(Current Directives\head_config_current)

<P> Note that all <EMPHASIS>(boolean) directives are <EMPHASIS>(disabled) (OFF)
by default.  This is done so that there can be no confusion about what is
enabled and disabled by default.  To use directive controlled facility it
<EMPHASIS>(must\BOLD) be explicitly enabled.

<LIST>(NUMBERED)

<LE> <EMPHASIS>(Accept <EMPHASIS>(host/domain name)\BOLD)
<EMPHASIS>((default: all))

<P> One or more (comma-separated if on the same line) internet host/domain 
names, with <QUOTE>(*) wildcarding for host/subdomain matching, to be 
explicitly allowed access. Also see the <EMPHASIS>(Reject) directive.  Reject 
directives have precedence over Accept directives.  The Accept directive may 
be used multiple times.  Also see <EMPHASIS>(DNSLookup) directive.

<P> Examples: 

<CODE_EXAMPLE>
  Accept *.remote.dsto.gov.au
  Accept *.hfrd.dsto.gov.au
<ENDCODE_EXAMPLE>

<LE> <EMPHASIS>(AddType <EMPHASIS>(.suffix) <EMPHASIS>(representation)
<EMPHASIS>(encoding) <EMPHASIS>(script-name) <EMPHASIS>(description) \BOLD)
<EMPHASIS>((no default)) 

<P> Binds a file suffix (extension, type) to a mime content type and encoding.  
The script name is used to auto-script (see <REFERENCE>(head_autoscripting)) 
a specified file type.  The description is used as documentation for 
directory listings. 

<LINE>(SMALLSKIP)
Examples:

<CODE_EXAMPLE>
  AddType  .html  text/html   8bit      -       HyperText Markup Language
  AddType  .txt   text/plain  8bit      -       plain text
  AddType  .gif   image/gif   binary    -       image (GIF)
  AddType  .hlb   text/x-script 7bit    /Conan  VMS Help library
  AddType  .decw$book   text/x-script   8bit    /HyperReader    Bookreader book
<ENDCODE_EXAMPLE>

<LE> <EMPHASIS>(AddIcon <EMPHASIS>(icon-URL) <EMPHASIS>(ALT-text) 
<EMPHASIS>(template) \BOLD) 
<EMPHASIS>((no default)) 

<P> Specifies a directory listing icon and alternative text for the mime 
content type specified in the template.

<LINE>(SMALLSKIP) Examples: 

<CODE_EXAMPLE>
  AddIcon  /icon/-/doc.xbm    [HTM]  text/html
  AddIcon  /icon/-/text.xbm   [TXT]  text/plain
  AddIcon  /icon/-/image.xbm  [IMG]  image/gif
<ENDCODE_EXAMPLE>

<LE> <EMPHASIS>(AddBlankIcon <EMPHASIS>(icon-URL)\BOLD)
<LINE> <EMPHASIS>(AddDirIcon <EMPHASIS>(icon-URL) <EMPHASIS>(ALT-text)\BOLD)
<LINE> <EMPHASIS>(AddParentIcon <EMPHASIS>(icon-URL) <EMPHASIS>(ALT-text)\BOLD)
<LINE> <EMPHASIS>(AddUnknownIcon <EMPHASIS>(icon-URL) <EMPHASIS>(ALT-text)\BOLD)
<EMPHASIS>((no defaults)) 
<LINE>(SMALLSKIP)
Specifies a directory listing icon for these non-content-type parts of the 
listing.
<LINE>(SMALLSKIP)
Examples:

<CODE_EXAMPLE>
  AddBlankIcon    /icon/-/blank.xbm
  AddDirIcon      /icon/-/dir.xbm      [DIR]
  AddParentIcon   /icon/-/back.xbm     [<--]
  AddUnknownIcon  /icon/-/unknown.xbm  [???] 
<ENDCODE_EXAMPLE>

<COMMENT>(HTML=<P>)

<LE> <EMPHASIS>(AuthLocal <EMPHASIS>(ON <VBAR> OFF\SMALLCAPS)\BOLD) 
<EMPHASIS>((default: OFF)) 

<P> Enables or disables SYSUAF user name authentication.  See 
<REFERENCE>(head_config_auth).

<COMMENT>(HTML=<P>)

<LE> <EMPHASIS>(Busy <EMPHASIS>(integer)\BOLD)
<EMPHASIS>((default: 10))

<P> The maximum number of concurrent client connections before a 
<QUOTE>(<EMPHASIS>(server too busy right now ... try again shortly)) error 
is returned to the client. 

<COMMENT>(HTML=<P>)

<LE> <EMPHASIS>(CommentedInfo <EMPHASIS>(ON <VBAR> OFF\SMALLCAPS)\BOLD) 
<EMPHASIS>((default: OFF)) 

<P> Includes, commented at the beginning of any HTML document, the software ID
of the server and any relevant VMS file specification for the document.

<COMMENT>(HTML=<P>)

<LE> <EMPHASIS>(DirAccess <EMPHASIS>(ON <VBAR> OFF <VBAR> 
SELECTIVE\SMALLCAPS)\BOLD) <EMPHASIS>((default: OFF)) 

<P> Controls directory listings.  <EMPHASIS>(SELECTIVE\SMALLCAPS) allows access 
only to those directories containing a file 
<EMPHASIS>(.WWW_BROWSABLE\SMALLCAPS).  The HFRD HTTPd directory access 
facility always ignores directories containing a file named 
<EMPHASIS>(.WWW_HIDDEN\SMALLCAPS).  Also see the <EMPHASIS>(DirWildcard)
directive.

<COMMENT>(HTML=<P>)

<LE> <EMPHASIS>(DirLayout <EMPHASIS>(string)\BOLD)
<EMPHASIS>((default: I__L__R__S__D)) 

<P> Allows specification of the directory listing layout.  This is a short 
string that specifies the included fields, relative placement and optionally 
the width of the fields in a directory listing.  Each field is controlled by a 
single letter and optional leading decimal number specifying its width.  If a 
width is not specified an appropriate default applies.  An underscore is used 
to indicate a single space and is used to separate the fields (two consecutive 
works well). 

<COMMENT>(HTML=<P>)

<LIST>(UNNUMBERED)
<LE><EMPHASIS>(C\BOLD) - creation date
<LE><EMPHASIS>(D\BOLD) - description (having no field-width attribute this is 
always best specified last)
<LE><EMPHASIS>(I\BOLD) - icon (takes no field-width attribute)
<LE><EMPHASIS>(L\BOLD) - link (highlighted anchor using the name of the file)
<LE><EMPHASIS>(N\BOLD) - name (no link, why bother? who knows!)
<LE><EMPHASIS>(O\BOLD) - owner (can be disabled)
<LE><EMPHASIS>(R\BOLD) - revision date
<LE><EMPHASIS>(S\BOLD) - size
<ENDLIST>

<COMMENT>(HTML=<P>)

<CP> As illustrated in the following examples:

<CODE_EXAMPLE>
  DirLayout       I__15L__S__D
  DirLayout       15L__9R__S
  DirLayout       15N_9C_9R_S
<ENDCODE_EXAMPLE>

<LE> <EMPHASIS>(DirOwner <EMPHASIS>(ON <VBAR> OFF \SMALLCAPS)\BOLD) 
<EMPHASIS>((default: OFF)) 

<P> Allows specification and display of the RMS file owner information.

<COMMENT>(HTML=<P>)

<LE> <EMPHASIS>(DirReadme <EMPHASIS>(TOP <VBAR> BOTTOM <VBAR>
OFF\SMALLCAPS)\BOLD) <EMPHASIS>((default: OFF)) 

<P> If any of the files provided using the <EMPHASIS>(DirReadMeFile) directive 
are located in the directory the contents are included at the top or bottom of 
the listing (or not at all).  Plain-text are included as plain-text, HTML are 
included as HTML allowing markup tags to be employed. 

<COMMENT>(HTML=<P>)

<LE> <EMPHASIS>(DirReadMeFile <EMPHASIS>(file.suffix\SMALLCAPS)\BOLD)
<EMPHASIS>((no default)) 

<P> Specifies the names and order in which a directory is checked for 
<EMPHASIS>(read-me) files.  This can be enabled or disabled using the 
<EMPHASIS>(DirReadme) directive.   Plain-text are included as plain-text, HTML 
are included as HTML allowing markup tags to be employed. 

<COMMENT>(HTML=<P>)

<P> Examples: 

<CODE_EXAMPLE>
  DirReadMeFile readme.html
  DirReadMeFile readme.htm
  DirReadMeFile readme.
  DirReadMeFile readme.txt
  DirReadMeFile readme.1st
<ENDCODE_EXAMPLE>

<COMMENT>(HTML=<P>)

<LE> <EMPHASIS>(DirWildcard <EMPHASIS>(OFF <VBAR> ON\SMALLCAPS)\BOLD)
<EMPHASIS>((default: OFF))

<P> This enables the facility to <EMPHASIS>(force) the server to provide a
directory listing by providing a wildcard file specification, even if there is
a home (welcome) document in the directory.  This should not be confused with
the <EMPHASIS>(DirAccess) directive which controls directory listing itself.

<COMMENT>(HTML=<P>)

<LE> <EMPHASIS>(DNSLookup <EMPHASIS>(ON <VBAR> OFF\SMALLCAPS)\BOLD)
<EMPHASIS>((default: OFF)) 

<P> Enables or disables connection request host name resolution.  This 
functionality may be expensive (in terms of processing overhead) and make 
serving granularity coarser if DNS is involved.  If not enabled and logging 
is, the entry is logged against the numeric internet address.  If not enabled 
any <EMPHASIS>(accept) or <EMPHASIS>(reject) directive must be expressed as 
numeric addresses. 

<COMMENT>(HTML=<P>)

<LE> <EMPHASIS>(InputTimeout <EMPHASIS>(integer)\BOLD)
<EMPHASIS>((default: 2)) 

<P> Number of minutes to allow a connection request to be in progress without 
submitting a complete request header before terminating it. 

<COMMENT>(HTML=<P>)

<LE> <EMPHASIS>(Logging <EMPHASIS>(ON <VBAR> OFF\SMALLCAPS)\BOLD)
<EMPHASIS>((default: OFF)) 

<P> Enables or disables the activity log.

<COMMENT>(HTML=<P>)

<LE> <EMPHASIS>(OutputTimeout <EMPHASIS>(integer)\BOLD)
<EMPHASIS>((default: 10)) 

<P> Number of minutes to allow a request to be output before terminating it.

<COMMENT>(HTML=<P>)

<LE> <EMPHASIS>(Port <EMPHASIS>(integer)\BOLD)
<EMPHASIS>((default: 80)) 

<P> IP port number for server to bind to.

<COMMENT>(HTML=<P>)

<LE> <EMPHASIS>(Recommend <EMPHASIS>(ON <VBAR> OFF\SMALLCAPS)\BOLD)
<EMPHASIS>((default: OFF)) 

<P> Provides a short message recommending action when reporting an error to a
client.  For example, if a document cannot be found it may say:

<CODE_EXAMPLE>
  <EMPHASIS>((document, or bookmark, requires revision))
<ENDCODE_EXAMPLE>

<COMMENT>(HTML=<P>)

<LE> <EMPHASIS>(Reject <EMPHASIS>(host/domain name)\BOLD)
<EMPHASIS>((default: none))

<P> One or more (comma-separated if on the same line) internet host/domain 
names, with <QUOTE>(*) wildcarding for host/subdomain matching, to be 
explicitly denied access.  Also see the <EMPHASIS>(Accept) directive.  Reject 
directives have precedence of Accept directives.  The Reject directive may be 
used multiple times.  Also see <EMPHASIS>(DNSLookup) directive.

<P> Example: 

<CODE_EXAMPLE>
  Reject *.remote.dsto.gov.au,*.hfrd.dsto.gov.au
<ENDCODE_EXAMPLE>

<LE> <EMPHASIS>(Search <EMPHASIS>(path)\BOLD)
<EMPHASIS>((no default)) 

<P> Specifies the <EMPHASIS>(physical path\BOLD) to the default query-string 
keyword search script.

<P> Examples: 

<CODE_EXAMPLE>
  Search /ht_root/script/query
<ENDCODE_EXAMPLE>

<LE> <EMPHASIS>(sHTML <EMPHASIS>(ON <VBAR> OFF\SMALLCAPS)\BOLD)
<EMPHASIS>((default: OFF)) 

<P> Enables or disables HTML pre-processing.

<COMMENT>(HTML=<P>)

<LE> <EMPHASIS>(sHTMLaccesses <EMPHASIS>(ON <VBAR> OFF\SMALLCAPS)\BOLD)
<EMPHASIS>((default: OFF)) 

<P> Enables or disables HTML pre-processing file access counter.

<COMMENT>(HTML=<P>)

<LE> <EMPHASIS>(sHTMLexec <EMPHASIS>(ON <VBAR> OFF\SMALLCAPS)\BOLD)
<EMPHASIS>((default: OFF)) 

<P> Enables or disables HTML pre-processing DCL execution functionality.

<COMMENT>(HTML=<P>)

<LE> <EMPHASIS>(Welcome <EMPHASIS>(file.suffix)\BOLD)
<EMPHASIS>((no default)) 

<P> Specifies the names and order in which a directory is checked for home page 
files.  If no home page is found a directory listing is generated.

<COMMENT>(HTML=<P>)

<P> Examples: 

<CODE_EXAMPLE>
  Welcome  home.html
  Welcome  home.htm
  Welcome  home.menu
  Welcome  home.mnu
<ENDCODE_EXAMPLE>

<ENDLIST>

<COMMENT>(....................................................................)

<COMMENT>(HTML=
<P> The example
<A HREF="/ht_root/example/httpd$config.conf">configuration file</A> for the
HFRD HTTPd server can be viewed.
)

<COMMENT>(HTML/OFF)
<P> The example configuration file for the HFRD HTTPd server can be viewed.
<P><BOX>(online hypertext link)
<COMMENT>(HTML/ON)

<COMMENT>(....................................................................)

<COMMENT>(--------------------------------------------------------------------)

<CHAPTER>(HTTPd Mapping Rules\chapter_mapping_rule)

<P> URLs paths are mapped to physical file system locations using mapping 
similar to other HTTPd implementations.  HFRD's is based on the CERN HTTPd 
implementation, with one additional script mapping rule.  By default, the 
system-table logical name <EMPHASIS>(HTTPD$MAP\SMALLCAPS) locates a common 
mapping rule file, unless an individual rule file is specified using a job-
table logical name.  Simple editing of the mapping file changes the rules.  
Comment lines may be included by prefixing them with the hash <QUOTE>(#) 
character.  Although, there is no fixed limit on the number of rules there are 
the processing implications of scanning a large, linear database. 

<P> Rules are given a basic consistency check when loaded (i.e. server 
startup, map reload, etc.)  If there is an obvious problem (unknown rule, 
missing component, etc.) a warning message is generated and the rule is not 
loaded into the database.  This will not cause the server startup to fail.  
These warning messages may be found in the server process log. 

<P> A command-line rule checking utility is available for assessing rule 
database files and the mappings generated against supplied paths.  See 
<REFERENCE>(head_chkmap). 

<P> Any changes to the mapping file may be (re)loaded into the running HTTPd 
server using the following command on the server system: 

<CODE_EXAMPLE>
  $ HTTPD /DO=MAP
<ENDCODE_EXAMPLE>

<P> Also see <REFERENCE>(chapter_config) for daemon configuration. 

<HEAD>(MAP, PASS, FAIL Rules\head_map_rules)

<LIST>(NUMBERED)

<LE> <EMPHASIS>(map <EMPHASIS>(template result)\BOLD)

<P> If the URL path matches the template, substitute the <EMPHASIS>(result) 
string for the path and use that for further rule processing. 

<COMMENT>(HTML=<P>)

<LE> <EMPHASIS>(pass <EMPHASIS>(template)\BOLD) 
<LINE> <EMPHASIS>(pass <EMPHASIS>(template result)\BOLD) 

<P> If the URL path matches the template, substitute the result if present (if 
not just use the original URL path), processing no further rules. 

<P> The <EMPHASIS>(result) should be a physical VMS file system specification 
in URL format.  If there is a direct correspondance between the 
<EMPHASIS>(template) and <EMPHASIS>(result) the result may be omitted. 

<P> The <EMPHASIS>(PASS\SMALLCAPS) directive is also used to 
<EMPHASIS>(reverse-map) VMS file specifications to the URL format equivalent. 

<COMMENT>(HTML=<P>)

<LE> <EMPHASIS>(fail <EMPHASIS>(template)\BOLD) 

<P> If the URL path matches the template, prohibit access, processing no 
further rules. 

<ENDLIST>

<HEAD>(REDIRECT Rule\head_map_rule_redirect)

<LIST>(NUMBERED)

<LE> <EMPHASIS>(redirect <EMPHASIS>(template) <EMPHASIS>(result)\BOLD)

<P> If the URL path matches the template, substitute the <EMPHASIS>(result) 
string for the path.  Process no further rules.  The <EMPHASIS>(result) must 
be a full URL (<EMPHASIS>(http://host/path)), and is used to redirect requests 
to another server on a separate host. 

<ENDLIST>

<P> This will probably not be used extensively on the internal HFRD network.

<HEAD>(EXEC and SCRIPT, Script Mapping Rules\head_map_rule_exec_script)

<P> Also see <REFERENCE>(chapter_scripting).

<P> The <EMPHASIS>(EXEC\SMALLCAPS) rules maps CGI script 
<EMPHASIS>(directories\BOLD).

<P> The <EMPHASIS>(SCRIPT\SMALLCAPS) rules maps CGI script <EMPHASIS>(file 
names\BOLD).  It is a little different to the <EMPHASIS>(EXEC\SMALLCAPS) rule 
and an extension to the CERN rules. 

<P> Both rules must have a <EMPHASIS>(template) and <EMPHASIS>(result), and 
both must end in a wildcard asterisk.  The placement of the wildcards and the 
subsequent functionality is slightly different however.

<LIST>(NUMBERED)

<LE> <EMPHASIS>(exec <EMPHASIS>(template result)\BOLD) 

<P> The <EMPHASIS>(EXEC\SMALLCAPS) rule requires the <EMPHASIS>(template)'s 
asterisk to immediately follow the slash terminating the directory 
specification containing the scripts.  The script name follows immediately as 
part of the wildcard-matched string.  For example:

<CODE_EXAMPLE>
  exec /htbin/* /ht_root/script/*
<ENDCODE_EXAMPLE>

<P> If the URL path matches the template, the result, including the first 
slash-terminated part of the wildcard-matched section, becomes the URL format 
physical VMS file specification for the DCL procedure of the script to be 
executed (the default file extension of <EMPHASIS>(.COM\SMALLCAPS) is not 
required).  What remains of the original URL path is used to create the path 
information.  Process no further rules. 

<P> Hence, the <EMPHASIS>(EXEC\SMALLCAPS) rule will match multiple script 
specifications without further rules, the script name being supplied with the 
URL path.  Hence any script (i.e. procedure, executable) in the specified 
directory is accessable, a possible security concern if script management is 
distributed.

<COMMENT>(HTML=<P>)

<LE> <EMPHASIS>(script <EMPHASIS>(template result)\BOLD) 

<P> The <EMPHASIS>(SCRIPT\SMALLCAPS) rule requires the <EMPHASIS>(template)'s 
asterisk to immediately follow the <EMPHASIS>(unique string) identifying the 
script in the URL path.  The wildcard-matched string is the following path, 
and supplied to the script.  For example: 

<CODE_EXAMPLE>
  script /conan* /ht_root/script/conan*
<ENDCODE_EXAMPLE>

<P> If the URL path matches the template, the result becomes the URL format 
physical VMS file specification for the DCL procedure of the script to be 
executed (the default file extension of <EMPHASIS>(.COM\SMALLCAPS) is not 
required).  What remains of the original URL path is used to create the path 
information.  Process no further rules. 

<NOTE>
The wildcard asterisk is best located immediately after the unique script 
identifier. In this way there does not need to be any path supplied with the 
script.  If even a slash follows the script identifier it may be mapped into a 
file specification that may or may not be meaningful to the script.
<ENDNOTE> 

<P> Hence, the <EMPHASIS>(SCRIPT\SMALLCAPS) rule will match only the script 
specified in the <EMPHASIS>(result), making for finely-granular scripting at 
the expense of a rule for each script thus specified.  It also implies that 
only the script name need precede any other path information.

<P> It may be thought of as a more efficient implementation of the equivalent 
functionlity using two CERN rules, as illustrated in the following example:

<CODE_EXAMPLE>
  map /conan* /script/conan*
  exec /script/* /ht_root/script/*
<ENDCODE_EXAMPLE>

<ENDLIST>

<HEAD>(Rule Interpretation\head_map_rule_interp)

<P> The rules are scanned from first towards last, until a <EMPHASIS>(pass), 
<EMPHASIS>(exec) or <EMPHASIS>(fail) is encountered, when processing ceases 
and final substitution occurs.  <EMPHASIS>(Map)ed rules substitute the 
<EMPHASIS>(template) with the <EMPHASIS>(result) and continue to the next 
rule. 

<P> Use of wildcards in <EMPHASIS>(template) and <EMPHASIS>(result):

<LIST>(UNNUMBERED)

<LE> The <EMPHASIS>(template) may contain one or more asterisk (<QUOTE>(*)) 
wildcard symbols.  These match zero or more characters up until the character 
following the wildcard (or end-of-string).  If no wildcard is present then the 
path must match the <EMPHASIS>(template) exactly. 

<LE> The <EMPHASIS>(result) may contain one or more asterisk (<QUOTE>(*)) 
wildcard symbols.  It must not contain more wildcards than the 
<EMPHASIS>(template).  The <EMPHASIS>(result) wildcards are expanded to 
replace the matching characters of the respective <EMPHASIS>(template) 
wildcards.  Characters represented by wildcards in the <EMPHASIS>(template) 
not represented by a corresponding wildcard in the <EMPHASIS>(result) are 
ignored.  Non-wildcard <EMPHASIS>(result) characters are directly inserted in 
reconstructed path.  Non-wildcard characters in the <EMPHASIS>(template) are 
ignored.  If the <EMPHASIS>(result) contains no wildcards it completely 
replaces the URL path. 

<ENDLIST>

<COMMENT>(....................................................................)

<COMMENT>(HTML=
<P> The example
<A HREF="/ht_root/example/httpd$map.conf">mapping rule file</A> for the HFRD
HTTPd server can be viewed.
)

<COMMENT>(HTML/OFF)
<P> The example mapping rule file for the HFRD HTTPd server can be viewed.
<P><BOX>(online hypertext link)
<COMMENT>(HTML/ON)

<COMMENT>(....................................................................)

<HEAD>(Examples of <EMPHASIS>(Map\BOLD) Rule\head_example_map)

<P> The <EMPHASIS>(result) string of these rules may or may not correspond to 
to a VMS physical file system path.  Either way the resulting rule is further 
processed before passing or failing. 

<LIST>(UNNUMBERED)

<LE> <EMPHASIS>(map /hyperdata/unix/* /hyperdata/software/unix/*\SMALLCAPS)

<P> If a URL path <EMPHASIS>(/hyperdata/unix/shells/c\SMALLCAPS) is being 
mapped the path would be replaced by 
<EMPHASIS>(/user$rts/hyperdata/software/unix/scripts/c\SMALLCAPS), and this 
used to  process further rules. 

<ENDLIST>

<HEAD>(Examples of <EMPHASIS>(Pass\BOLD) Rule\head_example_pass)

<P> The <EMPHASIS>(result) string of these rules should correspond to to a VMS 
physical file path. 

<LIST>(UNNUMBERED)

<LE> <EMPHASIS>(pass /hyperdata/rts/* /user$rts/hyperdata/*\SMALLCAPS)

<P> If a URL path <EMPHASIS>(/hyperdata/rts/home.html\SMALLCAPS) is being 
mapped the path would be replaced by 
<EMPHASIS>(/user$rts/hyperdata/home.html\SMALLCAPS), and this returned as the 
mapped path. 

<COMMENT>(HTML=<P>)

<LE> <EMPHASIS>(pass /icon/bhts/* /hyperdata/icon/bhts/*\SMALLCAPS)

<P> If a URL path <EMPHASIS>(/icon/bhts/dir.xbm\SMALLCAPS) is being mapped the 
path would be replaced by <EMPHASIS>(/hyperdata/icon/bhts/dir.xbm\SMALLCAPS), 
and this returned as the mapped path. 

<ENDLIST>

<HEAD>(Examples of <EMPHASIS>(Fail\BOLD) Rule\head_example_fail)

<LIST>(UNNUMBERED)

<LE> <EMPHASIS>(fail /hyperdata/private/*\SMALLCAPS)

<P> If a URL path <EMPHASIS>(/hyperdata/private/home.html\SMALLCAPS) is being 
mapped the path would immediately be failed. 

<COMMENT>(HTML=<P>)

<LE> <EMPHASIS>(fail /*\SMALLCAPS)

<P> To ensure all access fails, other than that explictly passed, this entry 
should be included the the rules. 

<ENDLIST>

<HEAD>(Examples of <EMPHASIS>(Exec\BOLD) and <EMPHASIS>(Script\BOLD) Rules
\head_example_exec)

<LIST>(UNNUMBERED)

<LE> <EMPHASIS>(exec /htbin/* /ht_root/script/*\SMALLCAPS)

<P> If a URL path <EMPHASIS>(/htbin/ismap/hyperdata/example.conf\SMALLCAPS) is 
being mapped the <QUOTE>(/ht_root/script/) must be the URL format equivalent 
of the physical VMS specification for the directory locating the script DCL 
procedure.  The <EMPHASIS>(/hyperdata/example.conf\SMALLCAPS) that followed 
the <EMPHASIS>(/htbin/ismap\SMALLCAPS) in the original URL becomes the 
translated path for the script.  See <REFERENCE>(chapter_scripting) for other 
information on scripting. 

<COMMENT>(HTML=<P>)

<LE> <EMPHASIS>(script /conan* /ht_root/script/conan*\SMALLCAPS)

<P> If a URL path <EMPHASIS>(/conan/hyperdata/example.hlb\SMALLCAPS) is being 
mapped the <QUOTE>(/ht_root/script/conan) must be the URL format equivalent of 
the physical VMS specification for the DCL procedure.  The 
<EMPHASIS>(/hyperdata/example.hlb\SMALLCAPS) that followed the 
<EMPHASIS>(/conan/\SMALLCAPS) in the original URL becomes the translated path 
for the script.  See <REFERENCE>(chapter_scripting) for other information on 
scripting. 

<ENDLIST>

<HEAD>(Examples of <EMPHASIS>(Redirect\BOLD) Rule\head_example_redirect)

<LIST>(UNNUMBERED)

<LE> <EMPHASIS>(redirect /AnotherGroup/* http://host/group/*\SMALLCAPS)

<P> If a URL path <EMPHASIS>(/AnotherGroup/this/that/other.html\SMALLCAPS) is 
being mapped the URL would be redirected to 
<EMPHASIS>(http://host/this/that/other.html\SMALLCAPS) 

<ENDLIST>

<COMMENT>(....................................................................)

<HEAD1>(Mapping User Directories (<EMPHASIS>(tilde) character (<QUOTE>(~)))
\head_user_dirs) 

<P> This server will map user directories using the same mechanisms as for 
any other.  No reference is made to SYSUAF.DAT, user support is accomplished 
via a combination of mapping rule and logical name.  This approach relies on a 
correspondance between the user name and the home directory name.  Hence users 
are made known by the HTTPd using the name of their top-level directory.  As 
the naming of home directories using the user name is a common practice this 
mechanism should suffice in the majority of cases.  Where there is no such 
correspondance individual rules could be used for each user. 

<P> The <EMPHASIS>(PASS\SMALLCAPS) rule provides a wildcard representation of 
users' directory paths.  As part of this mapping a subdirectory specifically 
for the hypertext data should be always included.  <EMPHASIS>(Never\BOLD) map 
users' top-level directories.   For instance if a user's account home 
directory was located in the area <EMPHASIS>(USER$DISK:[DANIEL]\SMALLCAPS) the 
following rule would potentially allow the user DANIEL to provide hypertext 
information from the home subdirectory <EMPHASIS>([.WWW]\SMALLCAPS) (if the 
user has created it) using the accompanying URL: 

<CODE_EXAMPLE>
  pass /~*/* /user$disk/*/www/*

  http://host/~daniel/
<ENDCODE_EXAMPLE>

<P> It is <EMPHASIS>(recommended\BOLD) that a separate logical name be created 
for locating user directories.  This helps hide the internal organisation of 
the file system.  The following logical name definition and mapping rule 
illustrate this point. 

<CODE_EXAMPLE>
  $ DEFINE /SYSTEM /EXEC /TRANSLATION=CONCEALED WWW_USER DSA811:[USER.]

  pass /~*/* /www_user/*/www/*
<ENDCODE_EXAMPLE>

<P> Where users are grouped into different areas of the file system a logical 
search list may be defined. 

<CODE_EXAMPLE>
  $ DEFINE /SYSTEM /EXEC /TRANSLATION=CONCEALED -
           WWW_USER -
           DISK1:[GROUP1.], -
           DISK1:[GROUP2.], -
           DISK2:[GROUP3.], -
           DISK2:[GROUP4.]
    
  pass /~*/* /www_user/*/www/*
<ENDCODE_EXAMPLE>

<P> As logical search lists have specific uses and some complications this is 
the only use for them recommended with this server, although it is 
specifically coded to allow for search lists in document specifications. 

<P> If only a subset of all users are to be provided with WWW publishing 
access either their account directories can be individually mapped (best used 
only with a small number) or a separate area of the file system be provided 
for this purpose and specifically mapped as user space. 

<P> Of course, user mapping is amenable to all other rule processing so it is 
a simple matter to redirect or otherwise process user paths.  For instance, 
the published user name does not need to, or need to continue to, correspond 
to any real user area, or the user's actual name or home area: 

<CODE_EXAMPLE>
  redirect /~doej/* http://a.nother.host/~doej/*
  pass /~doej/* /www/messages/deceased.html
  pass /~danielm/* /special$www$area/danielm/*
  pass /~Mark.Daniel/* /user$disk/danielm/www/*
  pass /~*/* /www_user/*/www/*
<ENDCODE_EXAMPLE>

<P> A user directory is always presented as a top-level directory (i.e. no 
parent directory is shown), although any subdirectory tree is accesssable by 
default.

<COMMENT>(--------------------------------------------------------------------)

<CHAPTER>(HTTPd Scripting\chapter_scripting)

<P> <EMPHASIS>(Scripts) are mechanisms for creating simple <QUOTE>(servers) 
sending data to a client, extending the services provided by the basic server.  
Anything that can write to <EMPHASIS>(SYS$OUTPUT\SMALLCAPS) can be used to 
generate script output.  A DCL procedure or an executable can be the basis for
a script.  Simply <EMPHASIS>(TYPE\SMALLCAPS)-ing a file can be provide script
output.

<P> Scripts are enabled using the <EMPHASIS>(exec) or <EMPHASIS>(script) rules in the mapping file 
(see <REFERENCE>(chapter_mapping_rule)).  The script portion of the 
<EMPHASIS>(result) must be a URL equivalent of the physical VMS procedure or
executable specification.  <EMPHASIS>(It is not necessary\BOLD) to supply the
<EMPHASIS>(.COM\SMALLCAPS) or <EMPHASIS>(.EXE\SMALLCAPS) file type (although
not forbidden either), the server will first check for a procedure and if none
found then check for an executable.

<HEAD1>(CGI Compliance\head_cgi_compliance)

<P> The HTTPd scripting mechanism is designed to be largely WWW CGI (Common 
Gateway Interface) compliant. 

<HEAD>(CGI Compliant Variables\head_cgi_variables)

<P> Environment variables are created in a similar way to the CERN VMS HTTPd 
implementation, where CGI environment variables are provided to the script via 
DCL global symbols.  Each CGI variable symbol name is prefixed with
<QUOTE>(WWW_) (by default, although this can be changed using the
<EMPHASIS>(/CGI_PREFIX\SMALLCAPS) qualifier, see
<REFERENCE>(head_httpd_command_line), this is not recommended if the HFRD VMS
scripts are to be used, as they expect CGI variable symbols to be prefixed in
this manner).

<HEAD>(Extensions to CGI Variables\head_cgi_extensions)

<P> In line with other CGI implemenations, additional, non-compliant variables
are provided to ease CGI interfacing.  These provide the various components of
the query string.  A <EMPHASIS>(keyword) query string and a <EMPHASIS>(form)
query string are parsed into separated variables, named

<CODE_EXAMPLE>
  WWW_KEY_<EMPHASIS>(number)
  WWW_KEY_COUNT
  WWW_FORM_<EMPHASIS>(form-element-name)
<ENDCODE_EXAMPLE>

<P> See the example below.

<HEAD>(CGI Variable Capacity\head_cgi_capacity)

<P> DCL symbol values are limited to approximately 1000 characters.  The CGI 
interface will provide symbols with values up to that limit if required.  This 
should be sufficient for most circumstances.

<P> The basic CGI symbol names are demonstrated here with a call to a script 
that simply executes the following DCL code:

<CODE_EXAMPLE>
  $ SHOW SYMBOL WWW_*
  $ SHOW SYMBOL *
<ENDCODE_EXAMPLE>

<CP> Note how the request components are represented for 
<EMPHASIS>(ISINDEX\SMALLCAPS)-style searching (third item) and a forms-based 
query (fourth item). 

<COMMENT>(....................................................................)

<COMMENT>(HTML=
<OL>
<LI> <A HREF="/script/cgi_symbols">&lt;A HREF="/script/cgi_symbols"&gt;</A>
<LI> <A HREF="/script/cgi_symbols/ht_root/doc/htd">
     &lt;A HREF="/script/cgi_symbols/ht_root/doc/htd"&gt;</A>
<LI> <A HREF="/script/cgi_symbols/ht_root/doc/htd/*.*?string1+string2">
     &lt;A HREF="/script/cgi_symbols/ht_root/doc/htd/*.*?string1+string2"&gt;</A>
<LI> <A HREF="/script/cgi_symbols/ht_root/doc/htd?FirstField=for&SecondField=this">
     &lt;A HREF="/script/cgi_symbols/ht_root/doc/htd?FirstField=for&SecondField=this"&gt;</A>
</OL>
)

<COMMENT>(HTML/OFF)
<P><BOX>(online hypertext demonstration)
<COMMENT>(HTML/ON)

<COMMENT>(....................................................................)

<HEAD>(CGI Compliant Output\head_cgi_output)

<P> Script output must behave in a CGI-compliant fashion (by way of contrast, 
see <REFERENCE>(head_non_cgi_compliance)).  That is, a CGI script may redirect 
the location of the document, using a <EMPHASIS>(Location:) header line, or 
may supply a data stream beginning with a <EMPHASIS>(Content-Type:) header 
line.  Both must be followed by a blank line. 

<P> If the script output begins with either of the these two lines HTTPd 
assumes that output will be line-oriented, without HTTP carriage-control (each 
line terminated by a carriage-return then a line-feed), and will thereafter 
ensure each record it receives is correctly terminated before passing it to 
the client.  In this way DCL procedure output (and the VMS CLI in general) is 
supported transparently. 

<COMMENT>(....................................................................)

<HEAD2>(Example DCL Scripts\head_example_dcl_scripts)

<P> A simple script to provide the system time might be:

<CODE_EXAMPLE>
  $ say = "write sys$output"
  $! the next two lines make it CGI-compliant
  $ say "Content-Type: text/plain"
  $ say ""
  $! start of plain-text script output
  $ show time
<ENDCODE_EXAMPLE>

<P> A script to provide the system time more elaborately (using HTML):

<CODE_EXAMPLE><LITERAL>
  $ say = "write sys$output"
  $! the next two lines make it CGI-compliant
  $ say "Content-Type: text/html"
  $ say ""
  $! start of HTML script output
  $ say "<HTML>"
  $ say "Hello ''WWW_REMOTE_HOST'"  !(CGI variable)
  $ say "<P>"
  $ say "System time on node ''f$getsyi("nodename")' is:"
  $ say "<H1>''f$cvtime()'</H1>"
  $ say "</HTML>"
<ENDLITERAL>
<ENDCODE_EXAMPLE>

<HEAD1>(Non-CGI Compliance Output\head_non_cgi_compliance)

<P> A script does not have to output a CGI-compliant data stream.  If it 
begins with a HTTP header status line (e.g. <QUOTE>(<EMPHASIS>(HTTP/1.0 200 
OK\SMALLCAPS))), HTTPd assumes it will supply a <EMPHASIS>(raw\BOLD) HTTP data 
stream, containing all the HTTP requirements. 

<P> Any such script must observe HyperText Transfer Protocol (for detailed 
information on HTTP see <REFERENCE>(chapter_references)).  Every line must be 
terminated by a <EMPHASIS>(carriage-return) and <EMPHASIS>(line-feed) 
(represented as <QUOTE>(<BACKSLASH>r)<QUOTE>(<BACKSLASH>n)), or as a minimum 
by a single <EMPHASIS>(line-feed).  In particular, the type of the data being 
returned by the scripts must be included in an HTTP header sent prior to the 
data itself.  Headers for the two most common data types will be illustrated 
here.  Note that the blank line is strictly necessary, it terminates the 
header. 

<HEAD>(Plain-Text\head_scripts_plain_text)

<CODE_EXAMPLE>
  HTTP/1.0 200 ok\r\n
  Content-Type: text/plain\r\n
  \r\n
<ENDCODE_EXAMPLE>

<HEAD>(HTML\head_scripts_html)

<CODE_EXAMPLE>
  HTTP/1.0 200 ok\r\n
  Content-Type: text/html\r\n
  \r\n
<ENDCODE_EXAMPLE>

<HEAD>(Non-CGI-Compliant DCL script\head_non_cgi_script)

<P> The following example show a non-CGI-compliant DCL script similar in 
function to the CGI-compliant one above.  Note the full HTTP header and each 
line explictly terminated with a carriage-return and line-feed pair. 

<CODE_EXAMPLE><LITERAL>
  $ cr[0,8] = %x0d
  $ lf[0,8] = %x0a
  $ say = "write sys$output"
  $! the next line makes it non-CGI-compliant
  $ say "HTTP/1.0 200 Time follows.''cr'''lf'"
  $ say "Content-Type: text/html''cr'''lf'"
  $ say "''cr'''lf'"
  $! start of HTML script output
  $ say "<HTML>''cr'''lf'"
  $ say "Hello ''WWW_REMOTE_HOST'''cr'''lf'"  !(CGI variable)
  $ say "<P>"
  $ say "System time on node ''f$getsyi("nodename")' is:''cr'''lf'"
  $ say "<H1>''f$cvtime()'</H1>''cr'''lf'"
  $ say "</HTML>''cr'''lf'"
<ENDLITERAL>
<ENDCODE_EXAMPLE>

<HEAD1>(Raw HTTP Input\head_raw_http_input)

<P> The logical name <EMPHASIS>(HTTP$INPUT\SMALLCAPS) defines a mailbox 
providing the raw HTTP input stream from the client.  This is available for 
procedures and executables to explictly open and read.

<P> Note that this is a raw stream, and HTTP <EMPHASIS>(lines) (carriage-
return/line-feed terminated sequences of characters) may have be blocked 
together for network transport.  These would need to be expliclty parsed by 
the program.

<P>
To make this stream implicitly available to an executable as the standard 
input stream the following DCL command should be executed immediately before 
invoking the image:

<CODE_EXAMPLE>
  DEFINE /USER_MODE SYS$INPUT HTTP$INPUT
<ENDCODE_EXAMPLE>

<COMMENT>(--------------------------------------------------------------------)

<CHAPTER>(Utilities\chapter_utilities)

<P> Foreign commands for these utilities (and the HTTPD control functionality) 
will need to be assigned in the adminstration users' LOGIN.COM, for example: 

<CODE_EXAMPLE>
  CHKMAP == "$HT_EXE:CHKMAP"
  HTTPD == "$HT_EXE:HTTPD"
  HTTPDMON == "$HT_EXE:HTTPDMON"
  WWWRKOUT == "$HT_EXE:WWWRKOUT"
<ENDCODE_EXAMPLE>

<HEAD1>(HTTPd Monitor\head_httpdmon)

<P> The HTTPd server may be monitored using the <EMPHASIS>(HTTPDMON\SMALLCAPS) 
utility.  This utility continuously displays a screen of information 
comprising three sections: 

<LIST>(NUMBERED)

<LE> <EMPHASIS>(Process Information\BOLD)
<LINE>
HTTPd process information includes its up-time, CPU-time consumed (excluding 
any subprocesses), I/O counts, and memory utilization.

<LE> <EMPHASIS>(Server Counters\BOLD)
<LINE>
The server counters keep track of the total connections received, accepted, 
rejected, etc., totals for each request type (file transfer, directory 
listing, image mapping, etc.).

<LE> <EMPHASIS>(Latest Request\BOLD)
<LINE>
This section provides the originating host, HTTP request, response status 
code, and some transaction statistics.

<ENDLIST>

<P> The following example provides sample output:

<CODE_EXAMPLE>
   Port: 80              HTTPDMON v1.0.0 AXP         Monday, 11-DEC-1995 11:12:12

   Process: HTTPd:80    PID: 2DE0143C  User: HTTP$SERVER
     Start: 1  Exit Status: n/a  (Zeroed: 0)
        Up: 0 04:05:55.47  CPU: 0 00:00:22.57
       BIO: 21493  DIO: 2760  Pg.Flts.: 3921  Pg.Used: 14%
    WsSize: 1240 (620kB)  WsPeak: 4368 (2184kB)  PeakVirt: 6848 (3424kB)
     Files: 1/100  Subprocesses: 0/50

   Connect: 19  Accept: 19  Reject: 0  Busy: 0  Current: 1  Peak: 2
     Error: 0  Parsed: 19 (1 redirect)  Forbidden: 0  RMS: 0
       GET: 16  HEAD: 2  POST: 1
      File: 13 (0)  Menu: 0 (0)  Directory: 5
     sHTML: 0  IsMap: 1  Internal: 0
    Script: 0  Auto.Script: 0  Subprocess: 0 (0)
       1xx: 0  2xx: 18  3xx: 0  4xx: 1  5xx: 0  (0 errors)  Redirect: 0
        Rx: 22806  Tx: 94757  (bytes)

      Time: 11 11:12:08  Host: beta.hfrd.dsto.gov.au
    Status: 200  Rx: 1198  Tx: 2274
   Request: GET /ht_root/doc/htd/htd_0700.html
<ENDCODE_EXAMPLE>

<P> The <EMPHASIS>(/HELP\SMALLCAPS) qualifier provides a brief usage summary.

<P> This information is, in part, obtained from the following logical names:

<LIST>(UNNUMBERED)
<LE> <EMPHASIS>(HTTPD<EMPHASIS>(port)$COUNT\BOLD)
<LE> <EMPHASIS>(HTTPD<EMPHASIS>(port)$PID\BOLD)
<LE> <EMPHASIS>(HTTPD<EMPHASIS>(port)$REQUEST\BOLD)
<ENDLIST>

<P> The server counter values are carried over when a server (re)starts 
(provided the system has stayed up).  To reset the counters use the following 
command on the server system: 

<CODE_EXAMPLE>
  $ HTTPD /DO=ZERO
<ENDCODE_EXAMPLE>

<COMMENT>(....................................................................)

<HEAD1>(Mapping Rule Checker\head_chkmap)

<P> The mapping rule database may be verified using the 
<EMPHASIS>(CHKMAP\SMALLCAPS) utility.  This utility will load the rule 
database, reporting on any rules that have an incorrect format, and then 
optionally map a URL format path to a VMS file specification, or reverse-map a 
VMS file specification to a URL format, or report an error!  A script path 
will be mapped into script and derived path components.  Various rules may be 
checked from the command-line for correct mapping to and from VMS 
specifications before committing them to a working server. 

<P> By default the utility will load any defined rule mapping database (i.e. 
HTTPD$MAP) via the appropriate logical name.  If a specific, non-default rule 
file is required it can be specified using the <EMPHASIS>(/MAP=\SMALLCAPS) 
qualifier. 

<P> URL format paths begin with, and contain forward-slashes.  For this 
reason, when being specified on the command line, must be enclosed within 
double-quotation marks.  This does not apply to VMS file specifications.

<CODE_EXAMPLE>
   $ CHKMAP
   $ CHKMAP /MAP=TEST.CONF
   $ CHKMAP "/hyperdata/home.html"
   $ CHKMAP USER$DISK:[DANIELM.WWW]HOME.HTML
<ENDCODE_EXAMPLE>

<P> The <EMPHASIS>(/HELP\SMALLCAPS) qualifier provides a brief usage summary.

<COMMENT>(....................................................................)

<HEAD1>(Server Workout <EMPHASIS>((stress-test))\head_wwwrkout)

<P> The <EMPHASIS>(WWWRKOUT\SMALLCAPS) (<QUOTE>(World Wide Web Workout)) 
utility exercises an HTTP server, both in the number of simultaneous 
connections maintained and in the number of back-to-back sequential connection 
requests and data transfers. 

<P> This utility can be used to <EMPHASIS>(stress-test) the HFRD VMS HTTP 
server (or any other), or to make comparisons between it and other servers.

<P> It sets up and maintains a specified number of simultaneous connections to 
a server.  It reads a buffer of data from each connection in turn, where data 
is waiting (does not block), until the document transfer is complete and the 
connection closed by the server.  It then closes the local end and immediately 
reuses the now-free socket to initiate another sequence.  If enabled (it is by 
default), the utility attempts to reflect the real-world in varying the data 
transfer rate for each connection, by setting the number of bytes read during 
each read loop differently for each connection.  All transfered data is 
discarded. 

<P> The data transfer rate for each connection is displayed at connection 
close.  It is by default it is the effective transfer rate, that is the rate 
from opening the connection to closing it, and so includes request processing 
time, etc.  If the <EMPHASIS>(/NOEFFECTIVE\SMALLCAPS) qualifier is employed 
it measures the document data transfer rate only. 

<P> Although a single document path may be specified on the command line it 
is preferable to supply a range of document paths, one per line in a plain 
text file.  Each document path and/or type specified should be different to 
the others, to exercise the server and file system cache.  Any number of paths 
may be specified in the file.  If the file is exhausted before the specified 
number of connections have been established the file contents are recycled 
from the first path.  If a path or a file of paths is not specified the 
utility just continually requests the welcome document. 

<P> To assess a server's total throughput choose paths that lead to large 
documents (<LITERAL>(>) 50K), where the overhead of connection setup, rule
processing and  transfer initiation are less significant than the data transfer
itself.  The  buffer size variation functionality should be disabled using the 
<EMPHASIS>(/NOVARY\SMALLCAPS) qualifier when assessing data transfer rates. 
Responsiveness is better assessed using small documents (<LITERAL>(<) 2K),
where the  overhead of the non-data-transfer activities is more significant.

<P><EMPHASIS>(WWWRKOUT [server_host_name[:port]] [path] [qualifiers]\BOLD)

<LIST>(UNNUMBERED)

<LE> <EMPHASIS>(/BUFFER=\BOLD) number of bytes to be read from server each 
time (default is 1024, will be modified by the default 
<EMPHASIS>(/VARY\SMALLCAPS) qualifier) 

<LE> <EMPHASIS>(/COUNT=\BOLD) total number of connections and requests to be 
done (default is 100)

<LE> <EMPHASIS>(/[NO]EFFECTIVE\BOLD) measures data transfer rate from request 
to close (if <EMPHASIS>(/NOEFFECTIVE\SMALLCAPS) is applied the rate is 
measured during data transfer only

<LE> <EMPHASIS>(/FILEOF=\BOLD) file name containing paths to documents

<LE> <EMPHASIS>(/HELP\BOLD) display brief usage information

<LE> <EMPHASIS>(/OUTPUT=\BOLD) file name for output

<LE> <EMPHASIS>(/PATH=\BOLD) single path to document to be retrieved

<LE> <EMPHASIS>(/PORT=\BOLD) IP port number of HTTP server (default is 80)

<LE> <EMPHASIS>(/SERVER=\BOLD) HTTP server host name

<LE> <EMPHASIS>(/SIMULTANEOUS=\BOLD) number of simultaneous connections to be 
set up at any one time (default is 10)

<LE> <EMPHASIS>(/[NO]VARY\BOLD) varies the size of the read buffer for each 
connection (default is vary)

<ENDLIST>

<P> Examples:

<CODE_EXAMPLE>
  $ WWWRKOUT
  $ WWWRKOUT www.server.host "/hyperdata/home.html"
  $ WWWRKOUT www.server.host:8080 /FILEOF=PATHS.TXT
  $ WWWRKOUT /SERVER=www.server.host /PORT=8080 /FILEOF=PATHS.TXT
  $ WWWRKOUT www.server.host:8080 /FILEOF=PATHS.TXT /NOEFFECTIVE /NOVARY
  $ WWWRKOUT www.server.host /FILEOF=PATHS.TXT /COUNT=500 /SIMUL=20 /BUFFER=512
<ENDCODE_EXAMPLE>

<P> The <EMPHASIS>(/HELP\SMALLCAPS) qualifier provides a brief usage summary.

<COMMENT>(--------------------------------------------------------------------)

<CHAPTER>(Organizing Data Areas\chapter_data_areas)

<P> This section deals with the provision of physical file space for HFRD 
hypertext-related data, and the integration of that within the logical 
organization of the HFRD hypertext data. 

<P> The logical organisation of served data is largely hierarchical, and is 
achieved via two mechanisms. 

<LIST>(NUMBERED)

<LE> The natural hierachy provided by a hierarchical file system.

<LE> The logical hierarchy possible using rules within the mapping file to 
place disparate physical areas into a single logical structure (see 
<REFERENCE>(chapter_mapping_rule)). 

<ENDLIST>

<P> HFRD has a single section of the file system for the core hypertext 
environment files, such as the Division home page, help areas, documentation, 
etc., it can be accessed using the logical area 
<EMPHASIS>(HYPERDATA:[000000]\SMALLCAPS).  Physically integrated with this are 
directories providing data storage for specific groupings of data, such as 
CEAC minutes, adminstration announcements, etc.  Physically distinct areas are 
also logical grouped into the hypertext environment, areas such as the IE 
group area, JFAS minutes, etc. 

<P> The reason for some areas being physically distinct are for legitimate 
physical reasons (e.g. the area can best be hosted on a group-local disk), for 
historical reasons (e.g. the area existed before any hypertext environment 
existed) or for reasons of convenience (e.g. lets put this where access 
controls already allow the maintainers to manage it). 

<P> The reasons for an area being physically integrated with the core 
hypertext data area can be legitimate (e.g. there is really nowhere else it 
reasonably belongs), convenience (e.g. lets quickly put it here) or by logical 
necessity (it really does below as part of the core hypertext enviroment). 

<HEAD>(Guidelines\head_data_area_guidelines)

<P> In general, only hypertext enviroment files need to go into the core 
physical hypertext file system.  All other groupings should, if possible, be 
decentralised into the portion of the file system they represent and logically 
placed in the hypertext area using rules in the mapping file.  That is, a 
given project's hypertext files should be located in the project's part of the 
file system.  If it doesn't have any then it may be a candidate for location 
in the core area. 

<P> When locating a hypertext area in a physically distinct area it is 
possible the managers of that data will already have the correct access 
controls.  If locating an area in the core hypertext environment it will be 
necessary to give the manager ownership of the directory area, or provide ACL 
access if multiple managers are involved. 

<P> When locating a hypertext area in a physically distinct area it will be 
necessary to update the mapping file with a new rule (see 
<REFERENCE>(chapter_mapping_rule)).  If located within the core hypertext area 
the rules do not need to be adjusted. 

<COMMENT>(--------------------------------------------------------------------)

<CHAPTER>(Brief Introduction to HTTPd Code\chapter_httpd_code)

<P> <EMPHASIS>(This section is designed to be only a broad overview of the 
basic functionality of the HTTPd server.  It also does not cover the full 
suite of HFRD VMS Hypertext Services software.) 

<COMMENT>(....................................................................)

<COMMENT>(HTML=
<I>The <A HREF="/ht_root/src/httpd/*.*">source code</A> should also be 
consulted.</I>
) 

<COMMENT>(HTML/OFF)
<EMPHASIS>(The source code should also be consulted.)
<P><BOX>(online hypertext link)
<COMMENT>(HTML/ON)

<COMMENT>(....................................................................)

<HEAD>(Multi-Threaded\head_multithreaded)

<P> The HFRD HTTPd is written to exploit VMS operating system characteristics 
allowing the straight-forward implementation of multi-threaded code.  The 
server is written to be I/O event driven.  Asynchronous System Traps (ASTs), 
or software interrupts, at the conclusion of an I/O (or other) event allow 
functions to be activated to post-process the event.  The event traps are 
automatically queued on a FIFO basis, allowing a series of events to be 
sequentially processed.  When not responding to an event the process is 
quiescent, or otherwise occupied, effectively interleaving I/O and processing, 
and allowing a sophisticated client multi-threading.

<P> When VMS supports kernel-threads (beginning v7.0 I believe) this may be 
enhanced to optionally support multiple threads over multiple CPUs within the 
one process, further extending server throughput. 

<P> Multi-threaded code is inherently more complex than single-threaded code, 
and there are issues involved in the synchronization of some activities in 
such an environment.  Fortunately VMS handles many of these issues internally. 
After connection acceptance, all of the processing done within HTTPd is at 
<EMPHASIS>(USER\SMALLCAPS) mode AST delivery level, and for all intents and 
purposes the processing done therein is <EMPHASIS>(atomic), implictly handling 
its own synchronization issues. 

<P> HTTPd is written to make longer duration activities, such as the transfer 
of a file's contents, event-driven.  Other, shorter duration activites, such 
as accepting a client connection request, are handled synchronously. 

<P> It is worth noting that with asynchronous, and AST-driven output, the 
data being written must be guaranteed to exist without modification for the 
duration of the write (until completion AST delivery).  This means data 
written must be static or in buffers that persist with the thread.
Function-local (automatic) storage cannot be used.  The HTTPd server allocates 
dynamic storage for general (e.g. output buffering) or specific (e.g. response 
headers) uses. 

<HEAD>(Tasks\head_tasks)

<P> Each request can have one or more <EMPHASIS>(tasks) executed sequentially 
to fullfil the request.  This occurs most obviously with the HTML
pre-processor, but also, to a more limited extent, with directory listing and 
its <EMPHASIS>(read-me) file inclusion.  A <EMPHASIS>(task) is defined as one 
of: 

<LIST>(UNNUMBERED)
<LE> send file
<LE> send menu
<LE> directory listing
<LE> DCL execution
<ENDLIST>

<CP> Each one of these modules executes relatively independently.  Before 
commencing a task, a next-task pointer can be set to the function required to 
execute at the conclusion of that task.  At that conclusion, the next-task 
functionality checks for a specified task to start or continue.  If it has 
been specified control is passed to that next-task function via an AST. 

<HEAD>(Memory Management\head_memory)

<P> Per-Thread memory is managed as two distinct portions.

<LIST>(NUMBERED)

<LE> A fixed-size structure of dynamic memory is used to contain the core 
request thread data.  This is released at thread disposal.

<COMMENT>(HTML=<P>)

<LE> A <EMPHASIS>(heap) of dynamically allocated memory is maintained during 
the life of a thread structure.

<P> When a dynamic structure is required this heap is expanded by 
<EMPHASIS>(calloc())ing memory, placing this in a double-linked list 
structure, and returning a pointer to the usable portion of the newly 
allocated memory.  This list is released in one operation at thread 
disposal, by traversing the list and <EMPHASIS>(free())ing each individual 
chunk (making it easier to avoid the memory leaks associated with making 
autononmous allocations for each dynamic memory structure required).

<ENDLIST>

<HEAD>(Output Buffering\head_buffer_output)

<P> To reduce the number of individual network writes, and thus provide 
significant improvements in efficiency, output generated from all modules 
except File(), is buffered into larger packets before sending to the client.  
All modules, including File(), work to implement a seamless integration of 
output via this mechanism (best seen in the Shtml() module). 

<P> The AST-driven nature of the server means this functionality is moderately 
complex.  A form of double buffering is employed, allowing the buffer to 
overflow and be flushed to the network asynchronously, without overwriting, 
losing or needing to re-request data.  Two buffer spaces are employed.  When 
one fills it is written to the network and a pointer to the two areas 
exchanged, allowing the supplied (and overflowed) data to be immediately 
buffered (without a synchronous wait for the network write to complete), and 
an immediate return to further AST-driven processing.  The alternate buffer 
continues to be used until it fills, when the process is repeated. 

<P> The possibility of an asynchronous write with every buffered output 
introduces complexity.  Every buffered output call must be used as if it is an 
asynchronous network write, an AST function address supplied with every call 
on the off-chance (and eventuality) that an actual network write will occur.  
If a network write does not occur (most of the time) the AST is explicitly 
declared for delivery.  This need to supply an AST function with every 
buffered write, basically means <EMPHASIS>(only one buffered write may occur 
per AST-executed function\BOLD). 

<HEAD>(Rule-Mapping\head_rule_mapping)

<P> A fundamental aspect of any HTTPd implementation is the rule mapping used 
to create a logical structure for the hypertext file system.  The HTTPd 
mapping function is designed to be flexible enough that script programs can 
also use it.  As a result it is text-file based, and opened and read when 
mapping.   This method of mapping provides a good deal of flexibility, coupled 
with acceptable performance.  The function has received a high level of 
attention in an effort to optimize it. 

<HEAD>(Auto-Scripting\head_autoscripting)

<P> The HFRD VMS HTTP server has the facility to automatically invoke a script 
to process a non-HTML document (file).  This facility is based on detecting 
the MIME content data type (via the file's extension) and causing a 
transparent, local redirection, invoking the script as if it was specified in 
the original request.

<HEAD>(Internal Directives\head_httpd_internal_directives)

<P> The HTTPd server detects certain strings as directives about its 
behaviour.  These directives are passed in the query string component of the 
request, and as reserved sequences cannot occur in normal requests (an 
unlikely combination of characters has been selected). 

<LIST>(UNNUMBERED)

<LE> <EMPHASIS>(?httpd=report\BOLD)
<LINE> Returns an HTML format report on server status and activity (see 
<RFERENCE>(head_httpdmon)).

<LE> <EMPHASIS>(?httpd=index\BOLD)
<LINE> Controls aspects of the directory listing module's behaviour:

<LIST>(UNNUMBERED)
<LE> &autoscript=no
<LE> &format=vms
<LE> &layout=<EMPHASIS>(string)
<LE> &readme=no
<LE> &script=<EMPHASIS>(string)
<ENDLIST>

<ENDLIST>

<HEAD>(HTTPd Modules\head_httpd_modules)

<P> The HTTPd server comprises eight main modules, implementing the obvious 
functionality of the server, and other, smaller, support modules. 

<LIST>(UNNUMBERED)
<LE> HTTPD.C (<REFERENCE>(head_httpd_c))
<LE> REQUEST.C (<REFERENCE>(head_request_c))
<LE> FILE.C (<REFERENCE>(head_file_c))
<LE> MENU.C (<REFERENCE>(head_menu_c))
<LE> DIR.C (<REFERENCE>(head_dir_c))
<LE> DCL.C (<REFERENCE>(head_dcl_c)
<LE> SHTML.C (<REFERENCE>(head_shtml_c))
<LE> ISMAP.C (<REFERENCE>(head_ismap_c))
<LE> LOGGING.C (<REFERENCE>(head_logging_c))
<ENDLIST>

<COMMENT>(--------------------------------------------------------------------)

<HEAD1>(HTTPD.C\head_httpd_c)

<COMMENT>(....................................................................)

<COMMENT>(HTML=Code for: <A HREF="/ht_root/src/httpd/HTTPd.c"><I>HTTPd.c</I></A>)

<COMMENT>(HTML/OFF)
<P><BOX>(online hypertext link)
<COMMENT>(HTML/ON)

<COMMENT>(....................................................................)

<P> This is the main module of the server.  It handles all TCP/IP network 
activites, from creating the server socket and listening on the port, to 
reading and writing network I/O.  The network read and write functions have 
provision for specifying I/O completion AST function addresses.  If these are 
provided then the function is called upon completion of the network I/O.  If 
not provided then the I/O completes without calling an AST routine. 

<P> The server begins by creating a network socket and then binding that to 
the HTTP port.  The server then enters an infinite loop, waiting for IP 
connections.

<P> When a connection request is received the remote host is checked as an 
allowed connection.  If allowed, a request data structure is created from 
dynamic memory, and an asynchronous read is queued from the network client.  
The pointer to this dynamic data structure becomes the request 
<EMPHASIS>(thread), and is passed from function to function, AST routine to 
AST routine.  The AST completion routine of the network read specifies a 
request analysis function.  The function then returns to the connection 
acceptance loop. 

<P> When the network read completes an AST completion function in the 
Request() module is called to process the HTTP request.

<COMMENT>(--------------------------------------------------------------------)

<HEAD1>(REQUEST.C\head_request_c)

<COMMENT>(....................................................................)

<COMMENT>(HTML=
Code for:<A HREF="/ht_root/src/httpd/Request.c"><I>Request.c</I></A>
)

<COMMENT>(HTML/OFF)
<P><BOX>(online hypertext link)
<COMMENT>(HTML/ON)

<COMMENT>(....................................................................)

<P> This module reads the request header from the client, parses this, and 
then calls the appropriate task function to execute the request (i.e. send a 
file, pre-process an HTML file, generate a directory listing, execute a 
script, etc.)

<P> The request header is contained in the network read buffer.  If it cannot
be completely read in the first chunk, the read buffer is dynamically expanded 
so as to be read in multiple chunks.  The request header is addressed by a 
specific pointer that allows the parse-and-execute function to process either 
a genuine, initial client request header, or a pseudo-header generated to 
effect a redirection. 

<P> The method, path information and query string are parsed from the first 
line of the header.  Other, specific request header fields are also parsed out 
and stored for later reference.  Once this has been done the header is not 
further used.

<P> Once the relevant information is obtained from the request header 
processing commences on implementing the request.  This comprsises the
rule-mapping of any path information, the RMS parsing of any resulting VMS 
file specification and decision-making on how to execute the request.

<LIST>(UNNUMBERED)

<LE> If an internal directive that is executed.

<LE> If the content-type of a supplied file specification is auto-scripting, 
an automatic redirection is generated.

<LE> If a file specification and no wildcards, the file is sent to the client.

<LE> If a wildcarded file specification, and no query string, a directory 
listing is generated.

<LE> If a directory specification (no file name), one of multiple, possible, 
home pages are attempted to be sent.  If no home page is found in the 
directory then a directory listing is generated. 

<LE> If a script specification prefixed the path information that script is 
executed.

<LE> If a query string is supplied (and it is not a server directive), and no 
script name was included in the path information, the server query script is 
automatically activated.

<ENDLIST>

<P> This functionality is used to parse and execute both the initial client 
request and any pseudo-request generated internally to effect a redirection.

<COMMENT>(--------------------------------------------------------------------)

<HEAD1>(FILE.C\head_file_c)

<COMMENT>(....................................................................)

<COMMENT>(HTML=Code for: <A HREF="/ht_root/src/httpd/File.c"><I>File.c</I></A>) 

<COMMENT>(HTML/OFF)
<P><BOX>(online hypertext link)
<COMMENT>(HTML/ON)

<COMMENT>(....................................................................)

<P> This module implements the file transfer functionality.  It obtains the 
file specification, mime content type and encoding (binary/text) information 
from the request data structure.  It handles record-oriented (text) files 
slightly differently to binary (e.g. image) files (specified using the 
<EMPHASIS>(AddType) configuration directive, see <REFERENCE>(chapter_config)).  
Record-oriented files will have multiple records buffered before writing them 
collectively to the network (improving efficiency).  Binary file reads are by 
Virtual Block, and are written to the network immediately.  The essential 
behaviour however is much the same. 

<LIST>(NUMBERED) 

<LE> The primary function attempts to open the file.  If unsuccessful it 
immediately returns the error status to the calling routine for further action 
(this behaviour is used to try each of multiple home pages by detecting file-
not-found, for example).

<COMMENT>(HTML=<P>)

<LE> After successfully opening the file it generates an HTTP response header 
if required.  It then calls one of either two functions to queue the first 
read from the file, one for text files (record-oriented transfer), another for 
binary files (block-oriented transfer).  After the read is queued it returns 
with a success status code. 

<COMMENT>(HTML=<P>)

<LE> When the asynchronous file read completes one of either two AST 
completion functions (one for text the other for binary) is called to
post-process the I/O.  Status is checked for success or otherwise.  If an 
error the status is reported to the client, the file closed, and the request 
thread concluded. 

<P> <EMPHASIS>(If end-of-file\BOLD), the file is closed, for record-oriented 
(text) files the buffer checked and if necessary flushed.  If an end task 
function was specified control is now passed to that, otherwise the thread is 
concluded. 

<P> <EMPHASIS>(If not end-of-file, for text files\BOLD) multiple records may be 
buffered before writing to the network.  If the buffer is full (the read was 
unsuccessful due to insufficient space) the contents are asynchronously 
written to the network, with the network write completion routine specifying a 
function to re-read the the file record that just failed.  If there is still 
space in the buffer another asynchronous read of the file is queued in an 
attempt to append the next record into the buffer.  After the read is queued 
the function completes. 

<P> <EMPHASIS>(If not end-of-file, for binary files\BOLD) a successful read 
results in a call to the network write function to send this to the client.  
This call contains the address of the function to read the next blocks from 
the file as an AST completion routine.  After the asynchronous network write 
is queued the function completes. 

<ENDLIST>

<P> For text files the contents can be encapsulated as plain text.  This 
involves prefixing the file send with a <EMPHASIS>(<LITERAL>(<PRE>)\SMALLCAPS) 
HTML tag and postfixing it with a <EMPHASIS>(<LITERAL>(</PRE>)\SMALLCAPS) tag.  
The buffer is filled as per normal but when ready to output a function is 
called that escapes all HTML-forbidden characters first (e.g. 
<QUOTE>(<LITERAL>(<)), <QUOTE>(<LITERAL>(>)), <QUOTE>(<LITERAL>(&)), etc.) 

<COMMENT>(--------------------------------------------------------------------)

<HEAD1>(MENU.C\head_menu_c)

<COMMENT>(....................................................................)

<COMMENT>(HTML=Code for: <A HREF="/ht_root/src/httpd/Menu.c"><I>Menu.c</I></A>)

<COMMENT>(HTML/OFF)
<P><BOX>(online hypertext link)
<COMMENT>(HTML/ON)

<COMMENT>(....................................................................)

<P> This module implements the HFRD menu interpretation functionality.  It 
obtains the file specification from the request data structure.  Output from 
this module is buffered to reduce network writes, improving I/O efficiency. 

<P> Essential behaviour: 

<LIST>(NUMBERED) 

<LE> The primary function attempts to open the file.  If unsuccessful it 
immediately returns the error status to the calling routine for further action 
(this behaviour is used to try multiple home pages, for example).

<COMMENT>(HTML=<P>)

<LE> After successfully opening the file it generates an HTTP response header 
if required.  A call is then made to asynchronously read a record from the 
file opened.  This call contains the address of a function to count the number 
of menu sections (blank-line delimited groups of lines) in the file.  After 
the asynchronous file read is queued the function returns with a success 
status code.

<COMMENT>(HTML=<P>)

<LE> When the asynchronous file read completes the AST completion function is 
called to count the sections.  Status is checked for success or otherwise.  If 
an error the status is reported to the client, the file closed, and the 
processing concluded. 

<P> If end-of-file, or up to three sections counted, the file is repositioned 
to the start and then another asynchronous file read is queued (starting with 
the first record again), with the AST completion routine specified as the menu 
interpretation function.  If still counting sections the completion AST 
routine specified is the same section counting function.  After the 
asynchronous file read is queued the function completes.

<COMMENT>(HTML=<P>)

<LE> When the asynchronous file read completes the AST completion function is 
called to interpret the line, dependant on the section number it occurs in.  
Status is checked for success or otherwise.  If an error the status is 
reported to the client, the file closed, and the request concluded.  If
end-of-file, the file is closed and the processing concluded.  For a 
successful record read the line can either be title, description or menu item.  
When the line is interpreted and written to the network another read is 
queued, with an AST completion routine again specifying the contents 
interpretation function.  The function then completes. 

<ENDLIST>

<COMMENT>(--------------------------------------------------------------------)

<HEAD1>(DIR.C\head_dir_c)

<COMMENT>(....................................................................)

<COMMENT>(HTML=Code for: <A HREF="/ht_root/src/httpd/Dir.c"><I>Dir.c</I></A>)

<COMMENT>(HTML/OFF)
<P><BOX>(online hypertext link)
<COMMENT>(HTML/ON)

<COMMENT>(....................................................................)

<P> This module implements the HTTPd directory listing functionality.  
Directories are listed first, then files.  File detail format customizable, 
with the default resembling the default CERN and NCSA server layout.  Output 
from this module is buffered to reduce network writes, improving I/O 
efficiency.  HTML files have the <LITERAL>(<TITLE></TITLE>) element extracted 
as a <QUOTE>(Description) item. 

<P> Essential behaviour: 

<LIST>(NUMBERED) 

<LE> The primary function obtains the file specification from the request data 
structure.  Server directives, controlling some features of the directory 
listing beahaviour, are checked for and parsed out if present.  The directory 
listing layout is initialized.  The directory specification (path information) 
is parsed to obtain the directory and file name/type components.  After 
successfully parsing the specification it generates an HTTP response header if 
required. 

<COMMENT>(HTML=<P>)

<LE> Column headings and (possibly) a parent directory item are buffered in an 
asynchronous function call.  An RMS structure is initialized to allow the 
asynchronous search for all files in the specified directory ending in ".DIR".  

<COMMENT>(HTML=<P>)

<LE> For each directory file found the directory search AST completion 
function is called.  Status is checked for success or otherwise.  If an error 
the status is reported to the client and the request processing concluded.  If 
the directory contained no directory files, or the directory files are 
exhausted a call to a function to begin a listing of non-directory files is 
made and the function then completes. 

<COMMENT>(HTML=<P>)

<P> If a directory file was returned a synchronous call to list the details of 
that directory is made and then another asynchronous search call made with an 
AST completion function again back to this function.

<COMMENT>(HTML=<P>)

<LE> When the directory files are exhausted the RMS structure is reinitialized 
to allow the search for all specification-matching, non-directory files in the 
directory.  An asynchronous search call is made.

<COMMENT>(HTML=<P>)

<LE> For each matching file found the file search AST completion function is 
called.  Status is checked for success or otherwise.  If an error the status 
is reported to the client and the processing concluded.  If the directory 
contained no matching files, or the files are exhausted, the processing is 
concluded and the function immediately completes. 

<P> If a file was returned a call is made to a function to check whether a 
file description can be obtained (HTML files only).  If it can then a function 
to initiate this is called and the function completes.  If no description can 
be obtained a synchronous call is made to a function to list the file details.  
After the file details are listed another asynchronous search call is made, 
with the same function specified for AST completion.  The function then 
immediately completes.

<COMMENT>(HTML=<P>)

<LE> To asynchronously locate a description in an HTML file, the file is 
opened and then each record asynchronously read and examined for the 
<LITERAL>(<TITLE>) element.   Once obtained a synchronous call is made to a 
function to list the file details.  After the file details are listed another 
asynchronous search call is made, with the file search function specified for 
AST completion.  The function then immediately completes.  

<ENDLIST> 

<COMMENT>(--------------------------------------------------------------------)

<HEAD1>(DCL.C\head_dcl_c)

<COMMENT>(....................................................................)

<COMMENT>(HTML=Code for: <A HREF="/ht_root/src/httpd/DCL.c"><I>DCL.c</I></A>)

<COMMENT>(HTML/OFF)
<P><BOX>(online hypertext link)
<COMMENT>(HTML/ON)

<COMMENT>(....................................................................)

<P> The DCL execution functionality must interface and coordinate with an 
external subprocess.  It too is asynchronously driven by I/O once the 
subprocess has been created and is executing independently.  Communication 
with the subprocess is via mailboxes.

<P> The DCL facility is used is two modes:

<LIST>(NUMBERED)

<LE> <EMPHASIS>(To execute independent DCL commands.\BOLD)
This is used to provide DCL output for pre-processed HTML.  In this mode 
multiple DCL commands may be executed within the one request. 

<LE> <EMPHASIS>(To execute CGI scripts.\BOLD)
In this mode only one DCL command is executed during the request.

<ENDLIST>

<P> DCL related structures and devices (e.g. mailboxes) are retained for 
life of the request, and may be reused if multiple commands are required.  
This reduces the overhead of DCL execution.

<LIST>(NUMBERED) 

<LE> The primary DCL function ensures any required file specification exists 
(e.g. script procedure).  The first time it is executed during an individual 
request it creates two or three mailboxes: 

<LIST>(NUMBERED)

<LE> for the subprocess' <EMPHASIS>(SYS$INPUT\SMALLCAPS) 

<LE> for the subprocess' <EMPHASIS>(SYS$OUTPUT\SMALLCAPS)

<LE> if CGI-script execution, available for the subprocess to explicitly open 
providing direct read access to the HTTP data stream (this could be done with 
a <EMPHASIS>(DEFINE /USER SYS$INPUT HTTPD$INPUT\SMALLCAPS)) 

<ENDLIST> 

<CP> A function writes to the <EMPHASIS>(SYS$INPUT\SMALLCAPS), creating a 
number of CGI-compliant symbol names and executing the command or invoking the 
execution of a DCL procedure, etc.

<P> A subprocess is spawned.  Input and output are then I/O AST completion 
routine driven, and the primary function returns to the calling routine. 

<COMMENT>(HTML=<P>)

<LE> When the subprocess writes to the <EMPHASIS>(SYS$OUTPUT\SMALLCAPS) stream 
the I/O completion AST routine associated with reading that mailbox is called.  

<P> If CGI-script execution, the first I/O from this stream is analyzed for 
CGI-compliance.  It is determined whether a raw HTTP data stream will be 
supplied by the script, or whether the script will be CGI-compliant (requiring 
the addition of HTTP header, etc.) and whether HTTP carriage-control needs to 
be checked/added for each record. 

<P> A CGI local redirection header (partial URL) is a special case.  When this 
is received <EMPHASIS>(all\BOLD) output from the subprocess is suppressed 
until the script processing is ready to be concluded.  At that time the 
<QUOTE>(Location:) information of the header is used to reinitiate the 
request, using the same thread data structure. 

<P> When normal <EMPHASIS>(SYS$OUTPUT\SMALLCAPS) processing is complete the 
record received can be handled in one of two ways.  If it is raw HTTP it is 
asynchronously written to the network.  The AST completion routine specified 
with the network write will queue another read from subprocess' 
<EMPHASIS>(SYS$OUTPUT\SMALLCAPS).  If it is record-oriented I/O (e.g. from DCL 
output), it is output buffered, into larger chunks, before these multiple 
records are written to the network at the one time.  This improves I/O 
efficiency. 

<P> The <EMPHASIS>(SYS$OUTPUT\SMALLCAPS) stream is a little problematic.  At 
subprocess exit there may be one or more records waiting in the mailbox for 
reading and subsequent writing to the client over the network, delaying 
processing conclusion.  Detection of completion is accomplished by making each 
QIO sensitive to mailbox status via the SS$_NOWRITER status, which indicates 
there is no channel assigned to the mailbox, and the mailbox buffer is empty.  
It then becomes safe to dispose of the client thread without loss of data. 

<COMMENT>(HTML=<P>)

<LE> If CGI-script execution, The HTTP data stream made available is also AST 
driven.  If the subprocess opens the stream and reads from it, the I/O 
completion routine called queues another asynchronous read from the network 
client. 

<ENDLIST>

<COMMENT>(--------------------------------------------------------------------)

<HEAD1>(SHTML.C\head_shtml_c)

<COMMENT>(....................................................................)

<COMMENT>(HTML=Code for: <A HREF="/ht_root/src/httpd/sHTML.c"><I>sHTML.c</I></A>)

<COMMENT>(HTML/OFF)
<P><BOX>(online hypertext link)
<COMMENT>(HTML/ON)

<COMMENT>(....................................................................)

<P> The HTML pre-processor module provides this functionality as an integrated 
part of the server.  Output from this module is buffered to reduce network 
writes, improving I/O efficiency.

<P> Essential behaviour: 

<LIST>(NUMBERED) 

<LE> The primary function attempts to open the file.  If unsuccessful it 
immediately returns the error status to the calling routine for further action 
(this behaviour is used to try multiple home pages, for example).

<COMMENT>(HTML=<P>)

<LE> After successfully opening the file it generates an HTTP response header 
if required.  A call is then made to asynchronously read a record from the 
file opened.  The record read AST function scans the record (line) looking for
pre-processor directives embedded in HTML comment directives.  If no directive 
is found the record is output buffered and another queued to be read.

<COMMENT>(HTML=<P>)

<LE> If a directive is detected any part of the line up the directive is 
output buffered and a function called to parse the directive.  This function 
reports an error if the directive specified is not supported (unknown, etc.) 
If a supported directive a specific function is called according to the 
directive specified.  These functions provide the pre-processor information in 
one of four ways: 

<LIST>(NUMBERED)

<COMMENT>(HTML=<P>)

<LE> <EMPHASIS>(Internally\BOLD)

<P> Information such as the system time, current document information, etc., 
can be provided from information contained in the request data, etc., or in 
the case of specified document/file information obtained via the file system. 
These directives have the relevant information buffered and then the function 
returns to the directive parsing function. 

<COMMENT>(HTML=<P>)

<LE> <EMPHASIS>(Via DCL Execution\BOLD)

<P> Information that must be obtained thorugh DCL execution is obtained using 
an asynchronous call to the Dcl() module.  The <EMPHASIS>(next-task) function 
is specified as the line parsing function.  When the DCL module has finished 
executing the required command control is passed back to this function. 

<COMMENT>(HTML=<P>)

<LE> <EMPHASIS>(Sending a File\BOLD)

<P> If a file is <EMPHASIS>(#include)d this is provided with an asynchronous 
call to the File() module.  The <EMPHASIS>(next-task) function is specified as 
the line parsing function.  When the File() module has finished transfering 
the included file control is passed back to this function. 

<COMMENT>(HTML=<P>)

<LE> <EMPHASIS>(Directory Listing\BOLD)

<P> If a directory listing is requested this is provided via an asynchronous 
call to the Dir() module.  The <EMPHASIS>(next-task) function is specified as 
the line parsing function.  When the Dir() module has finished generating the 
listing control is passed back to this function. 

<ENDLIST>

<COMMENT>(HTML=<P>)

<LE> Directives continue to be parsed, and executed, asynchronously if 
necessary (as just described), from within a line until the end-of-line is 
reached.  Any remaining characters are output buffered.  Lines continue to be 
read from the file using the AST mechanism until end-of-file. 

<ENDLIST>

<COMMENT>(--------------------------------------------------------------------)

<HEAD1>(ISMAP.C\head_ismap_c)

<COMMENT>(....................................................................)

<COMMENT>(HTML=Code for: <A HREF="/ht_root/src/httpd/IsMap.c"><I>IsMap.c</I></A>)

<COMMENT>(HTML/OFF)
<P><BOX>(online hypertext link)
<COMMENT>(HTML/ON)

<COMMENT>(....................................................................)

<P> The clickable-image support module provides this functionality as an 
integrated part of the server.  It supports image configuration file 
directives in either of NCSA or CERN formats.  Extensive configuration 
specification error reporting has been implemented.

<NOTE>(Acknowlegement:)

<P> Three coordinate mapping functions have been plagiarized from the NCSA 
<EMPHASIS>(imagemap.c\SMALLCAPS) script program.  These have been inserted 
unaltered in the module and an infrastructure built around the essential 
processing they provide.  Due acknowlegement to the original authors and 
maintainers of that application.  Any copyright over portions of that code is 
also acknowleged: 

<CODE_EXAMPLE>
  ** mapper 1.2
  ** 7/26/93 Kevin Hughes, kevinh@pulua.hcc.hawaii.edu
  ** "macmartinized" polygon code copyright 1992 by Eric Haines, erich@eye.com
<ENDCODE_EXAMPLE>

<ENDNOTE>

<P> Essential behaviour: 

<LIST>(NUMBERED) 

<LE> The primary function attempts to open the configuration file.  If 
unsuccessful it generates an error report and concludes processing.

<COMMENT>(HTML=<P>)

<LE> After successfully opening the configuration file it extracts the
client-supplied coordinate from the query string.  A call is then made to 
asynchronously read a record (line) from the configuration file.  
Configuration file processing is asynchronous from that point. 

<COMMENT>(HTML=<P>)

<LE> The record (line) read AST function checks for end-of-file, when it will 
return the <EMPHASIS>(default) URL (if supplied).  After end-of-file the 
file is closed and the processing is concluded.

<P> If not end-of-file, a function is called to parse the record for an image 
mapping directive.  When the components have been parsed the NCSA 
<EMPHASIS>(imagemap.c\SMALLCAPS) routines are used to determine if the click 
coordinates are within the specified region coordinates.

<P> If it is within the region the click has been mapped and the URL is placed 
in heap memory and the thread's redirection location pointer set to it.  The 
file is closed and the processing conclusion function called.  This function 
detects the redirection location and if a local URL instead of disposing of 
the thread generates a new, internal request from the redirection information.  
In a non-local URL the client is sent a redirection response and then the 
thread concluded. 

<P> If not within the region a call is made to asynchronous read the next 
record from the configuration file.

<ENDLIST>

<COMMENT>(--------------------------------------------------------------------)

<HEAD1>(LOGGING.C\head_logging_c)

<COMMENT>(....................................................................)

<COMMENT>(HTML=Code for: <A HREF="/ht_root/src/httpd/Logging.c"><I>Logging.c</I></A>)

<COMMENT>(HTML/OFF)
<P><BOX>(online hypertext link)
<COMMENT>(HTML/ON)

<COMMENT>(....................................................................)

<P> The logging module provides an <EMPHASIS>(access) log (server logs,
including error messages are generated by the detached HTTPd process, see
sections <REFERENCE>(head_logging_directory) and
<REFERENCE>(head_server_logging)).

<P> The access log format is that of the Web-standard, <QUOTE>(common)-format,
allowing processing by most log-analysis tools.  Each entry (record, line)
comprises:

<CODE_EXAMPLE>
  <EMPHASIS>(client_host) <EMPHASIS>(r_ident) <EMPHASIS>(auth_user) [<EMPHASIS>(time)] "<EMPHASIS>(request)" <EMPHASIS>(reponse_status) <EMPHASIS>(bytes_sent)
<ENDCODE_EXAMPLE>

<CP> where:

<LIST>(UNNUMBERED)

<LE> <EMPHASIS>(client_host\BOLD) is from where the request originated

<LE> <EMPHASIS>(r_ident\BOLD) is the user identified by the client host's
authentication daemon (RFC931), this is not available and is always a hyphen
(<QUOTE>(-))

<LE> <EMPHASIS>(auth_user\BOLD) the authenticated user-name associated with
the request, or a hyphen (<QUOTE>(-))

<LE> <EMPHASIS>(time\BOLD) the following format:
<EMPHASIS>(dd/mmm/yyyy:hh:mm:ss +/-GMT)
(e.g. <QUOTE>(16/Dec/1995:21:15:34 +10:30))

<LE> <EMPHASIS>(request\BOLD) the method, a space, then the path and any
query string

<LE> <EMPHASIS>(response_status\BOLD) the three digit response status code
(e.g. 200, 302)

<LE> <EMPHASIS>(bytes_sent\BOLD) the number of bytes sent to the client, or a
hyphen (<QUOTE>(-))

<ENDLIST>

<P> In addition to legitimate request entries the server adds <EMPHASIS>(bogus)
entries to time-stamp server startup, shutdown, and the log being explicitly
opened or closed (see <REFERENCE>(head_server_logging)).  These entries are
correctly formatted so as to be processed by a log analysis tool, and are
recognisable as being <QUOTE>(POST) method and coming from user <QUOTE>(HTTPd). 
The request path contains the event and a hexadecimal VMS status code, that
represents a valid exit status only in <QUOTE>(END) entries.

<P> Clickable-image requests are logged as <QUOTE>(302) entries, and the
resulting, redirected request entry logged as well.

<P> When a log entry is required the file is opened if closed.  The file is
again closed one minute after the initial request.  This flushes the contents
of the write-behind buffers.

<COMMENT>(--------------------------------------------------------------------)

<CHAPTER>(References\chapter_references)

<COMMENT>(....................................................................)

<COMMENT>(HTML=
<P> Available online within HFRD: 

<UL>

<LI> <A HREF="/hyperdata/technical/html-primer/html-primer.html">A Beginner's Guide
to HTML</A>

<LI> <A HREF="/hyperdata/technical/url/url_guide.html">Beginner's Buide to URLs</A>

<LI> <A HREF="/ht_root/doc/hfrdhyp/hfrdhyp_0000.html">
HFRD Hypertext Environment</A> 

<LI> <A HREF="/ht_root/doc/hfrd_primer/hfrd_primer_0000.html">
HFRD Hypertext Management Primer</A> 

<LI> <A HREF="/ht_root/doc/htd/htd_0000.html">
HFRD Hypertext Services - A Technical Overview</A> 

<LI> <A HREF="/hyperdata/technical/http/http.html">HTTP</A>

<LI> <A HREF="/hyperdata/technical/uri/uri-spec.html">URI Specification</A>

</UL>
)

<COMMENT>(HTML/OFF)

<P>
The following Hypertext documents can be found online within HFRD:

<LIST>(UNNUMBERED)
<LE> A beginner's Guide to HTML
<LE> Beginner's Guide to URLs
<LE> HFRD Hypertext Environment
<LE> HFRD Hypertext Management Primer (on <EMPHASIS>(menuing))
<LE> HTTP Hypertext Services - A Technical Overview
<LE> URI Specification
<ENDLIST> 

<COMMENT>(HTML/ON)

<COMMENT>(....................................................................)

<COMMENT>(--------------------------------------------------------------------)

