# WASD Hypertext Services - Technical Overview

## 2 - HTTPd - Overview

> 2.1 - Server Behaviour
> 2.2 - VMS Versions
> 2.3 - TCP/IP Packages
> 2.4 - International Features
> 2.5 - HTTP Methods Usage
> 2.5.1 - GET
> 2.5.2 - POST & PUT
> 2.5.3 - DELETE

[next] [previous] [contents] [full-page]

The most fundamental component of the WASD VMS Hypertext Services environment is the HTTPd, or HyperText Protocol Transport Daemon, or HTTP server. WASD has a single-process, multi-threaded, asynchronous I/O design.

###
General

- concurrent, multi-threaded client support

- "GET", "HEAD", "POST" and "PUT" HTTP method support

- virtual services (servers)

- concurrent IPv4 and IPv6 support (requires underlying TCP/IP support)

- requests above a configurable limit can be queued ("throttling")

- enhanced privacy using Secure Sockets Layer (SSL) technology, including OpenSSL Toolkit, WASD OpenSSL, and HP SSL (Secure Sockets Layer) for OpenVMS Alpha, IA64 and (now - late 2003) VAX product

- serves ODS-2 and ODS-5 (EFS) volumes, as well as file names encoded using the PATHWORKS 4/5, Advanced Server (PATHWORKS 6) and SRI (MultiNet NFS, etc.) schemas

- versatile directory listing (generic and VMS-style)

- Server-Side Includes (HTML pre-processing)

- configurable cache, with time-based and forced revalidation (reload)

- byte-range support with 206 partial responses (useful for PDF and restarting file download by modern browsers)

- proxy serving, with local caching for all HTTP/1.0 methods, plus the CONNECT method (allowing SSL tunnelling), along with FTP proxy (non-caching)

- gatewaying between protocols (HTTP-to-SSL, SSL-to-HTTP, HTTP-to-FTP, etc.)

- clickable-image support (both NCSA and CERN formats)

###
Scripting

- CGI-compliant scripting

- non-server and user account scripting

- "CGIplus" scripting (offering reduced latency, increased throughput and reduced system impact)

- "Persistent" scripting, Run-Time Environments (RTEs) that provide for simple persistent scripting

- "ISAPI" extensions/scripting (also offering reduced latency, increased throughput and reduced system impact)

- DECnet-based CGI scripting (with connection reuse)

- OSU (DECthreads server) scripting emulation, with connection reuse (as per OSU 3.3a), allowing many OSU scripts to be employed unmodified

- script processor (e.g. PERL) configurable on file type (suffix)

- configurable, automatic, MIME content-type initiated scripting ("presentation" scripting)

###
Access Control

- host-level, on per-host or per-domain

- "Basic" and "Digest" user authentication and path/group-based authorization

- WASD-specific user databases

- SYSUAF-authentication and VMS user security profile based file access control

- ACME service authentication (on applicable platforms)

- X.509 client certificate authentication (for SSL transactions)

- RFC1413 (ident daemon) "authentication"

###
Administration

- multiple instances (server processes) executing on the one system allow continuous availability via rolling restarts and "fail-through" processing

- "one-button" control of multiple instances on both single systems and across clusters

- online server configuration, including reports on requests, loaded configuration, mapping rules, authorization information and graphical activity displays

- online, live server processing event report (WATCH)

- Web-standard, "common" and "combined" access log formats (allowing processing by most log-analysis tools), along with a user-definition capability allowing custom log formats

- logging periods, where log files automatically change on a daily, weekly or monthly basis (keeps log files ordered and at a managable size)

- customizable message database (capable of supporting non-English and concurrent, multiple languages)

###
2.1 - Server Behaviour

The HTTPd executes permanently on the server host, listening for client connection requests on TCP/IP port 80 (by default). It provides concurrent services for a (technically) unlimitted number of clients (constrained only by the server resources). When a client connects the server performs the following tasks:

1. creates a thread for this request

2. reads and analyzes the HTTP request sent,
   depending on the nature of the request ...

   - transfer of a file (possibly from cache)

   - processing of an SSI HTML file

   - directory listing

   - file/directory create/update

   - server administration

   - processing of a clickable-image mapping file

   - create a process to to provide a CGI scripting environment:

     - standard CGI

     - WASD CGIplus

     - WASD Run-Time Environment

   - connects to a DECnet object to provide a scripting environment:

     - standard CGI

     - OSU (DECthreads) emulated

3. closes the connection to the client and disposes of the thread data structures

For I/O intensive activities like file transfer and directory listing, the AST-driven code provides an efficient, multi-threaded environment for the concurrent serving of multiple clients.

For scripts, the technique of using multi-threaded, concurrent, spawned subprocesses, detached autonomous processes, or network processes, attached to standard input/output streams, provides a versatile, extensible, powerful scripting environment. Any DCL procedure or image executing within the process can behave essentially as an HTTP server. This capability is employed to easily extend the basic services provided by the core daemon code. An HTTP script/server for this environment does not need to concern itself with network activities, it merely reads and writes from the standard I/O streams.

###
2.2 - VMS Versions

The WASD server is officially supported on any VMS version from V6.0 upwards, on Alpha, IA64 (Itanium) and VAX architectures. Obviously no guarantees can be made for yet-to-be-released versions but at a worst-case these should only require relinking. Pre 7.n versions of WASD have also been known to compile and run successfully under V5.5-n (4.11 - VMS 5.5-n ).

WASD on IA64 based systems has been initially ported and tested against "HP OpenVMS Industry Standard 64 Evaluation Release Version 8.1". Later VMS releases on this platform may require recompilation. No significant code changes were required for this port (as might be expected), mainly requirements for Alpha specific conditional compilation expanded to include IA64.

Non-server account scripting requires a minimum VMS V6.2, to provide the $PERSONA services required for this functionality. Equivalent functionality on earlier versions of VAX VMS (i.e. 6.0 and 6.1) is available using the PERSONA_MACRO build option (4.10 - VMS 6.0 and 6.1 ).

The WASD distribution and package organisation fully supports mixed-architecture clusters (Alpha, IA64 and/or VAX in the one cluster) as one integrated installation.

###
2.3 - TCP/IP Packages

The WASD server uses the Compaq TCP/IP Services (UCX) BG $QIO interface. The following packages support this interface and may be used.

- Compaq TCP/IP Services for OpenVMS (Hewlett Packard Corporation), any version

- Digital TCP/IP Services for OpenVMS (aka UCX), any version

- MultiNet for OpenVMS (Process Software Corporation), any version

- TCPware (Process Software Corporation), any version

To deploy IPv6 services this package must support IPv6 (needless-to-say).

###
2.4 - International Features

WASD provides a number of features that assist in the support of non-English and multi-language sites. These "international" features only apply to the server, not necessarily to any scripts!

- Language Variants

  A directory may contain language-specific variants of a basic document. When requesting the basic document name these variants are automatically and transparently provided as the response if one matches preferences expresses in the request's "Accept-Language:" request header field. Both text and non-text documents (e.g. images) may be provided using this mechanism.

  Configuration information is provided in 6.7 - Language Variants.

- Character Sets

  Generally the default character set for documents on the Web is ISO-8859-1 (Latin-1). The server allows the specification of any character set as a default for text document responses (plain and HTML). In addition, text document file types may be modified or additional ones specified that have a different character set associated with that type. Furthermore, specific character sets may be associated with mapping paths. A site can therefore relatively easily support multiple character set document resources.

  In addition the server may be configured to dynamically convert one character set to another during request processing. This is supported using the VMS standard NCS character set conversion library.

  For further information see [CharsetDefault], [CharsetConvert] and [AddType] in 8 - Server Configuration Directives.

- Server Messages

  The server uses an administrator-customizable database of messages that can contain multiple language instances of some or all messages, using the Latin-1 character set (ISO8859-1). Although the base English messages can be completely changed and/or translated to provide any message text required or desired, a more convenient approach is to supplement this base set with a language-specific one.

  One language is designated the prefered language. This would most commonly be the language appropriate to the geographical location and/or clientele of the server. Another language is designated the base language. This must have a complete set of messages and is a fall-back for any messages not configured for the additional language. Of course this base language would most commonly be the original English version.

  More than just two languages can be supported. If the browser has prefered languages set the server will attempt to match a message with a language in this preference list. If not, then the server-prefered and then the base language message would be issued, in that order. In this way it would be possible to simultaneously provide for English, French, German and Swedish audiences, just for example.

  For message configuration information see 10 - Message Configuration.

- Server Dates

  Dates appearing in server-generated, non-administrative content (e.g. directory listings, not META-tags, which use Web-standard time formats) will use the natural language specified by any SYS$LANGUAGE environment in use on the system or specifically created for the server.

- Virtual Services

  Virtual-server-associated mapping, authorization and character-sets allow for easy multiple language and environment sites. Further per-request tailoring may be deployed using conditional rule mapping described below. Single server can support multi-homed (host name) and multiple port services.

  For virtual services information see 6 - Server Configuration.

- Conditional Rule Mapping

  Mapping rules map requested URL paths to physical or other paths (13 - Mapping Rules). Conditional rules are only applied if the request matches criteria such as prefered language, host address (hence geographical location to a certain extent), etc. This allows requests for generic documents (e.g. home pages) to be mapped to language versions appropriate to the above criteria.

  For conditional mapping information see 13.7 - Conditional Mapping.

###
2.5 - HTTP Methods Usage

This section describes WASD-specific characteristics of the available HTTP/1.0 request methods.

###
2.5.1 - GET

Of course, the GET method is used to access documents supplied by the server. There is nothing WASD server-specific about this method.

###
2.5.2 - POST & PUT

The WASD HTTPd does not differentiate between POST and PUT methods, both are handled identically.

###
Script Handling

The "normal" usage of the POST method is to return data from a <FORM>..</FORM> construct to a script running on the server. In this regard WASD is no different to other any web server; the form data is delivered to the script's standard input as a stream of URL-encoded text. For example:

      name=Fred+Nurk&address=Fred%27s+House%0D%0A0+Nowhere+Lane&submit=Submit

Note that WWW_CONTENT_LENGTH will be the length of the form data. See "Scripting Environment" document for further information.

###
File Creation/Upload

If the client sends data back to the server using either the POST or the PUT methods, without a script being mapped to be executed in response to that data, the WASD HTTPd will create a file corresponding to the specified path. The data stream may be text or binary.

Of course, for the server to accept POST and PUT data in this manner, authentication and authorization must be enabled and allow such access to the request path.

The data stream is processed according to MIME content-type:

- application/x-www-form-urlencoded

  The server specially processes "application/x-www-form-urlencoded" POSTS (i.e. those generated by <FORM>...</FORM>, allowing files to be created directly from HTML forms. The processing eliminates any field names from the URL-encoded data stream, placing only field values into the file. This capability can be quite useful and is demonstrated in the Update HTTPd module (UPD.C).

- multipart/form-data

  This server can process a request body according to RFC-1867, "Form-based File Upload in HTML". As yet it is not a full implementation. It will not process "multipart/mixed" subsections. The implementation is basic, providing a facility to allow the upload of a file into the server administered file system. The ACTION= parameter of the <FORM> tag must specify the directory (as a URL path) in which the uploaded file will be created.

  The following example HTML illustrates how a form may be used to upload a file from the browser host file system:

        <FORM METHOD=POST ACTION="/web/directory/" ENCTYPE="multipart/form-data">
        <INPUT TYPE=submit VALUE=" Upload document ... ">
        <INPUT TYPE=file SIZE=50 NAME=uploadfile>
        </FORM>

  Again, the Update module (UPD.C) illustrates the use of this facility.

  > NOTE

  > This capability has only been tested against Netscape Navigator versions 2 and 3. VMS Netscape Navigator 3.0b5 hangs if an upload of a variable-record format file is attempted. Stick to STREAM-LF or fixed, or convert the file to STREAM-LF.

- text/html
  text/plain

  Text file created according to the path, VMS record type is STREAM-LF.

- image/gif
  application/octet-stream
  etc., etc., etc.

  Any other MIME type is considered binary and the created file is made an UNDEFINED record type.

###
Directory Creation

A directory will be created by the HTTPd if a directory path is provided with the POST or PUT methods. For example:

      /dir1/dir2/dir-to-be-created/

###
File Deletion

A file will be deleted by the HTTPd if the file path ending with a wildard version specification is provided with the POST or PUT methods. For example:

      /dir1/dir2/file-to-be.deleted;*

###
Directory Deletion

A directory will be deleted by the HTTPd if a directory path ending with a wildard version specification is provided with the POST or PUT methods. For example:

      /dir1/dir2/dir-to-be-deleted/;*

###
2.5.3 - DELETE

The DELETE method should delete the file or directory corresponding to the supplied path.

[next] [previous] [contents] [full-page]