# WASD Hypertext Services - Scripting Environment

## 7 - DECnet & OSU

> 7.1 - Script System Environment
> 7.1.1 - Proxy Access
> 7.1.2 - DECnet Objects
> 7.1.3 - Reducing Script Latency
> 7.1.4 - DECnet/OSU Startup
> 7.2 - CGI
> 7.3 - OSU (DECthreads) Emulation
> 7.4 - User Scripts

[next] [previous] [contents] [full-page]

"Imitation is the sincerest form of flattery" - proverb

> NOTE

> WASD requires no additional configuration to support (sub)process-based scripting. The following information applies only if DECnet-based scripting is desired.

By default WASD executes scripts within detached or subprocesses, but can also provide scripting using DECnet for the process management. DECnet scripting is not provided to generally supplant the (sub)process-based scripting but augment it for certain circumstances:

- To provide an environment within WASD where OSU-based scripts (both CGI and OSU-specific) may be employed without modification.

- To allow nodes without a full HTTP service to participate in providing resources via a well-known server, possibly resources that only they have access to.

- Load-sharing amongst cluster members for high-impact scripts or particularly busy sites.

- Provide user-account scripting.

###
DECnet Performance

Any DECnet based processing incurs some overheads:

  connection establishment
  NETSERVER image activation
  NETSERVER maintenance (such as logs, etc.)
  activation of DECnet object image or procedure
  DECnet object processing
  activation by object of image or procedure
  DECnet object run-down
  NETSERVER image reactivation

As of version 5.2 WASD provides reuse of DECnet connections for both CGI and OSU scripting, in-line with OSU v3.3 which provided reuse for OSU scripts. This means multiple script requests can be made for the cost of a single DECnet connection establishment and task object activation. Note that the OSU task procedure requires the definition of the logical name WWW_SCRIPT_MAX_REUSE representing the number of times a script may be reused. The WASD startup procedures can provide this.

In practice both the WASD CGI and OSU scripts seem to provide acceptable responsiveness.

###
Rule Mapping

DECnet-based scripts are mapped using the same rules as process-based scripts, using the SCRIPT and EXEC rules ( "Technical Overview, Mapping User Directories" for general information on mapping rules). DECnet scripts have a DECnet node and task specification string as part of the mapping rule. There are minor variations within these to further identify it as a WASD or an OSU script (7.4 - User Scripts).

The specification string follows basic VMS file system syntax (RMS), preceding the file components of the specification. The following example illustrates declaring that paths beginning with FRODO will allow the execution of scripts from the CGI-BIN:[000000] directory on DECnet node FRODO.

      exec /FRODO/* /FRODO::/cgi-bin/*

In similar fashion the following example illustrates a script "frodo_show" that might do a "SHOW SYSTEM" on node FRODO. Note that these rules are case-insensitive.

      script /frodo-showsys /frodo::/cgi-bin/showsys.com

Both of the above examples would use the WASD CGI DECnet environment (the default if no task specification string is provided). By including task information other environments, in particular the OSU scripting enviroment, can be specified for the script to be executed within. The default task is named CGIWASD and can also be explicitly specified (although this behaviour would be the same as that in the first example)

      exec /frodo/* /frodo::"task=cgiwasd"/cgi-bin/*

All task specification strings may also use zero as the task abbreviation.

      exec /frodo/* /frodo::"0=cgiwasd"/cgi-bin/*

To execute a script within the OSU environment specify the standard OSU task executive WWWEXEC, as in the following example:

      exec /osu/* /FRODO::"task=wwwexec"/htbin/*

This would allow any URL beginning with "/osu/" to execute a script in the OSU environment.

###
Scripting Account

By default the script process is created using the HTTPd scripting account (usually HTTP$NOBODY, although versions prior to 8.1 have used HTTP$SERVER). It is possible to specify alternate accounts for the scripts to be executed within.

The first examples are explicitly specifying an account in the script rule.

      exec /frodo/* /FRODO"ACCOUNT"::"0=cgiwasd"/cgi-bin/*
      script /frodo-whatever /FRODO"ACCOUNT"::/cgi-bin/whatever.com

It is also possible to have scripts that have been subject to SYSUAF authorization executed within the authenticated account. The dollar symbol in the following examples directs the server to substitute the authenticated username into the access string.

      exec /frodo/* /FRODO"$"::"0=cgiwasd"/cgi-bin/*
      script /frodo-whatever /FRODO"$"::/cgi-bin/whatever.com

The set script=as= rule used for PERSONA controlled process scripting can also be applied to DECnet scripts. This includes explicitly specified usernames as well as SYSUAF authenticated usernames. The server creates an appropriate access string when the script is activated.

      set /frodo* script=as=$
      exec /frodo/* /FRODO::"0=cgiwasd"/cgi-bin/*
      script /frodo-whatever /FRODO::/cgi-bin/whatever.com

User scripts can also be activated using these rules, either explicitly specifying the "~" in an access string or using the set script=as= mapping rule.

      exec /~*/cgi-bin/* /0"~"::/www_user/*/www/cgi-bin/*
      exec /~*/htbin/* /0"~"::"0=wwwexec"/www_user/*/www/htbin/*

See 7.4 - User Scripts for more detail.

###
Local System

To specify any script to execute on the same system as the HTTP server specify the node name as zero or SYS$NODE.

      exec /decnet/* /0::"task=cgiwasd"/cgi-bin/*
      exec /osu/* /sys$node::"task=wwwexec"/cgi-bin/*

Mapping rules are included in the examples ( HT_ROOT:[EXAMPLE]) providing this. After the DECnet environment has been started any CGI script may be executed on the local system via DECnet by substituting "/decnet/" for "/cgi-bin/" as the script path, and any OSU script available by using "/osu/". Behaviour is indeterminate, though it shouldn't be catastrophic, if one is invoked using the incorrect path (i.e. an OSU script using /decnet/ or a CGI script using /osu/).

###
7.1 - Script System Environment

The target system must have sufficient of the WASD server environment to support the required CGI script activation and activity. If the target system is actually the same system as the HTTP server then it already exists, or if part of the local system's cluster, then providing this should be relatively straight-forward. If the target system has none of the server environment then at a minimum it must have the logical name CGI-BIN defined representing the directory containing the required DECnet object procedure and scripts. The following fragment illustrates this:

      $ DEFINE /SYSTEM /TRANSLATION=(CONCEALED) CGI-BIN device:[dir.]

In this directory must be located the WASDCGI.COM and WWWEXEC.COM procedures required by the network task. Of course other parts of the environment may need to be provided depending on script requirements.

###
7.1.1 - Proxy Access

The local system must have proxy access to each target scripting system (even if that "target" system is the same system as the HTTP server). This involves creating a proxy entry in each target hosts's authorization database. The following example assumes the existance of a local HTTP$NOBODY account. If it does not exist on the target node then one must be created with the same security profile as the HTTP server's.

> Caution!

> If unsure of the security implications of this action consult the relevant VMS system management security documentation.

The zero represents the system the server is currently executing on.

      $ SET DEFAULT SYS$SYSTEM
      $ MCR AUTHORIZE
      UAF> ADD /PROXY 0::HTTP$SERVER HTTP$NOBODY /DEFAULT

It is necessary to ensure the account has permission to write into it's home directory. A network process creates a NETSERVER.LOG (Phase-IV) or NET$SERVER.LOG (DECnet-Plus) file in the home directory, and will fail to start if it cannot!

###
7.1.2 - DECnet Objects

To provide DECnet scripting DECnet object(s) must be specified for any system on which the scripts will be executed. The DECnet object is the program or procedure that is activated at the target system inside of a network-mode process to interact with the HTTP server.

###
DECnet-Plus (OSI/Phase-V)

DECnet-Plus uses the NCL utility to administer the network environment. The following NCL scripting shows the creation of a network application for the WASD CGI object:

      $ MCR NCL
      CREATE NODE 0 SESSION CONTROL APPLICATION CGIWASD
      SET NODE 0 SESSION CONTROL APPLICATION CGIWASD ADDRESSES = {NAME=CGIWASD} -
      ,CLIENT =  -
      ,INCOMING ALIAS = TRUE -
      ,INCOMING PROXY = TRUE -
      ,OUTGOING ALIAS = FALSE -
      ,OUTGOING PROXY = TRUE -
      ,NODE SYNONYM = TRUE -
      ,IMAGE NAME = CGI-BIN:[000000]CGIWASD.COM -
      ,INCOMING OSI TSEL =

To create a DECnet-Plus OSU WWWEXEC object:

      $ MCR NCL
      SET NODE 0 SESSION CONTROL APPLICATION WWWEXEC ADDRESSES = {NAME=WWWEXEC} -
      ,CLIENT =  -
      ,INCOMING ALIAS = TRUE -
      ,INCOMING PROXY = TRUE -
      ,OUTGOING ALIAS = FALSE -
      ,OUTGOING PROXY = TRUE -
      ,NODE SYNONYM = TRUE -
      ,IMAGE NAME = CGI-BIN:[000000]WWWEXEC.COM -
      ,INCOMING OSI TSEL =

These must be executed at each system (or server) startup, and may be executed standalone, as illustrated, or incorporated in the NCL script SYS$STARTUP:NET$APPLICATION_STARTUP.NCL for automatic creation at each system startup. Examples may be found in HT_ROOT:[EXAMPLE].

###
Phase-IV

DECnet Phase-IV uses the NCP utility to administer the network environment. The following NCP commands may be used each time during server startup to create the required DECnet objects. With Phase-IV the SET verb may be replaced with a DEFINE verb and the commands issued just once to permanently create the objects (a SET must also be done that first time to create working instances of the DEFINEd objects).

To create a DECnet CGI object:

      $ MCR NCP
      SET OBJECT CGIWASD NUMBER 0 FILE CGI-BIN:[000000]CGIWASD.COM

To create a DECnet OSU WWWEXEC object:

      $ MCR NCP
      SET OBJECT WWWEXEC NUMBER 0 FILE CGI-BIN:[000000]WWWEXEC.COM

Examples may be found in HT_ROOT:[EXAMPLE].

###
7.1.3 - Reducing Script Latency

Script system network process persistance may be configured using NETSERVER logical names. These can control the number and quiescent period of the server processes. These logical names must be defined in the LOGIN.COM of the HTTP server account on the target script system.

- NETSERVER$SERVERS_username - This logical controls the number of network server processes that are kept available at any one time. Defining this logical results in a minimum of the specified number of quiescent server processes maintained. This can improve script response latency by circumventing the need to create a process to service the request, at the cost of cluttering the system with NETSERVER processes.

        DEFINE /JOB NETSERVER$SERVERS_HTTP$NOBODY 5

- NETSERVER$TIMEOUT - This logical controls the duration a quiescent network process persists before being deleted. The default period is five minutes. The following examples first show reducing that to thirty seconds, the second increasing it to one hour. Again, this can improve script response latency by circumventing the need to create a process to service the request, at least during the period a previously created process continues to exist.

        DEFINE /JOB NETSERVER$TIMEOUT "0 00:00:30"
        DEFINE /JOB NETSERVER$TIMEOUT "0 01:00:00"

###
7.1.4 - DECnet/OSU Startup

The example STARTUP.COM and STARTUP_DECNET.COM procedures found in the HT_ROOT:[EXAMPLE] directory provide the essentials for DECnet/OSU scripting. If the INSTALL.COM startup environment is used setting the PROVIDE_DECNET symbol to 1 in STARTUP.COM will create the DECnet scripting environment during server startup.

###
7.2 - CGI

CGI scripts that use HTTP GET (the default) may be transparently executed within the DECnet scripting environment. This means that the script is executed within a network process, on the target system (which could be the local system), instead of within a process on the local system. Other than that the WASD DECnet CGI environment behaves identically to the standard (sub)process CGI environment. CGIplus scripting is not supported and if CGIplus-only scripts are executed the behaviour is indeterminate.

Scripts that wish to use HTTP POST will need to read the request body from the NET$LINK stream, rather than from HTTP$INPUT as with (sub)process based scripts. End of body is indicated by an empty record rather than EOF. Scripts may quite simply be made to function appropriately in both environments. The following C code fragment illustrates this.

      if (getenv ("NET$LINK") == NULL)
      {
         /* via process CGI */
         while (ReadCount = fread (Buffer, 1, sizeof(Buffer), stdin))
         {
            /* processing, processing ... */
         }
      }
      else
      {
         /* via DECnet CGI */
         if ((stdin = freopen ("NET$LINK", "r", stdin)) == NULL)
            exit (vaxc$errno);

         while (fgets (Buffer, sizeof(Buffer), stdin) != NULL)
         {
            /* check for end of stream */
            if (Buffer[0] == '\n' && Buffer[0] == '\0') break;

            /* processing, processing ... */
         }
      }

An example of making the HELP database on a system other than that hosting the HTTP server (using the CONAN script) would be done using the mapping rules

      map /FRODO/help /FRODO/help/
      script /FRODO/help/* /FRODO::/cgi-bin/conan/*

and for the example DCL SHOW script

      script /FRODO/show* /FRODO::/cgi-bin/show*

###
7.3 - OSU (DECthreads) Emulation

The OSU, or DECthreads server is the most widely deployed VMS HTTP server environment, authored by David Jones and copyright the Ohio State University. See http://kcgl1.eng.ohio-state.edu/www/doc/serverinfo.html for more information.

The WASD HTTP server provides an emulation of the OSU scripting environment. This is provided so that OSU-based scripts (both CGI-style and OSU-specific) may be employed by WASD with no modification. As this emulation has been designed through examining OSU code and lots of trial and error it's behaviour may be incomplete or present errors. A list of OSU scripts known to work with WASD is provided at the end of this section (Known Working Scripts).

Supported scripts include only those that depend on the OSU WWWEXEC object and dialog for all functionality. Any script that uses other OSU-specific functionality is not supported. Interactions between WASD's and OSU's authentication/authorization schemes may be expected.

OSU scripts expect to get the path information unmapped, whereas WASD always additionally maps any path after the script component has been derived from the request URI. It may be necessary to ensure OSU scripts are activated with the associated path SET to provide what they expect. For example:

      set /htbin/* mapONCE
      set /osu/* mapONCE

The author would like to know of any OSU scripts the WASD emulation barfs on, and will attempt to address the associated limitation(s) and/or problem(s).

###
OSU Setup

Software necessary for supporting the OSU scripting environment (e.g. WWWEXEC.COM) and selected OSU scripts (mainly for testing purposes) have been extracted from the OSU v3.4 package and included in the HT_ROOT:[SRC.OSU] directory. This has been done within the express OSU licensing conditions.

      Copyright 1994,1997 The Ohio State University.
      The Ohio State University will not assert copyright with respect
      to reproduction, distribution, performance and/or modification
      of this program by any person or entity that ensures that all
      copies made, controlled or distributed by or for him or it bear
      appropriate acknowlegement of the developers of this program.

An example DECnet and OSU scripting startup may be found in HT_ROOT:[EXAMPLE]. This should be called from or used within the HTTP server startup. It includes two logical definitions required for common OSU scripts. Other tailoring may be required for specific OSU scripts.

###
OSU - General Comments

David Jones, the author of the DECthreads (OSU) HTTP server, outlines his reasons for basing OSUs scripting on DECnet (reproduced from a USENET NEWS reply to a comment this author made about DECnet-based scripting).

      ------------------------------------------------------------------------

      From           JONESD@er6.eng.ohio-state.edu (David Jones)
      Organization   The Ohio State University
      Date           12 Aug 1997 09:04:11 GMT
      Newsgroups     vmsnet.sysmgt,comp.os.vms,comp.infosystems.www.servers.misc
      Message-ID     <5sp8ub$brs$1@charm.magnus.acs.ohio-state.edu>

      ------------------------------------------------------------------------

      ... some text omitted

      Since I was comfortable with DECnet, I based the scripting system
      for the OSU server around it.    The key reasons to use netserver
      processes rather than spawning sub-processes:

          1. DECnet automatically caches and re-uses netserver processes,
             whereas there were well-known performance problems with spawning
             sub-processes.

          2. DECnet processes are detached processes, so you don't worry about
             the effect of scripts consuming pooled quotas (e.g. bytlm) on
             the HTTP server process.

          3. Creation/connection with the DECnet server process is asynchronous
             with respect to the server so other operations can proceed concurrently.
             Spawning is done in supervisor mode, blocking the server's operation
             until the child process is completely initialized.

          4. With DECnet, scripts can be configured to run on different nodes
             for load balancing.

          5. In addition to the standard 'WWWEXEC' object, you can create
             other 'persistent' DECnet objects that the server communicates with
             as scripts. (this was implemented years before OpenMarket's FastCGI
             proposal).

          6. CGI is not the be-all end-all of scripting.  The dialog phase of
             OSU's scripting environment allows scripts to do things CGI
             is incapable of, such as ask the server to translate an arbitrary
             path and not just what followed the script name in the URL.

      People grouse all the time about the installation difficulties caused by
      it's reliance on DECnet,  the reason shown above were cited to show that it
      wasn't made so capricously.

      ... some text omitted

      David L. Jones               |      Phone:    (614) 292-6929
      Ohio State Unviversity       |      Internet:
      2070 Neil Ave. Rm. 122       |               jonesd@kcgl1.eng.ohio-state.edu
      Columbus, OH 43210           |               vman+@osu.edu

      Disclaimer: Dogs can't tell it's not bacon.

The OSU server's DECnet scripting is not based on arbitrary considerations. This author does not disagree with any of the concerns, and as may be seen from WASD documentation the design of WASD also directly addresses points 1, 3 and 5 with the use of persistant processes and CGIplus. Certainly DECnet-based scripting addresses the very legitimate point 4 (and also allows nodes with specific resources to participate without installing full HTTP server environments). Point 2 is not an issue with the use of detached scripting processes, or for all practical purposes addressed by adjusting account quotas to support the required number of subprocesses. Point 6 is only too true (possibly at least until Java servers and servlets become ubiquitous :^)

###
Known Working Scripts

The following is a list of OSU-specific scripts that the WASD implementation has either been developed or tested against, and any installation notes or other WASD specifics. The author would like to know of any OSU scripts the WASD emulation has problems or works successfully with.

- All of the scripts, etc. provided in the HT_ROOT:[SRC.OSU] directory. These include:

    cgi_symbols
    cgi-mailto
    html_preproc
    set_dcl_env
    testcgi
    testform
    tmail
    vmshelpgate
    webbook

- helpgate

  Comment out the Conan The Librarian mappings for the "/help" path and provide the following in HTTPD$MAP:

        # first make "/help" into a script specification
        map /help* /htbin/helpgate/help*
        # general rule mapping "/htbin" to OSU DECnet scripts
        exec /htbin/* /0::"0=wwwexec"/cgi-bin/*
        # map the non-script part of the path back to just "/help"
        pass /htbin/helpgate/help* /help*

  It is possible to support both HELP environments (although helpgate will not work without owning the "/help" path), merely provide another mapping for Conan with a slightly different path, for example:

        map /chelp /chelp/
        script /chelp/* /cgi-bin/conan/*

- HTML pre-processor

  Yes, backward compatibility can be provided for those old OSU .HTMLX files in your new WASD environment ;^) All that is needed is a file type mapping to the script in the HTTPD$CONFIG configuration file.

        [AddType]
        .HTMLX  text/html  /htbin/html_preproc  OSU SSI HTML

###
7.4 - User Scripts

The WASD DECnet environment provides a simple mechanism for executing scripts within accounts other than the server's. This allows configured users to write and maintain scripts within their own areas and have them execute as themselves. Both standard CGI and OSU scripting may be provided for with this facility.

Of course there is always a down-side. Be careful to whom this capability is granted. User scripts are executed within a user network-mode process created by DECnet. Script actions cannot generally affect server behaviour, but they can access any WORLD-readable and modify any WORLD-writable resource in the system/cluster, opening a window for information leakage or mischievous/malicious actions. Script authors should be aware of any potential side-effects of their scripts and Web administrators vigilant against possible destructive behaviours of scripts they do not author.

User scripting is not enabled by default. To provide this facility mapping rules into the user area must be provided in much the same way as for user directories, see "Technical Overview, Mapping User Directories".

The "EXEC" rule provides a wildcard representation of users' script paths. As part of this mapping a subdirectory specifically for the web scripts should always be included. Never map users' top-level directories. For instance if a user's account home directory was located in the area WWW_USER:[DANIEL] the following rule would potentially allow the user DANIEL to provide scripts from the home subdirectory [.WWW.CGI-BIN] using the accompanying rules (first for CGI, second for OSU scripts):

      exec /~*/cgi-bin/* /0"~"::/www_user/*/www/cgi-bin/*
      exec /~*/htbin/* /0"~"::"0=wwwexec"/www_user/*/www/htbin/*

Scripts located in these directories are accessable via paths such as the following:

      /~daniel/cgi-bin/cgi_symbols
      /~daniel/htbin/osu_testcgi

###
Proxy Access

For each user account permitted to execute local scripts proxy access to that account must be granted to the HTTP server account.

> Caution!

> If unsure of the security implications of this action consult the relevant VMS system management security documentation.

      $ SET DEFAULT SYS$SYSTEM
      $ MCR AUTHORIZE
      UAF> ADD /PROXY <node>::HTTP$NOBODY <account>

For example, the following would allow the HTTP server to execute scripts on behalf of the username DANIEL.

      UAF> ADD /PROXY 0::HTTP$NOBODY DANIEL

[next] [previous] [contents] [full-page]