Porting PVM to VMS

Dan L. Clark

1.0  Introduction

Please review and comment.

2.0  System Software Requirements

2.1  DEC TCP/IP Services for VMS - UCX

The DEC TCP/IP Services for VMS provides the C language socket programming inter-
face to the Internet communications subsystem. Sockets are utilized extensively by the 
PVM software for communication between the daemons and the application programs. By 
utilizing the Internet domain facilities available with this DEC layered product the com-
munications mechanism can be left in tact greatly simplifying the porting effort. The dis-
advantage to utilizing the UCX product is that it requires the purchase of an additional 
product to utilize the public domain PVM product. To remove the dependency on this lay-
ered product would be very difficult since all the communications interfaces would need to 
be re-written to utilize mailboxes. Furthermore, using mailboxes would not provide a very 
general method for connecting VMS nodes to other non-VMS nodes on the network. 

2.2  DEC Module Management System (MMS)

The DEC MMS product is a facility to update object and executable code based on date 
dependencies with other files on the system. The MMS system is very similar to the 
`make' facility available on many systems running the UNIX O/S. The MMS software is 
not required to port the PVM source code to a VMS platform. The current port does sup-
ply a MMS compatible file for creating the PVM executable for the VMS O/S. A com-
mand file could easily be constructed for all the files in the PVM distribution to create the 
object libraries and executable that are required to create distributed applications but such 
a command procedure has not been provided.

2.3  SUN External Data Representation (XDR) for VMS

No current public domain distribution of the SUN XDR utility libraries is available. Con-
tact the local SUN office for the set of files, included in the source code for the remote pro-
cedure call (RPC) programs, to port to the VMS.

2.4  General Setup

1.	Create a directory hierarchy on VMS. 
PVM
		SRC
			VMS
		LIB
			VMS

2.	Copy the current version of PVM for VMS to a VMS node. Note that only a subset of 
PVM 3.0 was ported to VMS. The additional functionality available under PVM, such 
as the group daemon and xab, have not been ported. Refer to the original distribution of 
PVM available from Oak Ridge National Labs to obtain the remaining portions of the 
PVM software.

3.0  Installation Instructions for PVM on VMS

4.0  PVM specific changes which result in impaired function

The following changes have been made to create an initial version of PVM on VMS. A 
more careful exploration of the facilities available under VMS may reveal that the func-
tionality impaired by these changes can be fully restored. Each of the items below is pre-
ceded by an assessment of the impact of this change on the overall operation of PVM.

4.1  unknown - No standard C interface for file control (fcntl)

The removal of the call to the file control function (fcntl) may result in the loss of power 
for the server to spawn new processes from the daemon that have all the files closed prop-
erly. If the flag F_SETFD is cleared the file descriptor remains open across a call to create 
a new executable (execve) and the new image has access to the file. Since the file descrip-
tor flag is updated to create non-blocking I/O on file descriptors the VMS daemon may 
hang when attempting to read when no data is available. The standard PVM release would 
return an error (-1) if no data was available. More work needs to be done should be done to 
determine if an equivalent scheme can be devised to cleanup the open files under the VMS 
O/S before creating new processes. (see both pvmd.c and lpvm.c)

4.2  major - No VMS rexec impairing creation of PVM slave daemons

The standard C call interface for creating remote executable (rexec) with a stream to the 
command is not available under VMS. The resulting loss in functionality is that a slave 
daemon can not be created on a remote node. Until a VMS specific mechanism is imple-
mented to create remote procedures the PVM daemon on each VMS node must be created 
on each machine for each user individually. This is not a desirable means of working with 
PVM on a multiple machine network. The standard release of the PVM start-up applica-
tion does have a provision to prompt the user to kick off the remote daemon. According to 
the documentation on the DEC TCP/IP connection product the rexec service is available 
and registered with a TCP/IP port number and several functions, such as getservbyname, 
are available. But the call to endserverent is not available and has been removed. The calls 
to these functions do not seem to return any valid values, perhaps due to the limited privi-
leges awarded to my account. Further experimentation is necessary to determine a feasible 
approach to creating PVM daemons on remote nodes. (see startup.c) 

4.3  unknown - No VMS unlink for file process authorization handshake

The UNIX file system call to remove the entry for a file path from its directory (unlink) is 
not available under VMS. The unlink call is used within PVM primarily on the authoriza-
tion file name during the initial creation of a new task. The loss of the unlink functionality 
my attribute to the problem associated with the authorization sequence. The VMS version 
uses a simplified authorization by ignoring the second round of verification (see tdpro.c). 
The impact of this change has not been thoroughly analyzed. (see lpvm.c, pvmd.c, star-
tup.c, tdpro.c, and cruft.c).

4.4  unknown - No Privilege to set socket options setsockopt

Skip calls to set to socket options for the communications port between the PVM daemon 
and the child process. Under VMS the user is not the owner of the socket and therefore the 
application does not have permission to change the attributes of the socket. The setsockopt 
call fails and causes PVM to abort the establishment of a connection. This code is easily 
bypassed by setting the NOSOCKOPT compiler flag under VMS. (see startup.c). The 
effect of not using this call is that the size of the socket buffer is not tuned to the size of the 
send and receive pack. As a result performance may not be as optimal on the system. Two 
actions can be taken to enhance performance by changing the internal packet size used by 
the socket. The privileged of the user account can be bumped up to allow the process to 
modify the socket. Or the standard PVM packet size can be set to the default packet size 
used on VMS.

5.0  PVM specific changes for VMS

5.1  Provide simultaneous access to a single file from multiple processes

During the authorization process both the library code of the child process and the daemon 
process open a file with shared read/write privilege. The utilization of the underlying secu-
rity of the file system provides a simple way of authenticating the child processes access to 
a PVM daemon. The VMS system requires the use of a C language extension to the open 
call to provide this functionality. (see lpvm.c and tdpro.c)
for example in lpvm.c 
if((authfd = open(autfn, O_RDONLY|O_CREATE|O_TRUNC, 0600, "shr=upd")) == -1){

5.2  Bypass second level authorization between PVM daemon and child

The child process fails authorization upon attempting to read what the daemon has written 
to the authorization file. The child returns and end of file error. This may be a result of the 
authorization file is not being rewound before it is read. A simple solution was imple-
mented by automatically assuming that the daemon approves access. This introduces a 
security risk to a VMS system running a PVM daemon. (see lpvm.c)
if((cc - read(authfd, (char *)&cc, 1)) -== -1) {
	log_perror("beatask() read auth file");
	cc = PvmSysErr;
	goto bail;
}
#ifdef VMS
		cc = 1;
#endif
if(cc != 1) {
		log_error("beatask() pvmd didn't validate itself\n");
		cc = PvmSysErr;
		goto bail;
}
and in tdpro.c
if ((cc = read(tp->t_authfd, &c, 1)) == -1) {
		log_perror("tm_conn2() can't read auth file");
		return 0;
}
#ifdef VMS
cc = 1; /* disable this authorization for VMS */
#endif
if (cc != 1) {
		log_error("tm_conn2() task didn't validate itself\n");
		goto bail;
}

5.3  Masking off high order bits of VMS process identification numbers

The VMS system returns process identification numbers that utilize full integers. Under 
most UNIX systems the process ID is fixed to a short integer. PVM, as documented in the 
user manual, utilizes a single long word to pack the process ID of both the daemon and the 
child. As a result the high order bits of the process ID returned by the system must be 
masked to 0. This change is implemented by implementing a new function for getpid 
under VMS, vmsgetpid, which returns the masked process ID. The second location of the 
change in the application occurs at the fork command when creating a new child process 
using the pvm_spawn function. (see vmsgetpid.c, pvmd.c, lpvm.c, startup.c)

5.4  Set the altpid to zero for the new process.

By setting the altpid to zero for the child process created under VMS the pvm_spawn 
command is fully operational. The impact of this change is not fully understood.
(see lpvm.c procedure beatask)



6.0  Conditional Code Changes for a UNIX to VMS port

The following changes are typical of any port from UNIX to VMS. These changes result 
in conditional code or new files. Once applied to the application no further changes should 
be necessary on subsequent ports. We recommend that these changes be carefully merged 
in with the master source since the conditional compiler flags and separate source code 
files isolate these changes from impacting the operation of PVM on other platforms.

6.1  Transforming a UNIX make file into MMS description file

1.	All dependencies in and MMS file must have a space before and after the colon. Typi-
cally, files created for the UNIX make program have no space preceding the colon 
delimiter. 

2.	Remove all the UNIX path specifications, such as "/", from the dependency lists. Gen-
erally specify all relative paths in VMS syntax. For example the variable SDIR is 
changed from ".." to "[-]".

3.	Replace all the UNIX object file suffixes of ".o" with the VMS object specification of 
".obj".
for example: 
old:
lpvm.o: 		$(SDIR)/lvpm.c
		$(CC) $(CFLAGS)	 $(SDIR)/lpvm.c
lpvm.o: 		$(SDIR)/pvm3.h
new:
lpvm.obj :		$(SDIR)lpvm.c
		$(CC) $(CLFAGS) $(SDIR)lpvm.c
lpvm.obj :		$(SDIR)pvm3.h

4.	Replace the references to UNIX object libraries with the specification of equivalent 
VMS object libraries. Three primary libraries are required for the PVM software. The 
standard VAX C library, the DEC TCP/IP connection library and the PVM utility 
library created from the make file and used for creating the programs. Typically the 
VAX C libraries are specified in a file dxm_link.opt located in the home directory for 
each persons account. 
ARCHLIB = ,libpvm3.obl/lib, sys$library:ucx$ipc/lib, sys$login:dxm_link.opt
or
ARCHLIB = ,libpvm3.obl/lib, sys$library:ucx$ipc/lib, sys$share:vaxcrtl/shareable

5.	Set the compiler flags to trigger the C preprocessor macros which most closely match 
the requirements for a VMS compilation of PVM. Note that the preprocessor macro 
"VMS" is already defined to be true by VMS. The other flags that were found to be use-
ful were:
 SYSVSTR=TRUE, ARCHCLASS=UVAX, IMA_UVAX=TRUE, and
NOSOCKOPT=TRUE

6.2  Find VMS file names for "standard" C include files

1.	Replace references to UNIX files with similar VMS files.
fcntl.h		replaced with file.h (see logging.c, lvpm.c, pvmd.c)
	ioctl.h		replaced with iodef.h (see host.c)

The removal of reference to the include file fcntl.h results in the loss of the ability to call fctnl function to 
change the attributes of a file.

2.	Add references to helpful include files
unixio.h		added (see host.c)

3.	Copy files from a UNIX system to supplement the VMS include files. 
xdr.h		(see umbuf.c ...)
types.h		(see pvmd3.c, startup.c, ...)

4.	Remove references to files not found nor needed under VMS.
param.h		not referenced
pwd.h		not referenced (see pvmd.c)
if.h		not referenced (see host.c)
wait.h		not referenced (see pvmd.c)

The Removal of the include file pwd.h forces the replacement of the call to get the process wide user identi-
fication code with a call to the cuserid function to obtain a user identification string.

6.3  Define constants not defined in VMS system include files

1.	add constants F_GETFL and F_SETFL 
defined in fcntl.h

2.	MAXPATHLEN 128

3.	MAXHOSTNAMELEN 64

4.	ARCHCLASS VMS

6.4  Create new PVM library functions to replace UNIX system calls.

1.	gettimeofday replaced with vmsgettimeofday 
The UNIX get time function is replaced with a subroutine which utilizes the time and 
localtime calls to determine the time. No major loss of precision has occurred as result 
of this call.

2.	getdtablesize replaced with vmsgetdtablesize
The UNIX get descriptor table size call is not available under VMS. This call has been 
replaced by a call under VMS which returns the constant 64. (see cruft.c)

3.	The UNIX function ffs is replaced with a PVM library function under VMS.
The UNIX function to find the first bit set and return the position of the bit is replaced 
with a simple C function. The ffs function was already implemented in the standard 
PVM release in the file cruft.c for the TITN architecture.

4.	Remove referenced to the signal handlers for SIGCHLD and SIGCLD. Many other sig-
nal types are commented out of the PVM signal handling code. There may not be any 
significant impact to this change. It appears as if the PVM does trap the signal under 
VMS that allows the daemon to clean up the tables for any child process which termi-
nates prematurely. This will occur any time a process exits without calling pvm_exit.

7.0  Unix compatible modifications typical of any Port to VMS

The following items were modified in the PVM 3.0 source code to port the application to 
VMS. All of the modification to source provide a single set of code for compilation under 
either the VMS or the UNIX O/S. It is therefore recommended that these changes be 
incorporated into the master PVM source. Removing UNIX path specification from the 
include files in the source code would require enhancing the make files for all other sys-
tems. Due to the larger impact of this change it may be advisable to create an small proce-
dure which would strip the includes for a dynamic creation of the VMS version. This 
could be achieved utilizing facilities such as awk or sed.

7.1  Remove UNIX path specification from C include file references

The unix path names included in the C preprocessor include directives are not required for 
any application under any operating system. These paths can be removed from the source 
code for both the VMS and the UNIX version by specifying the paths to be searched in the 
build procedures for the application (UNIX make or VMS MMS). Edit each file in the sys-
tem, both .c and .h files, and remove the unix paths from the file specification. For exam-
ple:
old:
#include <sys/stat.h>
#include <sys/socket.h>
new:
#include <stat.h>
#include <socket.h>

7.2  Replace UNIX subroutines and macros with VMS equivalents.

1.	fork replaced with vfork (see ddpro.c, lpvm.c, pvmd.c, startup.c)
the use of the standard UNIX fork which is used to create new processes to the VMS 
vfork routine. This can be achieved with a simple preprocessor definition changing all 
occurrences of fork to vfork.

2.	bzero replaced with memset
#define bzero(A,B)		memset(A, 0, B)

3.	bcopy replaced with memcpy
#define bcopy(A,B,C)		memcpy(B,A,C)

4.	bcmp replaced with memcmp
#define bcmp(A,B,C)		memcmp(A,B,C)

5.	wait3 replaced with wait
Use code emulating condition for SYSVSIGNAL handling in pvmd3.c
v                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            