IRIX Binary Compatibility, Part 2

Unix Program Startup

Now that our kernel is able to distinguish the difference between IRIX binaries and other programs, we need to arrange the program environment so that the IRIX binary is able to start up. (See Part 1 in this series for more on this.)

Generally speaking, Unix kernels have to communicate a few things to user programs in order to start them up. This includes the program's arguments and environment, and for dynamic binaries, the ELF auxiliary table, which is used by the dynamic linker to learn how to link the program. All this information is transmitted to the user program through the CPU registers and the stack.

If this information is corrupted, static binaries are still likely to work, but they will lose their arguments and environment. On the other hand, dynamic executables will not start at all if the ELF auxiliary table is screwed, because the dynamic linker will not be able to link them.

Therefore, it is a good idea to start with a simple static binary. We will use IRIX 5's sed(1) here. When we run it using the NetBSD native function to set up the stack and CPU register, it is able to start up. Then it gets a SIGSYS signal and it dumps core on the first system call, because our system call table for IRIX binaries is still empty. It is possible to check what the missing system call is with the ktrace(1) command on NetBSD:

$ ktrace -di /emul/irix/bin/sed /etc/passwd
Bad system call (core dumped)
$ kdump
  1209 ktrace   EMUL  "netbsd"
  1209 ktrace   CALL  execve(0x7fffea5f,0x7fffe99c,0x7fffe9a4)
  1209 ktrace   NAMI  "/emul/irix/bin/sed"
  1209 sed      EMUL  "irix o32"
  1209 sed      RET   execve 0
  1209 sed      CALL  #4 (unimplemented write)
  1209 sed      PSIG  SIGSYS

Most of the system calls first used by /bin/sed are plain SVR4, so it was easy to emulate them: just copy the system call definition from sys/svr4/syscall.master to sys/irix/syscall.master, issue a make to refresh the files generated from syscall.master, rebuild a kernel, reboot, and retry.

Within a few minutes, it was easy to get IRIX's /bin/sed nearly working. The next problem was to have it take its arguments correctly.

Setting Up the Stack for Program Startup

For static binaries, the stack is used to transmit arguments and the environment to the user program. The way it should be done is documented in the SVR4 ABI MIPS processor supplement.

The NetBSD kernel uses a function pointed to by the es_copyargs field of the struct execsw to set up the program stack on startup. Because NetBSD/mips ports conform to the SVR4 ABI, we could have expected the NetBSD version of this function (elf32_copyargs() from sys/kern/kern_exec_elf32.c) to just work with IRIX binaries. Unfortunately, this is not true. Using the NetBSD elf32_copyargs function with static o32 IRIX binary such as /bin/sed showed weird behavior with the way argument read: sometimes /bin/sed was reading the arguments correctly, sometimes it was not. The behavior was dependent upon the argument length. This suggested that something in the stack had to be aligned on a particular boundary; with some argument lengths it was being aligned, and with others it was not.

I already had to face this kind of situation when working on Linux/PowerPC binary compatibility on NetBSD (read the whole story). However, the situation is different here: IRIX is a closed source proprietary OS; therefore it is not possible to grab kernel sources and look at the way the IRIX kernel sets up the stack. Worse, because I was not able to build static binaries, it was impossible to make a static test program that dumped the stack and displayed argc, argv and envp to check what was wrong in the way the stack was set up.

The solution was to use gdb. With gdb we can run /bin/sed on IRIX, set a breakpoint at the beginning of the program and then examine the stack. The first thing to know is the program startup address. This information can be obtained using objdump on IRIX's sed:

$ objdump -f /bin/sed

/bin/sed:     file format elf32-bigmips
architecture: mips:3000, flags 0x00000102:
EXEC_P, D_PAGED
start address 0x100000c0

Then we can start gdb and set our breakpoint. It seems that it is not possible to break on the program's first instruction, but we can break on the second instruction. On the MIPS, all instructions are four bytes long, hence the second instruction is four bytes away from the program's start address, at 0x100000c0 + 0x4 = 0x100000c4:

$ gdb /bin/sed
(gdb) b *0x100000c4
Breakpoint 1 at 0x100000c4
(gdb) run aa aaa
Starting program: ./sed aa aaa

Breakpoint 1, 0x100000c4 in ?? ()
(gdb) x/16wx $sp
0x7fff2fa0:     0x00000003      0x7fff3000      0x7fff3027      0x7fff302a
0x7fff2fb0:     0x00000000      0x7fff302e      0x7fff3057      0x7fff306a
0x7fff2fc0:     0x7fff30a7      0x7fff30b4      0x7fff30be      0x7fff30d4
0x7fff2fd0:     0x7fff30e1      0x7fff30f6      0x7fff3109      0x7fff3113

In this dump, we recognize a standard startup stack layout--the argc value (three arguments: '/bin/sed', 'aa', and 'aaa'), followed by the argv array (a NULL terminated array of pointers to the argument strings), and then the envp array (a NULL terminated array of pointers to the environment strings). There are a lot of environment strings here, hence we do not see the trailing NULL here, it is a bit farther in the stack dump.

It is possible to dig up the value of an argument with its address:

(gdb) x/s 0x7fff3000
0x7fffea60:      "/bin/sed"
(gdb) x/s 0x7fff3027
0x7fffea72:      "aa"

Dumping the stack with various arguments to /bin/sed, it is possible to discover that, for an IRIX binary, the argv[0] must be aligned on a 16-byte boundary. The IRIX kernel sets the stack that way, and IRIX binaries depend on this particular layout.

It was not possible to modify elf32_copyargs() to implement this particular behavior because it is also used by native NetBSD binaries. Hence, the solution was to duplicate what was done in elf32_copyargs() in an irix_copyargs() function, which can be found in sys/compat/irix/irix_exec_elf32.c. This irix_copyargs() function just does elf32_copyargs() job and it enforces the 16-byte alignment of argv[0]. Of course, the irix_copyargs() function must be used in the es_copyargs field of the struct execsw for IRIX in sys/kern/exec_conf.c.

With this adjustment, static IRIX binaries were able to read their arguments and environment correctly.

The ELF Auxiliary Table

The ELF auxiliary table is used by dynamic linkers to gather information about the program they are about to link and launch. It is a table of pairs (type, value) stored on the stack. These pairs are called auxiliary vectors. Documentation of the available vector types can be found in NetBSD's /usr/include/elf.h:

#define AT_NULL         0       /* Marks end of array */
#define AT_IGNORE       1       /* No meaning, a_un is undefined */
#define AT_EXECFD       2       /* Open file descriptor of object file */
#define AT_PHDR         3       /* &phdr[0] */
#define AT_PHENT        4       /* sizeof(phdr[0]) */
#define AT_PHNUM        5       /* # phdr entries */
#define AT_PAGESZ       6       /* PAGESIZE */
#define AT_BASE         7       /* Interpreter base addr */

The ELF interpreter will use these to discover the address of the ELF program header, which lists the executable ELF sections in the executable, for instance. This is used to discover the list of required shared libraries and the symbol table location.

We use the same stack dumping technique as described in the previous section to discover what information the IRIX kernel lists in the ELF auxiliary table. Things are just a bit different: when running a dynamic executable, the kernel launches the interpreter first, not an ELF section from the program. Therefore, we cannot just set a breakpoint at a collected address using objdump(1) on our program. Instead we need to set the breakpoint at the interpreter's entry point. On IRIX, the interpreter is libc itself, hence we can discover the entry point of a dynamic binary by using objdump(1) on libc:

$ objdump -f /lib/libc.so.1

/lib/libc.so.1:     file format elf32-bigmips
architecture: mips:6000, flags 0x00000150:
HAS_SYMS, DYNAMIC, D_PAGED
start address 0x0fae0774

We just have to set up the breakpoint at 0x0fae0778, and we can see the stack as it is set up by the IRIX kernel. The ELF auxiliary table appears after the envp array. IRIX sets up the following vector types: AT_PHDR, AT_PHENT, AT_PHNUM, AT_ENTRY, AT_BASE, and AT_PAGESZ. Once we know what should be in it, then it is quite easy to add the code to copy this table to our irix_copyargs() function. This enables dynamic binaries to start, but they quickly die, complaining that a mysterious system call named syssgi() was not implemented. We will have a closer look to syssgi() in a future article.

Setting Up the CPU Registers on Startup

The CPU registers are set on startup by the function pointed by the e_setregs field of the struct emul. IRIX emulation uses the NetBSD native function, which is simply called setregs(), and this works for o32.

For n32 binaries, however, using setregs() led to an unpleasant crash before the first system call. The crash was caused by a SIGILL signal. This signal is sent by a trap raised by an illegal instruction.

As before, gdb is a good tool to help us understand what went wrong here. There are several reason why programs could issue an illegal instruction: Did we start the program at its entry point? Or did we corrupt the stack and return in a random place after a function call? Is it another problem?

Using gdb on a static n32 binary :
$ objdump -f sh
(snip)
start address 0x0e00ba44

$ gdb ./sh
(gdb) b *0x0e00ba48
Breakpoint 1 at 0xe00ba48
(gdb) run
Starting program: ./sh

Breakpoint 1, 0xe00ba48 in ?? ()
(gdb) info registers
          zero       at       v0       v1       a0       a1       a2       a3
 R0   00000000 00000000 00000000 00000000 7fffe9d8 00000000 00000000 0e090000
            t0       t1       t2       t3       t4       t5       t6       t7
 R8   00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
            s0       s1       s2       s3       s4       s5       s6       s7
 R16  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
            t8       t9       k0       k1       gp       sp       s8       ra
 R24  00000000 00000000 00000000 00000000 00000000 7fffe9d8 00000000 00000000
            sr       lo       hi      bad    cause       pc
      0000ff13 00000000 00000000 0e00ba44 00000024 0e00ba48
           fsr      fir       fp
      00000000 00000000 00000000
(gdb) x/26i 0xe00ba44
0xe00ba44:      lui     $a3,0xe09
0xe00ba48:      lw      $a0,0($sp)
0xe00ba4c:      addiu   $a3,$a3,200
0xe00ba50:      lw      $a3,0($a3)
0xe00ba54:      addiu   $a1,$sp,4
0xe00ba58:      li      $at,-16
0xe00ba5c:      lui     $gp,0xe09
0xe00ba60:      and     $sp,$sp,$at
0xe00ba64:      addiu   $a2,$a1,4
0xe00ba68:      sll     $v0,$a0,0x2
0xe00ba6c:      addiu   $gp,$gp,27744
0xe00ba70:      addiu   $sp,$sp,-16
0xe00ba74:      bnez    $a3,0xe00ba88
0xe00ba78:      addu    $a2,$a2,$v0
0xe00ba7c:      lui     $at,0xe09
0xe00ba80:      addiu   $at,$at,200
0xe00ba84:      sw      $a2,0($at)
0xe00ba88:      lui     $at,0xe09
0xe00ba8c:      addiu   $at,$at,15424
0xe00ba90:      sw      $a0,0($at)
0xe00ba94:      lui     $at,0xe09
0xe00ba98:      addiu   $at,$at,15456
0xe00ba9c:      sw      $a1,0($at)
0xe00baa0:      sd      $zero,8($sp)
0xe00baa4:      jal     0xe0715dc

(gdb) c
Continuing.

Program received signal SIGILL, Illegal instruction.
warning: Hit heuristic-fence-post without finding
warning: enclosing function for address 0xe00baa0
0xe00baa0 in ?? ()

The problem was caused by the sd instruction, which stands for "store double word". The credits for debugging this go to Wayne Knowles: the sd instruction is only allowed when the processor is running in 64-bit mode. Execution of sd in 32-bit mode causes a reserved instruction exception. The kernel turns this exception into a SIGILL signal.

The solution is to set up the processor in 64-bit mode for execution of n32 binaries. This is done by setting a flag in the SR register. This flag is called MIPS3_SR_UX in NetBSD's sys/arch/mips/include/psl.h. The fix to this problem is therefore to write a setregs_n32() function to set up the registers for IRIX n32 binaries. This function just sets the MIPS3_SR_UX flag and then calls the regular setregs().

In the exec switch from sys/exec_conf.c, IRIX has two entries: one for o32 binaries, which uses an o32 probe function called irix_elf32_probe_o32(); and the other the emul_irix_o32 struct emul (defined in sys/compat/irix/irix_exec.c). This struct emul contains a pointer to setregs(). The other entry is for n32 binaries; it uses irix_elf32_probe_n32() and the emul_irix_n32 struct emul. emul_irix_n32 contains a pointer to setregs_n32() as the function to set up CPU registers.

With this setregs_n32() function, n32 binaries are able to start up and do a few system calls. They crash on the first system call manipulating 64-bit data, which is the case for mmap(), lseek(), or stat(). Simple static n32 are hence actually able to work because they do not need mmap() to link. It is possible to run a static n32 /bin/sh and use it to launch shell commands, but it quickly dies (as soon as it hits a system call that uses 64-bit data, in fact).

To reliably run n32 binaries, we need 64-bit support in the kernel. This will be discussed in more details in a later article.

Emmanuel Dreyfus is a system and network administrator in Paris, France, and is currently a developer for NetBSD.

Return to the BSD DevCenter.

Copyright © 2007 O'Reilly Media, Inc.