IRIX Binary Compatibility, Part 6

In a previous article, we studied the IRIX threading model, focusing on how it was possible to emulate it on NetBSD. We now have a good idea of how to launch a native thread on NetBSD, but we still have to discover undocumented IRIX secrets such as the stack layout and the register setup when the native thread is launched by the IRIX kernel. To discover this, we will reverse engineer sproc(2).

The end of this part is about the emulation of IRIX oddities called share groups. We have to play a bit more than usual with the NetBSD virtual memory subsystem in order to get the work done. Working on IRIX turns into an adventure.

Reverse Engineering sproc(2)

We want the CPU register and stack setup of a process just created by sproc(2) on IRIX. We already faced this kind of situation when we had to discover the initial stack and register setup on program startup. If you forget how we handled this situation, please go back to part 2 of this series.

Of course the first idea is to use the same trick: if we use gdb to break at the beginning of the entry function in userland, we will be able to dump the stack and registers. We could imagine that we would set the breakpoint before entering sproc(2) and then continue to see the break in the child after the sproc(2) call.

Things are a bit more complicated now: we would like to set a breakpoint in the child process before it has even been created. This is not possible.

There is a technique for handling this kind of problem, which is to prepare an infinite empty loop at the beginning of the entry function. That way the child process gets caught on userland return, and we can attach gdb to it while it is running. We can see that with the following sample program:

/* sprocchild.c -- A sproc child test program */
#include <stdio.h>
#include <sys/types.h>
#include <sys/prctl.h>

void entry(void *);

int main(void) {
    pid_t pid;

    pid = sproc((void *)*entry, PR_SADDR, (void *)0x42534400);
    printf("parent: sproc() returned %d\n", pid);
    return 0;
}

void entry(void *args) {
    while(1); /* infinite loop */

    printf("child: args = %p\n", args);
    return; 
}

Note we gave the arg argument a funky value so that we can easily recognise it later. Everything is ready; let's start the game!

$ gdb ./sprocchild
(gdb) b sproc
Breakpoint 1 at 0x400aec
(gdb) r       
Starting program: ./sprocchild 
Breakpoint 1 at 0xfa5c0e0: file sproc.s, line 53. 

Breakpoint 1, _sproc () at sproc.s:58
58      sproc.s: No such file or directory.
Current language:  auto; currently asm
(gdb) show reg
(gdb) info reg   
        zero       at       v0       v1       a0       a1       a2       a3
 R0  00000000 100040b8 00000004 00000000 00400cc0 00000040 42534400 7fff2f7c 
(snip)

In registers A0 to A2, we find the arguments to sproc(). The entry function is at 0x0fa5c270, the inh flag is 0x40, and we recognize our arg argument: 0x42534400. Let's explore the entry function:

(gdb) x/10i $a0
0x400cc0 <entry>:       lui     $gp,0xfc1
0x400cc4 <entry+4>:     addiu   $gp,$gp,-19600
0x400cc8 <entry+8>:     addu    $gp,$gp,$t9
0x400ccc <entry+12>:    addiu   $sp,$sp,-32
0x400cd0 <entry+16>:    sw      $ra,28($sp)
0x400cd4 <entry+20>:    sw      $gp,24($sp)
0x400cd8 <entry+24>:    sw      $a0,32($sp)
0x400cdc <entry+28>:    b       0x400cdc <entry+28>
0x400ce0 <entry+32>:    nop

At 0x400cdc we have our infinite loop: a jump (b stands for the MIPS branch instruction) that loops to the current address. Let us remember this address and move forward. We are looking for the system call. Where is it?

 (gdb) x/4i $pc
0xfa5c0e0 <_sproc+20>:  b       0xfa5c138 <_nsproc+24>
0xfa5c0e4 <_sproc+24>:  li      $t0,1129
0xfa5c0e8 <_sprocsp>:   lui     $gp,0x10
0xfa5c0ec <_sprocsp+4>: addiu   $gp,$gp,-15880

No system call here, just a jump to another place. Obviously we are in a libc stub. We want to find the system call itself. Using the si (stands for stepi) command, we execute the branch instruction, and have a look at the destination:

 (gdb) si  
_nsproc () at sproc.s:110
110     in sproc.s
(gdb) x/400i $pc
0xfa5c138 <_nsproc+24>: lw      $t9,-31396($gp)
0xfa5c13c <_nsproc+28>: sw      $s0,56($sp)
0xfa5c140 <_nsproc+32>: sw      $s1,52($sp)
(snip)
0xfa5c1f4 <_nsproc+212>:        move    $s2,$a0
0xfa5c1f8 <_nsproc+216>:        lw      $v0,32($sp)
0xfa5c1fc <_nsproc+220>:        syscall
(snip)

This is probably what we are looking for. Now we can try to break at 0xfa5c1fc and check if we do get there or not.

 (gdb) b *0xfa5c1fc
Breakpoint 2 at 0xfa5c1fc: file sproc.s, line 155.
(gdb) c
Continuing.

Breakpoint 2, _nsproc () at sproc.s:155
155     in sproc.s
(gdb) info reg
        zero       at       v0       v1       a0       a1       a2       a3
 R0  00000000 0fb4f3d8 00000469 000000c9 0fa5c270 00000040 42534400 7fff2f7c 
(snip)

We got it! Maybe you remember that on the MIPS, the V0 register holds the system call number. IRIX system calls have an offset of 1000, so 0x469 (1129) is system call number 129, also known as sproc (remember, these are listed in IRIX's /usr/include/sys.s and in NetBSD's sys/compat/irix/syscall.master).

The second and third arguments to sproc have been left untouched, but the libc stub changed the pointer to the entry function. It was 0x400cc0 when we entered the libc stub, and it is now 0x0fa5c270. Where is this going?

 (gdb) x/4i $a0
0xfa5c270 <_nsproc+336>:        lui     $gp,0x10
0xfa5c274 <_nsproc+340>:        addiu   $gp,$gp,-16272
0xfa5c278 <_nsproc+344>:        addu    $gp,$gp,$s2
0xfa5c27c <_nsproc+348>:        lw      $t9,-29592($gp)

In fact, the libc stub requests the sproc(2) system call to return to another part of the stub. It will probably jump to the entry function at 0x400cc0; since tacks and registers may have changed in the meantime, we do not want to follow this path. No problem, we just have to change A0 to go directly to our infinite loop at 0x400cdc.

 (gdb) set $a0=0x400cdc
(gdb) info reg
        zero       at       v0       v1       a0       a1       a2       a3
 R0  00000000 0fb4f3d8 00000469 000000c9 00400cdc 00000040 42534400 7fff2f7c
(snip)
(gdb) c
Continuing.
parent: sproc() returned 761096

Program exited normally.

Our program went into the sproc() system call and then followed its normal code path and exited, after giving us the child PID. Now the child should be hung in the infinite loop, waiting for us to attach to it with gdb.

 (gdb) attach 761096
Attaching to program `./sprocchild', process 761096
Retry #1:
Retry #2:
Retry #3:
Retry #4:
[New Process 761096]
Symbols already loaded for /usr/lib/libc.so.1
entry () at sprocchild.c:16
16              while(1);
Current language:  auto; currently c
(gdb) x/3i $pc
0x400cdc <entry+28>:    b       0x400cdc <entry+28>
0x400ce0 <entry+32>:    nop
0x400ce4 <entry+36>:    nop

The child hung here just after the return to userland, so we have the virgin CPU registers and stack exactly as the kernel just prepared them. This is wonderful.

 (gdb) info reg
        zero       at       v0       v1       a0       a1       a2       a3
 R0  00000000 fffffffe 00000000 00000001 42534400 00000000 00000000 00000000 
          t0       t1       t2       t3       t4       t5       t6       t7
 R8  00000000 00000000 00000000 00000000 00000001 0000000b 00000001 ffffffff 
          s0       s1       s2       s3       s4       s5       s6       s7
 R16 00400cc0 00000040 0fa5c270 00000001 00000000 00000000 00000000 00000000 
          t8       t9       k0       k1       gp       sp       fp       ra
 R24 00000000 00000000 00000000 00000001 0fb582e0 7bff7fc0 00000000 00000000 
          pc    cause      bad       hi       lo      fsr      fir
     00400cdc 80008000 00000000 00000009 00000001 00000000 00000000 
(gdb) x/20w $sp-16
0x7bff7fb0:     0x00000000      0x00000000      0x00000000      0x00000000
0x7bff7fc0:     0x00000000      0x00000000      0x00000000      0x00000000
0x7bff7fd0:     0x00000000      0x00000000      0x00000000      0x00000000
0x7bff7fe0:     0x00000000      0x00000000      0x00000000      0x00000000
0x7bff7ff0:     0x00000000      0x00000000      0x00000000      0x00000000

At least the stack setup will not be difficult to emulate. On the register front, irix_sproc_child() must prepare the following:

• PC must be set up with the the entry function address
• SP must be set to the stack address
• A0 is set to the arg argument value
• RA and A1 to A3 are set to zero

Other values seems meaningless; they are equal to the registers' values in the parent or set to zero.

irix_sproc_child() uses the registers saved on the trap frame to set up the register values. We already saw, in part 4 of this series how this works, when we studied signal delivery emulation. Here is a code snippet from irix_sproc_child that does this.

struct frame *tf = (struct frame *)p2->p_md.md_regs;

tf->f_regs[PC] = (unsigned long)isc->isc_entry;
tf->f_regs[RA] = 0;

The last job of irix_sproc_child() is to map the new process stack. Once everything is done, the parent awakens, and the child_return() function is called to return to userland. The trap machinery will restore the register values we prepared in the trap frame.

This implementation led to a fair emulation of sproc(2); however, some bugs are awaiting us at the next stage, in the Process Data Area.

Related Reading The Complete FreeBSD Documentation from the Source By Greg Lehey Table of Contents (PDF) Index (PDF) Sample Chapter