August 29, 2025

Sun 3 bootrom souce - the Sun 3/80 - test the lance driver

What I want to do is to test the lance ethernet driver that I am able to compile from the bootrom source. The only hardware I have with a lance ethernet is my sun 3/80. The idea is not to monkey with the bootrom in my 3/80, but to use that bootrom to boot some standalone code I write. That standalone code will be a build of the minimal source I can prune down to run the lance ethernet driver as if was part of a bootrom.

So what will happen, when this all works, is that the sun 3/80 will network boot my code (using rarp and tftp). Then that code will turn around and do exactly the same thing -- run rarp and tftp to boot some other standalone code. It will need to be "some other" code because it will be loaded to and run at a different address than the 0x4000 that the test standalone will be running at.

Once all this works, I will take the next step (probably) and work up a DHCP client to replace RARP.

Doing all this has been an excellent way to really understand things that go on with the bootrom device drivers, especially the ethernet. It has also been a great chance to dig into some Sun 3/80 details I never looked at closely before.

Status report - 8-29-2025

The first step was to code up "fixture.c" which supplies a variety of things that are present in the bootrom code, but I don't want to import in their entirety. I also pull in inet.c and tftp.c verbatim, making a few minor changes. After a days work, it all compiles without errors.

Testing revealed some surprises with virtual address faults to unexpected addresses. The tftp code simply uses a block of memory at 0x3000 for the data structures it needs (a better scheme would use some kind of allocator). The code as trying to use the 3/160 rom base address to access the romvec table -- this was a simple change.

Now I have added a multitude of printf statements to see what the code is doing. It is trying to do the RARP, sending a packet and awaiting a reply.

I fire up wireshark and am pleased to find that "rarp" makes a perfectly good capture filter. It sees the real bootrom sending a rarp request, and my linux machine replying. But it does not see a second rarp request from my test code (though the printf statements indicate it things it sent a packet), so that is the problem I am now troubleshooting.

DMA addresses

The lance chip exposes itself as two 16 bit registers. It does most of its work by DMA interacting with structures in memory.

The lance chip has a 24 bit addressing capability. So what happens as far as the upper 8 bits in the Sun3 memory world. As near as I can tell this happens: 

on the sun 3/160 the upper 8 bits get set to 0x0f
on the sun 3/80 the upper 8 bits get set to 0xff
The bootrom code uses a simple memory allocator (in monalloc.c) to dole out memory suitable for DMA. It starts (on the 3/80) from a base address of 0xfff0_0000 -- note that the virtual address map has this mapped to physical memory at 0x00ff8000 which is near the top of the actual 16M of physical memory on my sun3/80. This is so for 3 pages (24K) then the mapping does other things. Perhaps 24K is adequate for the lance structures.

On the 3/160 the DMA memory allocator starts off at 0x0ff0_0000, but I have not worked out the full details of the address mapping as I have for the 3/80. But I do know the if_ie.c driver works just fine using that, so I am not worried much about it.

Debugging 9-12-2025

I perform the lance chip INIT, then poll watching the csr. I see status 0x0033 after I set the START bit. This is OK (tx-on, rx-on, start, and init). It turns out you cannot clear the init bit, so it is OK that it stays on. After a variable amount of time the status changes to 0x90b3. We get an INTR (interrupt) bit to tell us something happened. The upper 2 bits are ERR and MISS -- so we missed a packet.

We aren't expecting a packet, so this must be a broadcast that came our way. What does the driver do to filter out broadcast packets? When waiting for a REVARP reply, it does a full chip reset after any packet that it does not recognize. Certainly a brute force approach, but OK for a bootrom I suppose.

Once we get past REVARP and start running TFTP, we get input via ip_input, which gets packets from the lance "poll" routine. The poll routine only watches the RINT bit in the lance CSR, so it clearly doesn't expect the MISS bit to be set, nor does it check the ERR bit. It expects to receive a packet and then let TFTP sort out whether it is a broadcast packet that it doesn't care about. The tftp code has a list of reasons to ignore packets. It wants only in-sequence DATA packets.

But all this is somewhat beside the point. Our receive buffer ring must be faulty for the chip to decide there is no buffer for the packet, whatever it might be.

I start fooling around looking at the state of the lance chip and its memory data structures as left by the sun 3/80 bootrom. I see the CSR set to 0x4 (the chip is stopped). The buffer rings make sense and are in memory at 0xfff02000, at least the init block is. The Rx and Tx rings are at 0xfff05390 and 0xfff053d8. Interestingly, there are 8 buffers in the Rx ring (the driver I have source for only uses 2 buffers). Here is a clumsy dump of the Rx ring: 

RMD 0 FFF05890: 00002046 00000080 000000F0 0000F9C0 00000000
RMD 1 FFF05898: 00002686 00000080 000000F0 0000F9C0 00000000
RMD 2 FFF058A0: 00002CC6 00000080 000000F0 0000F9C0 00000000
RMD 3 FFF058A8: 00003306 00000080 000000F0 0000F9C0 00000000
RMD 4 FFF058B0: 00003946 00000080 000000F0 0000F9C0 00000000
RMD 5 FFF058B8: 00003F86 00000080 000000F0 0000F9C0 00000000
RMD 6 FFF058C0: 000045C6 00000080 000000F0 0000F9C0 00000000
RMD 7 FFF058C8: 00004C06 00000080 000000F0 0000F9C0 00000000
So, the buffers sit between the init block and the descriptors, which is fine of course. The flags are all 0x80, which is the OWN bit set (so the lance is free to use these).

The transmit descriptor looks like this: 

TMD 0 FFF058D8: 00005246 00000003 000000F0 0000FFC4 00000000 (60)

While the Receive buffers are all 1600 bytes in size, the Tx buffer is only 60 bytes! I suppose this is adequate for ARP, RARP, and TFTP. This is different than the code I have where both Rx and Tx buffers are 1600 bytes in size.

Here the Tx flags are 0x03 which are STP and ENP. The OWN bit is not set, which also makes sense, the LANCE has finished sending and has handed the buffer back to the driver.

Try some crazy thingss

Now we try just setting the START bit, we see this status immediately and endlessly. 
LE csr0 = 000090B2
Next I try repeating the initialization and then turning on the start bit. I see: 
tjt - lereset init loop start CSR0: 00000001
tjt - lereset init loop done CSR0, timeout: 00000181 999
tjt - lereset init loop done CSR0, timeout: 00000001 999
LE csr0 = 000090B3
LE csr0 = 000090B3
This is exactly what I see using the source driver. I am surprised. This makes me think that the chip setup and data structures are fine and there is some other magic that is needed that I need to learn about. This may require studying the 3/80 bootrom disassembly.

Unix sources

I take a look at the Sun SS411 source tree. I see: 
sys/sun3x/locore.s
sys/sunif/if_le.c
Doing grep on DVMA or dvma yields interesting things.

The "ENABLE" register is very different on the sun3x. For one thing is gets assigned a "plain old" address among the OBIO (on-board IO) registers, which end up needing to be mapped into the virtual address space when the MMU is enabled. SunOS maps it to 0xfedf8000, but the bootrom has a different place for it: 

Sys ENA Reg     0xFEF16000:      0x61000000:
On the sun3 architecture it is an 8 bit register at 0x40000000 in a special address space and must be accessed via a movsb instruction. On the sun3x, it is a 16 bit register at the address shown above.

Relevant files in an old version of the NetBSD sources are: 

NetBSD-2.0/usr/src/sys/arch/sun3/sun3x/dvma.c
NetBSD-2.0/usr/src/sys/arch/sun3/dev/if_le.c

I find this comment in the NetBSD sources: 

  * Enable DVMA in the System Enable register.
  * Note:  This is only necessary for VME slave accesses.
  *        On-board devices are always capable of DVMA.
Feedback? Questions? Drop me a line!

Tom's Computer Info / tom@mmto.org