Introduction

I put this little document together mostly to educate myself, but I presume it may be useful to others who are interested in understanding or meddling with a NetBSD device driver. A couple of publications with related information are worth mentioning. The first is The Design and Implementation of the 4.4BSD Operating System by McKusick et. al. Chapter 14 is of particular interest, as it discusses system startup and the related issues of kernel configuration, autoconfiguration, device probing, and attaching. Chapter 6 is the other place to look, but it is amazingly thin on real driver issues. What you really want is something like TCP/IP Illustrated, volume 2 by Wright and Stevens, but to my knowledge it does not exist. A distantly related document that is still quite useful is the "Writing Device Drivers" section of the SunOS manuals.

Building a kernel with config

As anyone who has ever built a NetBSD kernel knows, the place where the game starts is /usr/src/sys/arch/MACHINE/conf, where MACHINE is i386 or sun3 or some such. This is where the input file to the config program ( /usr/sbin/config ) lives, and any device will need to be named here, or you won't get far at all. There are actually two parts to the config process, the part that goes on when a kernel is built, and the autoconfiguration part that happens when the kernel actually runs and initializes itself.

You type "config TARGET" where TARGET is the name of the config file, and config creates a directory ../compile/TARGET and generates a number of files in that directory. The files generated are a Makefile, ioconf.c, swapnetbsd.c, and a bunch of include files. The file ioconf.c is of particular interest, as it holds data structures that govern autoconfiguration.

The include files are named xxx.h, where xxx is a device name and consist of a single line of the form:

#define NXXX 1

Autoconfiguration

When a netbsd kernel boots up, one of the most visible aspects of the process is the probing of devices done during what is known as autoconfiguration. Basically, the config file is just a list of devices that are potentially present in the system. What devices are actually present is worked out during autoconfiguration when the kernel probes each device.

NetBSD useds an interesting machine independent autoconfiguration scheme. This scheme (sometimes refereed to as the new config) was developed at Lawrence Berkeley Labs for the sparc port, but now is used for (all?) of the NetBSD ports.

Here is how the game is played: by some means (infinitely interesting, but not to be described in detail here and now), the kernel gets loaded somewhere in memory and is set running. We can traverse the following sequence of entry points, ignoring a lot of details along the way: 

start in sys/arch/MACHINE/MACHINE/locore.s
main() in sys/kern/init_main.c
config_init() in sys/kern/subr_autoconf.c --returns to main()
cpu_startup() in sys/arch/MACHINE/MACHINE/machdep.c
configure() in sys/arch/MACHINE/MACHINE/autoconf.c

Perhaps the most important thing to notice in all of this is the dance back and forth from machine independent to machine dependent code, this is not the last we will see of this.

The cfdata structure

The heart of ioconf.c is an array of cfdata structures, one per device configured in the system. Some of these devices are buses, others are controllers, others are things attached to buses or controllers. Everything is attached to something in a tree-like structure, anchored at the root (which is typically mainbus0 or some such, which is itself anchored to the root, but there is no entry for root in the cfdata array, and there may be multiple roots). The cfdata structure is defined in /usr/src/sys/sys/device.h and looks like this: 
struct cfdata {
	struct	cfattach *cf_attach;	/* config attachment */
	struct	cfdriver *cf_driver;	/* config driver */
	short	cf_unit;		/* unit number */
	short	cf_fstate;		/* finding state (below) */
	int	*cf_loc;		/* locators (machine dependent) */
	int	cf_flags;		/* flags from config */
	short	*cf_parents;		/* potential parents */
	void	(**cf_ivstubs)		/* config-generated vectors, if any */
			__P((void));
};

The first 2 entries point to structures that are within the specific device driver itself. Note that even buses must have device drivers with the entry points that support the autoconfiguration process. The unit number is the digit part of the name (i.e. 0 for sd0). State is either NORM (for fully specified devices like sd0) or STAR (for wildcard specifield devices like sd?). The loc field gives hardware address information. Flags is usually zero, by may be set by a flags entry in the config file. The parents field is almost universally zero. The ivstubs field point to an array of values, terminated by a -1 entry.

The cfattach and cfdriver structures

Each device driver must have an initialized structure of each of these present in it for the autoconfiguration code to latch onto. 
struct cfattach {
	size_t	  ca_devsize;		/* size of dev data (for malloc) */
	cfmatch_t ca_match;		/* returns a match level */
	void	(*ca_attach) __P((struct device *, struct device *, void *));
	/* XXX should have detach */
};

struct cfdriver {
	void	**cd_devs;		/* devices found */
	char	*cd_name;		/* device name */
	enum	devclass cd_class;	/* device classification */
	int	cd_indirect;		/* indirectly configure subdevices */
	int	cd_ndevs;		/* size of cd_devs array */
};

Root devices

The show really starts in configure() in the file: sys/arch/MACHINE/MACHINE/autoconf.c For each root device, this should call config_rootfound("mainbus",NULL) The config_rootfound() routine (in sys/kern/subr_autoconf.c) searches for the requested root device in the cfdata array, and once it finds it, attaches it using config_attach(). The config_attach() routine will ultimately call the xx_attach() entry point in the device driver. For things at the root level (like busses), this will launch a loop thru all devices attached to the bus. For some busses (like pci) it is possible to walk the bus, finding out what is there and then discovering if it matches some configured device. For other busses (like vme, isa) this is impossible, and the thing to do is to use the cfdata array to then probe all devices attached to the bus currently being attached.

Probing, matching, attaching

Each device configured into the system is probed during system boot. Probing is done by calling the match function (typically xx_match()) during the autoconfig process. These used to be called probe functions, but things have changed a bit. The match function may or may not actually touch hardware. Often for a controller, some kind of poking at the hardware is done. In some cases just verifying that a device register can be read or written is considered adequate to say that the probe has succeeded. Some devices are nice enough to have some kind of readable signature that can be verified by software. Ugly cases exist when the same address may contain any one of several devices. A probe for one type of device may severely annoy a device of a different type. (A case in point are certain software configurable wd8013 ethernet cards which can be probed into a state of non-functionality).

Once the probe for a device has succeeded, it is attached. This is basically an opportunity for the driver for the device to initalize itself, allocate resources, and perhaps to launch a new sequence of probing for devices that are attached to it (significantly true for buses and controllers).