VxWorks under Linux

Sometime in 1999

Introduction

Every now and then people stumble across this page and get the wrong idea, so let me lay a bit of groundwork here. First off, you cannot get VxWorks here; I cannot and will not give it away. It is not mine to give, it is a commercial product that you can purchase from Wind River Systems. VxWorks license costum plenty wampum. VxWorks is a great product and I definitely endorse it. However, if I were to give it away, mean people from Wind River would come and hurt me.

What I aim to do here is tell you how I configured the open source tools (binutils and gcc) from the Gnu project to allow me to do VxWorks development under linux. All this was necessary back in 1999 when Wind River was all but hostile (well, maybe they actually were hostile) towards linux. Then your options were to host VxWorks on sun workstations (which we did), or under windows (which was and is unthinkable). My understanding is that the management has changed and the new regime is quite enthusiastic about Linux --- and you can buy their new "workbench" product (which it seems replaces Tornado, or is Tornado under a new name, or maybe is some form of ecclipse) to run on linux (this is circa early 2005, at which time VxWorks is up to version 6.0). Workbench (Tornado?) is an integrated development environment, which I have always eschewed (and despised). Your mileage may vary, in which case you very well may want to avoid all of what follows and use their new products.

This cross compile setup has been in use for regular and serious production development for nearly 10 years (since 1999) and is well proven. It is wonderful how fast compilation is using even somewhat aging intel processors.

File layout

I am sure (positive in fact) that you could do things differently, but here is what I do.

I put all my vxworks stuff into /home/vxworks. On our systems, this ensures that it sticks around unmolested across system reinstalls and gets taken care of in backups, and so forth. It also gets to be on a nice big partition.

Then I create some links so the compilation system can find the components that it needs:

Doing this means that: The /u1/wind thing dates back to when I actually kept all this on a big /u1 partition. Sorry, but you are stuck with it. It would be better to use /opt/vxworks, but it isn't worth rebuilding all the tools for that.

The vxmake script

You could make permanent settings of environment variables in your bashrc or something like that. What I have done instead (to keep an unmolested environment for other projects) is to use a wrapper script I call vxmake instead of make.

For 531, it makes settings like: 

    export WIND_HOST_TYPE=linux
    export WIND_BASE=/u1/wind

    export VX_VW_BASE=/u1/wind/target

    export PATH=/u1/wind/host/linux/bin:$PATH
    export GCC_EXEC_PREFIX=/u1/wind/host/linux/lib/gcc-lib/

For 5.1, it makes settings like: 

    export VX_HOST_TYPE=linux
    export VX_VW_BASE=/opt/vxworks/VW
    export VX_BSP_BASE=/opt/vxworks/VW
    export VX_HSP_BASE=/opt/vxworks/VW

    export PATH=/opt/vxworks/VW/linux/bin:$PATH
    export GCC_EXEC_PREFIX=/opt/vxworks/VW/linux/lib/gcc-lib/

Also, set one of: 

export VX_TARGET_CPU=I80486
export VX_TARGET_CPU=MC68030

What I do, is have a script that looks for hidden files like .vw3 and .vwx86. By default, without any such hidden files, it builds for a mc68k target with vxworks 5.1

It keeps finding the wrong assembler!

The symptoms of trying to run the cross compiler without having the links and GCC_EXEC_PREFIX consistent can be extremely subtle. I got warnings from the assembler "as" about no such option as -mc68020 which was quite mystifying. Another install got warnings about -mno-68881, no such option.

What is happening is that it is finding the correct compiler front end, but then is running the native assembler, not the cross assembler. Running cc68k with the -v switch helped greatly to sort this mystery out. I mention all this in case you get things in a tangle as well.

Building the tools: How it was done back in 1999

This doesn't document my latest build, I was too lazy to do that, and now have forgotten the details, but here is how things were done a time or two before that:

Back in 1999, the native compiler was gcc 2.7.2.3 and I decided to be on the cutting edge and build gcc 2.8.1 as a cross compiler. The targets I wanted to support were the mvme147 and mvme167 boards (m68k CPUs) and the new pc486 target. I also wanted to keep running VxWorks 5.1 (a probably overcautious approach), while beginning to play with 5.3.1. It turns out the same cross compiler works just fine for both, and I deposit the cross compiler into the VW531 directory tree.

Years later the native compiler became some version of egcs, and later yet gcc 3.2 and eventually gcc 3.4. When I attempted to again build binutils-2.9.1 I got into a multitude of difficulties (ultimately best solved by using binutils-2.10.1).

The name of the game is to set up a development environment hosted on a fast Linux PC using the current Linux (some version of redhat, currently fedora core 3), and targetting m68k targets under VxWorks 5.1.1, and both m68k and pc486 targets under VxWorks 5.3.1. You might ask why you need a cross compiler if you are running on an x86 machine anyway, and it boils down to the fact that the native linux compiler generates ELF object files, and VxWorks expects a.out format object files.

Files you will need

In the instructions below, I mention a number of files that must be added or changed in the standard Gnu distribution. These are all available for you to browse through here.

The first thing is to put your Wind River distribution somewhere. I put the two I plan to work with into /opt/vxworks/VW51 and /opt/vxworks/VW531, in my case I just copying verbatim what we had installed on our sun host. This included both the m68k and pc486 packages and all the sun host executables (and tornado stuff, which will almost certainly be of no use on a linux system). I then made an empty directory /opt/vxworks/linux to hold the new linux host tools I am going to build.

The next thing is to get the Gnu tools. Get binutils-2.10.1 and gcc-2.8.1 and put them somewhere. I keep the pristine archives in /u1/archive/Gnu and untar these into /opt/vxworks/tools/src,

What about gdb? I don't use it; if you want it you are on your own. (I am also ignoring C++ for now and the forseeable future.)

After this, the first step is to build binutils ...

Binutils 2.9.1

The story here is simple. DO NOT USE binutils-2.9.1 !!

Back in 1999, I worked up a set of patches for binutils-2.9.1 (at that time 2.9.1 was cutting edge and all there was). Later in 2005, (foolishly) I worked through a bunch of issues with libiberty for binutils-2.9.1 and applied my patches and got it to work. Then I tried binutils-2.10.1 and found that my patches were all there, there were no libiberty issues, and it just WORKS right out of the box.

Binutils 2.10.1

At this point of time, I recommend using 2.10.1 since it fixes serious problems that exist with 2.9.1 and all the more recent versions will not build for me, as detailed below.

I do the build in a separate directory as follows:

A couple of side notes. First I am doing the build in the directory ./vx86 since I anticipate later creating a parallel directory ./vx68k to build the motorola cross tools from the same tree.
Seond, in my case, I often use "make -j 5" since I have an SMP system and want to hurry things along.

I do the m68k build as follows:

Binutils 2.11.2

What I do is to spot check two files: First bfd/config.bfd should have an entry for i[3456]86-*-vxworks that sets target_defvec to i386aout_vec, and an entry for m68k*-*-vxworks setting target_defvec to sunos_big_vec. Also ld/configure.tgt should have an entry for i[3456]86*-vxworks that sets target_emul=i386aout, and m68*-wrs-vxworks to target_emul=sum3.

Once I have verified that these things haven't changed, I go ahead and do the build:

These fail with a bizarre error: 
make[3]: Entering directory `/u1/vxworks/tools/src/binutils-2.15/vx86/gas'
/bin/sh ./libtool --mode=link gcc -W -Wall -Wstrict-prototypes -Wmissing-prototypes -g -O2   -o as-new  app.o as.o atof-generic.o bignum-copy.o cond.o depend.o dwarf2dbg.o dw2gencfi.o ecoff.o ehopt.o expr.o flonum-copy.o flonum-konst.o flonum-mult.o frags.o hash.o input-file.o input-scrub.o listing.o literal.o macro.o messages.o output-file.o read.o sb.o stabs.o subsegs.o symbols.o write.o tc-i386.o obj-elf.o atof-ieee.o  ../bfd/libbfd.la ../libiberty/libiberty.a
gcc -W -Wall -Wstrict-prototypes -Wmissing-prototypes -g -O2 -o as-new app.o as.o atof-generic.o bignum-copy.o cond.o depend.o dwarf2dbg.o dw2gencfi.o ecoff.o ehopt.o expr.o flonum-copy.o flonum-konst.o flonum-mult.o frags.o hash.o input-file.o input-scrub.o listing.o literal.o macro.o messages.o output-file.o read.o sb.o stabs.o subsegs.o symbols.o write.o tc-i386.o obj-elf.o atof-ieee.o  ../bfd/.libs/libbfd.a ../libiberty/libiberty.a
obj-elf.o(.text+0x6a8): In function `obj_elf_change_section':
../../gas/config/obj-elf.c:525: undefined reference to `_bfd_elf_get_sec_type_attr'
collect2: ld returned 1 exit status
make[3]: *** [as-new] Error 1
make[3]: Leaving directory `/u1/vxworks/tools/src/binutils-2.15/vx86/gas'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/u1/vxworks/tools/src/binutils-2.15/vx86/gas'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/u1/vxworks/tools/src/binutils-2.15/vx86/gas'
make: *** [all-gas] Error 2
(/opt/vxworks/tools/src/binutils-2.15/vx86) cholla $

I am too lazy to fight my way through this now, but what I will try quickly is the very latest version of binutils to see if this gets fixed:

Binutils 2.1.15

Doing the spot check on tgt/config.bfd reveals that things have changed since back in 2001 when 2.11.2 was around. Without a bit of hacking the i386 vxworks stuff will build for an ELF target, so I make changes as follows:

ld/configure.tgt

i[3-7]86-*-vxworks*)   targ_emul=elf_i386 ;;
 .... becomes ...
i[34]86-*-vxworks*)     targ_emul=i386aout ;;
i[5-7]86-*-vxworks*)    targ_emul=elf_i386 ;;

bfd/config.bfd

i[3-7]86-*-vxworks)
  targ_defvec=bfd_elf32_i386_vec
  targ_underscore=yes
  ;;
  .... becomes ...
i[34]86-*-vxworks)
 targ_defvec=i386aout_vec
 targ_underscore=yes
 ;;
i[5-7]86-*-vxworks)
  targ_defvec=bfd_elf32_i386_vec
  targ_underscore=yes
  ;;
Here I am letting i386-wrs-vxworks be my target and allowing i586-wrs-vxworks remain for whatever new things are going on.

The make blows up again as follows: 

obj-elf.o(.text+0x6a8): In function `obj_elf_change_section':
../../gas/config/obj-elf.c:525: undefined reference to `_bfd_elf_get_sec_type_attr'
collect2: ld returned 1 exit status
At this point I give up on this.
Something to do on a rainy day.

Gcc 2.8.1

First, I do this as a native compiler using gcc-3.4 (which is the compiler I have as the native compiler on my host system as of 2005). Note that I introduce the LANGUAGES=c argument. If you don't do this, it will try to build objective C and perhaps also C++ which I neither want nor need. Not only that, the attempt to build objective C fails and rather than try to debug something I am not even interested in, I just avoid it altogether. All this goes quite smooth, and this new native compiler handles some trivial test cases, so on to the cross compilers. Building gcc is pretty much the same game as for binutils. Three files need to be changed or added as follows.

A few lines need to be added to ./configure as shown in the following patch: 

diff -au zold/configure znew/configure
--- zold/configure	2005-02-25 18:33:26.911872848 -0700
+++ znew/configure	2005-02-25 18:33:30.072225357 -0700
@@ -3160,6 +3160,12 @@
 		;;
 	i370-*-mvs*)
 		;;
+	# tjt
+	i[34567]86-*-vxworks*)
+		tm_file=i386/vx386.h
+		tmake_file=i386/t-vxworks
+		use_collect2=yes
+		;;
 	i[34567]86-ibm-aix*)		# IBM PS/2 running AIX
                 if [ x$gas = xyes ]
 		then

The following two files are altogether new (get them via the link above). They aren't altogether new, they are modeled after other files for other systems -- but they do the job.

Once the files are in place, I again create a separate directory ./vx86 to build in, and do the following:

After doing this build again in February, 2005, I get object files the same sizes and contents as with the old compiler ... and no segfaults with missing include files, so it is a step forward!

gcc for the m68k targets

This is pretty simple after doing the x86. Binutils and gcc support the target m68k-wrs-vxworks right out of the box, and that is good enough for me.
We have already built binutils for the m68k, building gcc goes like this:

One obnoxious thing:

I always get an irritating error message from the m68k flavor of the cross compiler (not with the i386 flavor) and I hate having long builds cluttered up with this fussing about CPU being redefined ... 
$ vxmake main.o
cc68k -I/opt/vxworks/VW/h -I/opt/vxworks/VW/h/types -I/opt/vxworks/VW/h/netinet -DCPU=MC68030 -DVXWORKS -fno-builtin -nostdinc -ansi -pedantic -c main.c
*Initialization*:1: warning: `CPU' redefined
*Initialization*:1: warning: this is the location of the previous definition

Now that I can finally rebuild the cross compiler it is time to fix this problem.
I edit gcc-2.8.1/config/m68k/vxm68k.h and comment out a big block of code that sweats and fusses trying to define CPU as a default setting for the preprocessor. Heck with that, I will rely on my makefile to do this as I always have and voila!! Rebuild gcc and no more funky warnings.

Gcc 2.95.3

My initial attempts here failed, so the first thing to do as a sanity check would be to build a native compiler. The native compiler build seems to work just fine.
Now, I poke around in the source to see how the files to configure the vxworks cross compiler are set up. The relevant places to peek at are: As near as I can tell, the 2.95.3 release has proper vxworks targets set up, although it looks like they set up the CPU definitions, so I will either have to comment that stuff out, or change what I do in my makefiles. This grinds to a halt as follows: 
rm -f tmplibgcc2.a
for name in _muldi3 _divdi3 _moddi3 _udivdi3 _umoddi3 _negdi2 _lshrdi3 _ashldi3 _ashrdi3 _ffsdi2 _udiv_w_sdiv _udivmoddi4 _cmpdi2 _ucmpdi2 _floatdidf _floatdisf _fixunsdfsi _fixunssfsi _fixunsdfdi _fixdfdi _fixunssfdi _fixsfdi _fixxfdi _fixunsxfdi _floatdixf _fixunsxfsi _fixtfdi _fixunstfdi _floatditf __gcc_bcmp _varargs __dummy _eprintf _bb _shtab _clear_cache _trampoline __main _exit _ctors _pure; \
do \
  echo ${name}; \
  /opt/vxworks/tools/src/gcc-2.95.3/vx86/gcc/xgcc -B/opt/vxworks/tools/src/gcc-2.95.3/vx86/gcc/ -B/opt/vxworks/linux/i386-wrs-vxworks/bin/ -I/opt/vxworks/linux/i386-wrs-vxworks/include -O2  -DCROSS_COMPILE -DIN_GCC     -g -O2 -I./include   -g1 -DHAVE_GTHR_DEFAULT -DIN_LIBGCC2 -D__GCC_FLOAT_NOT_NEEDED   -I. -I../../gcc -I../../gcc/config -I../../gcc/../include -c -DL${name} \
       ../../gcc/libgcc2.c -o ${name}.o; \
  if [ $? -eq 0 ] ; then true; else exit 1; fi; \
  i386-wrs-vxworks-ar rc tmplibgcc2.a ${name}.o; \
  rm -f ${name}.o; \
done
_muldi3
../../gcc/libgcc2.c:41: stdlib.h: No such file or directory
../../gcc/libgcc2.c:42: unistd.h: No such file or directory
make[1]: *** [libgcc2.a] Error 1
make[1]: Leaving directory `/u1/vxworks/tools/src/gcc-2.95.3/vx86/gcc'
make: *** [all-gcc] Error 2
The usual mischief with include files ... I'll have to look at this later (or just stick with gcc 2.8.1). If I ever get to the 68k targets, they will go like this: Notice that using LANGUAGES is now deprecated, and you should use --enable-languages to avoid trying to build the other members of the GCC compiler family.

This build is still in progress and has yet to work properly.

Final Setup and a few last things

Now you are almost there. What I do next is to go into /opt/vxworks/linux/bin and make a bunch of links like: I have a script called LINKS that does all of this you should want to do. Then you are ready to try things out. What I did was try to rebuild a vxWorks kernel for the pc486 target. Some of the host utilities are missing to to do this. Some like vxsize are just shell scripts that can be copied from whatever host distribution you have. Others like munch and xsym I was able to hack together by dead reckoning and provide in my package of files.

I use a script called vxmake which is in my path, to invoke make with the appropriate environment. This script sets up an environment so that to build a kernel for instance, I just do this:

Have any comments? Questions? Drop me a line!

Adventures in Computing / tom@mmto.org