September 26, 2016

The Kyu real-time operating system -- C compilers

I do my development work under linux, like any sane person would do. I use Fedora linux (currently Fedora 24) and use the "out of the box" version of the gcc compiler. This is a cross compiler that runs on my x86_64 system and generates code for the ARM. I use: 
gcc-arm-linux-gnu-6.1.1
I was previously using version 5.3.1, but am now using 6.1.1. There is some danger in this, but my philosophy is to stick with the latest compiler and sort out any problems that come along. This might be more risky when active development stops and somebody tries to compile the Kyu sources with a much more up to date compiler. Some projects (like Xinu) bundle the compiler along with the project, and indeed if you want to cover all of your bases, this is the right thing.

There is also "arm-none-eabi-gcc-cs", which is the "Code Sourcery" compiler. Despite some efforts on my part to understand the differences, I have never straightened it out and just ignore it.

Linux include file headaches

The linux kernel include files are a complex system, full of obscure interdependencies, and relatively fragile. Why does this matter for Kyu?

Well it shouldn't, but it does, and eventually it won't.

I have "borrowed" a few library routines from the linux kernel source. You are allowed to do that with open source (that is what open source is all about, eh?). Why rewrite printf(), why not get some highly optimized memcpy() ans so forth. My approach to doing this was to pull in the C files for things like printf() and memset(), then import the various header files as whole directories, rather than trying to detangle and minimize the header file maze.

This works, but is not an elegant situation and is something I intend to reorganize. A recent compiler update has made this more urgent. Now that I am using the 6.1.1 compiler, I cannot build Kyu due to messages like this: 

arm-linux-gnu-gcc -DKYU -nostdinc -fno-builtin -Wno-implicit-function-declaration -fno-omit-frame-pointer -marm -march=armv7-a -msoft-float -I.. -Iinclude -Iinclude/uapi -Iinclude/generated/uapi
 -Iarch/arm/include -Iarch/arm/include/uapi -Iarch/arm/include/generated -Iarch/arm/include/generated/uapi -include include/linux/kconfig.h
 -isystem /usr/lib/gcc/arm-linux-gnueabi/4.9.2/include -isystem /usr/lib/gcc/arm-linux-gnueabi/6.1.1/include
 -D__KERNEL__ -D__LINUX_ARM_ARCH__=5 -DCC_HAVE_ASM_GOTO -o lib/string.o -c lib/string.c
In file included from include/linux/compiler.h:54:0,
                 from include/uapi/linux/stddef.h:1,
                 from include/linux/stddef.h:4,
                 from include/uapi/linux/posix_types.h:4,
                 from include/uapi/linux/types.h:13,
                 from include/linux/types.h:5,
                 from lib/string.c:22:
include/linux/compiler-gcc.h:107:30: fatal error: linux/compiler-gcc6.h: No such file or directory
 #include gcc_header(__GNUC__)
                              ^
compilation terminated.
../Makefile.inc:70: recipe for target 'lib/string.o' failed
The astute reader will notice an issue immediately in the above, namely the path "/usr/lib/gcc/arm-linux-gnueabi/4.9.2/include". Sadly though, just tidying this up does not fix the problem, it is more fundamental. The root of things here, insofar as I understand it is that the linux kernel performs checks on the compiler version and may refuse to be compiled by versions outside of some range (old or new).

Let's look into this more deeply. First of all, the file "compiler-gcc6.h" does not even exist on my system anywhere (based on running "locate compiler-gcc6.h"). Deeper investigation shows that it is gone from the linux kernel now; it and its brothers are gone, never to return, as you will learn if your continue to bear with me.

What about the file "include/linux/compiler.h". This would be (probably) "Kyu/src/linux/include/linux/compiler.h". It in turn includes "#include ", which is no doubt "Kyu/src/linux/include/linux/compiler-gcc.h" (at least it is if we have our compiler options right, in particular "-nostdinc"). The rub is this inside of compiler-gcc.h 

#define __gcc_header(x) #x
#define _gcc_header(x) __gcc_header(linux/compiler-gcc##x.h)
#define gcc_header(x) _gcc_header(x)
#include gcc_header(__GNUC__)
The include file is building the filename dynamically based on the compiler version and thus looking for the file "linux/compiler-gcc6.h", which does not exist in the kernel sources I "borrowed". The directory has the following:

[tom@trona linux]$ ls -l compiler* -rw-rw-r-- 1 tom tom 320 Jun 6 2015 compiler-clang.h -rw-rw-r-- 1 tom tom 635 Jun 6 2015 compiler-gcc3.h -rw-rw-r-- 1 tom tom 3089 Jun 6 2015 compiler-gcc4.h -rw-rw-r-- 1 tom tom 2484 Jun 6 2015 compiler-gcc5.h -rw-rw-r-- 1 tom tom 4280 Jun 6 2015 compiler-gcc.h -rw-rw-r-- 1 tom tom 14629 Jun 6 2015 compiler.h

It goes deeper than just finding a compiler-gcc6.h in the new kernel headers. There is no such thing in the new linux kernel. This whole scheme has been done away with.

So here is what I did, playing fast and loose: 

cd Kyu/src/linux/include/linux
mkdir compiler-gcc_OLD
mv compiler-gcc*.h compiler-gcc_OLD
cp /usr/src/kernels/4.7.4-200.fc24.x86_64+debug/include/linux/compiler-gcc.h .
An amazingly this works, and I can compile Kyu now. I do need to clean up this mess of "borrowed" library functions, but now we have the opportunity to procrastinate on that some more.

If it is any consolation, U-boot has the same issue in the linux headers it borrowed.

Have any comments? Questions? Drop me a line!

Kyu / tom@mmto.org