If my understanding of the problem is correct I have two alternatives. One is that I can modify my GIC400 driver so that it operates properly in non-secure mode. The other is that I can redo my U-Boot setup so that it does not run in non-secure mode. This is what I did for the Samsung s5p6818 on my NanoPi Fire3 board, and I was entirely satisfied with how things worked.
I should work up a whole page about what I have learned about U-Boot and this whole business, but here is a short note. When the H5 chip comes out of reset, it runs the on chip bootrom code, which looks for something to boot on MMC (the SD card). It it find something with a proper header it will load and run it in a special on chip static ram. This will be the U-Boot SPL. The U-Boot SPL will in turn look for something further along (32K further) and try to load it. What it finds is actually a package containing 3 things. the package is a FIT format image, which holds bl31.bin (the arm trusted firmware), the U-Boot image, and a DTB (device tree blob) that U-Boot needs. The bl31.bin file gets loaded to another special bit of on chip static RAM and it runs first. It will do its thing, then transfer to the main U-Boot image, which is in off chip RAM (where there is plenty of room). The DTB also ends up on off chip RAM.
The culprit as far as causing the transition from secure to non-secure mode (as well as from EL3 to EL2) is the bl31.bin executable that is built from the arm-trusted-firmware package. I could either omit this entirely, or modify it so that it is just a pass through and leaves the processor in secure mode. I am investigating the later option. I did finish this project and bypassed bl31 (but performed the transition from EL3 to EL2). This did not fix my interrupt problem. Too bad!
There is a lot of valuable information in the U-Boot distribution in the file u-boot-2018.09/board/sunxi/README.sunxi64. The instructions for building bl31.bin are:
export CROSS_COMPILE=aarch64-linux-gnu- make PLAT=sun50iw1p1 DEBUG=1 bl31The resulting binary is build/sun50iw1p1/debug/bl31.bin, which I copy into the U-Boot root. U-Boot finds it there and "does the right thing" with it at the end of the build process.
To rebuild bl31.bin, make clean does not quite cut it:
cd build/sun50iw1p1/debug rm bl31.bin rm -rf bl31Remarkably, this compiles quickly and without any compile errors, giving the following output:
CC drivers/arm/gic/arm_gic.c CC drivers/arm/gic/gic_v2.c CC drivers/arm/gic/gic_v3.c CC plat/common/plat_gic.c CC plat/sun50iw1p1/bl31_sunxi_setup.c CC plat/sun50iw1p1/plat_pm.c CC plat/sun50iw1p1/sunxi_security.c CC plat/sun50iw1p1/sunxi_power.c CC plat/sun50iw1p1/sunxi_cpu_ops.c CC plat/sun50iw1p1/plat_topology.c CC plat/sun50iw1p1/sunxi_clocks.c CC plat/sun50iw1p1/sunxi_sip_svc.c CC plat/sun50iw1p1/aarch64/sunxi_common.c CC bl31/bl31_main.c CC bl31/context_mgmt.c CC bl31/cpu_data_array.c CC bl31/runtime_svc.c CC bl31/interrupt_mgmt.c CC bl31/aarch64/bl31_arch_setup.c CC lib/locks/bakery/bakery_lock.c CC services/std_svc/std_svc_setup.c CC services/std_svc/psci/psci_afflvl_off.c CC services/std_svc/psci/psci_afflvl_on.c CC services/std_svc/psci/psci_afflvl_suspend.c CC services/std_svc/psci/psci_common.c CC services/std_svc/psci/psci_main.c CC services/std_svc/psci/psci_setup.c CC services/std_svc/psci/psci_system_off.c CC common/bl_common.c CC common/tf_printf.c CC lib/aarch64/xlat_helpers.c CC lib/stdlib/std.c CC lib/aarch64/xlat_tables.c CC plat/common/aarch64/plat_common.c CC plat/sun50iw1p1/drivers/uart/uart.c AS lib/cpus/aarch64/cortex_a53.S AS plat/common/aarch64/platform_mp_stack.S AS plat/sun50iw1p1/aarch64/plat_helpers.S AS bl31/aarch64/bl31_entrypoint.S AS bl31/aarch64/context.S AS bl31/aarch64/cpu_data.S AS bl31/aarch64/runtime_exceptions.S AS bl31/aarch64/crash_reporting.S AS lib/cpus/aarch64/cpu_helpers.S AS lib/locks/exclusive/spinlock.S AS services/std_svc/psci/psci_entry.S AS services/std_svc/psci/psci_helpers.S AS common/aarch64/debug.S AS lib/aarch64/cache_helpers.S AS lib/aarch64/misc_helpers.S AS plat/common/aarch64/platform_helpers.S PP bl31/bl31.ld.S LD build/sun50iw1p1/debug/bl31/bl31.elf BIN build/sun50iw1p1/debug/bl31.bin Built build/sun50iw1p1/debug/bl31.bin successfully OD build/sun50iw1p1/debug/bl31/bl31.dumpAlso worth recording here, I see the following messages when my current working setup of U-Boot starts up:
NOTICE: BL3-1: Running on H5 (1718) in SRAM A2 (@0x44000) NOTICE: Configuring SPC Controller NOTICE: BL3-1: v1.0(debug):c9f55c02 NOTICE: BL3-1: Built : 15:01:10, Nov 6 2018 NOTICE: DT: sun50i-h5-orangepi-pc2 NOTICE: SCPI: dummy stub handler, implementation level: 000000 INFO: BL3-1: Initializing runtime services INFO: BL3-1: Preparing for EL3 exit to normal world INFO: BL3-1: Next image address: 0x4a000000, SPSR: 0x3c9Here is the tail end of a U-boot build:
LD u-boot OBJCOPY u-boot.srec OBJCOPY u-boot-nodtb.bin start=$(aarch64-linux-gnu-nm u-boot | grep __rel_dyn_start | cut -f 1 -d ' '); end=$(aarch64-linux-gnu-nm u-boot | grep __rel_dyn_end | cut -f 1 -d ' '); tools/relocate-rela u-boot-nodtb.bin 0x4a000000 $start $end make[2]: 'arch/arm/dts/sun50i-h5-orangepi-pc2.dtb' is up to date. FDTGREP dts/dt-spl.dtb CAT u-boot-dtb.bin COPY u-boot.bin SYM u-boot.sym CC spl/arch/arm/cpu/armv8/fwcall.o LD spl/arch/arm/cpu/armv8/built-in.o CC spl/common/spl/spl.o LD spl/common/spl/built-in.o CC spl/lib/display_options.o LD spl/lib/built-in.o LD spl/u-boot-spl OBJCOPY spl/u-boot-spl-nodtb.bin COPY spl/u-boot-spl.bin MKSUNXI spl/sunxi-spl.bin MKIMAGE u-boot.img MKIMAGE u-boot-dtb.img ./"board/sunxi/mksunxi_fit_atf.sh" \ arch/arm/dts/sun50i-h5-orangepi-pc2.dtb > u-boot.its MKIMAGE u-boot.itb CAT u-boot-sunxi-with-spl.bin CHK include/config.h CFG u-boot.cfg CFGCHK u-boot.cfgHere is the tail end of a U-boot build with "make V=1"
make -f ./scripts/Makefile.build obj=spl/drivers/power
rm -f spl/drivers/power/built-in.o; aarch64-linux-gnu-ar rcs spl/drivers/power/built-in.o
make -f ./scripts/Makefile.build obj=spl/drivers/power/pmic
rm -f spl/drivers/power/pmic/built-in.o; aarch64-linux-gnu-ar rcs spl/drivers/power/pmic/built-in.o
make -f ./scripts/Makefile.build obj=spl/drivers/power/regulator
rm -f spl/drivers/power/regulator/built-in.o; aarch64-linux-gnu-ar rcs spl/drivers/power/regulator/built-in.o
make -f ./scripts/Makefile.build obj=spl/drivers/serial
aarch64-linux-gnu-gcc -Wp,-MD,spl/drivers/serial/.serial.o.d -nostdinc -isystem /usr/lib/gcc/aarch64-linux-gnu/7/include -Iinclude -I./arch/arm/include -include ./include/linux/kconfig.h -D__KERNEL__ -D__UBOOT__ -DCONFIG_SPL_BUILD -Wall -Wstrict-prototypes -Wno-format-security -fno-builtin -ffreestanding -std=gnu11 -fshort-wchar -fno-PIE -Os -fno-stack-protector -fno-delete-null-pointer-checks -g -fstack-usage -Wno-format-nonliteral -Werror=date-time -ffunction-sections -fdata-sections -D__ARM__ -mstrict-align -ffunction-sections -fdata-sections -fno-common -ffixed-r9 -fno-common -ffixed-x18 -pipe -march=armv8-a -D__LINUX_ARM_ARCH__=8 -I./arch/arm/mach-sunxi/include -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(serial)" -D"KBUILD_MODNAME=KBUILD_STR(serial)" -c -o spl/drivers/serial/serial.o drivers/serial/serial.c
aarch64-linux-gnu-gcc -Wp,-MD,spl/drivers/serial/.serial_ns16550.o.d -nostdinc -isystem /usr/lib/gcc/aarch64-linux-gnu/7/include -Iinclude -I./arch/arm/include -include ./include/linux/kconfig.h -D__KERNEL__ -D__UBOOT__ -DCONFIG_SPL_BUILD -Wall -Wstrict-prototypes -Wno-format-security -fno-builtin -ffreestanding -std=gnu11 -fshort-wchar -fno-PIE -Os -fno-stack-protector -fno-delete-null-pointer-checks -g -fstack-usage -Wno-format-nonliteral -Werror=date-time -ffunction-sections -fdata-sections -D__ARM__ -mstrict-align -ffunction-sections -fdata-sections -fno-common -ffixed-r9 -fno-common -ffixed-x18 -pipe -march=armv8-a -D__LINUX_ARM_ARCH__=8 -I./arch/arm/mach-sunxi/include -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(serial_ns16550)" -D"KBUILD_MODNAME=KBUILD_STR(serial_ns16550)" -c -o spl/drivers/serial/serial_ns16550.o drivers/serial/serial_ns16550.c
aarch64-linux-gnu-gcc -Wp,-MD,spl/drivers/serial/.ns16550.o.d -nostdinc -isystem /usr/lib/gcc/aarch64-linux-gnu/7/include -Iinclude -I./arch/arm/include -include ./include/linux/kconfig.h -D__KERNEL__ -D__UBOOT__ -DCONFIG_SPL_BUILD -Wall -Wstrict-prototypes -Wno-format-security -fno-builtin -ffreestanding -std=gnu11 -fshort-wchar -fno-PIE -Os -fno-stack-protector -fno-delete-null-pointer-checks -g -fstack-usage -Wno-format-nonliteral -Werror=date-time -ffunction-sections -fdata-sections -D__ARM__ -mstrict-align -ffunction-sections -fdata-sections -fno-common -ffixed-r9 -fno-common -ffixed-x18 -pipe -march=armv8-a -D__LINUX_ARM_ARCH__=8 -I./arch/arm/mach-sunxi/include -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(ns16550)" -D"KBUILD_MODNAME=KBUILD_STR(ns16550)" -c -o spl/drivers/serial/ns16550.o drivers/serial/ns16550.c
aarch64-linux-gnu-ld.bfd -r -o spl/drivers/serial/built-in.o spl/drivers/serial/serial.o spl/drivers/serial/serial_ns16550.o spl/drivers/serial/ns16550.o
aarch64-linux-gnu-ld.bfd -r -o spl/drivers/built-in.o spl/drivers/mmc/built-in.o spl/drivers/serial/built-in.o spl/drivers/gpio/built-in.o spl/drivers/power/built-in.o spl/drivers/power/pmic/built-in.o spl/drivers/power/regulator/built-in.o spl/drivers/block/built-in.o
make -f ./scripts/Makefile.build obj=spl/dts
rm -f spl/dts/built-in.o; aarch64-linux-gnu-ar rcs spl/dts/built-in.o
make -f ./scripts/Makefile.build obj=spl/fs
aarch64-linux-gnu-gcc -Wp,-MD,spl/fs/.fs_internal.o.d -nostdinc -isystem /usr/lib/gcc/aarch64-linux-gnu/7/include -Iinclude -I./arch/arm/include -include ./include/linux/kconfig.h -D__KERNEL__ -D__UBOOT__ -DCONFIG_SPL_BUILD -Wall -Wstrict-prototypes -Wno-format-security -fno-builtin -ffreestanding -std=gnu11 -fshort-wchar -fno-PIE -Os -fno-stack-protector -fno-delete-null-pointer-checks -g -fstack-usage -Wno-format-nonliteral -Werror=date-time -ffunction-sections -fdata-sections -D__ARM__ -mstrict-align -ffunction-sections -fdata-sections -fno-common -ffixed-r9 -fno-common -ffixed-x18 -pipe -march=armv8-a -D__LINUX_ARM_ARCH__=8 -I./arch/arm/mach-sunxi/include -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(fs_internal)" -D"KBUILD_MODNAME=KBUILD_STR(fs_internal)" -c -o spl/fs/fs_internal.o fs/fs_internal.c
aarch64-linux-gnu-ld.bfd -r -o spl/fs/built-in.o spl/fs/fs_internal.o
aarch64-linux-gnu-gcc -E -Wp,-MD,spl/.u-boot-spl.lds.d -D__KERNEL__ -D__UBOOT__ -DCONFIG_SPL_BUILD -D__ARM__ -mstrict-align -ffunction-sections -fdata-sections -fno-common -ffixed-r9 -fno-common -ffixed-x18 -pipe -march=armv8-a -D__LINUX_ARM_ARCH__=8 -I./arch/arm/mach-sunxi/include -Iinclude -I./arch/arm/include -include ./include/linux/kconfig.h -nostdinc -isystem /usr/lib/gcc/aarch64-linux-gnu/7/include -include ./include/u-boot/u-boot.lds.h -include ./include/config.h -DCPUDIR=arch/arm/cpu/armv8 -DLD_MAJOR=2 -DLD_MINOR=29 -ansi -D__ASSEMBLY__ -x assembler-with-cpp -P -o spl/u-boot-spl.lds arch/arm/cpu/armv8/u-boot-spl.lds
(cd spl && aarch64-linux-gnu-ld.bfd -T u-boot-spl.lds --gc-sections -Bstatic --gc-sections --no-dynamic-linker -Ttext 0x10060 arch/arm/cpu/armv8/start.o --start-group arch/arm/mach-sunxi/built-in.o arch/arm/cpu/armv8/built-in.o arch/arm/cpu/built-in.o arch/arm/lib/built-in.o board/sunxi/built-in.o common/spl/built-in.o common/init/built-in.o common/built-in.o cmd/built-in.o env/built-in.o lib/built-in.o disk/built-in.o drivers/built-in.o dts/built-in.o fs/built-in.o --end-group -L /usr/lib/gcc/aarch64-linux-gnu/7 -lgcc -Map u-boot-spl.map -o u-boot-spl)
aarch64-linux-gnu-objcopy -j .text -j .secure_text -j .secure_data -j .rodata -j .data -j .u_boot_list -j .rela.dyn -j .got -j .got.plt -j .binman_sym_table -j .text_rest -j .dtb.init.rodata -j .efi_runtime -j .efi_runtime_rel -O binary spl/u-boot-spl spl/u-boot-spl-nodtb.bin
cp spl/u-boot-spl-nodtb.bin spl/u-boot-spl.bin
./tools/mksunxiboot --default-dt "sun50i-h5-orangepi-pc2" spl/u-boot-spl.bin spl/sunxi-spl.bin
./tools/mkimage -f auto -A arm -T firmware -C none -O u-boot -a 0x4a000000 -e 0 -n "U-Boot 2018.09"" for sunxi board" -E -b arch/arm/dts/sun50i-h5-orangepi-pc2.dtb -d u-boot-nodtb.bin u-boot.img >/dev/null && cat /dev/null
cp dts/dt.dtb u-boot.dtb
./tools/mkimage -f auto -A arm -T firmware -C none -O u-boot -a 0x4a000000 -e 0 -n "U-Boot 2018.09"" for sunxi board" -E -b arch/arm/dts/sun50i-h5-orangepi-pc2.dtb -d u-boot-nodtb.bin u-boot-dtb.img >/dev/null && cat /dev/null
./"board/sunxi/mksunxi_fit_atf.sh" \
arch/arm/dts/sun50i-h5-orangepi-pc2.dtb > u-boot.its
./tools/mkimage -f u-boot.its -E u-boot.itb >/dev/null && cat /dev/null
cat spl/sunxi-spl.bin u-boot.itb > u-boot-sunxi-with-spl.bin
make -f ./scripts/Makefile.autoconf u-boot.cfg
if [ -d arch/arm/mach-sunxi/include/mach ]; then \
dest=../../mach-sunxi/include/mach; \
else \
dest=arch-sunxi; \
fi; \
ln -fsn $dest arch/arm/include/asm/arch
set -e; : ' CHK include/config.h'; mkdir -p include/; (echo "/* Automatically generated - do not edit */"; for i in $(echo "" | sed 's/,/ /g'); do echo \#define CONFIG_$i | sed '/=/ {s/=/ /;q; } ; { s/$/ 1/; }'; done; echo \#define CONFIG_BOARDDIR board/sunxi; echo \#include \; echo \#include \; echo \#include \; echo \#include \; echo \#include \; echo \#include \;) < scripts/Makefile.autoconf > include/config.h.tmp; if [ -r include/config.h ] && cmp -s include/config.h include/config.h.tmp; then rm -f include/config.h.tmp; else : ' UPD include/config.h'; mv -f include/config.h.tmp include/config.h; fi
aarch64-linux-gnu-gcc -E -Wall -Wstrict-prototypes -Wno-format-security -fno-builtin -ffreestanding -std=gnu11 -fshort-wchar -fno-PIE -Os -fno-stack-protector -fno-delete-null-pointer-checks -g -fstack-usage -Wno-format-nonliteral -Werror=date-time -D__KERNEL__ -D__UBOOT__ -D__ARM__ -fno-pic -mstrict-align -ffunction-sections -fdata-sections -fno-common -ffixed-r9 -fno-common -ffixed-x18 -pipe -Iinclude -I./arch/arm/include -include ./include/linux/kconfig.h -nostdinc -isystem /usr/lib/gcc/aarch64-linux-gnu/7/include -DDO_DEPS_ONLY -dM ./include/common.h > u-boot.cfg.tmp && { grep 'define CONFIG_' u-boot.cfg.tmp > u-boot.cfg; rm u-boot.cfg.tmp; } || { rm u-boot.cfg.tmp; false; }
./scripts/check-config.sh u-boot.cfg ./scripts/config_whitelist.txt .
Tom's electronics pages / tom@mmto.org