August 20, 2018

U-Boot - ARM D cache enable

All indications are that for the ARM to happily handle unaligned address accesses, two things are required. The "A" bit in the sctlr register must be cleared. Apparently also the D cache needs to be enabled. Of course just flipping the bit in the sctlr register to enable the D cache is only the smallest part of the story. The MMU also needs to be initialized, with due care to not enable caching for memory areas that contain device registers.

U-Boot does have a routine dcache_enable(), which in fact does do MMU setup. Is it just a simple matter of calling this from some appropriate place? It is worth noting that in the file arch/arm/mach-nexell/spl/spl.c this is called (but my Fire3 does not use an SPL, so maybe this is why I don't get the cache enabled). Also the code in this routine makes it dependent on CONFIG_SPL_CLI_FRAMEWORK which is mysterious.

These are the relevant bits in the SCTLR register (from arch/arm/include/asm/system.h): 

/*
 * SCTLR_EL1/SCTLR_EL2/SCTLR_EL3 bits definitions
 */
#define CR_M            (1 << 0)        /* MMU enable                   */
#define CR_A            (1 << 1)        /* Alignment abort enable       */
#define CR_C            (1 << 2)        /* Dcache enable                */
#define CR_SA           (1 << 3)        /* Stack Alignment Check Enable */
#define CR_I            (1 << 12)       /* Icache enable                */
#define CR_WXN          (1 << 19)       /* Write Permision Imply XN     */
#define CR_EE           (1 << 25)       /* Exception (Big) Endian       */
It looks like arch/arm/cpu/armv8/cache_v8.c is the code of interest. After tossing some printf statements in here, two things become clear:
  1. This does indeed contain the routine dcache_enable()
  2. The configuration variable CONFIG_SYS_DCACHE_OFF is defined.
With this config variable defined, a stub routine gets compiled. This variable is not in my .config file, so it is getting mysteriously generated. This gives us a choice. We can try to figure out what "clever" thing the U-Boot config system is doing -- or we can yank the code we want out, make it static in board.c and call it and force the issue -- a bad policy, but given the inscrutable nature of the U-Boot config process, very tempting.

There is some discussion of this in doc/README.arm-caches, which coaches us as follows: 

Enabling D-cache:
- Make sure CONFIG_SYS_DCACHE_OFF is not set and call dcache_enable().
If it were only that simple. How do we make sure? I certainly don't explicitly set this in the config file for my target. A grep turns this up: 
include/configs/s5p6818_nanopi3.h:#define CONFIG_SYS_DCACHE_OFF
I comment out the line in this file and rebuild. This does the trick -- I see the sctrl register has the following value: 0x30c51835, and I don't need to jam in my own call to dcache_enable(), it now gets done somewhere early in initialization. The only bad part is that my unaligned memory accesses still give my sychronous aborts.

Lessons Learned

Edit the file include/configs/s5p6818_nanopi3.h and comment out CONFIG_SYS_DCACHE_OFF to enable the D cache. No arcane hacking is required.

Many of the mysterious "hidden CONFIG settings" I have puzzled over are set in the file in include/configs.
The files in configs are a smaller subset, perhaps the settings people are usually expected to fiddle with.

Have any comments? Questions? Drop me a line!

Tom's electronics pages / tom@mmto.org