January 9, 2023

Kyu networking -- my introduction to MDIO

For months I have worked with Allwinner H3 code on the NanoPi Neo. Just for fun, I dug out a bona fide Orange Pi board (my Orange Pi PC plus). This has 1G of ram and is on a bigger PCB with more IO, but internally is the same as the Neo. I cable it up and in no time it boots "orange.bin" which is my latest build that I was running on the Neo. And indeed, a 200K transfer takes 1.11 seconds indicating that the network is running at 10 Mbit. It does say it is running the CPU clock at 1008 Mhz (as I remember, the Neo ran at 400). I see no messages about how the ethernet autonegotiation was handled.

I play around with U-Boot using the U-Boot console. For whatever reason "mdio" and "mii" are the names for the ethernet Phy in the U-Boot world.

=> mdio list
ethernet@1c30000:
1 - Generic PHY <--> ethernet@1c30000
=> mii device
MII devices: 'ethernet@1c30000'
Current device: 'ethernet@1c30000'
=> mii info
PHY 0x00: OUI = 0x50000, Model = 0x00, Rev = 0x00,  10baseT, HDX
PHY 0x01: OUI = 0x50000, Model = 0x00, Rev = 0x00,  10baseT, HDX
The last command indeed reports 10baseT, but deep knowledge would be required to be able to change it, if it is possible at all.

I spend some time looking over the board trying to identify a PHY chip. It turns out the PHY is in the H3 chip itself. The datasheet says (in the emac section) "10M/100M/1000M external PHY with MII/ RGMII interface", however the description of the "PC plus" only claims 10/100 for the network. My comments in the emac driver indicate that the internal PHY is only good for 10/100 and an external PHY is needed for gigabit. The Orange Pi plus (not the PC plus), which is a board I do not have, does have the external PHY.

My notes in my emac driver are depressing. I spent a lot of time trying to work with the H3 PHY and had lots of problems. What I see is that the call to "phy_init()" is commented out. I am apparently just inheriting and relying on whatever U-Boot did. On top of that, my comment indicates that I spent some time with U-Boot adding printf() statements to the phy code in U-Boot and concluded that U-Boot was also having trouble dealing with the H3 Phy and it it sheer luck that things work at all.

So we are looking at two options. One is to educate ourselves about this whole business of mii, mdio, and the H3 datasheet. The other is to find a linux driver for the H3 phy and see what we can learn from it. The latest linux kernel code has an emac driver:

drivers/net/ethernet/allwinner/sun4i-emac.c
drivers/net/ethernet/allwinner/sun4i-emac.h
However, as I look at this it becomes clear that this is something else entirely, the registers do not correspond at all.

Alphabet soup

MDIO - Management Data input/output - (aka SMI or MiiM). This is a serial bus that connects MAC devices (media access contro) with PHY devices (ethernet physical layer). The MAC device that controls the MDIO is called the SME (station management entity). There is a clock (MDC - MDIO inteface clock) driven by the MAC, along with a bidirectional data bus. There can be up to 32 PHY slaves.

MII - Media Independent interface - this has two interfaces. One is the data interface that transmits ethernet frame data to and fro between the MAC and the PHY. The other is a management interface (MDIO) used to read and write control registers in the PHY to configure and monitor it.

RMII - Reduced MII - reduces signals and pin count. Clocks jump from 25 Mhz to 50 Mhz while the data bus width drops from 4 to 2 bits.

RGMII - Reduced GMII, where GMII is Gigabit media independent interface. With GMII, the clock is 125 Mhz and there is an 8 bit data path. With RGMII, the clock is doubled and there is a 4 bit data path.

Other open source drivers

I use "locate emac" on my system, ignore all the hits on the emacs editor, and find the following as possible interesting code:
/u1/Projects/FreeBSD/freebsd/sys/arm/allwinner/if_emac.c
/u1/Projects/FreeBSD/freebsd/sys/arm/allwinner/if_emacreg.h

/u1/Projects/OrangePi/Armbian/sources/u-boot/v2016.11/drivers/net/davinci_emac.c
/u1/Projects/OrangePi/Armbian/sources/u-boot/v2016.11/drivers/net/davinci_emac.h
/u1/Projects/OrangePi/Armbian/sources/u-boot/v2016.11/drivers/net/sun8i_emac.c
/u1/Projects/OrangePi/Armbian/sources/u-boot/v2016.11/drivers/net/sunxi_emac.c

/u1/Projects/OrangePi/Xulong/linux-orangepi/drivers/net/ethernet/arc/emac.h
/u1/Projects/OrangePi/Xulong/linux-orangepi/drivers/net/ethernet/arc/emac_arc.c
/u1/Projects/OrangePi/Xulong/linux-orangepi/drivers/net/ethernet/arc/emac_main.c
/u1/Projects/OrangePi/Xulong/linux-orangepi/drivers/net/ethernet/arc/emac_mdio.c

/u1/linux/linux-3.4.113/drivers/net/ethernet/ibm/emac
/u1/linux/linux-3.4.113/drivers/net/ethernet/ibm/emac/Kconfig
/u1/linux/linux-3.4.113/drivers/net/ethernet/ibm/emac/Makefile
/u1/linux/linux-3.4.113/drivers/net/ethernet/ibm/emac/core.c
/u1/linux/linux-3.4.113/drivers/net/ethernet/ibm/emac/core.h
/u1/linux/linux-3.4.113/drivers/net/ethernet/ibm/emac/debug.c
/u1/linux/linux-3.4.113/drivers/net/ethernet/ibm/emac/debug.h
/u1/linux/linux-3.4.113/drivers/net/ethernet/ibm/emac/emac.h
/u1/linux/linux-3.4.113/drivers/net/ethernet/ibm/emac/mal.c
/u1/linux/linux-3.4.113/drivers/net/ethernet/ibm/emac/mal.h
/u1/linux/linux-3.4.113/drivers/net/ethernet/ibm/emac/phy.c
/u1/linux/linux-3.4.113/drivers/net/ethernet/ibm/emac/phy.h
/u1/linux/linux-3.4.113/drivers/net/ethernet/ibm/emac/rgmii.c
/u1/linux/linux-3.4.113/drivers/net/ethernet/ibm/emac/rgmii.h
/u1/linux/linux-3.4.113/drivers/net/ethernet/ibm/emac/tah.c
/u1/linux/linux-3.4.113/drivers/net/ethernet/ibm/emac/tah.h
/u1/linux/linux-3.4.113/drivers/net/ethernet/ibm/emac/zmii.c
/u1/linux/linux-3.4.113/drivers/net/ethernet/ibm/emac/zmii.h
/u1/linux/linux-3.4.113/drivers/net/ethernet/ti/davinci_emac.c

/u1/linux/linux-source-5.4.43-sunxi64/Documentation/devicetree/bindings/net/allwinner,sun8i-a83t-emac.yaml

/u1/linux/linux-sunxi/drivers/net/ethernet/allwinner/emac_readme.txt
/u1/linux/linux-sunxi/drivers/net/ethernet/allwinner/sunxi_emac.c
/u1/linux/linux-sunxi/drivers/net/ethernet/allwinner/sunxi_emac.h
The Freebsd code looks like the same inappropriate emac-4i driver that I found in the linux sources. I am going to ignore the Armbian U-boot sources given my notes from 5 years ago that U-boot was not handling the PHY any better than I was. The "arc" emac driver is some chip from Synopsis and no register layout is readily visible. The "ibm/emac" files pertain to a powerPC ethernet controller. The "sun8i yaml" file is nothing of interest. The very last thing (sunxi_emac) is once again the sun4i emac driver that is no use to me. So all of these possibilities missed the mark.

Not to give up. The emac driver must be kicking around in the linux sources somewhere.

cd /u1/linux/linux-source-5.4.43-sunxi64_PRISTINE/arch/arm/boot/dts
sunxi-h3-h5.dtsi
This has a big section that is appropriate:
	    emac: ethernet@1c30000 {
                        compatible = "allwinner,sun8i-h3-emac";
                        syscon = <&syscon>;
                        reg = <0x01c30000 0x10000>;
                        interrupts = ;
                        interrupt-names = "macirq";
                        resets = <&ccu RST_BUS_EMAC>;
                        reset-names = "stmmaceth";
                        clocks = <&ccu CLK_BUS_EMAC>;
                        clock-names = "stmmaceth";
                        status = "disabled";

                        mdio: mdio {
                                #address-cells = <1>;
                                #size-cells = <0>;
                                compatible = "snps,dwmac-mdio";
                        };

                        mdio-mux {
                                compatible = "allwinner,sun8i-h3-mdio-mux";
                                #address-cells = <1>;
                                #size-cells = <0>;

                                mdio-parent-bus = <&mdio>;
                                /* Only one MDIO is usable at the time */
                                internal_mdio: mdio@1 {
                                        compatible = "allwinner,sun8i-h3-mdio-internal";
                                        reg = <1>;
                                        #address-cells = <1>;
                                        #size-cells = <0>;

                                        int_mii_phy: ethernet-phy@1 {
                                                compatible = "ethernet-phy-ieee802.3-c22";
                                                reg = <1>;
                                                clocks = <&ccu CLK_BUS_EPHY>;
                                                resets = <&ccu RST_BUS_EPHY>;
                                        };
                                };

                                external_mdio: mdio@2 {
                                        reg = <2>;
                                        #address-cells = <1>;
                                        #size-cells = <0>;
                                };
                        };
                };
Doing a recursive grep in this kernel tree leads me to:
./drivers/net/ethernet/stmicro/stmmac/dwmac-sun8i.c
And by golly, the register layout looks just right. This same driver is in various linux source trees, including the very latest.
/u1/Projects/FPGA/Ebaz/Petalinux/linux-xlnx/drivers/net/ethernet/stmicro/stmmac/dwmac-sun8i.c
/u1/Projects/Femto/Openwrt/build_dir/target-mipsel_24kc_musl/linux-ramips_rt305x/linux-4.14.180/drivers/net/ethernet/stmicro/stmmac/dwmac-sun8i.c
/u1/Projects/Femto/Openwrt/build_dir/toolchain-mipsel_24kc_gcc-7.5.0_musl/linux-4.14.180/drivers/net/ethernet/stmicro/stmmac/dwmac-sun8i.c
/u1/Projects/OrangePi/Xulong/linux-orangepi/drivers/net/ethernet/stmicro/stmmac/dwmac-sun8i.c
/u1/linux/linux-source-5.4.43-sunxi64/drivers/net/ethernet/stmicro/stmmac/dwmac-sun8i.c
/u1/linux/linux-source-5.4.43-sunxi64_PRISTINE/drivers/net/ethernet/stmicro/stmmac/dwmac-sun8i.c
/u1/linux/linux-git/drivers/net/ethernet/stmicro/stmmac/dwmac-sun8i.c


Have any comments? Questions? Drop me a line!

Kyu / tom@mmto.org