Errors compiling sqrt function for RaspberryPI2 and BeableBoneBlack with arm-none-eabi-gcc compiler

Question

i'm new in this type of coding and i'm trying to do some test executing bare-metal software for Cortex-A processors. I have experience with Cortex-M MCU and i compiled code with an IDE like ARM-Keil and with SDK by Nordic for some BLE jobs. Now i want to try to understand better the world of Cortex-A and i would learn coding in a bare-metal way, starting from some examples that i found online. I had some experiences with freeRTOS with Cortex-M, so i found online some git repository from some good guy that makes a porting of freeRTOS for RaspberryPI and BeableBoneBlack. From now on i will just explain my problem for the BeableBoneBlack, beacouse for the RaspberryPI it's similar.

I'm on linux, i installed gcc-arm-none-eabi compiler, so i cloned the BeagleBone Black with freeRTOS repository from this link.

I usually use VS Code to write code, so in the integrated terminal when i run the command make everything it's working and the of my BBB flashes correctly. So now i would like to improve my code, and in order to do some tests i would like to use the rand() function, from stdlib.h. Unhappily I find that there are some errors: undefined reference to rand. The last months, before doing tests with the BBB, i found other repos for RPi2 and i learn something about the linker of the arm-none-eabi compiler, that needs the addition of some parameters in order link library files during the process. From this repo now i have two files: makedefs_ti and makefile. Opening the make file i found in line 26-27 the part of the generation of the file app, so where the linker is called. In this lines there are references to LIB_GCC and LIB_C, which are defined in the makedefs_ti. In lines 49-50 there are the references to the directory where the compiler is installed (I changed 4.7.3 with the correct one installed on my linux pc that is 9.2.1). In the makefile, after -L$(LIB_C), if i add the linker parameters like -lc or -lg and try to recompile, i had error like arm-none-eabi-ld: cannot find -lc. With some understanding from online resources i modified the the makedefs_ti as following:

# Toolchain/library path. LIB_PATH is an exported environmental variable which
# shall point to the installation of toolchain
#
#LIB_GCC=${LIB_PATH}/lib/gcc/arm-none-eabi/4.7.3/
#LIB_C=${LIB_PATH}/arm-none-eabi/lib/
LIB_GCC= /usr/lib/gcc/arm-none-eabi/9.2.1
LIB_C= /usr/lib/arm-none-eabi/lib

and the makefile :

$(LD) -o [email protected] $< -T bbb.ld -Map bbb.map $(APP_LIB) $(LDFLAGS) $(RUNTIMELIB) -L $(LPATH) \
   -L $(LIB_GCC) -lgcc -L $(LIB_C) -lc -lg

In this way my main.c file that calls a rand() function compile correctly. During another test i tried to compile another code. Now i wrote a pair of .c and .h file (i placed them in the rtos directory in order to exploit the same makefile that is present there) that have inside a function that calls sqrt(). So, calling the function in the main() and compiling, even if i include everywhere <math.h> i have the error:

undefined reference to `sqrt'

and obviously the compilation stops. Starting from this i tried to add another parameter to the linker, so i modified the makefile as following:

$(LD) -o [email protected] $< -T bbb.ld -Map bbb.map $(APP_LIB) $(LDFLAGS) $(RUNTIMELIB) -L $(LPATH) \
   -L $(LIB_GCC) -lgcc -L $(LIB_C) -lc -lg -lm

and i get other errors which i am unable to resolve in any way:

arm-none-eabi-ld: /usr/lib/arm-none-eabi/lib/libm.a(lib_a-w_sqrt.o): in function `sqrt':
/build/newlib-CVVEyx/newlib-3.3.0/build/arm-none-eabi/newlib/libm/math/../../../../../newlib/libm/math/w_sqrt.c:62: undefined reference to `__aeabi_dcmpun'
arm-none-eabi-ld: /build/newlib-CVVEyx/newlib-3.3.0/build/arm-none-eabi/newlib/libm/math/../../../../../newlib/libm/math/w_sqrt.c:63: undefined reference to `__aeabi_dcmplt'
arm-none-eabi-ld: /build/newlib-CVVEyx/newlib-3.3.0/build/arm-none-eabi/newlib/libm/math/../../../../../newlib/libm/math/w_sqrt.c:64: undefined reference to `__errno'
arm-none-eabi-ld: /build/newlib-CVVEyx/newlib-3.3.0/build/arm-none-eabi/newlib/libm/math/../../../../../newlib/libm/math/w_sqrt.c:65: undefined reference to `__aeabi_ddiv'
arm-none-eabi-ld: /usr/lib/arm-none-eabi/lib/libm.a(lib_a-e_sqrt.o): in function `__ieee754_sqrt':
/build/newlib-CVVEyx/newlib-3.3.0/build/arm-none-eabi/newlib/libm/machine/arm/../../../../../../newlib/libm/machine/arm/../../math/e_sqrt.c:110: undefined reference to `__aeabi_dmul'
arm-none-eabi-ld: /build/newlib-CVVEyx/newlib-3.3.0/build/arm-none-eabi/newlib/libm/machine/arm/../../../../../../newlib/libm/machine/arm/../../math/e_sqrt.c:110: undefined reference to `__aeabi_dadd'
arm-none-eabi-ld: /build/newlib-CVVEyx/newlib-3.3.0/build/arm-none-eabi/newlib/libm/machine/arm/../../../../../../newlib/libm/machine/arm/../../math/e_sqrt.c:117: undefined reference to `__aeabi_dsub'
arm-none-eabi-ld: /build/newlib-CVVEyx/newlib-3.3.0/build/arm-none-eabi/newlib/libm/machine/arm/../../../../../../newlib/libm/machine/arm/../../math/e_sqrt.c:117: undefined reference to `__aeabi_ddiv'
make: *** [makefile:26: app] Errore 1

I would remark that, calling the sqrt() in the main works correctly, the problem happens when i call the sqrt() in other .c file. Instead the rand() function in the .c additional file works correctly.

I don't know how to resolve those errors, someone could help me?

What i tried another day is to write the same code in a Code Composer Studio project and it worked (with same functions that call sqrt and rand in additional .c file. What is the difference?

I'm very sorry if this is a long explanation, but i didn't find another way to explain my situation.

Thanks a lot in advance.

Salvo

Answer 1

start.s

.globl _start
_start:
    ;@ enable fpu
    mrc p15, 0, r0, c1, c0, 2
    orr r0,r0,#0x300000 ;@ single precision
    orr r0,r0,#0xC00000 ;@ double precision
    mcr p15, 0, r0, c1, c0, 2
    mov r0,#0x40000000
    fmxr fpexc,r0

    mov sp,#0x8000
    bl notmain
hang: b hang

notmain.c

double notmain ( double a, double b )
{
    return(a+b);
}

(not a real baremetal app, derived from one)

build for hard float

arm-none-eabi-as --warn --fatal-warnings -mcpu=arm1176jzf-s -march=armv6 -mfpu=vfp start.s -o start.o
arm-none-eabi-gcc -Wall -O2 -ffreestanding -mcpu=arm1176jzf-s -mtune=arm1176jzf-s -mhard-float -mfpu=vfp -c notmain.c -o notmain.o
arm-none-eabi-ld -nostdlib -nostartfiles start.o notmain.o -T memmap -o notmain.elf
arm-none-eabi-objdump -D notmain.elf > notmain.list
arm-none-eabi-objcopy --srec-forceS3 notmain.elf -O srec notmain.srec
arm-none-eabi-objcopy notmain.elf -O binary kernel.img

hardware float used

00008024 <notmain>:
    8024:   ee300b01    vadd.f64    d0, d0, d1
    8028:   e12fff1e    bx  lr

build for soft float

arm-none-eabi-as --warn --fatal-warnings -mcpu=arm1176jzf-s -march=armv6 -mfpu=vfp start.s -o start.o
arm-none-eabi-gcc -Wall -O2 -ffreestanding -mcpu=arm1176jzf-s -mtune=arm1176jzf-s -c notmain.c -o notmain.o
arm-none-eabi-ld -nostdlib -nostartfiles start.o notmain.o -T memmap -o notmain.elf
arm-none-eabi-ld: notmain.o: in function `notmain':
notmain.c:(.text+0x4): undefined reference to `__aeabi_dadd'
Makefile:33: recipe for target 'notmain.elf' failed
make: *** [notmain.elf] Error 1

It needs gcclib.

Using gcc as a linker (I know how horrible a thought that is).

arm-none-eabi-as --warn --fatal-warnings -mcpu=arm1176jzf-s -march=armv6 -mfpu=vfp start.s -o start.o
arm-none-eabi-gcc -Wall -O2 -ffreestanding -mcpu=arm1176jzf-s -mtune=arm1176jzf-s -c notmain.c -o notmain.o
arm-none-eabi-gcc -Wall -O2 -ffreestanding -mcpu=arm1176jzf-s -mtune=arm1176jzf-s -nostdlib -nostartfiles start.o notmain.o -T memmap -o notmain.elf -lgcc
arm-none-eabi-objdump -D notmain.elf > notmain.list
arm-none-eabi-objcopy --srec-forceS3 notmain.elf -O srec notmain.srec
arm-none-eabi-objcopy notmain.elf -O binary kernel.img

...

00008024 <notmain>:
    8024:   e92d4010    push    {r4, lr}
    8028:   eb000003    bl  803c <__adddf3>
    802c:   e8bd8010    pop {r4, pc}

00008030 <__aeabi_drsub>:
    8030:   e2211102    eor r1, r1, #-2147483648    ; 0x80000000
    8034:   ea000000    b   803c <__adddf3>

00008038 <__aeabi_dsub>:
    8038:   e2233102    eor r3, r3, #-2147483648    ; 0x80000000

0000803c <__adddf3>:
    803c:   e92d4030    push    {r4, r5, lr}

gcc is passing these to ld.

[0][/opt/gnuarm/lib/gcc/arm-none-eabi/10.2.0/../../../../arm-none-eabi/bin/ld]
[1][-plugin]
[2][/opt/gnuarm/libexec/gcc/arm-none-eabi/10.2.0/liblto_plugin.so]
[3][-plugin-opt=/opt/gnuarm/libexec/gcc/arm-none-eabi/10.2.0/lto-wrapper]
[4][-plugin-opt=-fresolution=/tmp/ccRiWZtk.res]
[5][-X]
[6][-o]
[7][notmain.elf]
[8][-L/opt/gnuarm/lib/gcc/arm-none-eabi/10.2.0]
[9][-L/opt/gnuarm/lib/gcc/arm-none-eabi/10.2.0/../../../../arm-none-eabi/lib]
[10][start.o]
[11][notmain.o]
[12][-lgcc]
[13][-T]
[14][memmap]

find | grep libgcc
./lib/gcc/arm-none-eabi/10.2.0/libgcc.a
./lib/gcc/arm-none-eabi/10.2.0/thumb/libgcc.a
./lib/gcc/arm-none-eabi/10.2.0/thumb/autofp/v7/fpu/libgcc.a
./lib/gcc/arm-none-eabi/10.2.0/arm/autofp/v5te/fpu/libgcc.a

So to use ld then

arm-none-eabi-as --warn --fatal-warnings -mcpu=arm1176jzf-s -march=armv6 -mfpu=vfp start.s -o start.o
arm-none-eabi-gcc -Wall -O2 -ffreestanding -mcpu=arm1176jzf-s -mtune=arm1176jzf-s -c notmain.c -o notmain.o
arm-none-eabi-ld -nostdlib -nostartfiles start.o notmain.o -T memmap -o notmain.elf -L/opt/gnuarm/lib/gcc/arm-none-eabi/10.2.0/ -lgcc
arm-none-eabi-objdump -D notmain.elf > notmain.list
arm-none-eabi-objcopy --srec-forceS3 notmain.elf -O srec notmain.srec
arm-none-eabi-objcopy notmain.elf -O binary kernel.img

Why does it use addf3 now? do not know.

00008024 <notmain>:
    8024:   e92d4010    push    {r4, lr}
    8028:   eb000003    bl  803c <__adddf3>
    802c:   e8bd8010    pop {r4, pc}

00008030 <__aeabi_drsub>:
    8030:   e2211102    eor r1, r1, #-2147483648    ; 0x80000000
    8034:   ea000000    b   803c <__adddf3>

00008038 <__aeabi_dsub>:
    8038:   e2233102    eor r3, r3, #-2147483648    ; 0x80000000

0000803c <__adddf3>:
    803c:   e92d4030    push    {r4, r5, lr}
    8040:   e1a04081    lsl r4, r1, #1
    8044:   e1a05083    lsl r5, r3, #1

That is how you deal with the aeabi stuff, you need to either use hard float or include the path to gcclib or just add it to the line

arm-none-eabi-as --warn --fatal-warnings -mcpu=arm1176jzf-s -march=armv6 -mfpu=vfp start.s -o start.o
arm-none-eabi-gcc -Wall -O2 -ffreestanding -mcpu=arm1176jzf-s -mtune=arm1176jzf-s -c notmain.c -o notmain.o
arm-none-eabi-ld -nostdlib -nostartfiles start.o notmain.o -T memmap -o notmain.elf /opt/gnuarm/lib/gcc/arm-none-eabi/10.2.0/libgcc.a
#arm-none-eabi-gcc -Wall -O2 -ffreestanding -mcpu=arm1176jzf-s -mtune=arm1176jzf-s -nostdlib -nostartfiles start.o notmain.o -T memmap -o notmain.elf -lgcc
arm-none-eabi-objdump -D notmain.elf > notmain.list
arm-none-eabi-objcopy --srec-forceS3 notmain.elf -O srec notmain.srec
arm-none-eabi-objcopy notmain.elf -O binary kernel.img

Now sqrt() is a C library thing not a gcc library thing so you need a C library (looks like you are trying to use newlib) and you need to build the C library to match gcc so you need to build it for hard or soft float and need to include the right library. Which I do not have...well....

arm-none-eabi-as --warn --fatal-warnings -mcpu=arm1176jzf-s -march=armv6 -mfpu=vfp start.s -o start.o
arm-none-eabi-gcc -Wall -O2 -ffreestanding -mcpu=arm1176jzf-s -mtune=arm1176jzf-s -c notmain.c -o notmain.o
arm-none-eabi-ld -nostdlib -nostartfiles start.o notmain.o -T memmap -o notmain.elf /home/so/gcc-arm-none-eabi-9-2019-q4-major/lib/gcc/arm-none-eabi/9.2.1/libgcc.a
arm-none-eabi-objdump -D notmain.elf > notmain.list
arm-none-eabi-objcopy --srec-forceS3 notmain.elf -O srec notmain.srec
arm-none-eabi-objcopy notmain.elf -O binary kernel.img

so

double sqrt(double x);
double notmain ( double a )
{
    return(sqrt(a));
}

looks familiar

arm-none-eabi-gcc -Wall -O2 -ffreestanding -mcpu=arm1176jzf-s -mtune=arm1176jzf-s -c notmain.c -o notmain.o
arm-none-eabi-ld -nostdlib -nostartfiles start.o notmain.o -T memmap -o notmain.elf /home/so/gcc-arm-none-eabi-9-2019-q4-major/lib/gcc/arm-none-eabi/9.2.1/libgcc.a /home/so/gcc-arm-none-eabi-9-2019-q4-major/arm-none-eabi/lib/libm.a
arm-none-eabi-ld: /home/so/gcc-arm-none-eabi-9-2019-q4-major/arm-none-eabi/lib/libm.a(lib_a-w_sqrt.o): in function `sqrt':
w_sqrt.c:(.text.sqrt+0x34): undefined reference to `__aeabi_dcmpun'
arm-none-eabi-ld: w_sqrt.c:(.text.sqrt+0x6c): undefined reference to `__aeabi_dcmplt'
arm-none-eabi-ld: w_sqrt.c:(.text.sqrt+0xc4): undefined reference to `__aeabi_ddiv'
arm-none-eabi-ld: w_sqrt.c:(.text.sqrt+0xd8): undefined reference to `__errno'
arm-none-eabi-ld: w_sqrt.c:(.text.sqrt+0x120): undefined reference to `__errno'
arm-none-eabi-ld: /home/so/gcc-arm-none-eabi-9-2019-q4-major/arm-none-eabi/lib/libm.a(lib_a-e_sqrt.o): in function `__ieee754_sqrt':
e_sqrt.c:(.text.__ieee754_sqrt+0x200): undefined reference to `__aeabi_dmul'
arm-none-eabi-ld: e_sqrt.c:(.text.__ieee754_sqrt+0x20c): undefined reference to `__aeabi_dadd'
arm-none-eabi-ld: e_sqrt.c:(.text.__ieee754_sqrt+0x22c): undefined reference to `__aeabi_dsub'
arm-none-eabi-ld: e_sqrt.c:(.text.__ieee754_sqrt+0x238): undefined reference to `__aeabi_ddiv'
Makefile:33: recipe for target 'notmain.elf' failed
make: *** [notmain.elf] Error 1

find

./arm-none-eabi/lib/libm.a
./arm-none-eabi/lib/thumb/v7e-m/nofp/libm.a
./arm-none-eabi/lib/thumb/v6-m/nofp/libm.a
./arm-none-eabi/lib/thumb/v8-m.base/nofp/libm.a
./arm-none-eabi/lib/thumb/v7/nofp/libm.a
./arm-none-eabi/lib/thumb/v8-m.main+dp/hard/libm.a
./arm-none-eabi/lib/thumb/v8-m.main+dp/softfp/libm.a
./arm-none-eabi/lib/thumb/v7e-m+dp/hard/libm.a
./arm-none-eabi/lib/thumb/v7e-m+dp/softfp/libm.a
./arm-none-eabi/lib/thumb/v7-r+fp.sp/hard/libm.a
./arm-none-eabi/lib/thumb/v7-r+fp.sp/softfp/libm.a
./arm-none-eabi/lib/thumb/v7e-m+fp/hard/libm.a
./arm-none-eabi/lib/thumb/v7e-m+fp/softfp/libm.a
./arm-none-eabi/lib/thumb/v8-m.main+fp/hard/libm.a
./arm-none-eabi/lib/thumb/v8-m.main+fp/softfp/libm.a
./arm-none-eabi/lib/thumb/v7-m/nofp/libm.a
./arm-none-eabi/lib/thumb/nofp/libm.a
./arm-none-eabi/lib/thumb/v7+fp/hard/libm.a
./arm-none-eabi/lib/thumb/v7+fp/softfp/libm.a
./arm-none-eabi/lib/thumb/v8-m.main/nofp/libm.a
./arm-none-eabi/lib/arm/v5te/hard/libm.a
./arm-none-eabi/lib/arm/v5te/softfp/libm.a

armv5te should work for armv6

arm-none-eabi-ld -nostdlib -nostartfiles start.o notmain.o -T memmap -o notmain.elf /home/so/gcc-arm-none-eabi-9-2019-q4-major/lib/gcc/arm-none-eabi/9.2.1/libgcc.a /home/so/gcc-arm-none-eabi-9-2019-q4-major/arm-none-eabi/lib/arm/v5te/softfp/libm.a
arm-none-eabi-ld: /home/so/gcc-arm-none-eabi-9-2019-q4-major/arm-none-eabi/lib/arm/v5te/softfp/libm.a(lib_a-w_sqrt.o): in function `sqrt':
w_sqrt.c:(.text.sqrt+0x94): undefined reference to `__errno'
arm-none-eabi-ld: w_sqrt.c:(.text.sqrt+0xdc): undefined reference to `__errno'
Makefile:33: recipe for target 'notmain.elf' failed
make: *** [notmain.elf] Error 1

errno is a global variable. that the C library probably wants to use for the soft float.

int __errno;
double sqrt(double x);
double notmain ( double a )
{
    return(sqrt(a));
}

and now the tool is happy

arm-none-eabi-as --warn --fatal-warnings -mcpu=arm1176jzf-s -march=armv6 -mfpu=vfp start.s -o start.o
arm-none-eabi-gcc -Wall -O2 -ffreestanding -mcpu=arm1176jzf-s -mtune=arm1176jzf-s -c notmain.c -o notmain.o
arm-none-eabi-ld -nostdlib -nostartfiles start.o notmain.o -T memmap -o notmain.elf /home/so/gcc-arm-none-eabi-9-2019-q4-major/lib/gcc/arm-none-eabi/9.2.1/libgcc.a /home/so/gcc-arm-none-eabi-9-2019-q4-major/arm-none-eabi/lib/arm/v5te/softfp/libm.a
arm-none-eabi-objdump -D notmain.elf > notmain.list
arm-none-eabi-objcopy --srec-forceS3 notmain.elf -O srec notmain.srec
arm-none-eabi-objcopy notmain.elf -O binary kernel.img

but this is hard float not soft.

00008024 <notmain>:
    8024:   eaffffff    b   8028 <sqrt>

00008028 <sqrt>:
    8028:   e52de004    push    {lr}        ; (str lr, [sp, #-4]!)
    802c:   ed2d8b02    vpush   {d8}
    8030:   e1a02000    mov r2, r0
    8034:   e1a03001    mov r3, r1
    8038:   e24dd02c    sub sp, sp, #44 ; 0x2c
    803c:   ec432b18    vmov    d8, r2, r3
    8040:   eb000038    bl  8128 <__ieee754_sqrt>
    8044:   e59f30d4    ldr r3, [pc, #212]  ; 8120 <sqrt+0xf8>
    8048:   eeb48b48    vcmp.f64    d8, d8
    804c:   e1d3c0d0    ldrsb   ip, [r3]
    8050:   eef1fa10    vmrs    APSR_nzcv, fpscr

You get the idea though as to what the path to take to solve this is. You need a properly matched/built library then you need to link it. Keil and others simply do that for you in the toolchain, matching things based on the project. And/or a C library that is integrated with the toolchain.

Building for hard float

arm-none-eabi-as --warn --fatal-warnings -mcpu=arm1176jzf-s -march=armv6 -mfpu=vfp start.s -o start.o
arm-none-eabi-gcc -Wall -O2 -ffreestanding -mcpu=arm1176jzf-s -mtune=arm1176jzf-s -mhard-float -mfpu=vfp -c notmain.c -o notmain.o
arm-none-eabi-ld -nostdlib -nostartfiles start.o notmain.o -T memmap -o notmain.elf /home/so/gcc-arm-none-eabi-9-2019-q4-major/lib/gcc/arm-none-eabi/9.2.1/libgcc.a /home/so/gcc-arm-none-eabi-9-2019-q4-major/arm-none-eabi/lib/arm/v5te/hard/libm.a
arm-none-eabi-objdump -D notmain.elf > notmain.list
arm-none-eabi-objcopy --srec-forceS3 notmain.elf -O srec notmain.srec
arm-none-eabi-objcopy notmain.elf -O binary kernel.img

tool is happy.

00008024 <notmain>:
    8024:   eaffffff    b   8028 <sqrt>

00008028 <sqrt>:
    8028:   e52de004    push    {lr}        ; (str lr, [sp, #-4]!)
    802c:   ed2d8b02    vpush   {d8}
    8030:   eeb08b40    vmov.f64    d8, d0
    8034:   e24dd02c    sub sp, sp, #44 ; 0x2c
    8038:   eb000038    bl  8120 <__ieee754_sqrt>
    803c:   eeb48b48    vcmp.f64    d8, d8
    8040:   e59f30d0    ldr r3, [pc, #208]  ; 8118 <sqrt+0xf0>
    8044:   e1d310d0    ldrsb   

Would have to make this a real application main calls another function that has a call to sqrt in it. And then run it on hardware and/or examine these instructions. Or even better get a C library implementation of sqrt and build it in the project so that it matches the target and instruction sets. (same story with the libgcc calls as needed, test them or build them).