"There is no need to deallocate the stack at the end of the function when the inner-stack frame isn't modified", but it is being modified in this case

Question

Here is a simple function

#include <stdio.h>

int foo() {
    int a = 3;
    int b = 4;
    int c = 5;
    return a * b * c;
}

int main() {
    int a = foo();
}

And the assembly for foo() looks like

foo:
        push    rbp
        mov     rbp, rsp
        mov     DWORD PTR [rbp-4], 3
        mov     DWORD PTR [rbp-8], 4
        mov     DWORD PTR [rbp-12], 5
        mov     eax, DWORD PTR [rbp-4]
        imul    eax, DWORD PTR [rbp-8]
        imul    eax, DWORD PTR [rbp-12]
        pop     rbp
        ret

as seen with the rbp - N, the inner stack frame is being modified. So, why are there no leaves or add rsp, n?

Answer 1

It "isn't modified" in the sense that the stack pointer does not change. Since we never subtracted any offset from rsp on entry, of course we should not add any on exit.

And leave is simply the equivalent of mov rsp, rbp ; pop rbp. We did include pop rbp, and mov rsp, rbp would be redundant since as you can see, rsp and rbp are still equal at that point in the code.

In fact, this function stores its local variables on the stack without adjusting the stack pointer to "make space" for them, so they end up below the stack pointer. This is legal because the x86-64 SysV ABI mandates a red zone; code that does not make any function calls may safely use 128 bytes below the stack pointer. Signal handlers are guaranteed not to overwrite it.

Why does the x86-64 GCC function prologue allocate less stack than the local variables?

Where exactly is the red zone on x86-64?