Why both codes add -visibility=hidden, the normal functions compile incorrectly, but the virtual functions compile correctly?

Question

I reading this page.
I follow the answer step by step to test, alse add -fvisibility=hidden to make all symbols hidden, and then I extended the code that is in the answer.

//rectangle.h
#pragma once

#include <memory>

class Rectangle {
    int width, height;
  public:
    void set_values (int,int);
    int area();

};

std::shared_ptr<Rectangle> __attribute__((visibility("default"))) get_rectangle();

//rectangle.cpp
#include "rectangle.h"

int Rectangle::area() {return width*height;}

void Rectangle::set_values (int x, int y) {
  width = x;
  height = y;
}

std::shared_ptr<Rectangle> __attribute__((visibility("default"))) get_rectangle() {
    return std::make_shared<Rectangle>();
}

I build librectangle.so, then I code main.cpp to link librectangle.so

//main.cpp
#include "../rectangle/rectangle.h"

int main() {
    auto rectangle = get_rectangle();
    rectangle->set_values(2, 3);
    rectangle->area();
    return 0;
}

Compilation error

undefined reference to `Rectangle::set_values(int, int)'
undefined reference to `Rectangle::area()'

But, If I change the class member function to a virtual function, it will compile correctly

//changed rectangle.h
#pragma once

#include <memory>

class Rectangle {
    int width, height;
  public:
    virtual void set_values (int,int);
    virtual int area();

};

std::shared_ptr<Rectangle> __attribute__((visibility("default"))) get_rectangle();

Use Usenm -C librectangle.so to compare the normal function class and the virtual function class. The left is the normal function class. the right is the virtual function class.

nm -C librectangle.so | grep Rectangle

nm -C librectangle.so | grep area

nm -C librectangle.so | grep set_values

both demos add -visibility=hidden, the normal functions compile incorrectly, but the virtual functions compile correctly?

Answer 1

An library has a list of symbols (that is, functions and global variables) it contains, the exported symbols. An object file has a list of symbols it uses that aren't defined in that object file, the imported symbols.

Hidden visibility on a symbol means it isn't added to the list of exported symbols. Default visibility means it is.

When the linker creates a library/executable from a bunch of object files, it looks at the imported symbols of all the object files and resolves them according to the exported symbols of all object files and libraries.

Let's look at the first example. Your main.o is compiled to assembly roughly to this effect (assuming no callee-saved registers except stack pointer):

main:
  sub sp, 16 ; make space for shared_ptr on stack
  mov arg0, sp ; move address of that space to first argument register
  call get_rectangle
  mov arg0, [esp] ; load pointer to object behind shared_ptr into argument
  mov arg1, 2 ; load arguments
  mov arg2, 3
  call Rectangle::set_values
  mov arg0, [sp] ; load pointer again
  call Rectangle::area
  mov arg0, sp ; load shared_ptr address
  call shared_ptr::~shared_ptr ; destroy shared_ptr, end of scope

As you can see, it references three symbols that are used: get_rectangle, Rectangle::set_values and Rectangle::area. But only get_rectangle has default visibility, so the other two are not listed as exports in librectangle.so, so the linker can't find them. You get errors.

But that's not how virtual calls work. There, the code looks something like this:

main:
  sub sp, 16 ; as before
  mov arg0, sp
  call get_rectangle
  mov arg0, [esp] ; load pointer to object behind shared_ptr
  mov reg0, [arg0] ; load vtable pointer to call virtual function
  mov arg1, 2 ; load arguments
  mov arg2, 3
  call [reg0+0] ; call first entry in vtable, set_values
  mov arg0, [sp] ; load pointer again
  mov reg0, [arg0] ; load vtable pointer again
  call [reg0+8] ; call second entry in vtable, area
  mov arg0, sp ; load shared_ptr address
  call shared_ptr::~shared_ptr ; destroy shared_ptr, end of scope

As you can see, now the assembly only references a single symbol, get_rectangle. The others are only called through the pointers stored in the vtable. The vtable is a global constant stored in the librectangle.so, so it has access to the hidden functions. Also, the vtable itself is a hidden symbol, but it is only referenced by the constructor of Rectangle, which is implicitly defined by the compiler. It is an inline function and so exists in every compilation unit that includes the header, but the only place it is used is in get_rectangle, so there it has access to the vtable symbol.

Note that if you create a local variable of type Rectangle in main you will get a linker error referring to the vtable missing. Also, since the compiler might devirtualize the calls, you would also get errors for these symbols again as it tries the usual direct calls.