How to correctly write separate methods for rvalue and lvalue overloads?

Question

I have the following code.

#include <iostream>
#include <string>

struct A {
    int x;
    int y;
};

struct B {
    A a;
    ~B(){
        std::cout << "deleted" << std::endl;
        a.x = 99;
        a.y = 99;
    }
    operator A & () &   {
        std::cout << "cast A&" << std::endl;
        return a;
    }
    operator A () && {
        std::cout << "cast A" << std::endl;
        return a;
    }
};

B make_b(){
    return {0,1};
}

int main() {
    const A & a = make_b();
    std::cout << a.x << std::endl;
    std::cout << a.y << std::endl;
}

On clang it outputs

cast A
deleted
0
1

as expected.

On gcc it fails to compile with

Could not execute the program
Compiler returned: 1
Compiler stderr
<source>: In function 'int main()':
<source>:33:25: error: invalid user-defined conversion from 'B' to 'const A&' [-fpermissive]
   33 |     const A & a = make_b();
      |                   ~~~~~~^~
<source>:18:5: note: candidate is: 'B::operator A&() &' (near match)
   18 |     operator A & () &   {
      |     ^~~~~~~~
<source>:18:5: note:   passing 'B' as 'this' argument discards qualifiers
<source>:33:26: error: passing 'B' as 'this' argument discards qualifiers [-fpermissive]
   33 |     const A & a = make_b();
      |                          ^
<source>:18:5: note:   in call to 'B::operator A&() &'
   18 |     operator A & () &   {
      |     

Why does gcc not resolve the rvalue overload of the method?

Compiler explorer link to play with

https://godbolt.org/z/6v3GMocE3