Why do lambda functions drop deduced return type reference by default?

Question

In C++14, why do lambda functions with a deduced return type drop references from the return type by default? IIUC, since C++14 lambda functions with a deduced return type (without an explicit trailing return type) have a return type of auto, which drops references (among other things).

Why was this decision made? It seems to me like a gotcha to remove a reference when that's what your return statement returns.

This behavior caused the following nasty bug for me:

class Int {
public:
   Int(int i) : m_int{i} {}
   int m_int;
};

class C {
public:
    C(Int obj) : m_obj{obj} {}
    const auto& getObj() { return m_obj; }
    Int m_obj;
};

class D {
public:
    D(std::function<const Int&()> f) : m_f{f} {}
    std::function<const Int&()> m_f;
};

Int myint{5};
C c{myint};
D d{ [&c](){ return c.getObj(); } } // The deduced return type of the lambda is Int (with no reference)
const Int& myref = d.m_f(); // Instead of referencing myint, myref is a dangling reference; d.m_f() returned a copy of myint, which is subsequently destroyed.

Specifying the desired return type when initializing d resolves the issue:

D d{ [&c]() -> const Int& { return c.getObj(); } }

Interestingly, even if the auto return type deduction makes sense, isn't it a bug that std::function<const Int&> gets happily initialized with a function that returns a non-reference? I see this also by writing explicitly:

D d{ [&c]() -> Int { return c.getObj(); } }

which compiles without a problem. (on Xcode 8, clang 8.0.0)

Answer 1

I think the place you are stumbling is actually with the expression c.getObj() in the line return c.getObj();.

You think the expression c.getObj() has type const Int&. However that is not true; expressions never have reference type. As noted by Kerrek SB in comments, we sometimes talk about expressions as if they had reference type, as a shortcut to save on verbosity, but that leads to misconceptions so I think it is important to understand what is really going on.

The use of a reference type in a declaration (including as a return type as in getObj's declaration) affects how the thing being declared is initialized, but once it is initialized, there is no longer any evidence that it was originally a reference.

Here is a simpler example:

int a; int &b = a;  // 1

versus

int b; int &a = b;  // 2

These two codes are exactly identical ^{(except for the result of decltype(a) or decltype(b) which is a bit of a hack to the system)}. In both cases the expressions a and b both have type int and value category "lvalue" and denote the same object. It's not the case that a is the "real object" and b is some sort of disguised pointer to a. They are both on equal footing. It's one object with two names.

Going back to your code now: the expression c.getObj() has exactly the same behaviour as c.m_obj, apart from access rights. The type is Int and the value category is "lvalue". The & in the return type of getObj() only dictates that this is an lvalue and it will also designate an object that already existed (approximately speaking).

So the deduced return type from return c.getObj(); is the same as it would be for return c.m_obj; , which -- to be compatible with template type deduction, as mentioned elsewhere -- is not a reference type.

NB. If you understood this post you will also understand why I don't like the pedagogy of "references" being taught as "disguised pointers that auto dereference", which is somewhere between wrong and dangerous.

Answer 2

The standard (at least, the working draft) already gives you hints about what's happening and how to solve it:

The lambda return type is auto, which is replaced by the type specified by the trailing-return-type if provided and/or deduced from return statements as described in [dcl.spec.auto]. [ Example:

    auto x1 = [](int i){ return i; }; // OK: return type is int
    auto x2 = []{ return { 1, 2 }; }; // error: deducing return type from braced-init-list int j;
    auto x3 = []()->auto&& { return j; }; // OK: return type is int& 

— end example ]

Consider now the following template function:

template<typename T>
void f(T t) {}

// ....

int x = 42;
f(x);

What's t in f, a copy of x or a reference to it?
What happens if we change the function as it follows?

template<typename T>
void f(T &t) {}

The same applies more or less to the deduced return type of a lambda: if you want a reference, you must be explicit about that.

Why was this decision made? It seems to me like a gotcha to remove a reference when that's what your return statement returns.

The choice is consistent with how templates work since the beginning.
I would be surprised of the opposite instead.
Return type is deduced as well as the template parameter and it's a pretty good decision not to define different set of rules for them (at least from my point of view).

---

That said, to solve your problem you have several alternatives:

1. [&c]()->auto&&{ return c.getObj(); }
   

2. [&c]()->auto&{ return c.getObj(); }
   

3. [&c]()->decltype(c.getObj())&{ return c.getObj(); }
   

4. [&c]()->decltype(c.getObj())&&{ return c.getObj(); }
   

5. [&c]()->decltype(auto){ return c.getObj(); }
   

6. [&c]()->const Int &{ return c.getObj(); }
   

7. ...
   

Some of them are crazy, some of them are quite clear, all of them should work.
If the intended behavior is to return a reference, probably to be explicit about that is the best choice:

[&c]()->auto&{ return c.getObj(); }

Anyway, this is mostly opinion-based, so feel free to pick your preferred alternative up and use it.

---

Interestingly, even if the auto return type deduction makes sense, isn't it a bug that std::function gets happily initialized with a function that returns a non-reference?

Let's consider the code below (no reason to call in a std::function right now):

int f() { return 0; }
const int & g() { return f(); }
int main() { const int &x = g(); }

It gives you a few warnings but it compiles.
The reason is that a temporary is created from an rvalue and a temporary can bind to a const reference, so I'd say it's legal from the point of view of the standard.
The fact that it will explode at runtime is another problem.

Something similar happens when using a std:: function.
Anyway it's an abstraction over a generic callable object, so do not expect the same warnings.