Rust Async Borrow Lifetimes

Question

I'm trying to make a helper that allows asynchronously chaining side effects, but I'm unable to get the generic bounds correct so that the compiler understands the output of the future outlives a reference used to build it.

Playground Link

The gist of it comes down to:

struct Chain<T> {
    data: T
}
impl<T> Chain<T> {
    pub async fn chain<E, Fut, F>(self, effect: F) -> Result<T, E>
        where
            Fut: Future<Output=Result<(), E>>,
            F: FnOnce(&T) -> Fut
    {
        todo!()
    }
}

gives a compiler error of

error: lifetime may not live long enough
  --> src/main.rs:39:32
   |
39 |     let r = chain.chain(|this| this.good("bar")).await;
   |                          ----- ^^^^^^^^^^^^^^^^ returning this value requires that `'1` must outlive `'2`
   |                          |   |
   |                          |   return type of closure `impl Future` contains a lifetime `'2`
   |                          has type `&'1 MyData`

If we fix up chain so that it can infer that the reference is available for the same lifetime as the future:

impl<T> Chain<T> {
    pub async fn chain<'a, E, Fut, F>(self, effect: F) -> Result<T, E>
        where
            T: 'a, 
            Fut: 'a + Future<Output=Result<(), E>>,
            F: FnOnce(&'a T) -> Fut
    {
        effect(&self.data).await?;
        Ok(self.data)
    }
}

We get a new compiler error about moving self.data while it's still borrowed.

error[E0505]: cannot move out of `self.data` because it is borrowed
  --> src/main.rs:30:12
   |
23 |     pub async fn chain<'a, E, Fut, F>(self, effect: F) -> Result<T, E>
   |                        -- lifetime `'a` defined here
...
29 |         effect(&self.data).await?;
   |         ------------------
   |         |      |
   |         |      borrow of `self.data` occurs here
   |         argument requires that `self.data` is borrowed for `'a`
30 |         Ok(self.data)
   |            ^^^^^^^^^ move out of `self.data` occurs here

I guess there's a pathological closure along the lines of |this| futures::future::ready(Err(this)) that would cause an early return with the borrow still "alive".

Question

How can we get chain to work? My normal lifetime trick of block-scoping doesn't seem to help. Is there a set of where constraints that can be added to prove that the borrow and then eventual move are on disjoint lifetimes?

Answer 1

This particular situation is one where the current constraint syntax and lack of higher-kinded types does not let you express what you want.

You can use a higher-rank trait bound, the for<'a> syntax, to introduce an intermediate generic lifetime parameter 'a within a where clause to dictate that the constraint must be valid for any lifetime. This is necessary here and the reason your first fix didn't work was because 'a as a generic on chain meant that the lifetime was determined by the caller, however, lifetime of self is by construction less than any lifetime that could be picked by the caller. So the slightly more correct syntax (and identical to the de-sugared original code) would be:

pub async fn chain<E, Fut, F>(self, effect: F) -> Result<T, E>
    where
        Fut: Future<Output = Result<(), E>>,
        F: for<'a> FnOnce(&'a T) -> Fut
{
    ...

But this doesn't help at all, since there is still no association between Fut and 'a. There's unfortunately no way to use the same for<'a> across multiple constraints. You could try using impl Trait to define it all at once, but that isn't supported:

pub async fn chain<E, F>(self, effect: F) -> Result<T, E>
    where F: for<'a> FnOnce(&'a T) -> (impl Future<Output = Result<(), E>> + 'a)
{
    ...

error[E0562]: `impl Trait` not allowed outside of function and method return types
  --> src/lib.rs:35:44
   |
35 |         where F: for<'a> FnOnce(&'a T) -> (impl Future<Output = Result<(), E>> + 'a)
   |                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

There will hopefully be better support for higher-kinded types in the future. This particular case might have a solution on nightly by using the almost-complete generic associated types feature, but I haven't yet found it.

So the only real fix then is to use a named type as the return value, which really only leaves us with trait objects:

use std::pin::Pin;
use futures::future::FutureExt;

pub async fn chain<E, F>(self, effect: F) -> Result<T, E>
    where F: for<'a> FnOnce(&'a T) -> Pin<Box<dyn Future<Output = Result<(), E>> + 'a>>
{
    ...

let r = chain.chain(|this| this.good("bar").boxed()).await;

---

As a side note, your bad case still does not compile and indeed cannot work, since you'd be returning a reference to a local value.

Answer 2

It looks like you are trying to implement future.then()

If you are aware of that and you are doing it as an exercise, you probably should design it in a way that the effect method returns values, and use these values to return from chain method. That way you enforce proper order of operations. As far as I understand your design, you do not benefit from awaiting on effect inside the chain method, since as your skip function is also async and will return future (the actual return type of chain method is Future<Output=Result<T, E>>, since async works that way: it wraps your explicit return type in future).

So there is no point in awaiting on the effect inside the chain, you still have to await it whenever you use it - and nothing will happen until you actually await for it outside of the chain - futures are lazy that way.

TL;DR I would arange your effect methods to return values and arrange chain to just return these values