Lately I've been programming with Rust quite a bit. In my opinion, the language shows careful, thoughtful design in almost every decision point of its development. The syntax, semantics, standard library, and tooling are all truly top-notch (and a welcome change from my C++ work). I've also used Rust deeply enough to encounter the edges which aren't so smooth. Async code (using the Rust Future trait) can especially present difficult cases at times, like when dealing with more complicated lifetimes and when you want to be able to take parameterized futures (i.e. future generators) as arguments.

What do I mean by parameterized futures? Here's an example:

do_something_contextual(|context| async move {
    // async things using context
}).await

The function do_something_contextual is passed a closure which takes an argument and returns a future (which presumably will use that argument).

Rust Futures

In the design of the Rust Future trait, the interface was kept as simple as possible. This means that a type implementing Future simply needs to provide a way (the poll method) of trying to get the intended output (or at least make progress toward that goal). While this is satisfying in that it "does one thing", it does complicate code that wants to perform more complicated interactions involving futures. In the above example, the |..| and the async move are independent syntaxes which don't have any special interaction. As such, it creates a closure (the |...|) which will immediately return a future (the async [move] {...}).

Because closures and futures are separate concepts, to write a function which takes such a parameterized future (which really is more like a future generator), the generic constraints are a bit cumbersome:

use std::future::Future;
struct Context;

fn do_something_contextual<F, Fut>(generator: F)
  where
    F: Fn(Context) -> Fut,
    Fut: Future
{ ... }

More realistically, do_something_contextual is likely desired to be async itself, the returned future probably should also be Send (many async runtimes are multi-threaded and may end up polling a future on multiple threads), and you often want to use some returned value from the generator, too:

async fn do_something_contextual<F, Fut, R>(generator: F) -> (R, R)
  where
    F: Fn(Context) -> Fut,
    Fut: Future<Output = R> + Send
{
    let first = generator(some_context).await;
    let second = generator(some_context).await;
    (first,second)
}

This gets somewhat onerous and ugly, but it is still manageable. Moreover, you can make a trait (for each number of function arguments since Rust does not yet support variadic type parameters) with a blanket implementation to encapsulate these requirements. Without generic associated types (GATs), this can get a bit hairy with lifetime requirements, but we won't dive into that here.

Closure Captures and Generator Reuse

The problem I encountered in all of this was that I wanted to store and re-use these generator closures (type-erased) multiple times (modestly mocked by the above example calling generator twice), and the closures needed to be able to produce futures with static lifetimes (in part due to the lack of GATs). As you might imagine, to achieve this the closures would need to clone the necessary state (captures) each time they were called and move it into the returned future.

Such an approach works fine with our above definition of do_something_contextual: these details only matter at the generator creation site, not at the API interface. However, I quickly found that the ergonomics of such an interface were truly awful. I was using this extensively, and everything would basically end up like:

let a = ...; // something Clone
let b = ...; // something Clone
do_something_contextual(|context| {
    let a = a.clone();
    let b = b.clone();
    // etc, for each capture
    async move {
        // Use captures and context
    }
}).await

One great convenience of closures is that you don't need to explicitly list all the captures, it "just works" based on the lexical scope bindings. With this API, you end up needing to enumerate the captures anyway. It could really do with improvement.

The trick I found and took advantage of is that generated closures (by the grace of the Rust developers) derive Clone! So we can change our API to:

async fn do_something_contextual<F, Fut, R>(generator: F) -> (R, R)
  where
    F: FnOnce(Context) -> Fut + Clone,
    Fut: Future<Output = R> + Send
{
    let first = (generator.clone())(some_context).await;
    let second = generator(some_context).await;
    (first,second)
}

With this new API, the ergonomics improve significantly. Rather than a caller being required to make a closure that can be called multiple times to generate futures (thus needing to clone the captures manually), the caller simply needs to ensure that all the captures are Clone, and the Rust compiler will ensure that the closure is FnOnce and Clone for us. Thus, the call-site becomes pretty much as minimal as it can get:

let a = ...; // something Clone
let b = ...; // something Clone
do_something_contextual(|context| async move {
    // Use captures and context
}).await

One downside is that error messages may be a little bit less obvious, but to be fair, the error messages resulting from not cloning the captures in the prior (Fn) case would be similar, basically boiling down to an error saying the closure isn't Clone versus an error saying values are moved from an Fn. In both cases documentation can clarify the issue, and I far prefer the latter case using FnOnce and Clone as you can basically have these parameterized futures be just as convenient to write as normal closures!

I've encountered a few other troublesome edge cases of futures (which required larger refactoring or some very careful std::mem::transmute to promise that the lifetimes were in fact correct and sound), but this one ended up being very handily fixed with the language itself. It's very satisfying to discover that the features of a language allow for such ergonomic API improvements, a recurring pattern I've seen in my Rust development!