Rust Async Programming and Streams

An Async function returns a Future.

A Future can be seen as a task that will eventually return a value.

A Poll function polls a future, and returns a Result.

A Result can be seen as the result of a poll, indicating if the future is ready or not.

A Context is a struct that helps to prevent future polls from happening unnecessarily.

A Waker is a type that is used to notify the future when it is ready to make some progress.

When a future is polled, the future registers itself with the poller, by calling the waker() function.

The waker() function takes a future, and calls the poll() function on it.

If the poll() function returns a Result, the future is considered to be ready.

If the poll() function returns an Err, the future is not ready, and the poller will poll it again.

The poller will poll the future until it returns a Result.

Once the future returns a Result, the poller will stop polling it.

The Stream::iter function creates a stream that, whenever polled, returns a Poll::Ready(next value from the iterator).

The Stream::unfold function takes a state/seed as a parameter, and invokes an async future passing it, that is responsible for generating the next element of the stream and the new state that is going to be passed.

You can also create a state machine or sequence of different action with unfold using:

The Stream::iter function is useful for tests, when you don’t care about the async/await stuff, and are only interested in the stream of values.

Another interesting one is repeat_with, where you can pass a lambda/closure in order to generate the stream of values on demand/lazily:

If you take a look at the import, you will see that we use StreamExt, which is a shortcut for StreamExtension. It is a common practice in Rust to put only the minimal definition of a trait in a file, and additional/helper/nicer api in another Extension file.

The async-stream crate provides 2 macros, stream! and stream_from_iter.

The stream_from_iter macro takes an iterator and returns a stream that awaits something for real.

Streams are an efficient way to emit items asynchronously.

You can use them with for/loop, async functions or combinators.

You can consume them with next() or inspect().

Pinning is not necessary but is recommended.

There are many ways to consume streams, but read the documentation first.