# `tornado.gen` — Generator-based coroutines

`tornado.gen` implements generator-based coroutines.

Note

The “decorator and generator” approach in this module is a precursor to native coroutines (using `async def` and `await`) which were introduced in Python 3.5. Applications that do not require compatibility with older versions of Python should use native coroutines instead. Some parts of this module are still useful with native coroutines, notably `multi`, `sleep`, `WaitIterator`, and `with_timeout`. Some of these functions have counterparts in the `asyncio` module which may be used as well, although the two may not necessarily be 100% compatible.

Coroutines provide an easier way to work in an asynchronous environment than chaining callbacks. Code using coroutines is technically asynchronous, but it is written as a single generator instead of a collection of separate functions.

For example, here’s a coroutine-based handler:

    class GenAsyncHandler(RequestHandler):
        @gen.coroutine
        def get(self):
            http_client = AsyncHTTPClient()
            response = yield http_client.fetch("http://example.com")
            do_something_with_response(response)
            self.render("template.html")

Asynchronous functions in Tornado return an `Awaitable` or `Future`; yielding this object returns its result.

You can also yield a list or dict of other yieldable objects, which will be started at the same time and run in parallel; a list or dict of results will be returned when they are all finished:

    @gen.coroutine
    def get(self):
        http_client = AsyncHTTPClient()
        response1, response2 = yield [http_client.fetch(url1),
                                      http_client.fetch(url2)]
        response_dict = yield dict(response3=http_client.fetch(url3),
                                   response4=http_client.fetch(url4))
        response3 = response_dict['response3']
        response4 = response_dict['response4']

If `tornado.platform.twisted` is imported, it is also possible to yield Twisted’s `Deferred` objects. See the `convert_yielded` function to extend this mechanism.

Changed in version 3.2: Dict support added.

Changed in version 4.1: Support added for yielding `asyncio` Futures and Twisted Deferreds via `singledispatch`.

## Decorators

tornado.gen.coroutine(func: Callable[[...], Generator[Any, Any, _T]]) → Callable[[...], Future[_T]]
tornado.gen.coroutine(func: Callable[[...], _T]) → Callable[[...], Future[_T]]

  Decorator for asynchronous generators.

  For compatibility with older versions of Python, coroutines may also “return” by raising the special exception `Return(value)`.

  Functions with this decorator return a `Future`.

  Warning

  When exceptions occur inside a coroutine, the exception information will be stored in the `Future` object. You must examine the result of the `Future` object, or the exception may go unnoticed by your code. This means yielding the function if called from another coroutine, using something like `IOLoop.run_sync` for top-level calls, or passing the `Future` to `IOLoop.add_future`.

  Changed in version 6.0: The `callback` argument was removed. Use the returned awaitable object instead.

exception tornado.gen.Return(value: Any = None)

  Special exception to return a value from a `coroutine`.

  This exception exists for compatibility with older versions of Python (before 3.3). In newer code use the `return` statement instead.

  If this exception is raised, its value argument is used as the result of the coroutine:

      @gen.coroutine
      def fetch_json(url):
          response = yield AsyncHTTPClient().fetch(url)
          raise gen.Return(json_decode(response.body))

  By analogy with the return statement, the value argument is optional.

## Utility functions

tornado.gen.with_timeout(timeout: float | datetime.timedelta, future: Yieldable, quiet_exceptions: Type[Exception] | Tuple[Type[Exception], ...] = ())

  Wraps a `Future` (or other yieldable object) in a timeout.

  Raises `tornado.util.TimeoutError` if the input future does not complete before `timeout`, which may be specified in any form allowed by `IOLoop.add_timeout` (i.e. a `datetime.timedelta` or an absolute time relative to `IOLoop.time`)

  If the wrapped `Future` fails after it has timed out, the exception will be logged unless it is either of a type contained in `quiet_exceptions` (which may be an exception type or a sequence of types), or an `asyncio.CancelledError`.

  The wrapped `Future` is not canceled when the timeout expires, permitting it to be reused. `asyncio.wait_for` is similar to this function but it does cancel the wrapped `Future` on timeout.

  Added in version 4.0.

  Changed in version 4.1: Added the `quiet_exceptions` argument and the logging of unhandled exceptions.

  Changed in version 4.4: Added support for yieldable objects other than `Future`.

  Changed in version 6.0.3: `asyncio.CancelledError` is now always considered “quiet”.

  Changed in version 6.2: `tornado.util.TimeoutError` is now an alias to `asyncio.TimeoutError`.

tornado.gen.sleep(duration: float) → Future[None]

  Return a `Future` that resolves after the given number of seconds.

  When used with `yield` in a coroutine, this is a non-blocking analogue to `time.sleep` (which should not be used in coroutines because it is blocking):

      yield gen.sleep(0.5)

  Note that calling this function on its own does nothing; you must wait on the `Future` it returns (usually by yielding it).

  Added in version 4.1.

class tornado.gen.WaitIterator(*args: Future, **kwargs: Future)

  Provides an iterator to yield the results of awaitables as they finish.

  Yielding a set of awaitables like this:

  `results = yield [awaitable1, awaitable2]`

  pauses the coroutine until both `awaitable1` and `awaitable2` return, and then restarts the coroutine with the results of both awaitables. If either awaitable raises an exception, the expression will raise that exception and all the results will be lost.

  If you need to get the result of each awaitable as soon as possible, or if you need the result of some awaitables even if others produce errors, you can use `WaitIterator`:

      wait_iterator = gen.WaitIterator(awaitable1, awaitable2)
      while not wait_iterator.done():
          try:
              result = yield wait_iterator.next()
          except Exception as e:
              print("Error {} from {}".format(e, wait_iterator.current_future))
          else:
              print("Result {} received from {} at {}".format(
                  result, wait_iterator.current_future,
                  wait_iterator.current_index))

  Because results are returned as soon as they are available the output from the iterator will not be in the same order as the input arguments. If you need to know which future produced the current result, you can use the attributes `WaitIterator.current_future`, or `WaitIterator.current_index` to get the index of the awaitable from the input list. (if keyword arguments were used in the construction of the `WaitIterator`, `current_index` will use the corresponding keyword).

  `WaitIterator` implements the async iterator protocol, so it can be used with the `async for` statement (note that in this version the entire iteration is aborted if any value raises an exception, while the previous example can continue past individual errors):

      async for result in gen.WaitIterator(future1, future2):
          print("Result {} received from {} at {}".format(
              result, wait_iterator.current_future,
              wait_iterator.current_index))

  Added in version 4.1.

  Changed in version 4.3: Added `async for` support in Python 3.5.

  done() → bool

    Returns True if this iterator has no more results.

  next() → Future

    Returns a `Future` that will yield the next available result.

    Note that this `Future` will not be the same object as any of the inputs.

tornado.gen.multi(children: Sequence[None | Awaitable | List[Awaitable] | Dict[Any, Awaitable] | Future], quiet_exceptions: Type[Exception] | Tuple[Type[Exception], ...] = ()) → Future[List]
tornado.gen.multi(children: Mapping[Any, None | Awaitable | List[Awaitable] | Dict[Any, Awaitable] | Future], quiet_exceptions: Type[Exception] | Tuple[Type[Exception], ...] = ()) → Future[Dict]

  Runs multiple asynchronous operations in parallel.

  `children` may either be a list or a dict whose values are yieldable objects. `multi()` returns a new yieldable object that resolves to a parallel structure containing their results. If `children` is a list, the result is a list of results in the same order; if it is a dict, the result is a dict with the same keys.

  That is, `results = yield multi(list_of_futures)` is equivalent to:

      results = []
      for future in list_of_futures:
          results.append(yield future)

  If any children raise exceptions, `multi()` will raise the first one. All others will be logged, unless they are of types contained in the `quiet_exceptions` argument.

  In a `yield`-based coroutine, it is not normally necessary to call this function directly, since the coroutine runner will do it automatically when a list or dict is yielded. However, it is necessary in `await`-based coroutines, or to pass the `quiet_exceptions` argument.

  This function is available under the names `multi()` and `Multi()` for historical reasons.

  Cancelling a `Future` returned by `multi()` does not cancel its children. `asyncio.gather` is similar to `multi()`, but it does cancel its children.

  Changed in version 4.2: If multiple yieldables fail, any exceptions after the first (which is raised) will be logged. Added the `quiet_exceptions` argument to suppress this logging for selected exception types.

  Changed in version 4.3: Replaced the class `Multi` and the function `multi_future` with a unified function `multi`. Added support for yieldables other than `YieldPoint` and `Future`.

tornado.gen.multi_future(Union[List[Yieldable], Dict[Any, Yieldable]], quiet_exceptions: Union[Type[Exception], Tuple[Type[Exception], ...]] = ())

  Wait for multiple asynchronous futures in parallel.

  Since Tornado 6.0, this function is exactly the same as `multi`.

  Added in version 4.0.

  Changed in version 4.2: If multiple `Futures` fail, any exceptions after the first (which is raised) will be logged. Added the `quiet_exceptions` argument to suppress this logging for selected exception types.

  Deprecated since version 4.3: Use `multi` instead.

tornado.gen.convert_yielded(yielded: None | Awaitable | List[Awaitable] | Dict[Any, Awaitable] | Future) → Future

  Convert a yielded object into a `Future`.

  The default implementation accepts lists, dictionaries, and Futures. This has the side effect of starting any coroutines that did not start themselves, similar to `asyncio.ensure_future`.

  If the `singledispatch` library is available, this function may be extended to support additional types. For example:

      @convert_yielded.register(asyncio.Future)
      def _(asyncio_future):
          return tornado.platform.asyncio.to_tornado_future(asyncio_future)

  Added in version 4.1.

tornado.gen.maybe_future(x: Any) → Future

  Converts `x` into a `Future`.

  If `x` is already a `Future`, it is simply returned; otherwise it is wrapped in a new `Future`. This is suitable for use as `result = yield gen.maybe_future(f())` when you don’t know whether `f()` returns a `Future` or not.

  Deprecated since version 4.3: This function only handles `Futures`, not other yieldable objects. Instead of `maybe_future`, check for the non-future result types you expect (often just `None`), and `yield` anything unknown.

tornado.gen.is_coroutine_function(func: Any) → bool

  Return whether func is a coroutine function, i.e. a function wrapped with `coroutine`.

  Added in version 4.5.

tornado.gen.moment

  A special object which may be yielded to allow the IOLoop to run for one iteration.

  This is not needed in normal use but it can be helpful in long-running coroutines that are likely to yield Futures that are ready instantly.

  Usage: `yield gen.moment`

  In native coroutines, the equivalent of `yield gen.moment` is `await asyncio.sleep(0)`.

  Added in version 4.0.

  Deprecated since version 4.5: `yield None` (or `yield` with no argument) is now equivalent to `yield gen.moment`.