# `tornado.gen` — Simplify asynchronous code

`tornado.gen` is a generator-based interface to make it easier to work in an asynchronous environment. Code using the `gen` module is technically asynchronous, but it is written as a single generator instead of a collection of separate functions.

For example, the following asynchronous handler:

    class AsyncHandler(RequestHandler):
        @asynchronous
        def get(self):
            http_client = AsyncHTTPClient()
            http_client.fetch("http://example.com",
                              callback=self.on_fetch)

        def on_fetch(self, response):
            do_something_with_response(response)
            self.render("template.html")

could be written with `gen` as:

    class GenAsyncHandler(RequestHandler):
        @gen.coroutine
        def get(self):
            http_client = AsyncHTTPClient()
            response = yield http_client.fetch("http://example.com")
            do_something_with_response(response)
            self.render("template.html")

Most asynchronous functions in Tornado return a `Future`; yielding this object returns its `result`.

You can also yield a list or dict of `Futures`, which will be started at the same time and run in parallel; a list or dict of results will be returned when they are all finished:

    @gen.coroutine
    def get(self):
        http_client = AsyncHTTPClient()
        response1, response2 = yield [http_client.fetch(url1),
                                      http_client.fetch(url2)]
        response_dict = yield dict(response3=http_client.fetch(url3),
                                   response4=http_client.fetch(url4))
        response3 = response_dict['response3']
        response4 = response_dict['response4']

If the `singledispatch` library is available (standard in Python 3.4, available via the singledispatch package on older versions), additional types of objects may be yielded. Tornado includes support for `asyncio.Future` and Twisted’s `Deferred` class when `tornado.platform.asyncio` and `tornado.platform.twisted` are imported. See the `convert_yielded` function to extend this mechanism.

Changed in version 3.2: Dict support added.

Changed in version 4.1: Support added for yielding `asyncio` Futures and Twisted Deferreds via `singledispatch`.

## Decorators

`tornado.gen.``coroutine`(func, replace_callback=True)

  Decorator for asynchronous generators.

  Any generator that yields objects from this module must be wrapped in either this decorator or `engine`.

  Coroutines may “return” by raising the special exception `Return(value)`. In Python 3.3+, it is also possible for the function to simply use the `return value` statement (prior to Python 3.3 generators were not allowed to also return values). In all versions of Python a coroutine that simply wishes to exit early may use the `return` statement without a value.

  Functions with this decorator return a `Future`. Additionally, they may be called with a `callback` keyword argument, which will be invoked with the future’s result when it resolves. If the coroutine fails, the callback will not be run and an exception will be raised into the surrounding `StackContext`. The `callback` argument is not visible inside the decorated function; it is handled by the decorator itself.

  From the caller’s perspective, `@gen.coroutine` is similar to the combination of `@return_future` and `@gen.engine`.

  Warning

  When exceptions occur inside a coroutine, the exception information will be stored in the `Future` object. You must examine the result of the `Future` object, or the exception may go unnoticed by your code. This means yielding the function if called from another coroutine, using something like `IOLoop.run_sync` for top-level calls, or passing the `Future` to `IOLoop.add_future`.

`tornado.gen.``engine`(func)

  Callback-oriented decorator for asynchronous generators.

  This is an older interface; for new code that does not need to be compatible with versions of Tornado older than 3.0 the `coroutine` decorator is recommended instead.

  This decorator is similar to `coroutine`, except it does not return a `Future` and the `callback` argument is not treated specially.

  In most cases, functions decorated with `engine` should take a `callback` argument and invoke it with their result when they are finished. One notable exception is the `RequestHandler` HTTP verb methods, which use `self.finish()` in place of a callback argument.

## Utility functions

exception `tornado.gen.``Return`(value=None)

  Special exception to return a value from a `coroutine`.

  If this exception is raised, its value argument is used as the result of the coroutine:

      @gen.coroutine
      def fetch_json(url):
          response = yield AsyncHTTPClient().fetch(url)
          raise gen.Return(json_decode(response.body))

  In Python 3.3, this exception is no longer necessary: the `return` statement can be used directly to return a value (previously `yield` and `return` with a value could not be combined in the same function).

  By analogy with the return statement, the value argument is optional, but it is never necessary to `raise gen.Return()`. The `return` statement can be used with no arguments instead.

`tornado.gen.``with_timeout`(timeout, future, io_loop=None, quiet_exceptions=())

  Wraps a `Future` (or other yieldable object) in a timeout.

  Raises `TimeoutError` if the input future does not complete before `timeout`, which may be specified in any form allowed by `IOLoop.add_timeout` (i.e. a `datetime.timedelta` or an absolute time relative to `IOLoop.time`)

  If the wrapped `Future` fails after it has timed out, the exception will be logged unless it is of a type contained in `quiet_exceptions` (which may be an exception type or a sequence of types).

  Does not support `YieldPoint` subclasses.

  New in version 4.0.

  Changed in version 4.1: Added the `quiet_exceptions` argument and the logging of unhandled exceptions.

  Changed in version 4.4: Added support for yieldable objects other than `Future`.

exception `tornado.gen.``TimeoutError`

  Exception raised by `with_timeout`.

`tornado.gen.``sleep`(duration)

  Return a `Future` that resolves after the given number of seconds.

  When used with `yield` in a coroutine, this is a non-blocking analogue to `time.sleep` (which should not be used in coroutines because it is blocking):

      yield gen.sleep(0.5)

  Note that calling this function on its own does nothing; you must wait on the `Future` it returns (usually by yielding it).

  New in version 4.1.

`tornado.gen.``moment`

  A special object which may be yielded to allow the IOLoop to run for one iteration.

  This is not needed in normal use but it can be helpful in long-running coroutines that are likely to yield Futures that are ready instantly.

  Usage: `yield gen.moment`

  New in version 4.0.

  Deprecated since version 4.5: `yield None` is now equivalent to `yield gen.moment`.

class `tornado.gen.``WaitIterator`(*args, **kwargs)

  Provides an iterator to yield the results of futures as they finish.

  Yielding a set of futures like this:

  `results = yield [future1, future2]`

  pauses the coroutine until both `future1` and `future2` return, and then restarts the coroutine with the results of both futures. If either future is an exception, the expression will raise that exception and all the results will be lost.

  If you need to get the result of each future as soon as possible, or if you need the result of some futures even if others produce errors, you can use `WaitIterator`:

      wait_iterator = gen.WaitIterator(future1, future2)
      while not wait_iterator.done():
          try:
              result = yield wait_iterator.next()
          except Exception as e:
              print("Error {} from {}".format(e, wait_iterator.current_future))
          else:
              print("Result {} received from {} at {}".format(
                  result, wait_iterator.current_future,
                  wait_iterator.current_index))

  Because results are returned as soon as they are available the output from the iterator will not be in the same order as the input arguments. If you need to know which future produced the current result, you can use the attributes `WaitIterator.current_future`, or `WaitIterator.current_index` to get the index of the future from the input list. (if keyword arguments were used in the construction of the `WaitIterator`, `current_index` will use the corresponding keyword).

  On Python 3.5, `WaitIterator` implements the async iterator protocol, so it can be used with the `async for` statement (note that in this version the entire iteration is aborted if any value raises an exception, while the previous example can continue past individual errors):

      async for result in gen.WaitIterator(future1, future2):
          print("Result {} received from {} at {}".format(
              result, wait_iterator.current_future,
              wait_iterator.current_index))

  New in version 4.1.

  Changed in version 4.3: Added `async for` support in Python 3.5.

  `done`()

    Returns True if this iterator has no more results.

  `next`()

    Returns a `Future` that will yield the next available result.

    Note that this `Future` will not be the same object as any of the inputs.

`tornado.gen.``multi`(children, quiet_exceptions=())

  Runs multiple asynchronous operations in parallel.

  `children` may either be a list or a dict whose values are yieldable objects. `multi()` returns a new yieldable object that resolves to a parallel structure containing their results. If `children` is a list, the result is a list of results in the same order; if it is a dict, the result is a dict with the same keys.

  That is, `results = yield multi(list_of_futures)` is equivalent to:

      results = []
      for future in list_of_futures:
          results.append(yield future)

  If any children raise exceptions, `multi()` will raise the first one. All others will be logged, unless they are of types contained in the `quiet_exceptions` argument.

  If any of the inputs are `YieldPoints`, the returned yieldable object is a `YieldPoint`. Otherwise, returns a `Future`. This means that the result of `multi` can be used in a native coroutine if and only if all of its children can be.

  In a `yield`-based coroutine, it is not normally necessary to call this function directly, since the coroutine runner will do it automatically when a list or dict is yielded. However, it is necessary in `await`-based coroutines, or to pass the `quiet_exceptions` argument.

  This function is available under the names `multi()` and `Multi()` for historical reasons.

  Changed in version 4.2: If multiple yieldables fail, any exceptions after the first (which is raised) will be logged. Added the `quiet_exceptions` argument to suppress this logging for selected exception types.

  Changed in version 4.3: Replaced the class `Multi` and the function `multi_future` with a unified function `multi`. Added support for yieldables other than `YieldPoint` and `Future`.

`tornado.gen.``multi_future`(children, quiet_exceptions=())

  Wait for multiple asynchronous futures in parallel.

  This function is similar to `multi`, but does not support `YieldPoints`.

  New in version 4.0.

  Changed in version 4.2: If multiple `Futures` fail, any exceptions after the first (which is raised) will be logged. Added the `quiet_exceptions` argument to suppress this logging for selected exception types.

  Deprecated since version 4.3: Use `multi` instead.

`tornado.gen.``Task`(func, *args, **kwargs)

  Adapts a callback-based asynchronous function for use in coroutines.

  Takes a function (and optional additional arguments) and runs it with those arguments plus a `callback` keyword argument. The argument passed to the callback is returned as the result of the yield expression.

  Changed in version 4.0: `gen.Task` is now a function that returns a `Future`, instead of a subclass of `YieldPoint`. It still behaves the same way when yielded.

class `tornado.gen.``Arguments`

  The result of a `Task` or `Wait` whose callback had more than one argument (or keyword arguments).

  The `Arguments` object is a `collections.namedtuple` and can be used either as a tuple `(args, kwargs)` or an object with attributes `args` and `kwargs`.

`tornado.gen.``convert_yielded`(yielded)

  Convert a yielded object into a `Future`.

  The default implementation accepts lists, dictionaries, and Futures.

  If the `singledispatch` library is available, this function may be extended to support additional types. For example:

      @convert_yielded.register(asyncio.Future)
      def _(asyncio_future):
          return tornado.platform.asyncio.to_tornado_future(asyncio_future)

  New in version 4.1.

`tornado.gen.``maybe_future`(x)

  Converts `x` into a `Future`.

  If `x` is already a `Future`, it is simply returned; otherwise it is wrapped in a new `Future`. This is suitable for use as `result = yield gen.maybe_future(f())` when you don’t know whether `f()` returns a `Future` or not.

  Deprecated since version 4.3: This function only handles `Futures`, not other yieldable objects. Instead of `maybe_future`, check for the non-future result types you expect (often just `None`), and `yield` anything unknown.

`tornado.gen.``is_coroutine_function`(func)

  Return whether func is a coroutine function, i.e. a function wrapped with `coroutine`.

  New in version 4.5.

## Legacy interface

Before support for `Futures` was introduced in Tornado 3.0, coroutines used subclasses of `YieldPoint` in their `yield` expressions. These classes are still supported but should generally not be used except for compatibility with older interfaces. None of these classes are compatible with native (`await`-based) coroutines.

class `tornado.gen.``YieldPoint`

  Base class for objects that may be yielded from the generator.

  Deprecated since version 4.0: Use `Futures` instead.

  `start`(runner)

    Called by the runner after the generator has yielded.

    No other methods will be called on this object before `start`.

  `is_ready`()

    Called by the runner to determine whether to resume the generator.

    Returns a boolean; may be called more than once.

  `get_result`()

    Returns the value to use as the result of the yield expression.

    This method will only be called once, and only after `is_ready` has returned true.

class `tornado.gen.``Callback`(key)

  Returns a callable object that will allow a matching `Wait` to proceed.

  The key may be any value suitable for use as a dictionary key, and is used to match `Callbacks` to their corresponding `Waits`. The key must be unique among outstanding callbacks within a single run of the generator function, but may be reused across different runs of the same function (so constants generally work fine).

  The callback may be called with zero or one arguments; if an argument is given it will be returned by `Wait`.

  Deprecated since version 4.0: Use `Futures` instead.

class `tornado.gen.``Wait`(key)

  Returns the argument passed to the result of a previous `Callback`.

  Deprecated since version 4.0: Use `Futures` instead.

class `tornado.gen.``WaitAll`(keys)

  Returns the results of multiple previous `Callbacks`.

  The argument is a sequence of `Callback` keys, and the result is a list of results in the same order.

  `WaitAll` is equivalent to yielding a list of `Wait` objects.

  Deprecated since version 4.0: Use `Futures` instead.

class `tornado.gen.``MultiYieldPoint`(children, quiet_exceptions=())

  Runs multiple asynchronous operations in parallel.

  This class is similar to `multi`, but it always creates a stack context even when no children require it. It is not compatible with native coroutines.

  Changed in version 4.2: If multiple `YieldPoints` fail, any exceptions after the first (which is raised) will be logged. Added the `quiet_exceptions` argument to suppress this logging for selected exception types.

  Changed in version 4.3: Renamed from `Multi` to `MultiYieldPoint`. The name `Multi` remains as an alias for the equivalent `multi` function.

  Deprecated since version 4.3: Use `multi` instead.