# What’s new in Tornado 3.0

## Mar 29, 2013

### Highlights

- The `callback` argument to many asynchronous methods is now optional, and these methods return a `Future`. The `tornado.gen` module now understands `Futures`, and these methods can be used directly without a `gen.Task` wrapper.
- New function `IOLoop.current` returns the `IOLoop` that is running on the current thread (as opposed to `IOLoop.instance`, which returns a specific thread’s (usually the main thread’s) IOLoop.
- New class `tornado.netutil.Resolver` provides an asynchronous interface to DNS resolution. The default implementation is still blocking, but non-blocking implementations are available using one of three optional dependencies: `ThreadedResolver` using the `concurrent.futures` thread pool, `tornado.platform.caresresolver.CaresResolver` using the `pycares` library, or `tornado.platform.twisted.TwistedResolver` using `twisted`
- Tornado’s logging is now less noisy, and it no longer goes directly to the root logger, allowing for finer-grained configuration.
- New class `tornado.process.Subprocess` wraps `subprocess.Popen` with `PipeIOStream` access to the child’s file descriptors.
- `IOLoop` now has a static `configure` method like the one on `AsyncHTTPClient`, which can be used to select an `IOLoop` implementation other than the default.
- `IOLoop` can now optionally use a monotonic clock if available (see below for more details).

### Backwards-incompatible changes

- Python 2.5 is no longer supported. Python 3 is now supported in a single codebase instead of using `2to3`
- The `tornado.database` module has been removed. It is now available as a separate package, torndb
- Functions that take an `io_loop` parameter now default to `IOLoop.current()` instead of `IOLoop.instance()`.
- Empty HTTP request arguments are no longer ignored. This applies to `HTTPRequest.arguments` and `RequestHandler.get_argument[s]` in WSGI and non-WSGI modes.
- On Python 3, `tornado.escape.json_encode` no longer accepts byte strings.
- On Python 3, the `get_authenticated_user` methods in `tornado.auth` now return character strings instead of byte strings.
- `tornado.netutil.TCPServer` has moved to its own module, `tornado.tcpserver`.
- The Tornado test suite now requires `unittest2` when run on Python 2.6.
- `tornado.options.options` is no longer a subclass of `dict`; attribute-style access is now required.

### Detailed changes by module

#### Multiple modules

- Tornado no longer logs to the root logger. Details on the new logging scheme can be found under the `tornado.log` module. Note that in some cases this will require that you add an explicit logging configuration in order to see any output (perhaps just calling `logging.basicConfig()`), although both `IOLoop.start()` and `tornado.options.parse_command_line` will do this for you.
- On python 3.2+, methods that take an `ssl_options` argument (on `SSLIOStream`, `TCPServer`, and `HTTPServer`) now accept either a dictionary of options or an `ssl.SSLContext` object.
- New optional dependency on `concurrent.futures` to provide better support for working with threads. `concurrent.futures` is in the standard library for Python 3.2+, and can be installed on older versions with `pip install futures`.

#### `tornado.autoreload`

- `tornado.autoreload` is now more reliable when there are errors at import time.
- Calling `tornado.autoreload.start` (or creating an `Application` with `debug=True`) twice on the same `IOLoop` now does nothing (instead of creating multiple periodic callbacks). Starting autoreload on more than one `IOLoop` in the same process now logs a warning.
- Scripts run by autoreload no longer inherit `__future__` imports used by Tornado.

#### `tornado.auth`

- On Python 3, the `get_authenticated_user` method family now returns character strings instead of byte strings.
- Asynchronous methods defined in `tornado.auth` now return a `Future`, and their `callback` argument is optional. The `Future` interface is preferred as it offers better error handling (the previous interface just logged a warning and returned None).
- The `tornado.auth` mixin classes now define a method `get_auth_http_client`, which can be overridden to use a non-default `AsyncHTTPClient` instance (e.g. to use a different `IOLoop`)
- Subclasses of `OAuthMixin` are encouraged to override `OAuthMixin._oauth_get_user_future` instead of `_oauth_get_user`, although both methods are still supported.

#### `tornado.concurrent`

- New module `tornado.concurrent` contains code to support working with `concurrent.futures`, or to emulate future-based interface when that module is not available.

#### `tornado.curl_httpclient`

- Preliminary support for `tornado.curl_httpclient` on Python 3. The latest official release of pycurl only supports Python 2, but Ubuntu has a port available in 12.10 (`apt-get install python3-pycurl`). This port currently has bugs that prevent it from handling arbitrary binary data but it should work for textual (utf8) resources.
- Fix a crash with libcurl 7.29.0 if a curl object is created and closed without being used.

#### `tornado.escape`

- On Python 3, `json_encode` no longer accepts byte strings. This mirrors the behavior of the underlying json module. Python 2 behavior is unchanged but should be faster.

#### `tornado.gen`

- New decorator `@gen.coroutine` is available as an alternative to `@gen.engine`. It automatically returns a `Future`, and within the function instead of calling a callback you return a value with `raise gen.Return(value)` (or simply `return value` in Python 3.3).
- Generators may now yield `Future` objects.
- Callbacks produced by `gen.Callback` and `gen.Task` are now automatically stack-context-wrapped, to minimize the risk of context leaks when used with asynchronous functions that don’t do their own wrapping.
- Fixed a memory leak involving generators, `RequestHandler.flush`, and clients closing connections while output is being written.
- Yielding a large list no longer has quadratic performance.

#### `tornado.httpclient`

- `AsyncHTTPClient.fetch` now returns a `Future` and its callback argument is optional. When the future interface is used, any error will be raised automatically, as if `HTTPResponse.rethrow` was called.
- `AsyncHTTPClient.configure` and all `AsyncHTTPClient` constructors now take a `defaults` keyword argument. This argument should be a dictionary, and its values will be used in place of corresponding attributes of `HTTPRequest` that are not set.
- All unset attributes of `tornado.httpclient.HTTPRequest` are now `None`. The default values of some attributes (`connect_timeout`, `request_timeout`, `follow_redirects`, `max_redirects`, `use_gzip`, `proxy_password`, `allow_nonstandard_methods`, and `validate_cert` have been moved from `HTTPRequest` to the client implementations.
- The `max_clients` argument to `AsyncHTTPClient` is now a keyword-only argument.
- Keyword arguments to `AsyncHTTPClient.configure` are no longer used when instantiating an implementation subclass directly.
- Secondary `AsyncHTTPClient` callbacks (`streaming_callback`, `header_callback`, and `prepare_curl_callback`) now respect `StackContext`.

#### `tornado.httpserver`

- `HTTPServer` no longer logs an error when it is unable to read a second request from an HTTP 1.1 keep-alive connection.
- `HTTPServer` now takes a `protocol` keyword argument which can be set to `https` if the server is behind an SSL-decoding proxy that does not set any supported X-headers.
- `tornado.httpserver.HTTPConnection` now has a `set_close_callback` method that should be used instead of reaching into its `stream` attribute.
- Empty HTTP request arguments are no longer ignored. This applies to `HTTPRequest.arguments` and `RequestHandler.get_argument[s]` in WSGI and non-WSGI modes.

#### `tornado.ioloop`

- New function `IOLoop.current` returns the `IOLoop` that is running on the current thread (as opposed to `IOLoop.instance`, which returns a specific thread’s (usually the main thread’s) IOLoop).
- New method `IOLoop.add_future` to run a callback on the IOLoop when an asynchronous `Future` finishes.
- `IOLoop` now has a static `configure` method like the one on `AsyncHTTPClient`, which can be used to select an `IOLoop` implementation other than the default.
- The `IOLoop` poller implementations (`select`, `epoll`, `kqueue`) are now available as distinct subclasses of `IOLoop`. Instantiating `IOLoop` will continue to automatically choose the best available implementation.
- The `IOLoop` constructor has a new keyword argument `time_func`, which can be used to set the time function used when scheduling callbacks. This is most useful with the `time.monotonic` function, introduced in Python 3.3 and backported to older versions via the `monotime` module. Using a monotonic clock here avoids problems when the system clock is changed.
- New function `IOLoop.time` returns the current time according to the IOLoop. To use the new monotonic clock functionality, all calls to `IOLoop.add_timeout` must be either pass a `datetime.timedelta` or a time relative to `IOLoop.time`, not `time.time`. (`time.time` will continue to work only as long as the IOLoop’s `time_func` argument is not used).
- New convenience method `IOLoop.run_sync` can be used to start an IOLoop just long enough to run a single coroutine.
- New method `IOLoop.add_callback_from_signal` is safe to use in a signal handler (the regular `add_callback` method may deadlock).
- `IOLoop` now uses `signal.set_wakeup_fd` where available (Python 2.6+ on Unix) to avoid a race condition that could result in Python signal handlers being delayed.
- Method `IOLoop.running()` has been removed.
- `IOLoop` has been refactored to better support subclassing.
- `IOLoop.add_callback` and `add_callback_from_signal` now take `*args, **kwargs` to pass along to the callback.

#### `tornado.iostream`

- `IOStream.connect` now has an optional `server_hostname` argument which will be used for SSL certificate validation when applicable. Additionally, when supported (on Python 3.2+), this hostname will be sent via SNI (and this is supported by `tornado.simple_httpclient`)
- Much of `IOStream` has been refactored into a separate class `BaseIOStream`.
- New class `tornado.iostream.PipeIOStream` provides the IOStream interface on pipe file descriptors.
- `IOStream` now raises a new exception `tornado.iostream.StreamClosedError` when you attempt to read or write after the stream has been closed (by either side).
- `IOStream` now simply closes the connection when it gets an `ECONNRESET` error, rather than logging it as an error.
- `IOStream.error` no longer picks up unrelated exceptions.
- `BaseIOStream.close` now has an `exc_info` argument (similar to the one used in the `logging` module) that can be used to set the stream’s `error` attribute when closing it.
- `BaseIOStream.read_until_close` now works correctly when it is called while there is buffered data.
- Fixed a major performance regression when run on PyPy (introduced in Tornado 2.3).

#### `tornado.log`

- New module containing `enable_pretty_logging` and `LogFormatter`, moved from the options module.
- `LogFormatter` now handles non-ascii data in messages and tracebacks better.

#### `tornado.netutil`

- New class `tornado.netutil.Resolver` provides an asynchronous interface to DNS resolution. The default implementation is still blocking, but non-blocking implementations are available using one of three optional dependencies: `ThreadedResolver` using the `concurrent.futures` thread pool, `tornado.platform.caresresolver.CaresResolver` using the `pycares` library, or `tornado.platform.twisted.TwistedResolver` using `twisted`
- New function `tornado.netutil.is_valid_ip` returns true if a given string is a valid IP (v4 or v6) address.
- `tornado.netutil.bind_sockets` has a new `flags` argument that can be used to pass additional flags to `getaddrinfo`.
- `tornado.netutil.bind_sockets` no longer sets `AI_ADDRCONFIG`; this will cause it to bind to both ipv4 and ipv6 more often than before.
- `tornado.netutil.bind_sockets` now works when Python was compiled with `--disable-ipv6` but IPv6 DNS resolution is available on the system.
- `tornado.netutil.TCPServer` has moved to its own module, `tornado.tcpserver`.

#### `tornado.options`

- The class underlying the functions in `tornado.options` is now public (`tornado.options.OptionParser`). This can be used to create multiple independent option sets, such as for subcommands.
- `tornado.options.parse_config_file` now configures logging automatically by default, in the same way that `parse_command_line` does.
- New function `tornado.options.add_parse_callback` schedules a callback to be run after the command line or config file has been parsed. The keyword argument `final=False` can be used on either parsing function to supress these callbacks.
- `tornado.options.define` now takes a `callback` argument. This callback will be run with the new value whenever the option is changed. This is especially useful for options that set other options, such as by reading from a config file.
- `tornado.options.parse_command_line` `--help` output now goes to `stderr` rather than `stdout`.
- `tornado.options.options` is no longer a subclass of `dict`; attribute-style access is now required.
- `tornado.options.options` (and `OptionParser` instances generally) now have a `mockable()` method that returns a wrapper object compatible with `mock.patch`.
- Function `tornado.options.enable_pretty_logging` has been moved to the `tornado.log` module.

#### `tornado.platform.caresresolver`

- New module containing an asynchronous implementation of the `Resolver` interface, using the `pycares` library.

#### `tornado.platform.twisted`

- New class `tornado.platform.twisted.TwistedIOLoop` allows Tornado code to be run on the Twisted reactor (as opposed to the existing `TornadoReactor`, which bridges the gap in the other direction).
- New class `tornado.platform.twisted.TwistedResolver` is an asynchronous implementation of the `Resolver` interface.

#### `tornado.process`

- New class `tornado.process.Subprocess` wraps `subprocess.Popen` with `PipeIOStream` access to the child’s file descriptors.

#### `tornado.simple_httpclient`

- `SimpleAsyncHTTPClient` now takes a `resolver` keyword argument (which may be passed to either the constructor or `configure`), to allow it to use the new non-blocking `tornado.netutil.Resolver`.
- When following redirects, `SimpleAsyncHTTPClient` now treats a 302 response code the same as a 303. This is contrary to the HTTP spec but consistent with all browsers and other major HTTP clients (including `CurlAsyncHTTPClient`).
- The behavior of `header_callback` with `SimpleAsyncHTTPClient` has changed and is now the same as that of `CurlAsyncHTTPClient`. The header callback now receives the first line of the response (e.g. `HTTP/1.0 200 OK`) and the final empty line.
- `tornado.simple_httpclient` now accepts responses with a 304 status code that include a `Content-Length` header.
- Fixed a bug in which `SimpleAsyncHTTPClient` callbacks were being run in the client’s `stack_context`.

#### `tornado.stack_context`

- `stack_context.wrap` now runs the wrapped callback in a more consistent environment by recreating contexts even if they already exist on the stack.
- Fixed a bug in which stack contexts could leak from one callback chain to another.
- Yield statements inside a `with` statement can cause stack contexts to become inconsistent; an exception will now be raised when this case is detected.

#### `tornado.template`

- Errors while rendering templates no longer log the generated code, since the enhanced stack traces (from version 2.1) should make this unnecessary.
- The `{% apply %}` directive now works properly with functions that return both unicode strings and byte strings (previously only byte strings were supported).
- Code in templates is no longer affected by Tornado’s `__future__` imports (which previously included `absolute_import` and `division`).

#### `tornado.testing`

- New function `tornado.testing.bind_unused_port` both chooses a port and binds a socket to it, so there is no risk of another process using the same port. `get_unused_port` is now deprecated.
- New decorator `tornado.testing.gen_test` can be used to allow for yielding `tornado.gen` objects in tests, as an alternative to the `stop` and `wait` methods of `AsyncTestCase`.
- `tornado.testing.AsyncTestCase` and friends now extend `unittest2.TestCase` when it is available (and continue to use the standard `unittest` module when `unittest2` is not available)
- `tornado.testing.ExpectLog` can be used as a finer-grained alternative to `tornado.testing.LogTrapTestCase`
- The command-line interface to `tornado.testing.main` now supports additional arguments from the underlying `unittest` module: `verbose`, `quiet`, `failfast`, `catch`, `buffer`.
- The deprecated `--autoreload` option of `tornado.testing.main` has been removed. Use `python -m tornado.autoreload` as a prefix command instead.
- The `--httpclient` option of `tornado.testing.main` has been moved to `tornado.test.runtests` so as not to pollute the application option namespace. The `tornado.options` module’s new callback support now makes it easy to add options from a wrapper script instead of putting all possible options in `tornado.testing.main`.
- `AsyncHTTPTestCase` no longer calls `AsyncHTTPClient.close` for tests that use the singleton `IOLoop.instance`.
- `LogTrapTestCase` no longer fails when run in unknown logging configurations. This allows tests to be run under nose, which does its own log buffering (`LogTrapTestCase` doesn’t do anything useful in this case, but at least it doesn’t break things any more).

#### `tornado.util`

- `tornado.util.b` (which was only intended for internal use) is gone.

#### `tornado.web`

- `RequestHandler.set_header` now overwrites previous header values case-insensitively.
- `tornado.web.RequestHandler` has new attributes `path_args` and `path_kwargs`, which contain the positional and keyword arguments that are passed to the `get`/`post`/etc method. These attributes are set before those methods are called, so they are available during `prepare()`
- `tornado.web.ErrorHandler` no longer requires XSRF tokens on `POST` requests, so posts to an unknown url will always return 404 instead of complaining about XSRF tokens.
- Several methods related to HTTP status codes now take a `reason` keyword argument to specify an alternate “reason” string (i.e. the “Not Found” in “HTTP/1.1 404 Not Found”). It is now possible to set status codes other than those defined in the spec, as long as a reason string is given.
- The `Date` HTTP header is now set by default on all responses.
- `Etag`/`If-None-Match` requests now work with `StaticFileHandler`.
- `StaticFileHandler` no longer sets `Cache-Control: public` unnecessarily.
- When gzip is enabled in a `tornado.web.Application`, appropriate `Vary: Accept-Encoding` headers are now sent.
- It is no longer necessary to pass all handlers for a host in a single `Application.add_handlers` call. Now the request will be matched against the handlers for any `host_pattern` that includes the request’s `Host` header.

#### `tornado.websocket`

- Client-side WebSocket support is now available: `tornado.websocket.websocket_connect`
- `WebSocketHandler` has new methods `ping` and `on_pong` to send pings to the browser (not supported on the `draft76` protocol)