# PyPy v7.3.10: release of python 2.7, 3.8, and 3.9, released 2022-12-06

The PyPy team is proud to release version 7.3.10 of PyPy. We have some nice speedups and bugfixes we wish to share. The release includes three different interpreters:

> - PyPy2.7, which is an interpreter supporting the syntax and the features of Python 2.7 including the stdlib for CPython 2.7.18+ (the `+` is for backported security updates)

> - PyPy3.8, which is an interpreter supporting the syntax and the features of Python 3.8, including the stdlib for CPython 3.8.15.

> - PyPy3.9, which is an interpreter supporting the syntax and the features of Python 3.9, including the stdlib for CPython 3.9.15. We have gained confidence in the stability of this version, and are removing the "beta" label.

The interpreters are based on much the same codebase, thus the multiple release. This is a micro release, all APIs are compatible with the other 7.3 releases. Highlights of the release, since the release of 7.3.9 in March 2022 include:

> - A release of Apple Silicon M1 arm64 versions. This work was sponsored by an anonymous donor and is tested on our buildbots.

> - Many improvements to the basic interpreter to make it 15-20% faster

> - The conda-forge community has built over 1000 packages for PyPy3.8 and 3.9, making it easier than ever to use PyPy.

> - Update the packaged OpenSSL to 1.1.1s, sqlite3 to 3.39.4, and apply applicable security fixes from CPython 3.9.15 to PyPy2.7

> - Update the HPy backend in PyPy3.8 and PyPy3.9 to 0.0.4

We recommend updating. You can find links to download the v7.3.10 releases here:

> https://pypy.org/download.html

We would like to thank our donors for the continued support of the PyPy project. If PyPy is not quite good enough for your needs, we are available for direct consulting work. If PyPy is helping you out, we would love to hear about it and encourage submissions to our blog via a pull request to https://github.com/pypy/pypy.org

We would also like to thank our contributors and encourage new people to join the project. PyPy has many layers and we need help with all of them: bug fixes, PyPy and RPython documentation improvements, or general help with making RPython's JIT even better. Since the previous release, we have accepted contributions from five new contributors, thanks for pitching in, and welcome to the project!

If you are a python library maintainer and use C-extensions, please consider making a HPy / CFFI / cppyy version of your library that would be performant on PyPy. In any case, both cibuildwheel and the multibuild system support building wheels for PyPy.

## What is PyPy?

PyPy is a Python interpreter, a drop-in replacement for CPython 2.7, 3.8 and 3.9. It's fast (PyPy and CPython 3.7.4 performance comparison) due to its integrated tracing JIT compiler.

We also welcome developers of other dynamic languages to see what RPython can do for them.

We provide binary builds for:

> - x86 machines on most common operating systems (Linux 32/64 bits, Mac OS 64 bits, Windows 64 bits)

> - 64-bit ARM machines running Linux (`aarch64`).

> - Apple M1 arm64 machines (`macos_arm64`).

> - s390x running Linux

PyPy support Windows 32-bit, Linux PPC64 big- and little-endian, and Linux ARM 32 bit, but does not release binaries. Please reach out to us if you wish to sponsor binary releases for those platforms. Downstream packagers provide binary builds for debian, Fedora, conda, OpenBSD, FreeBSD, Gentoo, and more.

## Changelog

### Default version (2.7+)

#### Bugfixes shared across versions

- Fix zlib `ustart` handling for zlib v1.2.12 (issue 3717)

- Backport security fixes to Python2.7

- Structseq improvements: hide `structseqfield.__get__`, ignore any extra keys in the dict, preserve MapDict implementation strategy as the `__dict__`, make fields immutable and more

- Fix embedding startup code in CFFI (issue 3619)

- Fix `xmm` scratch register on win64 (issue 3753)

- Fix corner cases in method and code `__ne__` (issue 3759)

- In translation, if ctypes doesn't find the `C` library with `find_library('c')`, try to fallback to generic `libc.so`. This enables building with musl (issue 3559)

- Unbreak string formatting with mixed bytes/unicode (issue 3802)

- Pull in the `http.server` vulnerability fix from cpython-87389

- Raise if empty set contains unhashable (issue 3824)

- Support `class A(_RawIOBase, BytesIO)` (issue 3821)

- When raising an error: don't convert random things to unicode (issue 3828)

- Implement the `.description` attribute of sqlite3 cursors more carefully (issue 3840)

- Always use `-fPIC` when building shared objects on linux platforms

- Fix `MSG_TRUNC` socket weirdness on linux (issue 3864)

#### Speedups and enhancements shared across versions

- Update the HPy backend to 0.0.4

- Update CFFI to the latest HEAD (no new version was released)

- Speed up `dict.copy` and `emptydict.update(dict)`

- Optimize list sorting to allocate memory a bit less aggressively. Seems to give ~10% on sorting non-tiny lists of `ints`

- Speed up the Python interpreter (jitted code is unchanged) by auto-generating rpython-level shortcut methods for many special methods. This speeds up the interpreter greatly because we don't need to lookup the special method and don't need to go through the general call machinery at all. The effect is comparable to CPython's type slots, but all auto-generated from `TypeDefs`. It only works for built-in types at this point.

- Use structs to unpack `longlong` instead of casting to lltype Arrays

- Speed up the interpreter by caching global and builtin lookups on the code object

- Fix caching of reference constants in JitCodes

- Make the exception transformer not introduce calls to `ll_issubclass`, instead emit the correct `int_between` for the type check directly

- Instead of encoding the liveness of local registers in each jitcode as a dict mapping pc to a (shared) instance of a class with three strings, do the following: add a live instruction in the jitcode that that has as its argument an offset into a string that compactly encodes liveness.

- Fast path for `string[0]` to convert a `str` to a `char` for when `string` is already a char

- Clean up a few single-use specialized dictionaries in RPython, this reduces the binary size somewhat.

- Make `list.count` use the same fast paths as `list.index` (issue 3744)

- Improve `int.bit_length` for the jit: expose unwrapping and rewrapping to tracing

- Add a fast path for `getrandbits(n)` where `n <= 31` (issue 3733)

- Remove useless `cvt = converters.get(type(param))` from sqlite3: it was wrong and slowed things down

- Add two new hints to `rlib.jit`:

  - `record_exact_value(var, const)` tells the JIT that the box `var` must contain the value `const`.

  - `record_known_result(result, func, *args)` is a way to encode knowledge about the result of elidable functions. The hint means that the JIT can assume that if `func(*args)` will be called later, the outcome is `result`

  Typical usecases of this are: you can express this way that functions are inverses of each other, or that a function is idempotent. Both hints need to be used very carefully, because getting them wrong can really lead to miscompilation and crashes.

- Speed up `posix.stat` calls by directly constructing the output, avoiding a `structseq`

- Make PyPy available for Apple M1 (arm64)

  - Support JIT backend code generation

  - Handle the different FFI calling conventions

  - Widen support for packaging the build

  - Distinguish between the two macos builds

  - Set up a buildbot machine to run CI

- Add an optimization for `uint_rshift(0, x) -> 0` and `uint_rshift(x, 0) -> x`. Previously the optimization was only for `int_rshift`

- Make it possible to `@specialize.memo` on `rgc` custom trace hooks

- Use a more subtle condition to check whether aliasing is present when doing malloc removal in the static RPython optimizers.

- Micro-optimize `TraceIterator.next()` to not allocate quite so many intermediate lists in the JIT code that walks over an encoded trace.

- Only put `OptimizationResults` into the list for callbacks if the callback would actually do anything in the JIT optimizer.

- Small optimizations to improve tracing speed:

  - Have special versions of various record functions that take a fixed number of arguments. This makes it possible to not allocate arguments lists.

  - Don't lookup constant pointers that come from the jitcode in a dictionary again and again in opencoder.

- Make sure that `W_Root.getclass` does not exist in two versions, one for `access_directly=True`, one regular

- Two improvements to space operations:

  - rewrite the translation-time lookup caching to work on the RPython class instead of the `W_TypeObjects`. This makes the latter smaller and saves us having to call `space.type(w_obj)` first.

  - fix caching of binary ops by using a `@specialize`

- Clean up the number of `w_obj.getclass` variants in mapdict

- Use `append_char` where appropriate in unicode string builder

- Use a fast-path for `str.encode("utf-8")` (issue 3756)

- Optimize `float_abs(float_abs(x))` to `float_abs(x)` in the JIT

- Fix NFA generation in metaparser for grammar rules of form `foo: [a* b]` in the parser generator that is used for PyPy2.7 and PyPy3.8.

- Introduce `space.newtuple2` to save the list allocation when a specialized two-tuple is used anyway and use it in `.next` of `enumerate` and `zip`.

- Speed up using `warnings.warn` by making it more JIT friendly

- Add an option to the collect analyzer when defining a custom gc trace function

- Add a runtime JIT hook to disable tracing

- Add `PYPY_DISABLE_JIT` as an environment variable to disable the JIT (issue 3148)

- Fast-path finding whitespace in an ascii string inside `unicode.split()`

- Resync `_vmprof` with `vmprof-python`

- Replace the trie of names in unicodedata with a directed acyclic word graph to make it more compact. Also various other improvements to make unicodedata more compact. This change shrinks the PyPy2.7 binary by 2.1MiB, PyPy3.x by 2.6MiB.

- Review all the use cases of `jit.loop_unrolling_heuristic`, to unroll less aggressively (issue 3781)

- Inline `_fill_original_boxes` in the JIT to avoid creating variants in C

- Optimize `inline_call_*` in the JIT by filling in the new frame directly instead of creating an intermediate list of boxes

- Make sure the `LivenessIterator` gets inlined and optimized away in the JIT

- Speed up `append_slice` on unicode builders

- Make `list.__repr__` use a jit driver, and have implementations for a few of the strategies

- Expose a new function `__pypy__._raise_in_thread` that will raise an asynchronous exception in another thread the next time that thread runs. This also makes it possible to implement `PyThreadState_SetAsyncExc` (issue 3757)

- Make locals use an instance dict to speed them up

- Tiny warmup improvement: don't create the `recentops` in the JIT optimizer when looking for an existing operation, only when adding one

- Avoid using the pureop cache for `int_invert` and `float_neg`

- Speed up global dict reads by using the heapcache in the JIT frontend

- Constant-fold `ovf` operations in rpython

- Consider equal lltype ptrs to be equal constants in rpython constant-folding

#### C-API (cpyext) and C-extensions

- Make sure `decref` is being called even if a C-level call raises an exception (issue 3854)

### Python 3.8+

#### Python 3.8+ bugfixes

- Fix bug in the disassembler of py3 opcodes (issue 3700)

- Raise `ModuleNotFoundError` instead of `ImportError` in some cases

- Fix `.lineno`, `.col_offset` for decorated functions and classes

- Add a `name` to `sys.hash_info`

- Fix concurrency problem in buffered `io` reading (issue 3729)

- Make it possible to multiple-inherit from `KeyError` again (issue 3728)

- Check results from _openssl's `EVP_DigestInit_ex` and `EVP_DigestUpdate`, and fix some failing tests (issue 3741)

- Fix pickling of `filter` objects

- Fix the way that the lookup annotation optimization breaks python3 due to the way that module instances can change their class at runtime (issue 3758)

- Use the name mapping when creating new hashes for `_hashlib` (issue 3778)

- Expose `os.sendfile` on macos

- Do not override PyPy's `MAGIC_NUMBER` when using `importlib/_bootstrap_external.py` (issue 3783)

- Fix dictionary unpacking for `kwargs` (issue 3775)

- Add memory pressure when creating a tkinter image (issue 3798)

- Remove debug print from `_winapi` (issue 3819)

- Add `__contains__` to `array.array` type (issue 3820)

- Fix CVE-2022-37454 via porting CPython changes to _sha3/kcp/KeccakSponge.inc

- Make type lookups fill the `.name` field of `AttributeError`

- Check cursor lock in sqlite3 `Cursor.close`, also lock around `__fetch_one_row`

- Implement `os.get_native_thread`

- Fix setting a slice in a `memoryview` with non-unit strides (issue 3857)

- Fix the `__copy__` optimization of `itertools.tee`, which was copying the iterable, not the iterator (issue 3852)

- Fix `time.strftime` when the `format` contains unicode (issue 3862)

- Fix `time.strftime` formatting on windows

#### Python 3.8+ speedups and enhancements

- Speed up `fstrings` by making the parentstack a resizable list of chars

- Better error message when the `__iter__` of a class is set to `None` (issue 3716)

- Refactor the `package.py` script for better compatibility with conda-forge

- Add a jit driver for `filter` (issue 3745)

- Improve opcode handling: `jump_absolute`, `int_xor`, and others

- Don't make a JIT loop for one-arg `print()`

- Make float hashing elidable and avoid creating bridges

- Mimic CPython's `max_int_threshold` to limit the length of a string that that can be parsed into an int

#### Python 3.8+ C-API

- Add `PyReversed_Type`, `PyUnicode_EncodeCodePage`, `PyInterpreterState_GetID`, `PyErr_SetFromErrnoWithFilenameObjects`, `PyUnicode_Append`, `PyUnicode_AppendAndDel`, `PyClassMethod_Type`, `PyStructSequence_SetItem`, `PyStructSequence_GetItem`, `PyDictValues_Type`, `PyDictKeys_Type`,

- Map user defined python `__init__` to `tp_init` (issue 2806)

- Fix PyDict_Contains (issue 3742)

- Allow big ints in `PyNumber_ToBase` (issue 3765)

- Normalize OSErrors more consistently, may not be completely fixed on macos (issue 3786)

- Fix `PyDict_Contains` to raise on unhashable key

- Use `tp_itemsize==0` on `PyUnicode_Type`, even for compact forms (issue 3772)

- Include `<sys/time.h>` in headers, which fixes a problem with musl (issue 3801)

- Add missing incref in `PyObject_Init`, allocate `tp_basicsize` bytes when creating a `PyTypeObject` (issues 3844, 3847)

- Assign `tp_getset` to app-level type in `PyType_FromSpecWithBases` (issue 3851)

- Properly instantiate `PyFunction_Type`, `PyMethod_Type`, `PyRange_Type`, `PyTraceBack_Type` (issue 3776)

### Python 3.9+

#### Python 3.9+ bugfixes

- Fix `f-string` bug where the recursive tokenization was done incorrectly (issue 3751)

- Fixes to `repr` and slots of nested `GenericAliases` (issue 3720)

- Match CPython error messages for zip when `strict=True` (this is a backported 3.10 feature).

- Add `BASE_TYPE_ADAPTION` optimization to sqlite3, copied from CPython's approach

- Make `__file__` of the `__main__` module be an absolute path, if possible (issue 3766)

- Use an absolute path for the main module (issue 3792)

- Use an absolute path for `sys.path[0]` when running a directory from the cmdline (issue 3792)

- Fix first line number of `eval` to be reported as 0 (issue 3800)

- Implement `bitcount` for ints

- Check when unmarshalling `TYPE_SHORT_ASCII` that non-ascii bytes are not present

- Fix CVE-2022-42919 (str -> int parsing) as CPython did in cpython-97514

- Fix `DICT_MERGE` bytecode with objects that aren't dicts and don't implement `__len__` (issue 3841)

- Remove redundant pure-python `_functools.py` (issue 3861)

- Fix pure-python `functools.py` from CPython (CPython uses a c-extension)

#### Python 3.9+ speedups and enhancements

- Adopt CPython changes to speed up fractions (issue 3746, cpython-91851)

- Speed up `math.perm` and `math.comb` (issue 3859)

#### Python 3.9+ C-API

- Add `PyObject_VectorcallMethod`,

- Add `PyObject_Call` variants `*NoArgs`, `*OneArg`, `*MethodNoArgs`, `*MethodOneArg` (issue 3669)

- Handle vectorcall offset (issue 3845)