Real async file I/O for Python.

Other async file libraries dispatch every call to a thread pool. turbofile submits the I/O to the kernel (io_uring on Linux, POSIX AIO on macOS) and completes your await when the kernel says the data moved. Same API as aiofiles.

PIP INSTALL TURBOFILE VIEW ON GITHUB

PYTHON ≥ 3.12 · RUST CORE · MIT OR APACHE-2.0

term · turbofile

SAME CALLS AS AIOFILES · COMPLETED BY THE KERNEL

BUILT FOR:DATA LOADERSLOG TAILERSCHECKPOINT WRITERSWEB SERVERSETL PIPELINES

Fast where asyncio actually runs

An asyncio application does many small operations in flight, not one giant read. That is the workload where kernel submission beats a thread pool.

make bench · vs aiofiles
$ make bench
4 KiB whole-file read (read_bytes)  ... 2.5x
32 concurrent 4 KiB random reads    ... 15x
200 small files read concurrently   ... 3.6x
8 MiB whole-file read              ... 1.0x
8 MiB sequential write (1 MiB)      ... 1.0x

Apple-silicon Mac, macOS 26.4, POSIX AIO backend, page-cache-hot files. Large sequential transfers are memory-bandwidth-bound, so every implementation converges there; the wins are where per-op overhead and concurrency dominate. Run make bench for your hardware.

What it does differently

DROP-IN API

Keep your code

Binary and text modes, encoding/errors/newline, seek and tell, readline, async iteration, readinto, truncate, fsync. For the open API, import turbofile as aiofiles is the whole migration.

SWAP IT IN
ZERO-COPY

Bytes move once

Reads: the kernel fills the bytes object your await returns. Writes: your buffer is pinned and the kernel gets its pointer. No intermediate copies on the hot paths.

SEE THE NUMBERS
COMPLETION BATCHING

One wakeup per burst

A burst of completions costs a single event-loop wakeup, drained entirely in Rust. Large reads split into chunks the kernel fills concurrently into one buffer.

MEET THE BACKENDS

The fine print, up front

Known limitations, straight from the README

  • opener= and integer file descriptors are not supported.
  • read_bytes on very large files pays one buffer copy; prefer open(...).read() for multi-megabyte files.
  • Cancelling an await detaches the future; the kernel op still completes (and, for readinto, may still write into the buffer). Standard completion-model semantics.
■ BACKENDS

The best mechanism each OS has

Selection is automatic. TURBOFILE_BACKEND=compio overrides it on macOS. Windows (IOCP via compio) is planned.

uring [Linux]

COMPIO FUSION DRIVER · DEFAULT

io_uring submission and completion rings, with an automatic polling fallback under seccomp or older kernels.

aio [macOS]

POSIX AIO · DEFAULT

aio_read, aio_write and aio_fsync, served by XNU. Kernel-serviced async I/O, not a thread pool.

compio [macOS · opt-in]

KQUEUE POLLING DRIVER

The compio kqueue driver with thread dispatch, for benchmarking or as an escape hatch if POSIX AIO misbehaves on your setup.

FOR YOUR CODEBASE

One command, same API

GET IT ON PYPI

$ pip install turbofile

FOR THE CURIOUS

Read the source

GITHUB

RUST CORE · PYO3 · MIT OR APACHE-2.0