Comments on: Direct IO in Python

By: Reshmi

Reshmi — Wed, 02 Sep 2020 16:13:13 +0000

@Logan: this code does not seem to work in my system.

Traceback (most recent call last):
File “test.py”, line 4, in
f = os.fdopen(fd, “rb+”, 0)
OSError: [Errno 22] Invalid argument

By: Logan Gunthorpe

Logan Gunthorpe — Fri, 06 Dec 2019 17:36:14 +0000

This works for reads into an mmap buffer:

import mmap
import os

fd = os.open(“strace.txt”, os.O_DIRECT | os.O_RDWR)
f = os.fdopen(fd, “rb+”, 0)
m = mmap.mmap(-1, 4096)

print(f.readinto(m))

By: Brad Goodman

Brad Goodman — Tue, 30 Apr 2019 18:22:50 +0000

That’s what I needed – thanks!

By: Hilton Fernandes

Hilton Fernandes — Mon, 20 Nov 2017 11:28:14 +0000

Hello, @JPetazzo !

Where can I find documentation for the use of your library directio ?

Thanks in advance.

All the best,
Hilton

By: Matt C

Matt C — Fri, 16 May 2014 16:54:16 +0000

I found out how to do direct I/O reading from a file: convert it into a file object and the use the readinto() method of python file objects to read into the mmap’d buffer. To add onto the original code above:

fo = os.fdopen(f, ‘rw’)
fo.readinto(m)

By: Kenneth

Kenneth — Wed, 17 Apr 2013 21:10:29 +0000

How do you go about direct io reading from the file? I get the same silly error when I do an “data = os.read(4*1024)”.

By: Alexander Sandler

Alexander Sandler — Sun, 05 Feb 2012 21:05:35 +0000

@Sebastian
My pleasure. Please come again

By: Sebastian

Sebastian — Sat, 19 Nov 2011 15:24:27 +0000

Thanks for the hint to alignment as a reason for “Invalid argument”!

By: Alexander Sandler

Alexander Sandler — Mon, 19 Sep 2011 16:27:02 +0000

@Evan Jones
Good. Very interesting reading. Thanks a lot.

By: Evan Jones

Evan Jones — Mon, 19 Sep 2011 11:11:39 +0000

@Alexander Sandler – I actually did, but forgot to post something here. See Making Writes Durable. This is unfortunately written from the perspective of the underlying C system calls, and doesn’t discuss Python at all. However, it is relevant provided that you use the correct Python function calls.

http://www.evanjones.ca/durable-writes.html