Mercurial > p > roundup > code
comparison roundup/admin.py @ 8088:1045425c23b2
refactor!: replace os.listdir() with os.scandir()
In many places we did a listdir() then a stat to see if it's a file or
directory. This change removes the need for the stat call. Also for
larger directories, scandir() is an iterator, so less memory use.
There is one remnant of listdir used in an error handler. That
requires a stat on each element in the directory, so there is no
benefit to using scandir() other than a slight memory saving on a
rarely used piece of code.
BREAKING CHANGE:
Python 2 requires installation of scandir pip package after this
commit.
| author | John Rouillard <rouilj@ieee.org> |
|---|---|
| date | Tue, 16 Jul 2024 01:05:49 -0400 |
| parents | 851ddd72f9ce |
| children | 586f76eb33e8 |
comparison
equal
deleted
inserted
replaced
| 8087:0cb81ee2e572 | 8088:1045425c23b2 |
|---|---|
| 1224 | 1224 |
| 1225 class colon_separated(csv.excel): | 1225 class colon_separated(csv.excel): |
| 1226 delimiter = ':' | 1226 delimiter = ':' |
| 1227 | 1227 |
| 1228 # import all the files | 1228 # import all the files |
| 1229 for file in os.listdir(import_dir): | 1229 for dir_entry in os.scandir(import_dir): |
| 1230 classname, ext = os.path.splitext(file) | 1230 filename = dir_entry.name |
| 1231 classname, ext = os.path.splitext(filename) | |
| 1231 # we only care about CSV files | 1232 # we only care about CSV files |
| 1232 if ext != '.csv' or classname.endswith('-journals'): | 1233 if ext != '.csv' or classname.endswith('-journals'): |
| 1233 continue | 1234 continue |
| 1234 | 1235 |
| 1235 cl = self.get_class(classname) | 1236 cl = self.get_class(classname) |
| 1236 | 1237 |
| 1237 # ensure that the properties and the CSV file headings match | 1238 # ensure that the properties and the CSV file headings match |
| 1238 with open(os.path.join(import_dir, file), 'r') as f: | 1239 with open(os.path.join(import_dir, filename), 'r') as f: |
| 1239 reader = csv.reader(f, colon_separated, lineterminator='\n') | 1240 reader = csv.reader(f, colon_separated, lineterminator='\n') |
| 1240 file_props = None | 1241 file_props = None |
| 1241 maxid = 1 | 1242 maxid = 1 |
| 1242 # loop through the file and create a node for each entry | 1243 # loop through the file and create a node for each entry |
| 1243 for n, r in enumerate(reader): | 1244 for n, r in enumerate(reader): |