|
| 1 | +:mod:`btree` -- simple BTree database |
| 2 | +===================================== |
| 3 | + |
| 4 | +.. module:: btree |
| 5 | + :synopsis: simple BTree database |
| 6 | + |
| 7 | +The ``btree`` module implements a simple key-value database using external |
| 8 | +storage (disk files, or in general case, a random-access stream). Keys are |
| 9 | +stored sorted in the database, and besides efficient retrieval by a key |
| 10 | +value, a database also supports efficient ordered range scans (retrieval |
| 11 | +of values with the keys in a given range). On the application interface |
| 12 | +side, BTree database work as close a possible to a way standard `dict` |
| 13 | +type works, one notable difference is that both keys and values must |
| 14 | +be `bytes` objects (so, if you want to store objects of other types, you |
| 15 | +need to serialize them to `bytes` first). |
| 16 | + |
| 17 | +The module is based on the well-known BerkelyDB library, version 1.xx. |
| 18 | + |
| 19 | +Example:: |
| 20 | + |
| 21 | + import btree |
| 22 | + |
| 23 | + # First, we need to open a stream which holds a database |
| 24 | + # This is usually a file, but can be in-memory database |
| 25 | + # using uio.BytesIO, a raw flash section, etc. |
| 26 | + f = open("mydb", "w+b") |
| 27 | + |
| 28 | + # Now open a database itself |
| 29 | + db = btree.open(f) |
| 30 | + |
| 31 | + # The keys you add will be sorted internally in the database |
| 32 | + db[b"3"] = b"three" |
| 33 | + db[b"1"] = b"one" |
| 34 | + db[b"2"] = b"two" |
| 35 | + |
| 36 | + # Prints b'two' |
| 37 | + print(db[b"2"]) |
| 38 | + |
| 39 | + # Iterate over sorted keys in the database, starting from b"2" |
| 40 | + # until the end of the database, returning only values. |
| 41 | + # Mind that arguments passed to values() method are *key* values. |
| 42 | + # Prints: |
| 43 | + # b'two' |
| 44 | + # b'three' |
| 45 | + for word in db.values(b"2"): |
| 46 | + print(word) |
| 47 | + |
| 48 | + del db[b"2"] |
| 49 | + |
| 50 | + # No longer true, prints False |
| 51 | + print(b"2" in db) |
| 52 | + |
| 53 | + # Prints: |
| 54 | + # b"1" |
| 55 | + # b"3" |
| 56 | + for key in db: |
| 57 | + print(key) |
| 58 | + |
| 59 | + db.close() |
| 60 | + |
| 61 | + # Don't forget to close the underlying stream! |
| 62 | + f.close() |
| 63 | + |
| 64 | + |
| 65 | +Functions |
| 66 | +--------- |
| 67 | + |
| 68 | +.. function:: open(stream, \*, flags=0, cachesize=0, pagesize=0, minkeypage=0) |
| 69 | + |
| 70 | + Open a database from a random-access `stream` (like an open file). All |
| 71 | + other parameters are optional and keyword-only, and allow to tweak advanced |
| 72 | + paramters of the database operation (most users will not need them): |
| 73 | + |
| 74 | + * `flags` - Currently unused. |
| 75 | + * `cachesize` - Suggested maximum memory cache size in bytes. For a |
| 76 | + board with enough memory using larger values may improve performance. |
| 77 | + The value is only a recommendation, the module may use more memory if |
| 78 | + values set too low. |
| 79 | + * `pagesize` - Page size used for the nodes in BTree. Acceptable range |
| 80 | + is 512-65536. If 0, underlying I/O block size will be used (the best |
| 81 | + compromise between memory usage and performance). |
| 82 | + * `minkeypage` - Minimum number of keys to store per page. Default value |
| 83 | + of 0 equivalent to 2. |
| 84 | + |
| 85 | + Returns a `BTree` object, which implements a dictionary protocol (set |
| 86 | + of methods), and some additional methods described below. |
| 87 | + |
| 88 | +Methods |
| 89 | +------- |
| 90 | + |
| 91 | +.. method:: btree.close() |
| 92 | + |
| 93 | + Close the database. It's mandatory to close the database at the end of |
| 94 | + processing, as some unwritten data may be still in the cache. Note that |
| 95 | + this does not close underlying streamw with which the database was opened, |
| 96 | + it should be closed separately (which is also mandatory to make sure that |
| 97 | + data flushed from buffer to the underlying storage). |
| 98 | + |
| 99 | +.. method:: btree.flush() |
| 100 | + |
| 101 | + Flush any data in cache to the underlying stream. |
| 102 | + |
| 103 | +.. method:: btree.__getitem__(key) |
| 104 | +.. method:: btree.get(key, default=None) |
| 105 | +.. method:: btree.__setitem__(key, val) |
| 106 | +.. method:: btree.__detitem__(key) |
| 107 | +.. method:: btree.__contains__(key) |
| 108 | + |
| 109 | + Standard dictionary methods. |
| 110 | + |
| 111 | +.. method:: btree.__iter__() |
| 112 | + |
| 113 | + A BTree object can be iterated over directly (similar to a dictionary) |
| 114 | + to get access to all keys in order. |
| 115 | + |
| 116 | +.. method:: btree.keys([start_key, [end_key, [flags]]]) |
| 117 | +.. method:: btree.values([start_key, [end_key, [flags]]]) |
| 118 | +.. method:: btree.items([start_key, [end_key, [flags]]]) |
| 119 | + |
| 120 | + These methods are similar to standard dictionary methods, but also can |
| 121 | + take optional parameters to iterate over a key sub-range, instead of |
| 122 | + the entire database. Note that for all 3 methods, `start_key` and |
| 123 | + `end_key` arguments represent key values. For example, ``values()`` |
| 124 | + method will iterate over values corresponding to they key range |
| 125 | + given. None values for `start_key` means "from the first key", no |
| 126 | + `end_key` or its value of None means "until the end of database". |
| 127 | + By default, range is inclusive of `start_key` and exclusive of |
| 128 | + `end_key`, you can include `end_key` in iteration by passing `flags` |
| 129 | + of `btree.INCL`. You can iterate in descending key direction |
| 130 | + by passing `flags` of `btree.DESC`. The flags values can be ORed |
| 131 | + together. |
| 132 | + |
| 133 | +Constants |
| 134 | +--------- |
| 135 | + |
| 136 | +.. data:: INCL |
| 137 | + |
| 138 | + A flag for `keys()`, `values()`, `items()` methods to specify that |
| 139 | + scanning should be inclusive of the end key. |
| 140 | + |
| 141 | +.. data:: DESC |
| 142 | + |
| 143 | + A flag for `keys()`, `values()`, `items()` methods to specify that |
| 144 | + scanning should be in descending direction of keys. |
0 commit comments