Skip to content

Commit 2fcdae8

Browse files
dhondtadhondta
committed
Refactored documentation
1 parent 73d109e commit 2fcdae8

12 files changed

Lines changed: 313 additions & 83 deletions

File tree

docs/base.md renamed to docs/enc/base.md

File renamed without changes.

docs/binary.md renamed to docs/enc/binary.md

Lines changed: 1 addition & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,4 @@
1-
With `codext`, the `codecs` library has multiple new encodings in addition to [the native ones](https://docs.python.org/3.8/library/codecs.html#standard-encodings), like presented hereafter.
2-
3-
Unless explicitely specified, each codec supports writing to and reading from a file.
4-
5-
!!! warning "Lossy conversion"
6-
7-
Some encodings are lossy, meaning that it is not always possible to decode back to the exact start string. This should be considered especially when chaining codecs.
1+
`codext` also adds common binary encodings. For instance, the Manchester code, that encodes digits, is applied to the ordinals of the input text and the resulting binary stream is converted back to characters.
82

93
-----
104

docs/common.md renamed to docs/enc/common.md

Lines changed: 1 addition & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,4 @@
1-
With `codext`, the `codecs` library has multiple new encodings in addition to [the native ones](https://docs.python.org/3.8/library/codecs.html#standard-encodings), like presented hereafter.
2-
3-
Unless explicitely specified, each codec supports writing to and reading from a file.
4-
5-
!!! warning "Lossy conversion"
6-
7-
Some encodings are lossy, meaning that it is not always possible to decode back to the exact start string. This should be considered especially when chaining codecs.
1+
`codext` also provides some very common encodings, for the sake of simplicity (e.g. while chaining codecs with [the CLI tool](../cli.html)).
82

93
-----
104

docs/ciphers.md renamed to docs/enc/crypto.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
`codext` also implements several simple cryptographic ciphers that are comparable to encodings for each value of their keys as these are too weak.
1+
`codext` also implements several simple cryptographic ciphers. But how does it relate to encoding while a key is required ? `codext` focuses on ciphers that have a weak key. With dynamically named encodings, it is then possible to define a bunch of encodings, one for each value of the key. For instance, Barbie Typewriter has a key with only 4 possible values. The `barbie` codec can then be `barbie-1`, ..., `barbie-4`.
22

33
!!! note "Available masks"
44

docs/languages.md renamed to docs/enc/languages.md

Lines changed: 2 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,4 @@
1-
With `codext`, the `codecs` library has multiple new encodings in addition to [the native ones](https://docs.python.org/3.8/library/codecs.html#standard-encodings), like presented hereafter.
2-
3-
Unless explicitely specified, each codec supports writing to and reading from a file.
4-
5-
!!! warning "Lossy conversion"
6-
7-
Some encodings are lossy, meaning that it is not always possible to decode back to the exact start string. This should be considered especially when chaining codecs.
1+
`codext` also adds some common languages for encoding.
82

93
-----
104

@@ -71,7 +65,7 @@ It supports of course letters and digits, but also a few special characters: `.,
7165

7266
-----
7367

74-
### Navajo Code
68+
### Navajo
7569

7670
It implements the letters from the [Navajo Code Talkers' Dictionary](https://www.history.navy.mil/research/library/online-reading-room/title-list-alphabetically/n/navajo-code-talker-dictionary.html). It conserves digits and newlines.
7771

docs/others.md renamed to docs/enc/others.md

Lines changed: 1 addition & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,4 @@
1-
With `codext`, the `codecs` library has multiple new encodings in addition to [the native ones](https://docs.python.org/3.8/library/codecs.html#standard-encodings), like presented hereafter.
2-
3-
Unless explicitely specified, each codec supports writing to and reading from a file.
4-
5-
!!! warning "Lossy conversion"
6-
7-
Some encodings are lossy, meaning that it is not always possible to decode back to the exact start string. This should be considered especially when chaining codecs.
1+
All kinds of other codecs are categorized in "*Others*".
82

93
-----
104

docs/stegano.md renamed to docs/enc/stegano.md

Lines changed: 2 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,4 @@
1-
With `codext`, the `codecs` library has multiple new encodings in addition to [the native ones](https://docs.python.org/3.8/library/codecs.html#standard-encodings), like presented hereafter.
2-
3-
Unless explicitely specified, each codec supports writing to and reading from a file.
4-
5-
!!! warning "Lossy conversion"
6-
7-
Some encodings are lossy, meaning that it is not always possible to decode back to the exact start string. This should be considered especially when chaining codecs.
1+
`codext` defines a few steganography-related encodings. While encoding is not really steganography (that is, concealing data within data), the following codecs are worth creating this category as they relate to converting data into something that could mislead the unaware reader.
82

93
-----
104

@@ -48,7 +42,7 @@ This codec implements the SMS encoding, also caled T9, that is the conversion fr
4842

4943
This simple encoding replaces zeros and ones of the binary version of the input text with spaces and tabs. It is supported either with its original mapping or with the inverted mapping.
5044

51-
!!! warning "Do not confuse"
45+
!!! warning "Encoding, not programming !"
5246

5347
This should not be confused with the [whitespace esoteric language](https://en.wikipedia.org/wiki/Whitespace_(programming_language)).
5448

‎docs/encodings.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
With `codext`, the `codecs` library has multiple new encodings in addition to [the native ones](https://docs.python.org/3.8/library/codecs.html#standard-encodings).
2+
3+
Unless explicitely specified, each codec supports writing to and reading from a file.
4+
5+
!!! warning "Lossy conversion"
6+
7+
Some encodings are lossy, meaning that it is not always possible to decode back to the exact start string. This should be considered especially when chaining codecs.
8+

‎docs/features.md‎

Lines changed: 47 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -78,44 +78,13 @@ In this second example, we can see that:
7878

7979
- Only the encoding function is defined.
8080
- A pattern is defined to match the prefix "`mydyn-`" and then an integer which is captured and used with `mydyncodec_encode(i)`.
81-
- The `text=True` keyword-argument could have been used to only support text de/encoding and not file handling.
8281

8382
!!! warning "Pattern capture group"
8483

8584
A capture group means that the parameter will be used with a dynamic (decorated) encoding function. In order to avoid this, i.e. for matching multiple names leading to the same encoding while calling a static encoding function, we can simply define a non-capturing group, e.g. "`(?:my|special_)codec`".
8685

8786
-----
8887

89-
## Search for encodings
90-
91-
Natively, `codecs` provides a `lookup` function that allows to get the `CodecInfo` object for the desired encoding. This performs a lookup in the registry based on an exact match. Sometimes, it can be useful to search for available encodings based on a regular expression. Therefore, a `search` function is added by `codext` to allow to get a list of encoding names matching the input regex.
92-
93-
```python
94-
>>> codext.search("baudot")
95-
['baudot', 'baudot_spaced', 'baudot_tape']
96-
>>> codext.search("al")
97-
['capitalize', 'octal', 'octal_spaced', 'ordinal', 'ordinal_spaced', 'radio']
98-
>>> codext.search("white")
99-
['whitespace', 'whitespace_after_before']
100-
```
101-
102-
!!! note "For new encodings only"
103-
104-
The `search` function relies on `codext`'s registry and not the native one, hence returning only regex matches for new encodings added with `codext` and an exact match by also using the `lookup` function.
105-
106-
Also, `codext` provides an `examples` function to get some examples of valid encoding names. This is especially useful when it concerns dynamicly named encodings (e.g. `rot`, `shift` or `dna`).
107-
108-
```python
109-
>>> codext.examples("rot")
110-
['rot-14', 'rot-24', 'rot-7', 'rot18', 'rot3', 'rot4', 'rot6', 'rot_1', 'rot_12', 'rot_2']
111-
>>> codext.examples("dna")
112-
['dna-1', 'dna-2', 'dna-5', 'dna1', 'dna4', 'dna5', 'dna6', 'dna8', 'dna_3', 'dna_5']
113-
>>> codext.examples("barbie", 5)
114-
['barbie-1', 'barbie1', 'barbie4', 'barbie_2', 'barbie_4']
115-
```
116-
117-
-----
118-
11988
## Add a custom map encoding
12089

12190
New codecs using encoding maps can be added easily using the new function `add_map`.
@@ -125,21 +94,22 @@ New codecs using encoding maps can be added easily using the new function `add_m
12594
>>> help(codext.add)
12695
Help on function add_map in module codext.__common__:
12796

128-
add_map(ename, encmap, repl_char='?', sep='', ignore_case=False, no_error=False, binary=False, **kwargs)
97+
add_map(ename, encmap, repl_char='?', sep='', ignore_case=None, no_error=False, intype=None, outype=None, **kwargs)
12998
This adds a new mapping codec (that is, declarable with a simple character mapping dictionary) to the codecs module
13099
dynamically setting its encode and/or decode functions, eventually dynamically naming the encoding with a pattern
131100
and with file handling (if text is True).
132101

133102
:param ename: encoding name
134103
:param encmap: characters encoding map ; can be a dictionary of encoding maps (for use with the first capture
135-
group of the regex pattern)
104+
group of the regex pattern) or a function building the encoding map
136105
:param repl_char: replacement char (used when errors handling is set to "replace")
137106
:param sep: string of possible character separators (hence, only single-char separators are considered) ;
138107
- while encoding, the first separator is used
139108
- while decoding, separators can be mixed in the input text
140-
:param ignore_case: ignore text case
109+
:param ignore_case: ignore text case while encoding and/or decoding
141110
:param no_error: this encoding triggers no error (hence, always in "leave" errors handling)
142-
:param binary: encoding applies to the binary string of the input text
111+
:param intype: specify the input type for pre-transforming the input text
112+
:param outype: specify the output type for post-transforming the output text
143113
:param pattern: pattern for dynamically naming the encoding
144114
:param text: specify whether the codec is a text encoding
145115
:param add_to_codecs: also add the search function to the native registry
@@ -175,7 +145,7 @@ ENCMAP = [
175145
{'00': "D", '01': "C", '10': "B", '11': "A"},
176146
]
177147

178-
codext.add("mydyncodec", ENCMAP, "#", ignore_case=True, binary=True, pattern=r"mydyn-(\d+)$")
148+
codext.add("mydyncodec", ENCMAP, "#", ignore_case=True, intype="bin", pattern=r"mydyn-(\d+)$")
179149
```
180150

181151
In this second example, we can see that:
@@ -185,6 +155,47 @@ In this second example, we can see that:
185155
- The case is ignored ; decoding either "`abcd`" or "`ABCD`" will succeed.
186156
- The binary mode is enabled, meaning that the input text is converted to a binary string for encoding, while it is converted from binary to text when decoding.
187157

158+
!!! warning "Input/Output types"
159+
160+
By default, when `intype` is defined, `outype` takes the same value. So, if the new encoding uses a pre-conversion to bits (`intype="bin"`) but maps bits to characters (therefore binary conversion to text is not needed), `outype` shall then be set to "`str`" (or if it maps bits to ordinals, use `outype="ord"`).
161+
162+
-----
163+
164+
## List new encodings
165+
166+
New codecs added with `codext` can be listed with the related function. Note that it only lists new codecs, not native ones.
167+
168+
```python
169+
>>> codext.list()
170+
['ascii85', 'base85', 'base100', 'base122', 'base2', ..., 'scytale', 'shift', 'xor', 'braille', 'leet', 'morse', 'navajo', 'radio', 'southpark', ..., 'markdown', 'url', 'resistor', 'sms', 'whitespace', 'whitespace-after-before']
171+
```
172+
173+
-----
174+
175+
## Search for encodings
176+
177+
Natively, `codecs` provides a `lookup` function that allows to get the `CodecInfo` object for the desired encoding. This performs a lookup in the registry based on an exact match. Sometimes, it can be useful to search for available encodings based on a regular expression. Therefore, a `search` function is added by `codext` to allow to get a list of encoding names matching the input regex.
178+
179+
```python
180+
>>> codext.search("baudot")
181+
['baudot', 'baudot_spaced', 'baudot_tape']
182+
>>> codext.search("al")
183+
['capitalize', 'octal', 'octal_spaced', 'ordinal', 'ordinal_spaced', 'radio']
184+
>>> codext.search("white")
185+
['whitespace', 'whitespace_after_before']
186+
```
187+
188+
Also, `codext` provides an `examples` function to get some examples of valid encoding names. This is especially useful when it concerns dynamicly named encodings (e.g. `rot`, `shift` or `dna`).
189+
190+
```python
191+
>>> codext.examples("rot")
192+
['rot-14', 'rot-24', 'rot-7', 'rot18', 'rot3', 'rot4', 'rot6', 'rot_1', 'rot_12', 'rot_2']
193+
>>> codext.examples("dna")
194+
['dna-1', 'dna-2', 'dna-5', 'dna1', 'dna4', 'dna5', 'dna6', 'dna8', 'dna_3', 'dna_5']
195+
>>> codext.examples("barbie", 5)
196+
['barbie-1', 'barbie1', 'barbie4', 'barbie_2', 'barbie_4']
197+
```
198+
188199
-----
189200

190201
## Remove a custom encoding

0 commit comments

Comments
 (0)