Skip to content

Commit af4786c

Browse files
dhondtadhondta
committed
Improved CLI script
1 parent dd595c7 commit af4786c

4 files changed

Lines changed: 84 additions & 59 deletions

File tree

‎README.md‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -22,31 +22,31 @@ $ pip install codext
2222
## Usage (CLI tool)
2323

2424
```sh
25-
$ codext dna-1 -i test.txt
25+
$ codext -i test.txt encode dna-1
2626
GTGAGCGGGTATGTGA
27-
$ echo -en "test" | codext morse
27+
$ echo -en "test" | codext encode morse
2828
- . ... -
2929
```
3030

3131
Python 3 (includes Ascii85, Base85, Base100 and braille):
3232

3333
```sh
34-
$ echo -en "test" | codext braille
34+
$ echo -en "test" | codext encode braille
3535
⠞⠑⠎⠞
36-
$ echo -en "test" | codext base100
36+
$ echo -en "test" | codext encode base100
3737
👫👜👪👫
3838
```
3939

4040
Using codecs chaining:
4141

4242
```sh
43-
$ echo -en "Test string" | codext reverse
43+
$ echo -en "Test string" | codext encode reverse
4444
gnirts tseT
45-
$ echo -en "Test string" | codext reverse morse
45+
$ echo -en "Test string" | codext encode reverse morse
4646
--. -. .. .-. - ... / - ... . -
47-
$ echo -en "Test string" | codext reverse morse dna-2
47+
$ echo -en "Test string" | codext encode reverse morse dna-2
4848
AGTCAGTCAGTGAGAAAGTCAGTGAGAAAGTGAGTGAGAAAGTGAGTCAGTGAGAAAGTCAGAAAGTGAGTGAGTGAGAAAGTTAGAAAGTCAGAAAGTGAGTGAGTGAGAAAGTGAGAAAGTC
49-
$ echo -en "Test string" | codext reverse morse dna-2 octal
49+
$ echo -en "Test string" | codext encode reverse morse dna-2 octal
5050
101107124103101107124103101107124107101107101101101107124103101107124107101107101101101107124107101107124107101107101101101107124107101107124103101107124107101107101101101107124103101107101101101107124107101107124107101107124107101107101101101107124124101107101101101107124103101107101101101107124107101107124107101107124107101107101101101107124107101107101101101107124103
5151
$ echo -en "AGTCAGTCAGTGAGAAAGTCAGTGAGAAAGTGAGTGAGAAAGTGAGTCAGTGAGAAAGTCAGAAAGTGAGTGAGTGAGAAAGTTAGAAAGTCAGAAAGTGAGTGAGTGAGAAAGTGAGAAAGTC" | codext -d dna-2 morse reverse
5252
test string

‎codext/__common__.py‎

Lines changed: 17 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -27,9 +27,9 @@
2727
maketrans = str.maketrans
2828

2929

30-
__all__ = ["add", "add_map", "b", "clear", "codecs", "decode", "encode", "ensure_str", "examples",
31-
"generate_strings_from_regex", "get_alphabet_from_mask", "guess", "handle_error", "list_encodings", "lookup",
32-
"maketrans", "re", "register", "remove", "reset", "s2i", "search", "BytesIO", "MASKS", "PY3"]
30+
__all__ = ["add", "add_map", "b", "clear", "codecs", "decode", "encode", "ensure_str", "examples", "guess",
31+
"generate_strings_from_regex", "get_alphabet_from_mask", "handle_error", "list_categories", "list_encodings",
32+
"lookup", "maketrans", "re", "register", "remove", "reset", "s2i", "search", "BytesIO", "MASKS", "PY3"]
3333
CODECS_REGISTRY = None
3434
MASKS = {
3535
'a': printable,
@@ -436,14 +436,20 @@ def examples(encoding, number=10):
436436
codecs.examples = examples
437437

438438

439-
def list_encodings(*categories):
440-
""" Get a list of all codecs. """
441-
# first, determine the list of valid categories
442-
valid_categories = ["native"]
439+
def list_categories():
440+
""" Get a list of all codec categories. """
441+
c = ["native"]
443442
root = os.path.dirname(__file__)
444443
for d in os.listdir(root):
445444
if os.path.isdir(os.path.join(root, d)) and not d.startswith("__"):
446-
valid_categories.append(d.rstrip("s"))
445+
c.append(d.rstrip("s"))
446+
return c
447+
448+
449+
def list_encodings(*categories):
450+
""" Get a list of all codecs. """
451+
# first, determine the list of valid categories
452+
valid_categories = list_categories()
447453
# then, if "non-native" is in the input list, extend the list with the whole categories but "native"
448454
categories = list(categories)
449455
for c in categories[:]:
@@ -856,6 +862,9 @@ def __score(input, codec):
856862
new_input = decode(input, encoding)
857863
except:
858864
continue
865+
# ignore encodings that give an output identical to the input (identity transformation)
866+
if b(input) == b(new_input):
867+
continue
859868
score = 1.0
860869
#FIXME: score the input/new_input to establish priorities of the depth-first search
861870
#This could rely on a series of weighted features:

‎codext/__init__.py‎

Lines changed: 38 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -41,31 +41,43 @@ def main():
4141
"\nThis tool allows to encode/decode input strings/files with an extended set of codecs.\n\n" \
4242
.format(__version__, __author__, __email__, __copyright__, __license__, __source__)
4343
examples = "usage examples:\n- " + "\n- ".join([
44-
"codext --search bitcoin",
45-
"codext -d base32 -i file.b32",
46-
"codext morse < to_be_encoded.txt",
47-
"echo \"test\" | codext base100",
48-
"echo -en \"test\" | codext braille -o test.braille",
49-
"codext base64 < to_be_encoded.txt > text.b64",
50-
"echo -en \"test\" | codext base64 | codext base32",
51-
"echo -en \"mrdvm6teie6t2cq=\" | codext upper | codext -d base32 | codext -d base64",
52-
"echo -en \"test\" | codext upper reverse base32 | codext -d base32 reverse lower",
53-
"echo -en \"test\" | codext upper reverse base32 base64 morse",
44+
"codext search bitcoin",
45+
"codext decode base32 -i file.b32",
46+
"codext encode morse < to_be_encoded.txt",
47+
"echo \"test\" | codext encode base100",
48+
"echo -en \"test\" | codext encode braille -o test.braille",
49+
"codext encode base64 < to_be_encoded.txt > text.b64",
50+
"echo -en \"test\" | codext encode base64 | codext encode base32",
51+
"echo -en \"mrdvm6teie6t2cq=\" | codext encode upper | codext decode base32 | codext decode base64",
52+
"echo -en \"test\" | codext encode upper reverse base32 | codext decode base32 reverse lower",
53+
"echo -en \"test\" | codext encode upper reverse base32 base64 morse",
54+
"echo -en \"test\" | codext encode base64 gzip | codext guess",
55+
"echo -en \"test\" | codext encode base64 gzip | codext guess gzip",
5456
])
5557
parser = argparse.ArgumentParser(description=descr, epilog=examples, formatter_class=argparse.RawTextHelpFormatter)
56-
parser.add_argument("encoding", nargs="+", help="list of encodings to apply")
57-
parser.add_argument("-d", "--decode", action="store_true", help="set decode mode")
58-
parser.add_argument("-e", "--errors", default="strict", choices=["ignore", "leave", "replace", "strict"],
59-
help="error handling")
58+
sparsers = parser.add_subparsers(dest="command", help="command to be executed")
6059
parser.add_argument("-i", "--input-file", dest="infile", help="input file (if none, take stdin as input)")
6160
parser.add_argument("-o", "--output-file", dest="outfile", help="output file (if none, display result to stdout)")
6261
parser.add_argument("-s", "--strip-newlines", action="store_true", dest="strip", help="strip newlines from input")
63-
parser.add_argument("--search", action="store_true", help="search for encoding names")
62+
encode = sparsers.add_parser("encode", help="encode input using the specified codecs")
63+
encode.add_argument("encoding", nargs="+", help="list of encodings to apply")
64+
encode.add_argument("-e", "--errors", default="strict", choices=["ignore", "leave", "replace", "strict"],
65+
help="error handling")
66+
decode = sparsers.add_parser("decode", help="decode input using the specified codecs")
67+
decode.add_argument("encoding", nargs="+", help="list of encodings to apply")
68+
decode.add_argument("-e", "--errors", default="strict", choices=["ignore", "leave", "replace", "strict"],
69+
help="error handling")
70+
guess = sparsers.add_parser("guess", help="try guessing the decoding codecs")
71+
guess.add_argument("encoding", nargs="*", help="list of known encodings to apply")
72+
guess.add_argument("-c", "--category", choices=list_categories(), nargs="*", help="codec categories to search in")
73+
guess.add_argument("-d", "--depth", default=3, type=int, help="maximum codec search depth")
74+
search = sparsers.add_parser("search", help="search for codecs")
75+
search.add_argument("pattern", nargs="+", help="encoding pattern to search")
6476
args = parser.parse_args()
6577
# if a search pattern is given, only handle it
66-
if args.search:
78+
if args.command == "search":
6779
results = []
68-
for enc in args.encoding:
80+
for enc in args.pattern:
6981
results.extend(codecs.search(enc))
7082
print(", ".join(results) or "No encoding found")
7183
return
@@ -79,18 +91,18 @@ def main():
7991
c += line
8092
if args.strip:
8193
c = re.sub(r"\r?\n", "", c)
82-
# encode or decode
83-
for encoding in args.encoding:
84-
c = getattr(codecs, ["encode", "decode"][args.decode])(c, encoding, args.errors)
94+
if args.command in ["decode", "encode"]:
95+
# encode or decode
96+
for encoding in args.encoding:
97+
c = getattr(codecs, ["encode", "decode"][args.command == "decode"])(c, encoding, args.errors)
98+
elif args.command == "guess":
99+
c, e = codecs.guess(c, max_depth=args.depth, codec_categories=args.category, found=args.encoding)
100+
if e:
101+
print("Encodings: %s" % ", ".join(e))
85102
# handle output file or stdout
86103
if args.outfile:
87104
with open(args.outfile, 'wb') as f:
88105
f.write(c)
89106
else:
90-
if PY3:
91-
try:
92-
c = c.decode("utf-8")
93-
except:
94-
c = c.decode("latin-1")
95-
print(c, end="")
107+
print(ensure_str(c), end="")
96108

‎docs/cli.md‎

Lines changed: 21 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -7,12 +7,9 @@
77
The help message describes everything to know:
88

99
```sh
10-
$ codext --help
11-
usage: codext [-h] [-d] [-e {ignore,leave,replace,strict}] [-i INFILE]
12-
[-o OUTFILE] [-s]
13-
encoding [encoding ...]
10+
usage: codext [-h] [-i INFILE] [-o OUTFILE] [-s] {encode,decode,guess,search} ...
1411

15-
Codecs Extension (CodExt) 1.4.0
12+
Codecs Extension (CodExt) 1.6.2
1613

1714
Author : Alexandre D'Hondt (alexandre.dhondt@gmail.com)
1815
Copyright: © 2019-2020 A. D'Hondt
@@ -22,28 +19,34 @@ Source : https://github.com/dhondta/python-codext
2219
This tool allows to encode/decode input strings/files with an extended set of codecs.
2320

2421
positional arguments:
25-
encoding list of encodings to apply
22+
{encode,decode,guess,search}
23+
command to be executed
24+
encode encode input using the specified codecs
25+
decode decode input using the specified codecs
26+
guess try guessing the decoding codecs
27+
search search for codecs
2628

2729
optional arguments:
2830
-h, --help show this help message and exit
29-
-d, --decode set decode mode
30-
-e {ignore,leave,replace,strict}, --errors {ignore,leave,replace,strict}
31-
error handling
3231
-i INFILE, --input-file INFILE
3332
input file (if none, take stdin as input)
3433
-o OUTFILE, --output-file OUTFILE
3534
output file (if none, display result to stdout)
3635
-s, --strip-newlines strip newlines from input
3736

3837
usage examples:
39-
- codext -d base32 -i file.b32
40-
- codext morse < to_be_encoded.txt
41-
- echo "test" | codext base100
42-
- echo -en "test" | codext braille -o test.braille
43-
- codext base64 < to_be_encoded.txt > text.b64
44-
- echo -en "test" | codext base64 | codext base32
45-
- echo -en "mrdvm6teie6t2cq=" | codext upper | codext -d base32 | codext -d base64
46-
- echo -en "test" | codext upper reverse base32 base64 morse
38+
- codext search bitcoin
39+
- codext decode base32 -i file.b32
40+
- codext encode morse < to_be_encoded.txt
41+
- echo "test" | codext encode base100
42+
- echo -en "test" | codext encode braille -o test.braille
43+
- codext encode base64 < to_be_encoded.txt > text.b64
44+
- echo -en "test" | codext encode base64 | codext encode base32
45+
- echo -en "mrdvm6teie6t2cq=" | codext encode upper | codext decode base32 | codext decode base64
46+
- echo -en "test" | codext encode upper reverse base32 | codext decode base32 reverse lower
47+
- echo -en "test" | codext encode upper reverse base32 base64 morse
48+
- echo -en "test" | codext encode base64 gzip | codext guess
49+
- echo -en "test" | codext encode base64 gzip | codext guess gzip
4750
```
4851

4952
!!! note "Input/output"
@@ -55,3 +58,4 @@ usage examples:
5558
!!! note "Encodings chaining"
5659

5760
Encodings can be chained as shown in the last examples of the help message. This can be practical for quickly manipulating data.
61+

0 commit comments

Comments
 (0)