Mercurial > p > roundup > code
annotate roundup/msgfmt.py @ 5525:bb7865241f8a
Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
The roundup-admin export and import commands are used for migrating
between different database backends. It is desirable that they should
be usable also for migrations between Python 2 and Python 3, and in
some cases (e.g. with the anydbm backend) this may be required.
To be usable for such migrations, the format of the generated CSV
files needs to be stable, meaning the same as currently used with
Python 2. The export process uses repr() to produce the fields in the
CSV files and eval() to convert them back to Python data structures.
repr() of strings with non-ASCII characters produces different results
for Python 2 and Python 3.
This patch adds repr_export and eval_import functions to
roundup/anypy/strings.py which provide the required operations that
are just repr() and eval() in Python 2, but are more complicated in
Python 3 to use data representations compatible with Python 2. These
functions are then used in the required places for export and import.
repr() and eval() are also used in storing the dict of changed values
in the journal for the RDBMS backends. It is similarly desirable that
the database be compatible between Python 2 and Python 3, so that
export and import do not need to be used for a migration between
Python versions for non-anydbm back ends. Thus, this patch changes
rdbms_common.py in the places involved in storing journals in the
database, not just in those involved in import/export.
Given this patch, import/export with non-ASCII characters appear based
on some limited testing to work across Python versions, and an
instance using the sqlite backend appears to be compatible between
Python versions without needing import/export, *if* the sessions/otks
databases (which use anydbm) are deleted when changing Python version.
| author | Joseph Myers <jsm@polyomino.org.uk> |
|---|---|
| date | Sun, 02 Sep 2018 23:48:04 +0000 |
| parents | f2fade4552c5 |
| children | 4d2e1fa03f0f |
| rev | line source |
|---|---|
|
2784
40643f8b41a0
catalog compilation utility for message translations
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff
changeset
|
1 #! /usr/bin/env python |
|
40643f8b41a0
catalog compilation utility for message translations
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff
changeset
|
2 # -*- coding: iso-8859-1 -*- |
|
5450
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
3 # Written by Martin v. Loewis <loewis@informatik.hu-berlin.de> |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
4 # |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
5 # Changed by Christian 'Tiran' Heimes <tiran@cheimes.de> for the placeless |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
6 # translation service (PTS) of Zope |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
7 # |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
8 # Fixed some bugs and updated to support msgctxt |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
9 # by Hanno Schlichting <hanno@hannosch.eu> |
|
2784
40643f8b41a0
catalog compilation utility for message translations
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff
changeset
|
10 |
|
40643f8b41a0
catalog compilation utility for message translations
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff
changeset
|
11 """Generate binary message catalog from textual translation description. |
|
40643f8b41a0
catalog compilation utility for message translations
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff
changeset
|
12 |
|
40643f8b41a0
catalog compilation utility for message translations
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff
changeset
|
13 This program converts a textual Uniforum-style message catalog (.po file) into |
|
5450
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
14 a binary GNU catalog (.mo file). This is essentially the same function as the |
|
2784
40643f8b41a0
catalog compilation utility for message translations
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff
changeset
|
15 GNU msgfmt program, however, it is a simpler implementation. |
|
40643f8b41a0
catalog compilation utility for message translations
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff
changeset
|
16 |
|
5450
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
17 This file was taken from Python-2.3.2/Tools/i18n and altered in several ways. |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
18 Now you can simply use it from another python module: |
|
2784
40643f8b41a0
catalog compilation utility for message translations
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff
changeset
|
19 |
|
5450
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
20 from msgfmt import Msgfmt |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
21 mo = Msgfmt(po).get() |
|
2784
40643f8b41a0
catalog compilation utility for message translations
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff
changeset
|
22 |
|
5450
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
23 where po is path to a po file as string, an opened po file ready for reading or |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
24 a list of strings (readlines of a po file) and mo is the compiled mo file as |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
25 binary string. |
|
2784
40643f8b41a0
catalog compilation utility for message translations
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff
changeset
|
26 |
|
5450
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
27 Exceptions: |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
28 |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
29 * IOError if the file couldn't be read |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
30 |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
31 * msgfmt.PoSyntaxError if the po file has syntax errors |
|
2784
40643f8b41a0
catalog compilation utility for message translations
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff
changeset
|
32 """ |
|
40643f8b41a0
catalog compilation utility for message translations
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff
changeset
|
33 |
|
5450
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
34 import array |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
35 from ast import literal_eval |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
36 import codecs |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
37 from email.parser import HeaderParser |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
38 import struct |
|
2784
40643f8b41a0
catalog compilation utility for message translations
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff
changeset
|
39 import sys |
|
40643f8b41a0
catalog compilation utility for message translations
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff
changeset
|
40 |
|
5450
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
41 PY3 = sys.version_info[0] == 3 |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
42 if PY3: |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
43 def header_charset(s): |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
44 p = HeaderParser() |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
45 return p.parsestr(s).get_content_charset() |
|
2784
40643f8b41a0
catalog compilation utility for message translations
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff
changeset
|
46 |
|
5450
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
47 import io |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
48 BytesIO = io.BytesIO |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
49 FILE_TYPE = io.IOBase |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
50 else: |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
51 def header_charset(s): |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
52 p = HeaderParser() |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
53 return p.parsestr(s.encode('utf-8', 'ignore')).get_content_charset() |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
54 |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
55 from cStringIO import StringIO as BytesIO |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
56 FILE_TYPE = file |
|
2784
40643f8b41a0
catalog compilation utility for message translations
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff
changeset
|
57 |
|
40643f8b41a0
catalog compilation utility for message translations
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff
changeset
|
58 |
|
5450
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
59 class PoSyntaxError(Exception): |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
60 """ Syntax error in a po file """ |
|
2784
40643f8b41a0
catalog compilation utility for message translations
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff
changeset
|
61 |
|
5450
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
62 def __init__(self, msg): |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
63 self.msg = msg |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
64 |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
65 def __str__(self): |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
66 return 'Po file syntax error: %s' % self.msg |
|
2784
40643f8b41a0
catalog compilation utility for message translations
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff
changeset
|
67 |
|
40643f8b41a0
catalog compilation utility for message translations
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff
changeset
|
68 |
|
5450
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
69 class Msgfmt: |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
70 |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
71 def __init__(self, po, name='unknown'): |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
72 self.po = po |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
73 self.name = name |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
74 self.messages = {} |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
75 self.openfile = False |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
76 # Start off assuming latin-1, so everything decodes without failure, |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
77 # until we know the exact encoding |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
78 self.encoding = 'latin-1' |
|
2784
40643f8b41a0
catalog compilation utility for message translations
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff
changeset
|
79 |
|
5450
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
80 def readPoData(self): |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
81 """ read po data from self.po and return an iterator """ |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
82 output = [] |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
83 if isinstance(self.po, str): |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
84 output = open(self.po, 'rb') |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
85 elif isinstance(self.po, FILE_TYPE): |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
86 self.po.seek(0) |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
87 self.openfile = True |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
88 output = self.po |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
89 elif isinstance(self.po, list): |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
90 output = self.po |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
91 if not output: |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
92 raise ValueError("self.po is invalid! %s" % type(self.po)) |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
93 if isinstance(output, FILE_TYPE): |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
94 # remove BOM from the start of the parsed input |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
95 first = output.readline() |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
96 if len(first) == 0: |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
97 return output.readlines() |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
98 if first.startswith(codecs.BOM_UTF8): |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
99 first = first.lstrip(codecs.BOM_UTF8) |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
100 return [first] + output.readlines() |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
101 return output |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
102 |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
103 def add(self, context, id, string, fuzzy): |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
104 "Add a non-empty and non-fuzzy translation to the dictionary." |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
105 if string and not fuzzy: |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
106 # The context is put before the id and separated by a EOT char. |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
107 if context: |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
108 id = context + u'\x04' + id |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
109 if not id: |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
110 # See whether there is an encoding declaration |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
111 charset = header_charset(string) |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
112 if charset: |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
113 # decode header in proper encoding |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
114 string = string.encode(self.encoding).decode(charset) |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
115 if not PY3: |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
116 # undo damage done by literal_eval in Python 2.x |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
117 string = string.encode(self.encoding).decode(charset) |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
118 self.encoding = charset |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
119 self.messages[id] = string |
|
5237
c643d693d91e
Iniitalize two variables to None.
John Rouillard <rouilj@ieee.org>
parents:
3563
diff
changeset
|
120 |
|
5450
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
121 def generate(self): |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
122 "Return the generated output." |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
123 # the keys are sorted in the .mo file |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
124 keys = sorted(self.messages.keys()) |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
125 offsets = [] |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
126 ids = strs = b'' |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
127 for id in keys: |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
128 msg = self.messages[id].encode(self.encoding) |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
129 id = id.encode(self.encoding) |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
130 # For each string, we need size and file offset. Each string is |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
131 # NUL terminated; the NUL does not count into the size. |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
132 offsets.append((len(ids), len(id), len(strs), |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
133 len(msg))) |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
134 ids += id + b'\0' |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
135 strs += msg + b'\0' |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
136 output = b'' |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
137 # The header is 7 32-bit unsigned integers. We don't use hash tables, |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
138 # so the keys start right after the index tables. |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
139 keystart = 7 * 4 + 16 * len(keys) |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
140 # and the values start after the keys |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
141 valuestart = keystart + len(ids) |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
142 koffsets = [] |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
143 voffsets = [] |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
144 # The string table first has the list of keys, then the list of values. |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
145 # Each entry has first the size of the string, then the file offset. |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
146 for o1, l1, o2, l2 in offsets: |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
147 koffsets += [l1, o1 + keystart] |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
148 voffsets += [l2, o2 + valuestart] |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
149 offsets = koffsets + voffsets |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
150 # Even though we don't use a hashtable, we still set its offset to be |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
151 # binary compatible with the gnu gettext format produced by: |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
152 # msgfmt file.po --no-hash |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
153 output = struct.pack("Iiiiiii", |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
154 0x950412de, # Magic |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
155 0, # Version |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
156 len(keys), # # of entries |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
157 7 * 4, # start of key index |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
158 7 * 4 + len(keys) * 8, # start of value index |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
159 0, keystart) # size and offset of hash table |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
160 if PY3: |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
161 output += array.array("i", offsets).tobytes() |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
162 else: |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
163 output += array.array("i", offsets).tostring() |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
164 output += ids |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
165 output += strs |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
166 return output |
|
2784
40643f8b41a0
catalog compilation utility for message translations
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff
changeset
|
167 |
|
5450
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
168 def get(self): |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
169 """ """ |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
170 self.read() |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
171 # Compute output |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
172 return self.generate() |
|
2784
40643f8b41a0
catalog compilation utility for message translations
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff
changeset
|
173 |
|
5450
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
174 def read(self, header_only=False): |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
175 """ """ |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
176 ID = 1 |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
177 STR = 2 |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
178 CTXT = 3 |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
179 |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
180 section = None |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
181 fuzzy = 0 |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
182 msgid = msgstr = msgctxt = u'' |
|
2784
40643f8b41a0
catalog compilation utility for message translations
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff
changeset
|
183 |
|
5450
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
184 # Parse the catalog |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
185 lno = 0 |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
186 for l in self.readPoData(): |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
187 l = l.decode(self.encoding) |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
188 lno += 1 |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
189 # If we get a comment line after a msgstr or a line starting with |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
190 # msgid or msgctxt, this is a new entry |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
191 if section == STR and (l[0] == '#' or (l[0] == 'm' and |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
192 (l.startswith('msgctxt') or l.startswith('msgid')))): |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
193 self.add(msgctxt, msgid, msgstr, fuzzy) |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
194 section = None |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
195 fuzzy = 0 |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
196 # If we only want the header we stop after the first message |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
197 if header_only: |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
198 break |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
199 # Record a fuzzy mark |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
200 if l[:2] == '#,' and 'fuzzy' in l: |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
201 fuzzy = 1 |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
202 # Skip comments |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
203 if l[0] == '#': |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
204 continue |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
205 # Now we are in a msgctxt section |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
206 if l.startswith('msgctxt'): |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
207 section = CTXT |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
208 l = l[7:] |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
209 msgctxt = u'' |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
210 # Now we are in a msgid section, output previous section |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
211 elif (l.startswith('msgid') and |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
212 not l.startswith('msgid_plural')): |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
213 if section == STR: |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
214 self.add(msgid, msgstr, fuzzy) |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
215 section = ID |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
216 l = l[5:] |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
217 msgid = msgstr = u'' |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
218 is_plural = False |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
219 # This is a message with plural forms |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
220 elif l.startswith('msgid_plural'): |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
221 if section != ID: |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
222 raise PoSyntaxError( |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
223 'msgid_plural not preceeded by ' |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
224 'msgid on line %d of po file %s' % |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
225 (lno, repr(self.name))) |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
226 l = l[12:] |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
227 msgid += u'\0' # separator of singular and plural |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
228 is_plural = True |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
229 # Now we are in a msgstr section |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
230 elif l.startswith('msgstr'): |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
231 section = STR |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
232 if l.startswith('msgstr['): |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
233 if not is_plural: |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
234 raise PoSyntaxError( |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
235 'plural without msgid_plural ' |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
236 'on line %d of po file %s' % |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
237 (lno, repr(self.name))) |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
238 l = l.split(']', 1)[1] |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
239 if msgstr: |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
240 # Separator of the various plural forms |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
241 msgstr += u'\0' |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
242 else: |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
243 if is_plural: |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
244 raise PoSyntaxError( |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
245 'indexed msgstr required for ' |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
246 'plural on line %d of po file %s' % |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
247 (lno, repr(self.name))) |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
248 l = l[6:] |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
249 # Skip empty lines |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
250 l = l.strip() |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
251 if not l: |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
252 continue |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
253 # TODO: Does this always follow Python escape semantics? |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
254 try: |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
255 l = literal_eval(l) |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
256 except Exception as msg: |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
257 raise PoSyntaxError( |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
258 '%s (line %d of po file %s): \n%s' % |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
259 (msg, lno, repr(self.name), l)) |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
260 if isinstance(l, bytes): |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
261 l = l.decode(self.encoding) |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
262 if section == CTXT: |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
263 msgctxt += l |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
264 elif section == ID: |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
265 msgid += l |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
266 elif section == STR: |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
267 msgstr += l |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
268 else: |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
269 raise PoSyntaxError( |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
270 'error on line %d of po file %s' % |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
271 (lno, repr(self.name))) |
|
2784
40643f8b41a0
catalog compilation utility for message translations
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff
changeset
|
272 |
|
5450
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
273 # Add last entry |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
274 if section == STR: |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
275 self.add(msgctxt, msgid, msgstr, fuzzy) |
|
2784
40643f8b41a0
catalog compilation utility for message translations
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff
changeset
|
276 |
|
5450
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
277 if self.openfile: |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
278 self.po.close() |
|
2784
40643f8b41a0
catalog compilation utility for message translations
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff
changeset
|
279 |
|
5450
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
280 def getAsFile(self): |
|
f2fade4552c5
replaced msgfmt.py with latest version supporting Python 3
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5395
diff
changeset
|
281 return BytesIO(self.get()) |
