Mercurial > p > roundup > code
annotate scripts/import_sf.py @ 5414:3fa026621f69
Python 3 preparation: comparisons.
Python 3 no longer has the cmp function, or cmp= arguments to sorting
functions / methods (key= must be used instead), and requires rich
comparison methods such as __lt__ to be defined instead of using
__cmp__. All of the comparison mechanisms supported in Python 3 are
also supported in Python 2.
This patch makes the corresponding changes in Roundup to use key
functions and rich comparison methods. In the case of the
JournalPassword and Permission classes, only __eq__ and __ne__ are
defined as I don't see ordered comparisons as useful there (and for
Permission, the old __cmp__ function didn't try to provide a valid
ordering). In the case of the Date class, I kept the __cmp__ method
and implemented the others in terms of it, to avoid excess
repetitiveness in duplicating implementation code for all six rich
comparison methods.
In roundup/admin.py, help_commands_html used operator.attrgetter to
produce the second argument of sorted() - which would be reasonable
for a key function, but the second argument is the cmp function in
Python 2, not a key function (and the key function must be a named
argument not a positional argument in Python 3). That function
appears to be completely unused, so I expect that code never worked.
This patch adds the missing key= to that sorted() call, but it would
also be reasonable to remove the unused function completely instead.
| author | Joseph Myers <jsm@polyomino.org.uk> |
|---|---|
| date | Wed, 25 Jul 2018 00:39:37 +0000 |
| parents | f004824563a3 |
| children | 56c9bcdea47f |
| rev | line source |
|---|---|
|
4089
eddb82d0964c
Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents:
3904
diff
changeset
|
1 """ Import tracker data from Sourceforge.NET |
|
3536
00a7983b4666
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
3534
diff
changeset
|
2 |
|
00a7983b4666
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
3534
diff
changeset
|
3 This script needs four steps to work: |
|
00a7983b4666
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
3534
diff
changeset
|
4 |
|
00a7983b4666
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
3534
diff
changeset
|
5 1. Export the project XML data using the admin web interface at sf.net |
|
00a7983b4666
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
3534
diff
changeset
|
6 2. Run the file fetching (these are not included in the XML): |
|
00a7983b4666
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
3534
diff
changeset
|
7 |
|
00a7983b4666
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
3534
diff
changeset
|
8 import_sf.py files <path to XML> <path to files dir> |
|
00a7983b4666
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
3534
diff
changeset
|
9 |
|
00a7983b4666
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
3534
diff
changeset
|
10 this will place all the downloaded files in the files dir by file id. |
|
00a7983b4666
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
3534
diff
changeset
|
11 3. Convert the sf.net XML to Roundup "export" format: |
|
00a7983b4666
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
3534
diff
changeset
|
12 |
|
00a7983b4666
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
3534
diff
changeset
|
13 import_sf.py import <tracker home> <path to XML> <path to files dir> |
|
00a7983b4666
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
3534
diff
changeset
|
14 |
|
00a7983b4666
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
3534
diff
changeset
|
15 this will generate a directory "/tmp/imported" which contains the |
|
00a7983b4666
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
3534
diff
changeset
|
16 data to be imported into a Roundup tracker. |
|
00a7983b4666
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
3534
diff
changeset
|
17 4. Import the data: |
|
00a7983b4666
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
3534
diff
changeset
|
18 |
|
00a7983b4666
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
3534
diff
changeset
|
19 roundup-admin -i <tracker home> import /tmp/imported |
|
00a7983b4666
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
3534
diff
changeset
|
20 |
|
00a7983b4666
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
3534
diff
changeset
|
21 And you're done! |
|
4089
eddb82d0964c
Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents:
3904
diff
changeset
|
22 """ |
|
3536
00a7983b4666
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
3534
diff
changeset
|
23 |
|
5407
f004824563a3
Python 3 preparation: update httplib imports.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5402
diff
changeset
|
24 import sys, os, csv, time, mimetypes |
|
3542
a2fc38fd0142
fixes and speedup
Richard Jones <richard@users.sourceforge.net>
parents:
3541
diff
changeset
|
25 |
|
a2fc38fd0142
fixes and speedup
Richard Jones <richard@users.sourceforge.net>
parents:
3541
diff
changeset
|
26 try: |
|
a2fc38fd0142
fixes and speedup
Richard Jones <richard@users.sourceforge.net>
parents:
3541
diff
changeset
|
27 import cElementTree as ElementTree |
|
a2fc38fd0142
fixes and speedup
Richard Jones <richard@users.sourceforge.net>
parents:
3541
diff
changeset
|
28 except ImportError: |
|
a2fc38fd0142
fixes and speedup
Richard Jones <richard@users.sourceforge.net>
parents:
3541
diff
changeset
|
29 from elementtree import ElementTree |
|
3534
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
30 |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
31 from roundup import instance, hyperdb, date, support, password |
|
5407
f004824563a3
Python 3 preparation: update httplib imports.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5402
diff
changeset
|
32 from roundup.anypy import http_, urllib_ |
|
3534
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
33 |
|
3538
7f7902af7864
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
3537
diff
changeset
|
34 today = date.Date('.') |
|
7f7902af7864
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
3537
diff
changeset
|
35 |
|
3537
d819ff1b3116
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
3536
diff
changeset
|
36 DL_URL = 'http://sourceforge.net/tracker/download.php?group_id=%(group_id)s&atid=%(atid)s&aid=%(aid)s' |
|
d819ff1b3116
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
3536
diff
changeset
|
37 |
|
3534
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
38 def get_url(aid): |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
39 """ so basically we have to jump through hoops, given an artifact id, to |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
40 figure what the URL should be to access that artifact, and hence any |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
41 attached files.""" |
|
3538
7f7902af7864
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
3537
diff
changeset
|
42 # first we hit this URL... |
|
5407
f004824563a3
Python 3 preparation: update httplib imports.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5402
diff
changeset
|
43 conn = http_.client.HTTPConnection("sourceforge.net") |
|
3534
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
44 conn.request("GET", "/support/tracker.php?aid=%s"%aid) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
45 response = conn.getresponse() |
|
3538
7f7902af7864
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
3537
diff
changeset
|
46 # which should respond with a redirect to the correct url which has the |
|
7f7902af7864
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
3537
diff
changeset
|
47 # magic "group_id" and "atid" values in it that we need |
|
3534
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
48 assert response.status == 302, 'response code was %s'%response.status |
|
3537
d819ff1b3116
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
3536
diff
changeset
|
49 location = response.getheader('location') |
|
5402
88dbacd11cd1
Python 3 preparation: update urllib / urllib2 / urlparse imports.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5381
diff
changeset
|
50 query = urllib_.urlparse(response.getheader('location'))[-2] |
|
3537
d819ff1b3116
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
3536
diff
changeset
|
51 info = dict([param.split('=') for param in query.split('&')]) |
| 3541 | 52 return DL_URL%info |
|
3534
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
53 |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
54 def fetch_files(xml_file, file_dir): |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
55 """ Fetch files referenced in the xml_file into the dir file_dir. """ |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
56 root = ElementTree.parse(xml_file).getroot() |
|
4089
eddb82d0964c
Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents:
3904
diff
changeset
|
57 to_fetch = set() |
|
eddb82d0964c
Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents:
3904
diff
changeset
|
58 deleted = set() |
|
3534
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
59 for artifact in root.find('artifacts'): |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
60 for field in artifact.findall('field'): |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
61 if field.get('name') == 'artifact_id': |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
62 aid = field.text |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
63 for field in artifact.findall('field'): |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
64 if field.get('name') != 'artifact_history': continue |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
65 for event in field.findall('history'): |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
66 d = {} |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
67 for field in event.findall('field'): |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
68 d[field.get('name')] = field.text |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
69 if d['field_name'] == 'File Added': |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
70 fid = d['old_value'].split(':')[0] |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
71 to_fetch.add((aid, fid)) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
72 if d['field_name'] == 'File Deleted': |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
73 fid = d['old_value'].split(':')[0] |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
74 deleted.add((aid, fid)) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
75 to_fetch = to_fetch - deleted |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
76 |
|
4089
eddb82d0964c
Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents:
3904
diff
changeset
|
77 got = set(os.listdir(file_dir)) |
|
3534
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
78 to_fetch = to_fetch - got |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
79 |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
80 # load cached urls (sigh) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
81 urls = {} |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
82 if os.path.exists(os.path.join(file_dir, 'urls.txt')): |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
83 for line in open(os.path.join(file_dir, 'urls.txt')): |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
84 aid, url = line.strip().split() |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
85 urls[aid] = url |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
86 |
|
3544
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3543
diff
changeset
|
87 for aid, fid in support.Progress('Fetching files', list(to_fetch)): |
|
3534
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
88 if fid in got: continue |
|
5381
0942fe89e82e
Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents:
4687
diff
changeset
|
89 if aid not in urls: |
|
3534
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
90 urls[aid] = get_url(aid) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
91 f = open(os.path.join(file_dir, 'urls.txt'), 'a') |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
92 f.write('%s %s\n'%(aid, urls[aid])) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
93 f.close() |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
94 url = urls[aid] + '&file_id=' + fid |
|
5402
88dbacd11cd1
Python 3 preparation: update urllib / urllib2 / urlparse imports.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5381
diff
changeset
|
95 f = urllib_.urlopen(url) |
|
3534
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
96 data = f.read() |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
97 n = open(os.path.join(file_dir, fid), 'w') |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
98 n.write(data) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
99 f.close() |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
100 n.close() |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
101 |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
102 def import_xml(tracker_home, xml_file, file_dir): |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
103 """ Generate Roundup tracker import files based on the tracker schema, |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
104 sf.net xml export and downloaded files from sf.net. """ |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
105 tracker = instance.open(tracker_home) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
106 db = tracker.open('admin') |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
107 |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
108 resolved = db.status.lookup('resolved') |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
109 unread = db.status.lookup('unread') |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
110 chatting = db.status.lookup('unread') |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
111 critical = db.priority.lookup('critical') |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
112 urgent = db.priority.lookup('urgent') |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
113 bug = db.priority.lookup('bug') |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
114 feature = db.priority.lookup('feature') |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
115 wish = db.priority.lookup('wish') |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
116 adminuid = db.user.lookup('admin') |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
117 anonuid = db.user.lookup('anonymous') |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
118 |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
119 root = ElementTree.parse(xml_file).getroot() |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
120 |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
121 def to_date(ts): |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
122 return date.Date(time.gmtime(float(ts))) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
123 |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
124 # parse out the XML |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
125 artifacts = [] |
|
4089
eddb82d0964c
Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents:
3904
diff
changeset
|
126 categories = set() |
|
eddb82d0964c
Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents:
3904
diff
changeset
|
127 users = set() |
|
eddb82d0964c
Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents:
3904
diff
changeset
|
128 add_files = set() |
|
eddb82d0964c
Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents:
3904
diff
changeset
|
129 remove_files = set() |
|
3534
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
130 for artifact in root.find('artifacts'): |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
131 d = {} |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
132 op = {} |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
133 artifacts.append(d) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
134 for field in artifact.findall('field'): |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
135 name = field.get('name') |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
136 if name == 'artifact_messages': |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
137 for message in field.findall('message'): |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
138 l = d.setdefault('messages', []) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
139 m = {} |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
140 l.append(m) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
141 for field in message.findall('field'): |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
142 name = field.get('name') |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
143 if name == 'adddate': |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
144 m[name] = to_date(field.text) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
145 else: |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
146 m[name] = field.text |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
147 if name == 'user_name': users.add(field.text) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
148 elif name == 'artifact_history': |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
149 for event in field.findall('history'): |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
150 l = d.setdefault('history', []) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
151 e = {} |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
152 l.append(e) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
153 for field in event.findall('field'): |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
154 name = field.get('name') |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
155 if name == 'entrydate': |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
156 e[name] = to_date(field.text) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
157 else: |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
158 e[name] = field.text |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
159 if name == 'mod_by': users.add(field.text) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
160 if e['field_name'] == 'File Added': |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
161 add_files.add(e['old_value'].split(':')[0]) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
162 elif e['field_name'] == 'File Deleted': |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
163 remove_files.add(e['old_value'].split(':')[0]) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
164 elif name == 'details': |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
165 op['body'] = field.text |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
166 elif name == 'submitted_by': |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
167 op['user_name'] = field.text |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
168 d[name] = field.text |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
169 users.add(field.text) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
170 elif name == 'open_date': |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
171 thedate = to_date(field.text) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
172 op['adddate'] = thedate |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
173 d[name] = thedate |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
174 else: |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
175 d[name] = field.text |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
176 |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
177 categories.add(d['category']) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
178 |
|
5381
0942fe89e82e
Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents:
4687
diff
changeset
|
179 if 'body' in op: |
|
3534
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
180 l = d.setdefault('messages', []) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
181 l.insert(0, op) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
182 |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
183 add_files -= remove_files |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
184 |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
185 # create users |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
186 userd = {'nobody': '2'} |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
187 users.remove('nobody') |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
188 data = [ |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
189 {'id': '1', 'username': 'admin', 'password': password.Password('admin'), |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
190 'roles': 'Admin', 'address': 'richard@python.org'}, |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
191 {'id': '2', 'username': 'anonymous', 'roles': 'Anonymous'}, |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
192 ] |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
193 for n, user in enumerate(list(users)): |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
194 userd[user] = n+3 |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
195 data.append({'id': str(n+3), 'username': user, 'roles': 'User', |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
196 'address': '%s@users.sourceforge.net'%user}) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
197 write_csv(db.user, data) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
198 users=userd |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
199 |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
200 # create categories |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
201 categoryd = {'None': None} |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
202 categories.remove('None') |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
203 data = [] |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
204 for n, category in enumerate(list(categories)): |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
205 categoryd[category] = n |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
206 data.append({'id': str(n), 'name': category}) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
207 write_csv(db.keyword, data) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
208 categories = categoryd |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
209 |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
210 # create issues |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
211 issue_data = [] |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
212 file_data = [] |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
213 message_data = [] |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
214 issue_journal = [] |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
215 message_id = 0 |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
216 for artifact in artifacts: |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
217 d = {} |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
218 d['id'] = artifact['artifact_id'] |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
219 d['title'] = artifact['summary'] |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
220 d['assignedto'] = users[artifact['assigned_to']] |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
221 if d['assignedto'] == '2': |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
222 d['assignedto'] = None |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
223 d['creation'] = artifact['open_date'] |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
224 activity = artifact['open_date'] |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
225 d['creator'] = users[artifact['submitted_by']] |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
226 actor = d['creator'] |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
227 if categories[artifact['category']]: |
|
3904
91008ec8f9a0
retire "topic" usage
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3579
diff
changeset
|
228 d['keyword'] = [categories[artifact['category']]] |
|
3534
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
229 issue_journal.append(( |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
230 d['id'], d['creation'].get_tuple(), d['creator'], "'create'", {} |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
231 )) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
232 |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
233 p = int(artifact['priority']) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
234 if artifact['artifact_type'] == 'Feature Requests': |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
235 if p > 3: |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
236 d['priority'] = feature |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
237 else: |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
238 d['priority'] = wish |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
239 else: |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
240 if p > 7: |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
241 d['priority'] = critical |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
242 elif p > 5: |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
243 d['priority'] = urgent |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
244 elif p > 3: |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
245 d['priority'] = bug |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
246 else: |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
247 d['priority'] = feature |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
248 |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
249 s = artifact['status'] |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
250 if s == 'Closed': |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
251 d['status'] = resolved |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
252 elif s == 'Deleted': |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
253 d['status'] = resolved |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
254 d['is retired'] = True |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
255 else: |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
256 d['status'] = unread |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
257 |
|
4089
eddb82d0964c
Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents:
3904
diff
changeset
|
258 nosy = set() |
|
3534
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
259 for message in artifact.get('messages', []): |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
260 authid = users[message['user_name']] |
|
3542
a2fc38fd0142
fixes and speedup
Richard Jones <richard@users.sourceforge.net>
parents:
3541
diff
changeset
|
261 if not message['body']: continue |
|
3534
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
262 body = convert_message(message['body'], message_id) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
263 if not body: continue |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
264 m = {'content': body, 'author': authid, |
|
3579
7f1e2d650486
fix blank-title subject line handling [SF#1442121]
Richard Jones <richard@users.sourceforge.net>
parents:
3544
diff
changeset
|
265 'date': message['adddate'], |
|
3534
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
266 'creation': message['adddate'], } |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
267 message_data.append(m) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
268 if authid not in (None, '2'): |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
269 nosy.add(authid) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
270 activity = message['adddate'] |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
271 actor = authid |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
272 if d['status'] == unread: |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
273 d['status'] = chatting |
|
3538
7f7902af7864
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
3537
diff
changeset
|
274 |
|
3579
7f1e2d650486
fix blank-title subject line handling [SF#1442121]
Richard Jones <richard@users.sourceforge.net>
parents:
3544
diff
changeset
|
275 # add import message |
|
3538
7f7902af7864
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
3537
diff
changeset
|
276 m = {'content': 'IMPORT FROM SOURCEFORGE', 'author': '1', |
|
3579
7f1e2d650486
fix blank-title subject line handling [SF#1442121]
Richard Jones <richard@users.sourceforge.net>
parents:
3544
diff
changeset
|
277 'date': today, 'creation': today} |
|
3538
7f7902af7864
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
3537
diff
changeset
|
278 message_data.append(m) |
|
7f7902af7864
*** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents:
3537
diff
changeset
|
279 |
|
3579
7f1e2d650486
fix blank-title subject line handling [SF#1442121]
Richard Jones <richard@users.sourceforge.net>
parents:
3544
diff
changeset
|
280 # sort messages and assign ids |
|
7f1e2d650486
fix blank-title subject line handling [SF#1442121]
Richard Jones <richard@users.sourceforge.net>
parents:
3544
diff
changeset
|
281 d['messages'] = [] |
|
5414
3fa026621f69
Python 3 preparation: comparisons.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5407
diff
changeset
|
282 message_data.sort(key=lambda a:a['date']) |
|
3579
7f1e2d650486
fix blank-title subject line handling [SF#1442121]
Richard Jones <richard@users.sourceforge.net>
parents:
3544
diff
changeset
|
283 for message in message_data: |
|
7f1e2d650486
fix blank-title subject line handling [SF#1442121]
Richard Jones <richard@users.sourceforge.net>
parents:
3544
diff
changeset
|
284 message_id += 1 |
|
7f1e2d650486
fix blank-title subject line handling [SF#1442121]
Richard Jones <richard@users.sourceforge.net>
parents:
3544
diff
changeset
|
285 message['id'] = str(message_id) |
|
7f1e2d650486
fix blank-title subject line handling [SF#1442121]
Richard Jones <richard@users.sourceforge.net>
parents:
3544
diff
changeset
|
286 d['messages'].append(message_id) |
|
7f1e2d650486
fix blank-title subject line handling [SF#1442121]
Richard Jones <richard@users.sourceforge.net>
parents:
3544
diff
changeset
|
287 |
|
3534
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
288 d['nosy'] = list(nosy) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
289 |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
290 files = [] |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
291 for event in artifact.get('history', []): |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
292 if event['field_name'] == 'File Added': |
|
3542
a2fc38fd0142
fixes and speedup
Richard Jones <richard@users.sourceforge.net>
parents:
3541
diff
changeset
|
293 fid, name = event['old_value'].split(':', 1) |
|
3534
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
294 if fid in add_files: |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
295 files.append(fid) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
296 name = name.strip() |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
297 try: |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
298 f = open(os.path.join(file_dir, fid)) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
299 content = f.read() |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
300 f.close() |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
301 except: |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
302 content = 'content missing' |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
303 file_data.append({ |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
304 'id': fid, |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
305 'creation': event['entrydate'], |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
306 'creator': users[event['mod_by']], |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
307 'name': name, |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
308 'type': mimetypes.guess_type(name)[0], |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
309 'content': content, |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
310 }) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
311 continue |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
312 elif event['field_name'] == 'close_date': |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
313 action = "'set'" |
| 3543 | 314 info = { 'status': unread } |
|
3534
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
315 elif event['field_name'] == 'summary': |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
316 action = "'set'" |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
317 info = { 'title': event['old_value'] } |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
318 else: |
| 3543 | 319 # not an interesting / translatable event |
|
3534
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
320 continue |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
321 row = [ d['id'], event['entrydate'].get_tuple(), |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
322 users[event['mod_by']], action, info ] |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
323 if event['entrydate'] > activity: |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
324 activity = event['entrydate'] |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
325 issue_journal.append(row) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
326 d['files'] = files |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
327 |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
328 d['activity'] = activity |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
329 d['actor'] = actor |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
330 issue_data.append(d) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
331 |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
332 write_csv(db.issue, issue_data) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
333 write_csv(db.msg, message_data) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
334 write_csv(db.file, file_data) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
335 |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
336 f = open('/tmp/imported/issue-journals.csv', 'w') |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
337 writer = csv.writer(f, colon_separated) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
338 writer.writerows(issue_journal) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
339 f.close() |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
340 |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
341 def convert_message(content, id): |
|
4089
eddb82d0964c
Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents:
3904
diff
changeset
|
342 """ Strip off the useless sf message header crap """ |
|
3534
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
343 if content[:14] == 'Logged In: YES': |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
344 return '\n'.join(content.splitlines()[3:]).strip() |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
345 return content |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
346 |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
347 class colon_separated(csv.excel): |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
348 delimiter = ':' |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
349 |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
350 def write_csv(klass, data): |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
351 props = klass.getprops() |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
352 if not os.path.exists('/tmp/imported'): |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
353 os.mkdir('/tmp/imported') |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
354 f = open('/tmp/imported/%s.csv'%klass.classname, 'w') |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
355 writer = csv.writer(f, colon_separated) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
356 propnames = klass.export_propnames() |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
357 propnames.append('is retired') |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
358 writer.writerow(propnames) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
359 for entry in data: |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
360 row = [] |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
361 for name in propnames: |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
362 if name == 'is retired': |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
363 continue |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
364 prop = props[name] |
|
5381
0942fe89e82e
Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents:
4687
diff
changeset
|
365 if name in entry: |
|
3534
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
366 if isinstance(prop, hyperdb.Date) or \ |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
367 isinstance(prop, hyperdb.Interval): |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
368 row.append(repr(entry[name].get_tuple())) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
369 elif isinstance(prop, hyperdb.Password): |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
370 row.append(repr(str(entry[name]))) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
371 else: |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
372 row.append(repr(entry[name])) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
373 elif isinstance(prop, hyperdb.Multilink): |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
374 row.append('[]') |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
375 elif name in ('creator', 'actor'): |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
376 row.append("'1'") |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
377 elif name in ('created', 'activity'): |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
378 row.append(repr(today.get_tuple())) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
379 else: |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
380 row.append('None') |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
381 row.append(entry.get('is retired', False)) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
382 writer.writerow(row) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
383 |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
384 if isinstance(klass, hyperdb.FileClass) and entry.get('content'): |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
385 fname = klass.exportFilename('/tmp/imported/', entry['id']) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
386 support.ensureParentsExist(fname) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
387 c = open(fname, 'w') |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
388 if isinstance(entry['content'], unicode): |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
389 c.write(entry['content'].encode('utf8')) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
390 else: |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
391 c.write(entry['content']) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
392 c.close() |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
393 |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
394 f.close() |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
395 f = open('/tmp/imported/%s-journals.csv'%klass.classname, 'w') |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
396 f.close() |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
397 |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
398 if __name__ == '__main__': |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
399 if sys.argv[1] == 'import': |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
400 import_xml(*sys.argv[2:]) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
401 elif sys.argv[1] == 'files': |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
402 fetch_files(*sys.argv[2:]) |
|
ad9f06e32f7c
scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
403 |
