Mercurial > p > roundup > code

annotate scripts/import_sf.py @ 5416:56c9bcdea47f

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Python 3 preparation: unicode. This patch introduces roundup/anypy/strings.py, which has a comment explaining the string representations generally used and common functions to handle the required conversions. Places in the code that explicitly reference the "unicode" type / built-in function are generally changed to use the new functions (or, in a few places where those new functions don't seem to fit well, other approaches such as references to type(u'') or use of the codecs module). This patch does not generally attempt to address text conversions in any places not currently referencing the "unicode" type (although scripts/import_sf.py is made to use binary I/O in places as fixing the "unicode" reference didn't seem coherent otherwise).
author Joseph Myers <jsm@polyomino.org.uk>
date Wed, 25 Jul 2018 09:05:58 +0000
parents 3fa026621f69
children fed0f839c260
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
4089
eddb82d0964c Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents: 3904
diff changeset
1 """ Import tracker data from Sourceforge.NET
3536
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
2
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
3 This script needs four steps to work:
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
4
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
5 1. Export the project XML data using the admin web interface at sf.net
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
6 2. Run the file fetching (these are not included in the XML):
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
7
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
8 import_sf.py files <path to XML> <path to files dir>
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
9
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
10 this will place all the downloaded files in the files dir by file id.
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
11 3. Convert the sf.net XML to Roundup "export" format:
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
12
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
13 import_sf.py import <tracker home> <path to XML> <path to files dir>
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
14
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
15 this will generate a directory "/tmp/imported" which contains the
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
16 data to be imported into a Roundup tracker.
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
17 4. Import the data:
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
18
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
19 roundup-admin -i <tracker home> import /tmp/imported
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
20
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
21 And you're done!
4089
eddb82d0964c Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents: 3904
diff changeset
22 """
3536
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
23
5407
f004824563a3 Python 3 preparation: update httplib imports.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5402
diff changeset
24 import sys, os, csv, time, mimetypes
3542
a2fc38fd0142 fixes and speedup
Richard Jones <richard@users.sourceforge.net>
parents: 3541
diff changeset
25
a2fc38fd0142 fixes and speedup
Richard Jones <richard@users.sourceforge.net>
parents: 3541
diff changeset
26 try:
a2fc38fd0142 fixes and speedup
Richard Jones <richard@users.sourceforge.net>
parents: 3541
diff changeset
27 import cElementTree as ElementTree
a2fc38fd0142 fixes and speedup
Richard Jones <richard@users.sourceforge.net>
parents: 3541
diff changeset
28 except ImportError:
a2fc38fd0142 fixes and speedup
Richard Jones <richard@users.sourceforge.net>
parents: 3541
diff changeset
29 from elementtree import ElementTree
3534
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
30
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
31 from roundup import instance, hyperdb, date, support, password
5407
f004824563a3 Python 3 preparation: update httplib imports.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5402
diff changeset
32 from roundup.anypy import http_, urllib_
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5414
diff changeset
33 from roundup.anypy.strings import s2b, us2s
3534
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
34
3538
7f7902af7864 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3537
diff changeset
35 today = date.Date('.')
7f7902af7864 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3537
diff changeset
36
3537
d819ff1b3116 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3536
diff changeset
37 DL_URL = 'http://sourceforge.net/tracker/download.php?group_id=%(group_id)s&atid=%(atid)s&aid=%(aid)s'
d819ff1b3116 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3536
diff changeset
38
3534
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
39 def get_url(aid):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
40 """ so basically we have to jump through hoops, given an artifact id, to
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
41 figure what the URL should be to access that artifact, and hence any
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
42 attached files."""
3538
7f7902af7864 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3537
diff changeset
43 # first we hit this URL...
5407
f004824563a3 Python 3 preparation: update httplib imports.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5402
diff changeset
44 conn = http_.client.HTTPConnection("sourceforge.net")
3534
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
45 conn.request("GET", "/support/tracker.php?aid=%s"%aid)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
46 response = conn.getresponse()
3538
7f7902af7864 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3537
diff changeset
47 # which should respond with a redirect to the correct url which has the
7f7902af7864 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3537
diff changeset
48 # magic "group_id" and "atid" values in it that we need
3534
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
49 assert response.status == 302, 'response code was %s'%response.status
3537
d819ff1b3116 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3536
diff changeset
50 location = response.getheader('location')
5402
88dbacd11cd1 Python 3 preparation: update urllib / urllib2 / urlparse imports.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5381
diff changeset
51 query = urllib_.urlparse(response.getheader('location'))[-2]
3537
d819ff1b3116 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3536
diff changeset
52 info = dict([param.split('=') for param in query.split('&')])
3541
07072b1b796c fix typo
Richard Jones <richard@users.sourceforge.net>
parents: 3538
diff changeset
53 return DL_URL%info
3534
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
54
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
55 def fetch_files(xml_file, file_dir):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
56 """ Fetch files referenced in the xml_file into the dir file_dir. """
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
57 root = ElementTree.parse(xml_file).getroot()
4089
eddb82d0964c Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents: 3904
diff changeset
58 to_fetch = set()
eddb82d0964c Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents: 3904
diff changeset
59 deleted = set()
3534
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
60 for artifact in root.find('artifacts'):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
61 for field in artifact.findall('field'):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
62 if field.get('name') == 'artifact_id':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
63 aid = field.text
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
64 for field in artifact.findall('field'):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
65 if field.get('name') != 'artifact_history': continue
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
66 for event in field.findall('history'):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
67 d = {}
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
68 for field in event.findall('field'):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
69 d[field.get('name')] = field.text
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
70 if d['field_name'] == 'File Added':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
71 fid = d['old_value'].split(':')[0]
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
72 to_fetch.add((aid, fid))
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
73 if d['field_name'] == 'File Deleted':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
74 fid = d['old_value'].split(':')[0]
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
75 deleted.add((aid, fid))
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
76 to_fetch = to_fetch - deleted
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
77
4089
eddb82d0964c Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents: 3904
diff changeset
78 got = set(os.listdir(file_dir))
3534
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
79 to_fetch = to_fetch - got
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
80
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
81 # load cached urls (sigh)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
82 urls = {}
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
83 if os.path.exists(os.path.join(file_dir, 'urls.txt')):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
84 for line in open(os.path.join(file_dir, 'urls.txt')):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
85 aid, url = line.strip().split()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
86 urls[aid] = url
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
87
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3543
diff changeset
88 for aid, fid in support.Progress('Fetching files', list(to_fetch)):
3534
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
89 if fid in got: continue
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 4687
diff changeset
90 if aid not in urls:
3534
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
91 urls[aid] = get_url(aid)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
92 f = open(os.path.join(file_dir, 'urls.txt'), 'a')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
93 f.write('%s %s\n'%(aid, urls[aid]))
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
94 f.close()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
95 url = urls[aid] + '&file_id=' + fid
5402
88dbacd11cd1 Python 3 preparation: update urllib / urllib2 / urlparse imports.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5381
diff changeset
96 f = urllib_.urlopen(url)
3534
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
97 data = f.read()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
98 n = open(os.path.join(file_dir, fid), 'w')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
99 n.write(data)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
100 f.close()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
101 n.close()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
102
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
103 def import_xml(tracker_home, xml_file, file_dir):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
104 """ Generate Roundup tracker import files based on the tracker schema,
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
105 sf.net xml export and downloaded files from sf.net. """
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
106 tracker = instance.open(tracker_home)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
107 db = tracker.open('admin')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
108
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
109 resolved = db.status.lookup('resolved')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
110 unread = db.status.lookup('unread')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
111 chatting = db.status.lookup('unread')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
112 critical = db.priority.lookup('critical')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
113 urgent = db.priority.lookup('urgent')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
114 bug = db.priority.lookup('bug')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
115 feature = db.priority.lookup('feature')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
116 wish = db.priority.lookup('wish')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
117 adminuid = db.user.lookup('admin')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
118 anonuid = db.user.lookup('anonymous')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
119
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
120 root = ElementTree.parse(xml_file).getroot()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
121
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
122 def to_date(ts):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
123 return date.Date(time.gmtime(float(ts)))
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
124
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
125 # parse out the XML
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
126 artifacts = []
4089
eddb82d0964c Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents: 3904
diff changeset
127 categories = set()
eddb82d0964c Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents: 3904
diff changeset
128 users = set()
eddb82d0964c Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents: 3904
diff changeset
129 add_files = set()
eddb82d0964c Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents: 3904
diff changeset
130 remove_files = set()
3534
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
131 for artifact in root.find('artifacts'):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
132 d = {}
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
133 op = {}
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
134 artifacts.append(d)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
135 for field in artifact.findall('field'):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
136 name = field.get('name')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
137 if name == 'artifact_messages':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
138 for message in field.findall('message'):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
139 l = d.setdefault('messages', [])
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
140 m = {}
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
141 l.append(m)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
142 for field in message.findall('field'):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
143 name = field.get('name')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
144 if name == 'adddate':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
145 m[name] = to_date(field.text)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
146 else:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
147 m[name] = field.text
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
148 if name == 'user_name': users.add(field.text)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
149 elif name == 'artifact_history':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
150 for event in field.findall('history'):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
151 l = d.setdefault('history', [])
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
152 e = {}
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
153 l.append(e)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
154 for field in event.findall('field'):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
155 name = field.get('name')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
156 if name == 'entrydate':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
157 e[name] = to_date(field.text)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
158 else:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
159 e[name] = field.text
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
160 if name == 'mod_by': users.add(field.text)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
161 if e['field_name'] == 'File Added':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
162 add_files.add(e['old_value'].split(':')[0])
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
163 elif e['field_name'] == 'File Deleted':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
164 remove_files.add(e['old_value'].split(':')[0])
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
165 elif name == 'details':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
166 op['body'] = field.text
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
167 elif name == 'submitted_by':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
168 op['user_name'] = field.text
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
169 d[name] = field.text
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
170 users.add(field.text)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
171 elif name == 'open_date':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
172 thedate = to_date(field.text)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
173 op['adddate'] = thedate
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
174 d[name] = thedate
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
175 else:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
176 d[name] = field.text
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
177
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
178 categories.add(d['category'])
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
179
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 4687
diff changeset
180 if 'body' in op:
3534
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
181 l = d.setdefault('messages', [])
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
182 l.insert(0, op)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
183
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
184 add_files -= remove_files
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
185
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
186 # create users
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
187 userd = {'nobody': '2'}
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
188 users.remove('nobody')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
189 data = [
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
190 {'id': '1', 'username': 'admin', 'password': password.Password('admin'),
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
191 'roles': 'Admin', 'address': 'richard@python.org'},
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
192 {'id': '2', 'username': 'anonymous', 'roles': 'Anonymous'},
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
193 ]
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
194 for n, user in enumerate(list(users)):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
195 userd[user] = n+3
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
196 data.append({'id': str(n+3), 'username': user, 'roles': 'User',
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
197 'address': '%s@users.sourceforge.net'%user})
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
198 write_csv(db.user, data)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
199 users=userd
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
200
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
201 # create categories
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
202 categoryd = {'None': None}
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
203 categories.remove('None')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
204 data = []
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
205 for n, category in enumerate(list(categories)):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
206 categoryd[category] = n
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
207 data.append({'id': str(n), 'name': category})
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
208 write_csv(db.keyword, data)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
209 categories = categoryd
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
210
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
211 # create issues
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
212 issue_data = []
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
213 file_data = []
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
214 message_data = []
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
215 issue_journal = []
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
216 message_id = 0
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
217 for artifact in artifacts:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
218 d = {}
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
219 d['id'] = artifact['artifact_id']
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
220 d['title'] = artifact['summary']
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
221 d['assignedto'] = users[artifact['assigned_to']]
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
222 if d['assignedto'] == '2':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
223 d['assignedto'] = None
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
224 d['creation'] = artifact['open_date']
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
225 activity = artifact['open_date']
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
226 d['creator'] = users[artifact['submitted_by']]
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
227 actor = d['creator']
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
228 if categories[artifact['category']]:
3904
91008ec8f9a0 retire "topic" usage
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3579
diff changeset
229 d['keyword'] = [categories[artifact['category']]]
3534
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
230 issue_journal.append((
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
231 d['id'], d['creation'].get_tuple(), d['creator'], "'create'", {}
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
232 ))
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
233
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
234 p = int(artifact['priority'])
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
235 if artifact['artifact_type'] == 'Feature Requests':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
236 if p > 3:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
237 d['priority'] = feature
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
238 else:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
239 d['priority'] = wish
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
240 else:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
241 if p > 7:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
242 d['priority'] = critical
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
243 elif p > 5:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
244 d['priority'] = urgent
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
245 elif p > 3:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
246 d['priority'] = bug
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
247 else:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
248 d['priority'] = feature
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
249
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
250 s = artifact['status']
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
251 if s == 'Closed':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
252 d['status'] = resolved
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
253 elif s == 'Deleted':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
254 d['status'] = resolved
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
255 d['is retired'] = True
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
256 else:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
257 d['status'] = unread
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
258
4089
eddb82d0964c Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents: 3904
diff changeset
259 nosy = set()
3534
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
260 for message in artifact.get('messages', []):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
261 authid = users[message['user_name']]
3542
a2fc38fd0142 fixes and speedup
Richard Jones <richard@users.sourceforge.net>
parents: 3541
diff changeset
262 if not message['body']: continue
3534
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
263 body = convert_message(message['body'], message_id)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
264 if not body: continue
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
265 m = {'content': body, 'author': authid,
3579
7f1e2d650486 fix blank-title subject line handling [SF#1442121]
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
266 'date': message['adddate'],
3534
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
267 'creation': message['adddate'], }
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
268 message_data.append(m)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
269 if authid not in (None, '2'):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
270 nosy.add(authid)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
271 activity = message['adddate']
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
272 actor = authid
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
273 if d['status'] == unread:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
274 d['status'] = chatting
3538
7f7902af7864 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3537
diff changeset
275
3579
7f1e2d650486 fix blank-title subject line handling [SF#1442121]
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
276 # add import message
3538
7f7902af7864 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3537
diff changeset
277 m = {'content': 'IMPORT FROM SOURCEFORGE', 'author': '1',
3579
7f1e2d650486 fix blank-title subject line handling [SF#1442121]
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
278 'date': today, 'creation': today}
3538
7f7902af7864 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3537
diff changeset
279 message_data.append(m)
7f7902af7864 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3537
diff changeset
280
3579
7f1e2d650486 fix blank-title subject line handling [SF#1442121]
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
281 # sort messages and assign ids
7f1e2d650486 fix blank-title subject line handling [SF#1442121]
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
282 d['messages'] = []
5414
3fa026621f69 Python 3 preparation: comparisons.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5407
diff changeset
283 message_data.sort(key=lambda a:a['date'])
3579
7f1e2d650486 fix blank-title subject line handling [SF#1442121]
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
284 for message in message_data:
7f1e2d650486 fix blank-title subject line handling [SF#1442121]
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
285 message_id += 1
7f1e2d650486 fix blank-title subject line handling [SF#1442121]
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
286 message['id'] = str(message_id)
7f1e2d650486 fix blank-title subject line handling [SF#1442121]
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
287 d['messages'].append(message_id)
7f1e2d650486 fix blank-title subject line handling [SF#1442121]
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
288
3534
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
289 d['nosy'] = list(nosy)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
290
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
291 files = []
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
292 for event in artifact.get('history', []):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
293 if event['field_name'] == 'File Added':
3542
a2fc38fd0142 fixes and speedup
Richard Jones <richard@users.sourceforge.net>
parents: 3541
diff changeset
294 fid, name = event['old_value'].split(':', 1)
3534
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
295 if fid in add_files:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
296 files.append(fid)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
297 name = name.strip()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
298 try:
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5414
diff changeset
299 f = open(os.path.join(file_dir, fid), 'rb')
3534
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
300 content = f.read()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
301 f.close()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
302 except:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
303 content = 'content missing'
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
304 file_data.append({
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
305 'id': fid,
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
306 'creation': event['entrydate'],
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
307 'creator': users[event['mod_by']],
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
308 'name': name,
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
309 'type': mimetypes.guess_type(name)[0],
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
310 'content': content,
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
311 })
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
312 continue
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
313 elif event['field_name'] == 'close_date':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
314 action = "'set'"
3543
Richard Jones <richard@users.sourceforge.net>
parents: 3542
diff changeset
315 info = { 'status': unread }
3534
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
316 elif event['field_name'] == 'summary':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
317 action = "'set'"
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
318 info = { 'title': event['old_value'] }
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
319 else:
3543
Richard Jones <richard@users.sourceforge.net>
parents: 3542
diff changeset
320 # not an interesting / translatable event
3534
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
321 continue
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
322 row = [ d['id'], event['entrydate'].get_tuple(),
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
323 users[event['mod_by']], action, info ]
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
324 if event['entrydate'] > activity:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
325 activity = event['entrydate']
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
326 issue_journal.append(row)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
327 d['files'] = files
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
328
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
329 d['activity'] = activity
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
330 d['actor'] = actor
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
331 issue_data.append(d)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
332
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
333 write_csv(db.issue, issue_data)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
334 write_csv(db.msg, message_data)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
335 write_csv(db.file, file_data)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
336
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
337 f = open('/tmp/imported/issue-journals.csv', 'w')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
338 writer = csv.writer(f, colon_separated)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
339 writer.writerows(issue_journal)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
340 f.close()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
341
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
342 def convert_message(content, id):
4089
eddb82d0964c Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents: 3904
diff changeset
343 """ Strip off the useless sf message header crap """
3534
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
344 if content[:14] == 'Logged In: YES':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
345 return '\n'.join(content.splitlines()[3:]).strip()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
346 return content
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
347
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
348 class colon_separated(csv.excel):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
349 delimiter = ':'
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
350
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
351 def write_csv(klass, data):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
352 props = klass.getprops()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
353 if not os.path.exists('/tmp/imported'):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
354 os.mkdir('/tmp/imported')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
355 f = open('/tmp/imported/%s.csv'%klass.classname, 'w')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
356 writer = csv.writer(f, colon_separated)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
357 propnames = klass.export_propnames()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
358 propnames.append('is retired')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
359 writer.writerow(propnames)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
360 for entry in data:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
361 row = []
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
362 for name in propnames:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
363 if name == 'is retired':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
364 continue
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
365 prop = props[name]
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 4687
diff changeset
366 if name in entry:
3534
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
367 if isinstance(prop, hyperdb.Date) or \
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
368 isinstance(prop, hyperdb.Interval):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
369 row.append(repr(entry[name].get_tuple()))
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
370 elif isinstance(prop, hyperdb.Password):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
371 row.append(repr(str(entry[name])))
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
372 else:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
373 row.append(repr(entry[name]))
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
374 elif isinstance(prop, hyperdb.Multilink):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
375 row.append('[]')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
376 elif name in ('creator', 'actor'):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
377 row.append("'1'")
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
378 elif name in ('created', 'activity'):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
379 row.append(repr(today.get_tuple()))
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
380 else:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
381 row.append('None')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
382 row.append(entry.get('is retired', False))
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
383 writer.writerow(row)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
384
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
385 if isinstance(klass, hyperdb.FileClass) and entry.get('content'):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
386 fname = klass.exportFilename('/tmp/imported/', entry['id'])
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
387 support.ensureParentsExist(fname)
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5414
diff changeset
388 c = open(fname, 'wb')
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5414
diff changeset
389 if isinstance(entry['content'], bytes):
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5414
diff changeset
390 c.write(entry['content'])
3534
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
391 else:
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5414
diff changeset
392 c.write(s2b(us2s(entry['content'])))
3534
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
393 c.close()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
394
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
395 f.close()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
396 f = open('/tmp/imported/%s-journals.csv'%klass.classname, 'w')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
397 f.close()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
398
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
399 if __name__ == '__main__':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
400 if sys.argv[1] == 'import':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
401 import_xml(*sys.argv[2:])
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
402 elif sys.argv[1] == 'files':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
403 fetch_files(*sys.argv[2:])
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
404
Roundup Issue Tracker: http://roundup-tracker.org/