Mercurial > p > roundup > code

annotate scripts/import_sf.py @ 3536:00a7983b4666

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
*** empty log message ***
author Richard Jones <richard@users.sourceforge.net>
date Fri, 03 Feb 2006 04:12:36 +0000
parents ad9f06e32f7c
children d819ff1b3116
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3536
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
1 ''' Import tracker data from Sourceforge.NET
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
2
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
3 This script needs four steps to work:
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
4
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
5 1. Export the project XML data using the admin web interface at sf.net
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
6 2. Run the file fetching (these are not included in the XML):
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
7
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
8 import_sf.py files <path to XML> <path to files dir>
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
9
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
10 this will place all the downloaded files in the files dir by file id.
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
11 3. Convert the sf.net XML to Roundup "export" format:
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
12
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
13 import_sf.py import <tracker home> <path to XML> <path to files dir>
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
14
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
15 this will generate a directory "/tmp/imported" which contains the
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
16 data to be imported into a Roundup tracker.
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
17 4. Import the data:
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
18
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
19 roundup-admin -i <tracker home> import /tmp/imported
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
20
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
21 And you're done!
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
22 '''
00a7983b4666 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3534
diff changeset
23
3534
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
24 import sys, sets, os, csv, time, urllib2, httplib, mimetypes
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
25 from elementtree import ElementTree
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
26
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
27 from roundup import instance, hyperdb, date, support, password
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
28
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
29 def get_url(aid):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
30 """ so basically we have to jump through hoops, given an artifact id, to
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
31 figure what the URL should be to access that artifact, and hence any
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
32 attached files."""
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
33 conn = httplib.HTTPConnection("sourceforge.net")
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
34 conn.request("GET", "/support/tracker.php?aid=%s"%aid)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
35 response = conn.getresponse()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
36 assert response.status == 302, 'response code was %s'%response.status
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
37 return 'http://sourceforge.net' + response.getheader('location')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
38
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
39 def fetch_files(xml_file, file_dir):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
40 """ Fetch files referenced in the xml_file into the dir file_dir. """
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
41 root = ElementTree.parse(xml_file).getroot()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
42 to_fetch = sets.Set()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
43 deleted = sets.Set()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
44 for artifact in root.find('artifacts'):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
45 for field in artifact.findall('field'):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
46 if field.get('name') == 'artifact_id':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
47 aid = field.text
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
48 for field in artifact.findall('field'):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
49 if field.get('name') != 'artifact_history': continue
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
50 for event in field.findall('history'):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
51 d = {}
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
52 for field in event.findall('field'):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
53 d[field.get('name')] = field.text
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
54 if d['field_name'] == 'File Added':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
55 fid = d['old_value'].split(':')[0]
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
56 to_fetch.add((aid, fid))
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
57 if d['field_name'] == 'File Deleted':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
58 fid = d['old_value'].split(':')[0]
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
59 deleted.add((aid, fid))
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
60 to_fetch = to_fetch - deleted
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
61
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
62 got = sets.Set(os.listdir(file_dir))
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
63 to_fetch = to_fetch - got
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
64
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
65 # load cached urls (sigh)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
66 urls = {}
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
67 if os.path.exists(os.path.join(file_dir, 'urls.txt')):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
68 for line in open(os.path.join(file_dir, 'urls.txt')):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
69 aid, url = line.strip().split()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
70 urls[aid] = url
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
71
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
72 for aid, fid in Progress('Fetching files', list(to_fetch)):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
73 if fid in got: continue
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
74 if not urls.has_key(aid):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
75 urls[aid] = get_url(aid)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
76 f = open(os.path.join(file_dir, 'urls.txt'), 'a')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
77 f.write('%s %s\n'%(aid, urls[aid]))
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
78 f.close()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
79 url = urls[aid] + '&file_id=' + fid
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
80 f = urllib2.urlopen(url)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
81 data = f.read()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
82 n = open(os.path.join(file_dir, fid), 'w')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
83 n.write(data)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
84 f.close()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
85 n.close()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
86
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
87 def import_xml(tracker_home, xml_file, file_dir):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
88 """ Generate Roundup tracker import files based on the tracker schema,
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
89 sf.net xml export and downloaded files from sf.net. """
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
90 tracker = instance.open(tracker_home)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
91 db = tracker.open('admin')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
92
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
93 resolved = db.status.lookup('resolved')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
94 unread = db.status.lookup('unread')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
95 chatting = db.status.lookup('unread')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
96 critical = db.priority.lookup('critical')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
97 urgent = db.priority.lookup('urgent')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
98 bug = db.priority.lookup('bug')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
99 feature = db.priority.lookup('feature')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
100 wish = db.priority.lookup('wish')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
101 adminuid = db.user.lookup('admin')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
102 anonuid = db.user.lookup('anonymous')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
103
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
104 root = ElementTree.parse(xml_file).getroot()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
105
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
106 def to_date(ts):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
107 return date.Date(time.gmtime(float(ts)))
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
108
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
109 # parse out the XML
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
110 artifacts = []
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
111 categories = sets.Set()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
112 users = sets.Set()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
113 add_files = sets.Set()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
114 remove_files = sets.Set()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
115 for artifact in root.find('artifacts'):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
116 d = {}
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
117 op = {}
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
118 artifacts.append(d)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
119 for field in artifact.findall('field'):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
120 name = field.get('name')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
121 if name == 'artifact_messages':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
122 for message in field.findall('message'):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
123 l = d.setdefault('messages', [])
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
124 m = {}
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
125 l.append(m)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
126 for field in message.findall('field'):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
127 name = field.get('name')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
128 if name == 'adddate':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
129 m[name] = to_date(field.text)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
130 else:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
131 m[name] = field.text
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
132 if name == 'user_name': users.add(field.text)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
133 elif name == 'artifact_history':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
134 for event in field.findall('history'):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
135 l = d.setdefault('history', [])
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
136 e = {}
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
137 l.append(e)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
138 for field in event.findall('field'):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
139 name = field.get('name')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
140 if name == 'entrydate':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
141 e[name] = to_date(field.text)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
142 else:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
143 e[name] = field.text
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
144 if name == 'mod_by': users.add(field.text)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
145 if e['field_name'] == 'File Added':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
146 add_files.add(e['old_value'].split(':')[0])
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
147 elif e['field_name'] == 'File Deleted':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
148 remove_files.add(e['old_value'].split(':')[0])
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
149 elif name == 'details':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
150 op['body'] = field.text
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
151 elif name == 'submitted_by':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
152 op['user_name'] = field.text
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
153 d[name] = field.text
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
154 users.add(field.text)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
155 elif name == 'open_date':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
156 thedate = to_date(field.text)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
157 op['adddate'] = thedate
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
158 d[name] = thedate
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
159 else:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
160 d[name] = field.text
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
161
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
162 categories.add(d['category'])
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
163
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
164 if op.has_key('body'):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
165 l = d.setdefault('messages', [])
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
166 l.insert(0, op)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
167
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
168 add_files -= remove_files
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
169
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
170 # create users
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
171 userd = {'nobody': '2'}
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
172 users.remove('nobody')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
173 data = [
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
174 {'id': '1', 'username': 'admin', 'password': password.Password('admin'),
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
175 'roles': 'Admin', 'address': 'richard@python.org'},
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
176 {'id': '2', 'username': 'anonymous', 'roles': 'Anonymous'},
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
177 ]
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
178 for n, user in enumerate(list(users)):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
179 userd[user] = n+3
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
180 data.append({'id': str(n+3), 'username': user, 'roles': 'User',
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
181 'address': '%s@users.sourceforge.net'%user})
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
182 write_csv(db.user, data)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
183 users=userd
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
184
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
185 # create categories
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
186 categoryd = {'None': None}
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
187 categories.remove('None')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
188 data = []
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
189 for n, category in enumerate(list(categories)):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
190 categoryd[category] = n
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
191 data.append({'id': str(n), 'name': category})
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
192 write_csv(db.keyword, data)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
193 categories = categoryd
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
194
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
195 # create issues
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
196 issue_data = []
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
197 file_data = []
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
198 message_data = []
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
199 issue_journal = []
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
200 message_id = 0
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
201 for artifact in artifacts:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
202 d = {}
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
203 d['id'] = artifact['artifact_id']
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
204 d['title'] = artifact['summary']
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
205 d['assignedto'] = users[artifact['assigned_to']]
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
206 if d['assignedto'] == '2':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
207 d['assignedto'] = None
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
208 d['creation'] = artifact['open_date']
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
209 activity = artifact['open_date']
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
210 d['creator'] = users[artifact['submitted_by']]
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
211 actor = d['creator']
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
212 if categories[artifact['category']]:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
213 d['topic'] = [categories[artifact['category']]]
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
214 issue_journal.append((
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
215 d['id'], d['creation'].get_tuple(), d['creator'], "'create'", {}
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
216 ))
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
217
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
218 p = int(artifact['priority'])
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
219 if artifact['artifact_type'] == 'Feature Requests':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
220 if p > 3:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
221 d['priority'] = feature
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
222 else:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
223 d['priority'] = wish
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
224 else:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
225 if p > 7:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
226 d['priority'] = critical
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
227 elif p > 5:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
228 d['priority'] = urgent
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
229 elif p > 3:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
230 d['priority'] = bug
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
231 else:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
232 d['priority'] = feature
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
233
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
234 s = artifact['status']
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
235 if s == 'Closed':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
236 d['status'] = resolved
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
237 elif s == 'Deleted':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
238 d['status'] = resolved
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
239 d['is retired'] = True
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
240 else:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
241 d['status'] = unread
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
242
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
243 messages = []
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
244 nosy = sets.Set()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
245 for message in artifact.get('messages', []):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
246 message_id += 1
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
247 authid = users[message['user_name']]
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
248 body = convert_message(message['body'], message_id)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
249 if not body: continue
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
250 m = {'content': body, 'author': authid,
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
251 'date': message['adddate'], 'id': str(message_id),
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
252 'creation': message['adddate'], }
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
253 message_data.append(m)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
254 messages.append(message_id)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
255 if authid not in (None, '2'):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
256 nosy.add(authid)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
257 activity = message['adddate']
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
258 actor = authid
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
259 if d['status'] == unread:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
260 d['status'] = chatting
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
261 d['messages'] = messages
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
262 d['nosy'] = list(nosy)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
263
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
264 files = []
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
265 for event in artifact.get('history', []):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
266 if event['field_name'] == 'File Added':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
267 fid, name = event['old_value'].split(':')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
268 if fid in add_files:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
269 files.append(fid)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
270 name = name.strip()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
271 try:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
272 f = open(os.path.join(file_dir, fid))
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
273 content = f.read()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
274 f.close()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
275 except:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
276 content = 'content missing'
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
277 file_data.append({
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
278 'id': fid,
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
279 'creation': event['entrydate'],
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
280 'creator': users[event['mod_by']],
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
281 'name': name,
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
282 'type': mimetypes.guess_type(name)[0],
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
283 'content': content,
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
284 })
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
285 continue
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
286 elif event['field_name'] == 'close_date':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
287 action = "'set'"
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
288 info = { 'status': 'unread' }
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
289 elif event['field_name'] == 'summary':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
290 action = "'set'"
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
291 info = { 'title': event['old_value'] }
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
292 else:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
293 continue
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
294 row = [ d['id'], event['entrydate'].get_tuple(),
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
295 users[event['mod_by']], action, info ]
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
296 if event['entrydate'] > activity:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
297 activity = event['entrydate']
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
298 issue_journal.append(row)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
299 d['files'] = files
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
300
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
301 d['activity'] = activity
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
302 d['actor'] = actor
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
303 issue_data.append(d)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
304
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
305 write_csv(db.issue, issue_data)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
306 write_csv(db.msg, message_data)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
307 write_csv(db.file, file_data)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
308
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
309 f = open('/tmp/imported/issue-journals.csv', 'w')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
310 writer = csv.writer(f, colon_separated)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
311 writer.writerows(issue_journal)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
312 f.close()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
313
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
314 def convert_message(content, id):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
315 ''' Strip off the useless sf message header crap '''
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
316 if content[:14] == 'Logged In: YES':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
317 return '\n'.join(content.splitlines()[3:]).strip()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
318 return content
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
319
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
320 class colon_separated(csv.excel):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
321 delimiter = ':'
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
322
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
323 today = date.Date('.')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
324 def write_csv(klass, data):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
325 props = klass.getprops()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
326 if not os.path.exists('/tmp/imported'):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
327 os.mkdir('/tmp/imported')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
328 f = open('/tmp/imported/%s.csv'%klass.classname, 'w')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
329 writer = csv.writer(f, colon_separated)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
330 propnames = klass.export_propnames()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
331 propnames.append('is retired')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
332 writer.writerow(propnames)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
333 for entry in data:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
334 row = []
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
335 for name in propnames:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
336 if name == 'is retired':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
337 continue
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
338 prop = props[name]
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
339 if entry.has_key(name):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
340 if isinstance(prop, hyperdb.Date) or \
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
341 isinstance(prop, hyperdb.Interval):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
342 row.append(repr(entry[name].get_tuple()))
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
343 elif isinstance(prop, hyperdb.Password):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
344 row.append(repr(str(entry[name])))
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
345 else:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
346 row.append(repr(entry[name]))
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
347 elif isinstance(prop, hyperdb.Multilink):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
348 row.append('[]')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
349 elif name in ('creator', 'actor'):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
350 row.append("'1'")
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
351 elif name in ('created', 'activity'):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
352 row.append(repr(today.get_tuple()))
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
353 else:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
354 row.append('None')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
355 row.append(entry.get('is retired', False))
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
356 writer.writerow(row)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
357
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
358 if isinstance(klass, hyperdb.FileClass) and entry.get('content'):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
359 fname = klass.exportFilename('/tmp/imported/', entry['id'])
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
360 support.ensureParentsExist(fname)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
361 c = open(fname, 'w')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
362 if isinstance(entry['content'], unicode):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
363 c.write(entry['content'].encode('utf8'))
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
364 else:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
365 c.write(entry['content'])
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
366 c.close()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
367
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
368 f.close()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
369 f = open('/tmp/imported/%s-journals.csv'%klass.classname, 'w')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
370 f.close()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
371
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
372 class Progress:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
373 '''Progress display for console applications.
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
374
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
375 See __main__ block at end of file for sample usage.
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
376 '''
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
377 def __init__(self, info, sequence):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
378 self.info = info
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
379 self.sequence = iter(sequence)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
380 self.total = len(sequence)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
381 self.start = self.now = time.time()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
382 self.num = 0
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
383 self.stepsize = self.total / 100 or 1
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
384 self.steptimes = []
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
385 self.display()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
386
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
387 def __iter__(self): return self
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
388
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
389 def next(self):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
390 self.num += 1
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
391
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
392 if self.num > self.total:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
393 print self.info, 'done', ' '*(75-len(self.info)-6)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
394 sys.stdout.flush()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
395 return self.sequence.next()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
396
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
397 if self.num % self.stepsize:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
398 return self.sequence.next()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
399
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
400 self.display()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
401 return self.sequence.next()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
402
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
403 def display(self):
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
404 # figure how long we've spent - guess how long to go
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
405 now = time.time()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
406 steptime = now - self.now
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
407 self.steptimes.insert(0, steptime)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
408 if len(self.steptimes) > 5:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
409 self.steptimes.pop()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
410 steptime = sum(self.steptimes) / len(self.steptimes)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
411 self.now = now
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
412 eta = steptime * ((self.total - self.num)/self.stepsize)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
413
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
414 # tell it like it is (or might be)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
415 if now - self.start > 3:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
416 M = eta / 60
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
417 H = M / 60
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
418 M = M % 60
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
419 S = eta % 60
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
420 s = '%s %2d%% (ETA %02d:%02d:%02d)'%(self.info,
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
421 self.num * 100. / self.total, H, M, S)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
422 else:
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
423 s = '%s %2d%%'%(self.info, self.num * 100. / self.total)
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
424 sys.stdout.write(s + ' '*(75-len(s)) + '\r')
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
425 sys.stdout.flush()
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
426
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
427 if __name__ == '__main__':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
428 if sys.argv[1] == 'import':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
429 import_xml(*sys.argv[2:])
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
430 elif sys.argv[1] == 'files':
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
431 fetch_files(*sys.argv[2:])
ad9f06e32f7c scripts/import_sf.py will import a tracker from Sourceforge.NET
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
432
Roundup Issue Tracker: http://roundup-tracker.org/