Mercurial > p > roundup > code

annotate roundup/backends/blobfiles.py @ 645:e6d1c6d66de3

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
add module blobfiles in backends with file access functions. not yet activated.
author Engelbert Gruber <grubert@users.sourceforge.net>
date Mon, 25 Feb 2002 14:25:41 +0000
parents
children 9b2575610953
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
645
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
1 #
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
2 # Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/)
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
3 # This module is free software, and you may redistribute it and/or modify
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
4 # under the same terms as Python, so long as this copyright message and
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
5 # disclaimer are retained in their original form.
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
6 #
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
7 # IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
8 # DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
9 # OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
10 # POSSIBILITY OF SUCH DAMAGE.
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
11 #
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
12 # BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING,
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
13 # BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
14 # FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS"
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
15 # BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE,
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
16 # SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
17 #
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
18 #$Id: blobfiles.py,v 1.1 2002-02-25 14:25:41 grubert Exp $
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
19 '''
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
20 This module exports file storage for roundup backends.
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
21 Files are stored into a directory hierarchy.
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
22 '''
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
23
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
24 import os, os.path
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
25
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
26 class FileStorage:
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
27 """Store files in some directory structure"""
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
28 def __init__(self):
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
29 # maybe set "files"
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
30 pass
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
31
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
32 def filename(self, classname, nodeid, property=None):
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
33 '''Determine what the filename for the given node and optionally
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
34 property is.
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
35 '''
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
36 if property:
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
37 name = '%s%s.%s'%(classname, nodeid, property)
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
38 else:
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
39 # roundupdb.FileClass never specified the property name, so don't
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
40 # include it
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
41 name = '%s%s'%(classname, nodeid)
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
42
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
43 # have a separate subdir for every thousand messages
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
44 subdir = str(int(nodeid) / 1000)
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
45 return os.path.join(self.dir, 'files', classname, subdir, name)
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
46
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
47 def filename_flat(self, classname, nodeid, property=None):
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
48 '''Determine what the filename for the given node and optionally
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
49 property is.
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
50 '''
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
51 if property:
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
52 return os.path.join(self.dir, 'files', '%s%s.%s'%(classname,
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
53 nodeid, property))
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
54 else:
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
55 # roundupdb.FileClass never specified the property name, so don't
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
56 # include it
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
57 return os.path.join(self.dir, 'files', '%s%s'%(classname,
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
58 nodeid))
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
59
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
60 def storefile(self, classname, nodeid, property, content):
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
61 '''Store the content of the file in the database. The property may be
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
62 None, in which case the filename does not indicate which property
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
63 is being saved.
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
64 '''
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
65 name = self.filename(classname, nodeid, property)
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
66 if not os.path.exists(os.path.dirname(name)):
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
67 os.makedirs(os.path.dirname(name))
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
68 open(name + '.tmp', 'wb').write(content)
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
69 self.transactions.append((self._doStoreFile, (name, )))
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
70
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
71
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
72 def getfile(self, classname, nodeid, property):
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
73 '''Get the content of the file in the database.
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
74 '''
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
75 filename = self.filename(classname, nodeid, property)
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
76 try:
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
77 return open(filename, 'rb').read()
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
78 except:
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
79 try:
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
80 return open(filename+'.tmp', 'rb').read()
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
81 except:
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
82 # fallback to flat file storage
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
83 filename = self.filename_flat(classname, nodeid, property)
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
84 return open(filename, 'rb').read()
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
85
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
86 def numfiles(self):
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
87 '''Get number of files in storage, even across subdirectories.
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
88 '''
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
89 files_dir = os.path.join(self.dir, 'files')
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
90
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
91 def files_in_dir(dir):
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
92 if not os.path.exists(dir):
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
93 return 0
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
94 num_files = 0
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
95 for dir_entry in os.listdir(dir):
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
96 full_filename = os.path.join(dir,dir_entry)
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
97 if os.path.isfile(full_filename):
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
98 num_files = num_files + 1
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
99 elif os.path.isdir(full_filename):
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
100 num_files = num_files + files_in_dir(full_filename)
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
101 return num_files
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
102
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
103 return files_in_dir(files_dir)
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
104
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
105
e6d1c6d66de3 add module blobfiles in backends with file access functions. not yet activated.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
diff changeset
106
Roundup Issue Tracker: http://roundup-tracker.org/