Mercurial > p > roundup > code

annotate test/session_common.py @ 6806:bdd28b244839

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
- issue2551223 - fix timestamp truncation in mysql and postgresql The data types used to represent timestamps in pg and mysql for ephemeral tables: sessions and otks don't have enough signifcant digits to work. As a result the timestamps are rounduped (up/down) rsuling in the stored timestamp being 2 minutes (pg) or 2-3 hours(mysql) off from what it should be. Modify db schema to use a numeric type that preserves more significant figures. Implement schema upgrade. Document need for upgrade in upgrading.txt. Write tests for schema upgrade. Implement test for updateTimestamp method on BasicDatabase that showed this issue in the first place. Write overrides for test for anydbm/memorydb which store timestamp properly or not at all.
author John Rouillard <rouilj@ieee.org>
date Mon, 25 Jul 2022 17:20:20 -0400
parents 044dcf3608a2
children 375d40a9e730
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
6802
044dcf3608a2 update session db tests
John Rouillard <rouilj@ieee.org>
parents: 5794
diff changeset
1 import os, shutil, time, unittest
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2
5388
d26921b851c3 Python 3 preparation: make relative imports explicit.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5319
diff changeset
3 from .db_test_base import config
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
4
5033
63c79c0992ae Update tests to work with py.test
John Kristensen <john@jerrykan.com>
parents: 4386
diff changeset
5
63c79c0992ae Update tests to work with py.test
John Kristensen <john@jerrykan.com>
parents: 4386
diff changeset
6 class SessionTest(object):
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
7 def setUp(self):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
8 # remove previous test, ignore errors
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
9 if os.path.exists(config.DATABASE):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
10 shutil.rmtree(config.DATABASE)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
11 os.makedirs(config.DATABASE + '/files')
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
12 self.db = self.module.Database(config, 'admin')
5319
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5033
diff changeset
13 self.sessions = self.db.getSessionManager()
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5033
diff changeset
14 self.otks = self.db.getOTKManager()
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
15
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
16 def tearDown(self):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
17 if hasattr(self, 'db'):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
18 self.db.close()
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
19 if os.path.exists(config.DATABASE):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
20 shutil.rmtree(config.DATABASE)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
21
4386
cc33dc9aa3f2 moar test coverage
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
22 def testList(self):
6802
044dcf3608a2 update session db tests
John Rouillard <rouilj@ieee.org>
parents: 5794
diff changeset
23 '''Under dbm/memory sessions store, keys are returned as
044dcf3608a2 update session db tests
John Rouillard <rouilj@ieee.org>
parents: 5794
diff changeset
24 byte strings. self.s2b converts string to byte under those
044dcf3608a2 update session db tests
John Rouillard <rouilj@ieee.org>
parents: 5794
diff changeset
25 backends but is a no-op for rdbms based backends.
044dcf3608a2 update session db tests
John Rouillard <rouilj@ieee.org>
parents: 5794
diff changeset
26
044dcf3608a2 update session db tests
John Rouillard <rouilj@ieee.org>
parents: 5794
diff changeset
27 Unknown why keys can be strings not bytes for get/set
044dcf3608a2 update session db tests
John Rouillard <rouilj@ieee.org>
parents: 5794
diff changeset
28 and work correctly.
044dcf3608a2 update session db tests
John Rouillard <rouilj@ieee.org>
parents: 5794
diff changeset
29 '''
4386
cc33dc9aa3f2 moar test coverage
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
30 self.sessions.list()
cc33dc9aa3f2 moar test coverage
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
31 self.sessions.set('random_key', text='hello, world!')
6802
044dcf3608a2 update session db tests
John Rouillard <rouilj@ieee.org>
parents: 5794
diff changeset
32 self.sessions.set('random_key2', text='hello, world!')
044dcf3608a2 update session db tests
John Rouillard <rouilj@ieee.org>
parents: 5794
diff changeset
33 self.assertEqual(self.sessions.list().sort(),
044dcf3608a2 update session db tests
John Rouillard <rouilj@ieee.org>
parents: 5794
diff changeset
34 [self.s2b('random_key'), self.s2b('random_key2')].sort())
4386
cc33dc9aa3f2 moar test coverage
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
35
cc33dc9aa3f2 moar test coverage
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
36 def testGetAll(self):
6802
044dcf3608a2 update session db tests
John Rouillard <rouilj@ieee.org>
parents: 5794
diff changeset
37 self.sessions.set('random_key', text='hello, world!', otherval='bar')
4386
cc33dc9aa3f2 moar test coverage
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
38 self.assertEqual(self.sessions.getall('random_key'),
6802
044dcf3608a2 update session db tests
John Rouillard <rouilj@ieee.org>
parents: 5794
diff changeset
39 {'text': 'hello, world!', 'otherval': 'bar'})
4386
cc33dc9aa3f2 moar test coverage
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
40
cc33dc9aa3f2 moar test coverage
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
41 def testDestroy(self):
cc33dc9aa3f2 moar test coverage
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
42 self.sessions.set('random_key', text='hello, world!')
5794
95a366d46065 Replace deprecated assertEquals with assertEqual and failUnlessRaises
John Rouillard <rouilj@ieee.org>
parents: 5388
diff changeset
43 self.assertEqual(self.sessions.getall('random_key'),
4386
cc33dc9aa3f2 moar test coverage
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
44 {'text': 'hello, world!'})
cc33dc9aa3f2 moar test coverage
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
45 self.sessions.destroy('random_key')
cc33dc9aa3f2 moar test coverage
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
46 self.assertRaises(KeyError, self.sessions.getall, 'random_key')
cc33dc9aa3f2 moar test coverage
Richard Jones <richard@users.sourceforge.net>
parents: 2089
diff changeset
47
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
48 def testSetSession(self):
6802
044dcf3608a2 update session db tests
John Rouillard <rouilj@ieee.org>
parents: 5794
diff changeset
49 self.sessions.set('random_key', text='hello, world!', otherval='bar')
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
50 self.assertEqual(self.sessions.get('random_key', 'text'),
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
51 'hello, world!')
6802
044dcf3608a2 update session db tests
John Rouillard <rouilj@ieee.org>
parents: 5794
diff changeset
52 self.assertEqual(self.sessions.get('random_key', 'otherval'),
044dcf3608a2 update session db tests
John Rouillard <rouilj@ieee.org>
parents: 5794
diff changeset
53 'bar')
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
54
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
55 def testUpdateSession(self):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
56 self.sessions.set('random_key', text='hello, world!')
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
57 self.assertEqual(self.sessions.get('random_key', 'text'),
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
58 'hello, world!')
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
59 self.sessions.set('random_key', text='nope')
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
60 self.assertEqual(self.sessions.get('random_key', 'text'), 'nope')
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
61
6806
bdd28b244839 - issue2551223 - fix timestamp truncation in mysql and postgresql
John Rouillard <rouilj@ieee.org>
parents: 6802
diff changeset
62 # overridden in dbm and memory backends
bdd28b244839 - issue2551223 - fix timestamp truncation in mysql and postgresql
John Rouillard <rouilj@ieee.org>
parents: 6802
diff changeset
63 def testUpdateTimestamp(self):
bdd28b244839 - issue2551223 - fix timestamp truncation in mysql and postgresql
John Rouillard <rouilj@ieee.org>
parents: 6802
diff changeset
64 def get_ts_via_sql(self):
bdd28b244839 - issue2551223 - fix timestamp truncation in mysql and postgresql
John Rouillard <rouilj@ieee.org>
parents: 6802
diff changeset
65 sql = '''select %(name)s_time from %(name)ss
bdd28b244839 - issue2551223 - fix timestamp truncation in mysql and postgresql
John Rouillard <rouilj@ieee.org>
parents: 6802
diff changeset
66 where %(name)s_key = '%(session)s';'''% \
bdd28b244839 - issue2551223 - fix timestamp truncation in mysql and postgresql
John Rouillard <rouilj@ieee.org>
parents: 6802
diff changeset
67 {'name': self.sessions.name,
bdd28b244839 - issue2551223 - fix timestamp truncation in mysql and postgresql
John Rouillard <rouilj@ieee.org>
parents: 6802
diff changeset
68 'session': 'random_session'}
bdd28b244839 - issue2551223 - fix timestamp truncation in mysql and postgresql
John Rouillard <rouilj@ieee.org>
parents: 6802
diff changeset
69
bdd28b244839 - issue2551223 - fix timestamp truncation in mysql and postgresql
John Rouillard <rouilj@ieee.org>
parents: 6802
diff changeset
70 self.sessions.cursor.execute(sql)
bdd28b244839 - issue2551223 - fix timestamp truncation in mysql and postgresql
John Rouillard <rouilj@ieee.org>
parents: 6802
diff changeset
71 db_tstamp = self.sessions.cursor.fetchone()
bdd28b244839 - issue2551223 - fix timestamp truncation in mysql and postgresql
John Rouillard <rouilj@ieee.org>
parents: 6802
diff changeset
72 return db_tstamp
bdd28b244839 - issue2551223 - fix timestamp truncation in mysql and postgresql
John Rouillard <rouilj@ieee.org>
parents: 6802
diff changeset
73
bdd28b244839 - issue2551223 - fix timestamp truncation in mysql and postgresql
John Rouillard <rouilj@ieee.org>
parents: 6802
diff changeset
74 # make sure timestamp is older than one minute so update will apply
bdd28b244839 - issue2551223 - fix timestamp truncation in mysql and postgresql
John Rouillard <rouilj@ieee.org>
parents: 6802
diff changeset
75 timestamp = time.time() - 62
bdd28b244839 - issue2551223 - fix timestamp truncation in mysql and postgresql
John Rouillard <rouilj@ieee.org>
parents: 6802
diff changeset
76 self.sessions.set('random_session', text='hello, world!',
bdd28b244839 - issue2551223 - fix timestamp truncation in mysql and postgresql
John Rouillard <rouilj@ieee.org>
parents: 6802
diff changeset
77 __timestamp=timestamp)
bdd28b244839 - issue2551223 - fix timestamp truncation in mysql and postgresql
John Rouillard <rouilj@ieee.org>
parents: 6802
diff changeset
78
bdd28b244839 - issue2551223 - fix timestamp truncation in mysql and postgresql
John Rouillard <rouilj@ieee.org>
parents: 6802
diff changeset
79 self.sessions.updateTimestamp('random_session')
bdd28b244839 - issue2551223 - fix timestamp truncation in mysql and postgresql
John Rouillard <rouilj@ieee.org>
parents: 6802
diff changeset
80 # this doesn't work as the rdbms backends have a
bdd28b244839 - issue2551223 - fix timestamp truncation in mysql and postgresql
John Rouillard <rouilj@ieee.org>
parents: 6802
diff changeset
81 # session_time, otk_time column and the timestamp in the
bdd28b244839 - issue2551223 - fix timestamp truncation in mysql and postgresql
John Rouillard <rouilj@ieee.org>
parents: 6802
diff changeset
82 # session marshalled payload isn't updated. The dbm
bdd28b244839 - issue2551223 - fix timestamp truncation in mysql and postgresql
John Rouillard <rouilj@ieee.org>
parents: 6802
diff changeset
83 # backend does update the __timestamp value so it works
bdd28b244839 - issue2551223 - fix timestamp truncation in mysql and postgresql
John Rouillard <rouilj@ieee.org>
parents: 6802
diff changeset
84 # for dbm.
bdd28b244839 - issue2551223 - fix timestamp truncation in mysql and postgresql
John Rouillard <rouilj@ieee.org>
parents: 6802
diff changeset
85 #self.assertNotEqual (self.sessions.get('random_session',
bdd28b244839 - issue2551223 - fix timestamp truncation in mysql and postgresql
John Rouillard <rouilj@ieee.org>
parents: 6802
diff changeset
86 # '__timestamp'),
bdd28b244839 - issue2551223 - fix timestamp truncation in mysql and postgresql
John Rouillard <rouilj@ieee.org>
parents: 6802
diff changeset
87 # timestamp)
bdd28b244839 - issue2551223 - fix timestamp truncation in mysql and postgresql
John Rouillard <rouilj@ieee.org>
parents: 6802
diff changeset
88
bdd28b244839 - issue2551223 - fix timestamp truncation in mysql and postgresql
John Rouillard <rouilj@ieee.org>
parents: 6802
diff changeset
89 # use 61 to allow a fudge factor
bdd28b244839 - issue2551223 - fix timestamp truncation in mysql and postgresql
John Rouillard <rouilj@ieee.org>
parents: 6802
diff changeset
90 self.assertGreater(get_ts_via_sql(self)[0] - timestamp, 61)
Roundup Issue Tracker: http://roundup-tracker.org/