view scripts/dump_dbm_sessions_db.py @ 6823:fe0091279f50

Refactor session db logging and key generation for sessions/otks While I was working on the redis sessiondb stuff, I noticed that log_wanrning, get_logger ... was duplicated. Also there was code to generate a unique key for otks that was duplicated. Changes: creating new sessions_common.py and SessionsCommon class to provide methods: log_warning, log_info, log_debug, get_logger, getUniqueKey getUniqueKey method is closer to the method used to make session keys in client.py. sessions_common.py now report when random_.py chooses a weak random number generator. Removed same from rest.py. get_logger reconciles all logging under roundup.hyperdb.backends.<name of BasicDatabase class> some backends used to log to root logger. have BasicDatabase in other sessions_*.py modules inherit from SessionCommon. change logging to use log_* methods. In addition: remove unused imports reported by flake8 and other formatting changes modify actions.py, rest.py, templating.py to use getUniqueKey method. add tests for new methods test_redis_session.py swap out ModuleNotFoundError for ImportError to prevent crash in python2 when redis is not present. allow injection of username:password or just password into redis connection URL. set pytest_redis_pw envirnment variable to password or user:password when running test.
author John Rouillard <rouilj@ieee.org>
date Sun, 07 Aug 2022 01:51:11 -0400
parents 1188bb423f92
children 9ba04f37896f
line wrap: on
line source

#! /usr/bin/env python3
"""Usage: dump_dbm_sessions_db.py [filename]

Simple script to dump the otks and sessions dbm databases.  Dumps
sessions db in current directory if no argument is given.

Dump format:

   key: <timestamp> data

where <timestamp> is the human readable __timestamp decoded from the
data object. Data object is dumped in json format. With pretty print

   key:
     <timestamp>
       {
          key: val,
          ...
       }

if data is not a python object, print will be key: data or
   key:
     data

if pretty printed.
"""

import argparse, dbm, json, marshal, os, sys
from datetime import datetime

def indent(text, amount, ch=" "):
  """ Found at: https://stackoverflow.com/a/8348914
  """
  padding = amount * ch
  return ''.join(padding+line for line in text.splitlines(True))

def print_marshal(k):
  d = marshal.loads(db[k])
  try:
    t = datetime.fromtimestamp(d['__timestamp'])
  except (KeyError, TypeError):
    # TypeError raised if marshalled data is not a dict (list, tuple etc)
    t = "no_timestamp"
  if args.pretty:
    print("%s:\n  %s\n%s"%(k, t, indent(json.dumps(
      d, sort_keys=True, indent=4), 4)))
  else:
    print("%s: %s %s"%(k, t, d))

def print_raw(k):
  if args.pretty:
    print("%s:\n  %s"%(k, db[k]))
  else:
    print("%s: %s"%(k, db[k]))

parser = argparse.ArgumentParser(
  description='Dump DBM files used by Roundup in storage order.')
parser.add_argument('-k', '--key', action="append",
    help='dump the entry for a key, can be used multiple times.')
parser.add_argument('-K', '--keysonly', action='store_true',
    help='print the database keys, sorted in byte order.')
parser.add_argument('-p', '--pretty', action='store_true',
    help='pretty print the output rather than printing on one line.')
parser.add_argument('file', nargs='?',
                    help='file to be dumped ("sessions" if not provided)')
args = parser.parse_args()

if args.file:
  file = args.file
else:
  file="sessions"

try:
   db = dbm.open(file)
except Exception as e:
  print("Unable to open database for %s: %s"%(file, e))
  try:
    os.stat(file)
    print("  perhaps file is invalid or was created with a different version of Python?")
  except OSError:
    # the file does exist on disk.
    pass
  exit(1)

if args.keysonly:
  for k in sorted(db.keys()):
    print("%s"%k)
  exit(0)

if args.key:
  for k in args.key:
    try:
      print_marshal(k)
    except (ValueError):
      print_raw(k)
  exit(0)

k = db.firstkey()
while k is not None:
  try:
    print_marshal(k)
  except (ValueError):  # ValueError marshal.loads failed
    print_raw(k)

  k = db.nextkey(k)

Roundup Issue Tracker: http://roundup-tracker.org/