changeset 5973:fe334430ca07

issue2550919 - Anti-bot signup using 4 second delay Took the code by erik forsberg and massaged it into the core. So this is no longer needed in the tracker. Updated devel and responsive trackers to remove timestamp.py and update input field name. Docs, changes and tests complete. Hopefully these tracker changes won't cause an issue for other tests.
author John Rouillard <rouilj@ieee.org>
date Sat, 09 Nov 2019 00:30:37 -0500
parents ae35daa5baab
children 98a8509ce45c
files CHANGES.txt doc/upgrading.txt roundup/cgi/actions.py roundup/cgi/templating.py share/roundup/templates/classic/html/user.register.html share/roundup/templates/devel/extensions/timestamp.py share/roundup/templates/devel/html/user.register.html share/roundup/templates/jinja2/html/user.register.html share/roundup/templates/minimal/html/user.register.html share/roundup/templates/responsive/extensions/timestamp.py share/roundup/templates/responsive/html/user.register.html test/test_cgi.py
diffstat 12 files changed, 145 insertions(+), 68 deletions(-) [+]
line wrap: on
line diff
--- a/CHANGES.txt	Thu Nov 07 21:09:02 2019 -0500
+++ b/CHANGES.txt	Sat Nov 09 00:30:37 2019 -0500
@@ -22,6 +22,10 @@
   (Ralf Schlatterbeck)
 - issue2550926 - Original author adding a second message shouldn't set
   status to 'chatting'. See upgrading.txt for details. (John Rouillard)
+- issue2550919 - Anti-bot signup using 4 second delay. New config.ini
+  param [web] registration_delay must be set to 0 if template
+  user.register.html is not modified.  See upgrading.txt for details.
+
 Fixed:
 
 - issue2550996 - Give better error message when running with -c
--- a/doc/upgrading.txt	Thu Nov 07 21:09:02 2019 -0500
+++ b/doc/upgrading.txt	Sat Nov 09 00:30:37 2019 -0500
@@ -25,7 +25,7 @@
 
 Upgrade tracker's config.ini file
 --------------------------------------
-Once you have installed the new roundup, use:
+Once you have installed the new roundup, use::
 
   roundup-admin -i /path/to/tracker updateconfig newconfig.ini
 
@@ -41,11 +41,14 @@
 Many of the ``.html`` and ``.py`` files from Roundup that are copied
 into tracker directories have changed for Python 3 support.  If you
 wish to move an existing tracker to Python 3, you need to merge in
-those changes.  If your tracker uses the ``anydbm`` or ``mysql``
-backends, you also need to export the tracker contents using
-``roundup-admin export`` running under Python 2, and them import them
-using ``roundup-admin import`` running under Python 3, as for a
-migration to a different backend.  If using the ``sqlite`` backend,
+those changes. Also you need to make sure that locally created python
+code in the tracker is correct for Python 3.
+
+If your tracker uses the ``anydbm`` or ``mysql`` backends, you also
+need to export the tracker contents using ``roundup-admin export``
+running under Python 2, and them import them using ``roundup-admin
+import`` running under Python 3. This is detailed in the documention
+for migrating to a different backend.  If using the ``sqlite`` backend,
 you do not need to export and import, but need to delete the
 ``db/otks`` and ``db/sessions`` files when changing Python version.
 If using the ``postgresql`` backend, you do not need to export and
@@ -65,6 +68,40 @@
 the same steps as moving from 2 to 3 except using Python 3 to perform
 the export.)
 
+Rate Limit New User Registration
+--------------------------------
+
+The new user registration form can be abused by bots to allow
+automated registration for spamming. This can be limited by using the
+new ``config.ini`` ``[web]`` option called
+``registration_delay``. The default is 4 and is the number of seconds
+between the time the form was generated and the time the form is
+processed.
+
+If you do not modify the ``user.register.html`` template in your
+tracker's html directory, you *must* set this to 0. Otherwise you will
+see the error:
+
+  Form is corrupted, missing: opaqueregister.
+
+If set to 0, the rate limit check is disabled.
+
+If you want to use this, you can change your ``user.register.html``
+file to include::
+
+ <input type="hidden" name="opaqueregister" tal:attributes="value python: utils.timestamp()">
+
+The hidden input field can be placed right after the form declaration
+that starts with::
+
+   <form method="POST" onSubmit="return submit_once()"
+
+If you have applied Erik Forsberg's tracker level patch to implement
+(see: https://hg.python.org/tracker/python-dev/rev/83477f735132), you
+can back the code out of the tracker. You must change the name of the
+field in the html template to ``opaqueregistration`` from ``opaque``
+in order to use the core code.
+
 PGP mail processing
 -------------------
 
@@ -120,8 +157,8 @@
 Update userauditor.py to restrict usernames
 -------------------------------------------
 
-A username can be created with embedded commas and &lt; and &gt;
-characters. Even though the &lt; and &gt; are usually escaped when
+A username can be created with embedded commas and < and >
+characters. Even though the < and > are usually escaped when
 displayed, the embedded comma makes it difficult to edit lists of
 users as they are comma separated.
 
--- a/roundup/cgi/actions.py	Thu Nov 07 21:09:02 2019 -0500
+++ b/roundup/cgi/actions.py	Sat Nov 09 00:30:37 2019 -0500
@@ -6,6 +6,7 @@
 from roundup.cgi import exceptions, templating
 from roundup.mailgw import uidFromAddress
 from roundup.rate_limit import Gcra, RateLimit
+from roundup.cgi.timestamp import Timestamped
 from roundup.exceptions import Reject, RejectRaw
 from roundup.anypy import urllib_
 from roundup.anypy.strings import StringIO
@@ -1036,7 +1037,7 @@
             return
         return self.finishRego()
 
-class RegisterAction(RegoCommon, EditCommon):
+class RegisterAction(RegoCommon, EditCommon, Timestamped):
     name = 'register'
     permissionType = 'Register'
 
@@ -1050,6 +1051,15 @@
         if self.client.env['REQUEST_METHOD'] != 'POST':
             raise Reject(self._('Invalid request'))
 
+        # try to make sure user is not a bot by checking the
+        # hidden field opaqueregister to make sure it's at least
+        # WEB_REGISTRATION_DELAY seconds. If set to 0,
+        # disable the check.
+        delaytime = self.db.config['WEB_REGISTRATION_DELAY']
+
+        if delaytime > 0:
+            self.timecheck('opaqueregister', delaytime)
+        
         # parse the props from the form
         try:
             props, links = self.client.parsePropsFromForm(create=1)
--- a/roundup/cgi/templating.py	Thu Nov 07 21:09:02 2019 -0500
+++ b/roundup/cgi/templating.py	Sat Nov 09 00:30:37 2019 -0500
@@ -35,6 +35,8 @@
 
 from .KeywordsExpr import render_keywords_expression_editor
 
+from roundup.cgi.timestamp import pack_timestamp
+
 import roundup.anypy.random_ as random_
 try:
     import cPickle as pickle
@@ -3090,6 +3092,9 @@
     def anti_csrf_nonce(self, lifetime=None):
         return anti_csrf_nonce(self.client, lifetime=lifetime)
 
+    def timestamp(self):
+        return pack_timestamp()
+    
     def url_quote(self, url):
         """URL-quote the supplied text."""
         return urllib_.quote(url)
--- a/share/roundup/templates/classic/html/user.register.html	Thu Nov 07 21:09:02 2019 -0500
+++ b/share/roundup/templates/classic/html/user.register.html	Sat Nov 09 00:30:37 2019 -0500
@@ -12,6 +12,8 @@
       enctype="multipart/form-data"
       tal:attributes="action context/designator">
 
+<input type="hidden" name="opaqueregister"
+	 tal:attributes="value python: utils.timestamp()">
 <table class="form">
  <tr>
   <th i18n:translate="">Name</th>
--- a/share/roundup/templates/devel/extensions/timestamp.py	Thu Nov 07 21:09:02 2019 -0500
+++ /dev/null	Thu Jan 01 00:00:00 1970 +0000
@@ -1,28 +0,0 @@
-import time, struct, base64
-from roundup.cgi.actions import RegisterAction
-from roundup.cgi.exceptions import *
-
-def timestamp():
-    return base64.encodestring(struct.pack("i", time.time())).strip()
-
-def unpack_timestamp(s):
-    return struct.unpack("i",base64.decodestring(s))[0]
-
-class Timestamped:
-    def check(self):
-        try:
-            created = unpack_timestamp(self.form['opaque'].value)
-        except KeyError:
-            raise FormError("somebody tampered with the form")
-        if time.time() - created < 4:
-            raise FormError("responding to the form too quickly")
-        return True
-
-class TimestampedRegister(Timestamped, RegisterAction):
-    def permission(self):
-        self.check()
-        RegisterAction.permission(self)
-
-def init(instance):
-    instance.registerUtil('timestamp', timestamp)
-    instance.registerAction('register', TimestampedRegister)
--- a/share/roundup/templates/devel/html/user.register.html	Thu Nov 07 21:09:02 2019 -0500
+++ b/share/roundup/templates/devel/html/user.register.html	Sat Nov 09 00:30:37 2019 -0500
@@ -18,7 +18,9 @@
       enctype="multipart/form-data"
       tal:attributes="action context/designator">
 
-<input type="hidden" name="opaque" tal:attributes="value python: utils.timestamp()" />
+<input type="hidden" name="opaqueregister"
+	 tal:attributes="value python: utils.timestamp()">
+
 <table class="form">
  <tr>
   <th i18n:translate="">Name</th>
--- a/share/roundup/templates/jinja2/html/user.register.html	Thu Nov 07 21:09:02 2019 -0500
+++ b/share/roundup/templates/jinja2/html/user.register.html	Sat Nov 09 00:30:37 2019 -0500
@@ -14,6 +14,8 @@
         name ="itemSynopsis"
         enctype ="multipart/form-data"
         action ='{{  context.designator() }}'>
+    <input type="hidden" name="opaqueregister"
+           value="{{ utils.timestamp() }}" >
     <table>
       <tr>
         <th>{{ i18n.gettext('Name')|u }}</th>
--- a/share/roundup/templates/minimal/html/user.register.html	Thu Nov 07 21:09:02 2019 -0500
+++ b/share/roundup/templates/minimal/html/user.register.html	Sat Nov 09 00:30:37 2019 -0500
@@ -19,6 +19,8 @@
 
 <tal:block tal:condition="editok">
 <form method="POST" onSubmit="return submit_once()" enctype="multipart/form-data">
+<input type="hidden" name="opaqueregister"
+       tal:attributes="value python: utils.timestamp()" >
 <input type="hidden" name=":template" value="register">
 <input type="hidden" name=":required" value="username">
 <input type="hidden" name=":required" value="password">
--- a/share/roundup/templates/responsive/extensions/timestamp.py	Thu Nov 07 21:09:02 2019 -0500
+++ /dev/null	Thu Jan 01 00:00:00 1970 +0000
@@ -1,28 +0,0 @@
-import time, struct, base64
-from roundup.cgi.actions import RegisterAction
-from roundup.cgi.exceptions import *
-
-def timestamp():
-    return base64.encodestring(struct.pack("i", time.time())).strip()
-
-def unpack_timestamp(s):
-    return struct.unpack("i",base64.decodestring(s))[0]
-
-class Timestamped:
-    def check(self):
-        try:
-            created = unpack_timestamp(self.form['opaque'].value)
-        except KeyError:
-            raise FormError("somebody tampered with the form")
-        if time.time() - created < 4:
-            raise FormError("responding to the form too quickly")
-        return True
-
-class TimestampedRegister(Timestamped, RegisterAction):
-    def permission(self):
-        self.check()
-        RegisterAction.permission(self)
-
-def init(instance):
-    instance.registerUtil('timestamp', timestamp)
-    instance.registerAction('register', TimestampedRegister)
--- a/share/roundup/templates/responsive/html/user.register.html	Thu Nov 07 21:09:02 2019 -0500
+++ b/share/roundup/templates/responsive/html/user.register.html	Sat Nov 09 00:30:37 2019 -0500
@@ -18,7 +18,8 @@
       enctype="multipart/form-data"
       tal:attributes="action context/designator">
 
-<input type="hidden" name="opaque" tal:attributes="value python: utils.timestamp()" />
+<input type="hidden" name="opaqueregister"
+       tal:attributes="value python: utils.timestamp()" >
 <table class="form">
  <tr>
   <th i18n:translate="">Name</th>
--- a/test/test_cgi.py	Thu Nov 07 21:09:02 2019 -0500
+++ b/test/test_cgi.py	Sat Nov 09 00:30:37 2019 -0500
@@ -14,7 +14,7 @@
 import pytest
 
 from roundup.cgi import client, actions, exceptions
-from roundup.cgi.exceptions import FormError, NotFound
+from roundup.cgi.exceptions import FormError, NotFound, Redirect
 from roundup.exceptions import UsageError
 from roundup.cgi.templating import HTMLItem, HTMLRequest, NoTemplate
 from roundup.cgi.templating import HTMLProperty, _HTMLItem, anti_csrf_nonce
@@ -22,6 +22,8 @@
 from roundup import init, instance, password, hyperdb, date
 from roundup.anypy.strings import StringIO, u2s, b2s
 
+from time import sleep
+
 # For testing very simple rendering
 from roundup.cgi.engine_zopetal import RoundupPageTemplate
 
@@ -1540,6 +1542,72 @@
         k = self.db.keyword.getnode('2')
         self.assertEqual(k.name, 'newkey2')
 
+    def testRegisterAction(self):
+        from roundup.cgi.timestamp import pack_timestamp
+
+        # need to set SENDMAILDEBUG to prevent
+        # downstream issue when email is sent on successful
+        # issue creation. Also delete the file afterwards
+        # just tomake sure that someother test looking for
+        # SENDMAILDEBUG won't trip over ours.
+        if 'SENDMAILDEBUG' not in os.environ:
+            os.environ['SENDMAILDEBUG'] = 'mail-test1.log'
+        SENDMAILDEBUG = os.environ['SENDMAILDEBUG']
+
+        
+        # missing opaqueregister
+        cl = self._make_client({'username':'new_user1', 'password':'secret',
+                 '@confirm@password':'secret', 'address':'new_user@bork.bork'},
+                                nodeid=None, userid='2')
+        with self.assertRaises(FormError) as cm:
+            actions.RegisterAction(cl).handle()
+        self.assertEqual(cm.exception.args,
+                    ('Form is corrupted, missing: opaqueregister.',))
+
+        # broken/invalid opaqueregister
+        # strings chosen to generate:
+        #   binascii.Error Incorrect padding
+        #   struct.error requires a string argument of length 4
+        cl = self._make_client({'username':'new_user1',
+                                'password':'secret',
+                                '@confirm@password':'secret',
+                                'address':'new_user@bork.bork',
+                                'opaqueregister': 'zzz' },
+                               nodeid=None, userid='2')
+        with self.assertRaises(FormError) as cm:
+            actions.RegisterAction(cl).handle()
+        self.assertEqual(cm.exception.args, ('Form is corrupted.',))
+
+        cl = self._make_client({'username':'new_user1',
+                                'password':'secret',
+                                '@confirm@password':'secret',
+                                'address':'new_user@bork.bork',
+                                'opaqueregister': 'xyzzyzl=' },
+                               nodeid=None, userid='2')
+        with self.assertRaises(FormError) as cm:
+            actions.RegisterAction(cl).handle()
+        self.assertEqual(cm.exception.args, ('Form is corrupted.',))
+
+        # valid opaqueregister
+        cl = self._make_client({'username':'new_user1', 'password':'secret',
+                 '@confirm@password':'secret', 'address':'new_user@bork.bork',
+                                'opaqueregister': pack_timestamp() },
+                               nodeid=None, userid='2')
+        # submitted too fast, so raises error
+        with self.assertRaises(FormError) as cm:
+            actions.RegisterAction(cl).handle()
+        self.assertEqual(cm.exception.args,
+                    ('Responding to form too quickly.',))
+
+        sleep(4.1) # sleep as requested so submit will take long enough
+        self.assertRaises(Redirect, actions.RegisterAction(cl).handle)
+
+        # FIXME check that email output makes sense at some point
+        
+        # clean up from email log
+        if os.path.exists(SENDMAILDEBUG):
+            os.remove(SENDMAILDEBUG)
+
     def testserve_static_files(self):
         # make a client instance
         cl = self._make_client({})

Roundup Issue Tracker: http://roundup-tracker.org/