issue2551203 - Add support for CORS preflight request Add support for unauthenticated CORS preflight and fix headers for CORS. client.py: pass through unauthenticated CORS preflight to rest backend. Normal rest OPTION handlers (including tracker defined extensions) can see and handle the request. make some error cases return error json with crrect mime type rather than plain text tracebacks. create new functions to verify origin and referer that filter using allowed origins setting. remove tracker base url from error message is referer is not at an allowed origin. rest.py: fix up OPTION methods handlers to include Access-Control-Allow-Methods that are the same as the Allow header. set cache to one week for all Access-Control headers for CORS preflight only. remove self.client.setHeader("Access-Control-Allow-Origin", "*") and set Access-Control-Allow-Origin to the client supplied origin if it passes allowed origin checks. Required for CORS otherwise data isn't available to caller. Set for all responses. set Vary header now includes Origin as responses can differ based on Origin for all responses. set Access-Control-Allow-Credentials to true on all responses. test_liveserver.py: run server with setting to enforce origin csrf header check run server with setting to enforce x-requested-with csrf header check run server with setting for allowed_api_origins requests now set required csrf headers test preflight request on collections check new headers and Origin is no longer '*' rewrite all compression checks to use a single method with argument to use different compression methods. Reduce a lot of code duplication and makes updating for new headers easier. test_cgi: test new error messages in client.py account for new headers test preflight and new code paths

# Auditor for patch files
# Patches should be declared as text/plain (also .py files),
# independent of what the browser says, and
# the "patch" keyword should get set automatically.

import posixpath

patchtypes = ('.diff', '.patch')
sourcetypes = ('.diff', '.patch', '.py')

def ispatch(file, types):
    return posixpath.splitext(file)[1] in types

def patches_text_plain(db, cl, nodeid, newvalues):
    if ispatch(newvalues['name'], sourcetypes):
        newvalues['type'] = 'text/plain'

def patches_keyword(db, cl, nodeid, newvalues):
    # Check whether there are any new files
    newfiles = set(newvalues.get('files',()))
    if nodeid:
        newfiles -= set(db.issue.get(nodeid, 'files'))
    # Check whether any of these is a patch
    newpatch = False
    for fileid in newfiles:
        if ispatch(db.file.get(fileid, 'name'), patchtypes):
            newpatch = True
            break
    if newpatch:
        # Add the patch keyword if its not already there
        patchid = db.keyword.lookup("patch")
        oldkeywords = []
        if nodeid:
            oldkeywords = db.issue.get(nodeid, 'keywords')
            if patchid in oldkeywords:
                # This is already marked as a patch
                return
        if 'keywords' not in newvalues:
            newvalues['keywords'] = oldkeywords
        newvalues['keywords'].append(patchid)

def init(db):
    db.file.audit('create', patches_text_plain)
    db.issue.audit('create', patches_keyword)
    db.issue.audit('set', patches_keyword)