annotate .github/workflows/codeql-analysis.yml @ 6433:c1d3fbcdbfbd

issue2551142 - Import of retired node ... unique constraint failure. Title: Import of retired node with username after active node fails with unique constraint failure. More fixes needed for mysql and postgresql. mysql: add unique constraint for (keyvalue, __retired__) when creating class in the database. On schema change if class is changed, remove the unique constraint too. upgrade version of rdbms database from 5 to 6 to add constraint to all version 5 databases that were created as version 5 and didn't get the unique constraint. Make no changes on version 5 databases upgraded from version 4, the upgrade process to 5 added the constraint. Make no changes to other databases (sqlite, postgres) during upgrade from version 5 to 6. postgres: Handle the exception raised on unique constraint violation. The exception invalidates the database connection so it can't be used to recover from the exception. Added two new database methods: checkpoint_data - performs a db.commit under postgres does nothing on other backends restore_connection_on_error - does a db.rollback on postgres, does nothing on other backends with the rollback() done on the connection I can use the database connection to fixup the import that failed on the unique constraint. This makes postgres slower but without the commit after every imported object, the rollback will delete all the entries done up to this point. Trying to figure out how to make the caller do_import batch and recover from this failure is beyond me. Also dismissed having to process the export csv file before importing. Pushing that onto a user just seems wrong. Also since import/export isn't frequently done the lack of surprise on having a failing import and reduced load/frustration for the user seems worth it. Also the import can be run in verbose mode where it prints out a row as it is processed, so it may take a while, ut the user can get feedback. db_test-base.py: add test for upgrade from 5 to 6.
author John Rouillard <rouilj@ieee.org>
date Thu, 10 Jun 2021 12:52:05 -0400
parents 9972e26ab140
children 432edc4d8123
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
6286
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
1 # For most projects, this workflow file will not need changing; you simply need
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
2 # to commit it to your repository.
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
3 #
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
4 # You may wish to alter this file to override the set of languages analyzed,
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
5 # or to provide custom queries or build logic.
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
6 #
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
7 # ******** NOTE ********
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
8 # We have attempted to detect the languages in your repository. Please check
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
9 # the `language` matrix defined below to confirm you have the correct set of
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
10 # supported CodeQL languages.
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
11 # ******** NOTE ********
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
12
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
13 name: "CodeQL"
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
14
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
15 on:
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
16 push:
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
17 branches: [ master ]
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
18 pull_request:
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
19 # The branches below must be a subset of the branches above
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
20 branches: [ master ]
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
21 schedule:
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
22 - cron: '28 17 * * 1'
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
23
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
24 jobs:
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
25 analyze:
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
26 name: Analyze
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
27 runs-on: ubuntu-latest
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
28
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
29 strategy:
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
30 fail-fast: false
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
31 matrix:
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
32 language: [ 'javascript', 'python' ]
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
33 # CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python' ]
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
34 # Learn more...
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
35 # https://docs.github.com/en/github/finding-security-vulnerabilities-and-errors-in-your-code/configuring-code-scanning#overriding-automatic-language-detection
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
36
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
37 steps:
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
38 - name: Checkout repository
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
39 uses: actions/checkout@v2
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
40
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
41 # Initializes the CodeQL tools for scanning.
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
42 - name: Initialize CodeQL
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
43 uses: github/codeql-action/init@v1
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
44 with:
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
45 languages: ${{ matrix.language }}
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
46 # If you wish to specify custom queries, you can do so here or in a config file.
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
47 # By default, queries listed here will override any specified in a config file.
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
48 # Prefix the list here with "+" to use these queries and those in the config file.
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
49 # queries: ./path/to/local/query, your-org/your-repo/queries@main
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
50
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
51 # Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
52 # If this step fails, then you should remove it and run the build manually (see below)
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
53 - name: Autobuild
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
54 uses: github/codeql-action/autobuild@v1
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
55
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
56 # â„šī¸ Command-line programs to run using the OS shell.
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
57 # 📚 https://git.io/JvXDl
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
58
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
59 # âœī¸ If the Autobuild fails above, remove it and uncomment the following three lines
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
60 # and modify them (or add more) to build your code if your project
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
61 # uses a compiled language
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
62
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
63 #- run: |
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
64 # make bootstrap
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
65 # make release
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
66
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
67 - name: Perform CodeQL Analysis
9972e26ab140 add security analysis workflow
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
68 uses: github/codeql-action/analyze@v1

Roundup Issue Tracker: http://roundup-tracker.org/