Is the issue here that it's using the bundle from the virtual environment instead of whatever is the latest for this branch? Why did that cause a failure? Can you link to the logs as I didn't spot any failures?

Note there's another job in this workflow. You've only changed the job for windows. I'd expect both want to be kept in sync.
I haven't really thought it through but I'd argue that perhaps all the CI jobs in this repo should be specifying "latest".

Also cc. @Daverlo on this PR

Is the issue here that it's using the bundle from the virtual environment instead of whatever is the latest for this branch? Why did that cause a failure?

It does neither of those. It always uses the codeql-bundle-20200826 bundle. It's a hard-coded string in

Can you link to the logs as I didn't spot any failures?

https://github.com/github/codeql-action/actions/runs/328696002

Note there's another job in this workflow. You've only changed the job for windows. I'd expect both want to be kept in sync.
I haven't really thought it through but I'd argue that perhaps all the CI jobs in this repo should be specifying "latest".

The other one always uses the version from the virtual environment so it doesn't break as easily (though it could still break arbitrarily when the environment is updated). It does seem like probably both the jobs are wrong, but only one is broken.

I've updated both the Windows and Linux tests to always use the latest bundle. I think this is probably the best thing to test because it ensures new bundles work before we start using them rather than after. It also means tests won't break when the environment updates.

The Linux job previously used whatever CodeQL bundle was included in the Actions environment it was running in, meaning:

1. Whenever the CodeQL bundle is updated, the new bundle is not tested against the Python setup scripts.
2. If the environment is updated with an incompatible CodeQL bundle the CI on this repository will break.

Both of those seem like undesirable properties.

If the environment is updated with an incompatible CodeQL bundle the CI on this repository will break.

This is the part that I'm not understanding. How would the environment be updated?

The default CodeQL Bundle comes from the Actions virtual environments. They are periodically updated, getting their bundle from whatever is the latest bundle referenced in main. So what would happen is:

1. Someone updates the CodeQL Bundle with a pull request against main. This would previously be untested because the tests use the CodeQL Bundle from the environment, not the one referenced in main.
2. A new virtual environment is built using the new bundle. This is also untested because the CodeQL Action CI does not get run on virtual environment updates.
3. The new virtual environment is rolled out on ring 0 and the tests now start trying to use it for all new PRs. The best case is it's a problem with the tests and the CI on this repository is failing until the problem is fixed. The worst case is it's a problem with the new bundle that no tests were run for until after we started rolling it out.

I knew there were some changes in the process of releasing a new bundle, but I wasn't aware all this amazing work was done and now we are using the Actions virtual environments. That explains it, thanks!

Lets merge this?