You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Fix security issues with the Trusted Publishing example #8
Another suggestion is to fix the GitHub Environment name to represent the deployment target as it's meant to. I usually go for pypi and testpypi so it's obvious that uploading to both is separate.
In that example the build step happens in an entirely different job, which uploads the built asset as an artifact. Then a separate job with id-token: write downloads that artifact and posts it to PyPI.
Is there a simple pattern for this? I'm happy to go with that if it's the right thing to do.
Yes, the PyPUG guide is mine. So it already shows what I believe is best: separate jobs, environments etc.
The GH doc was authored by @woodruffw and we're trying to get more updates into it but their processes are very slow so I only hope it'll get merged this year..
GitHub's starter workflows also have a lot of outdated stuff (like pinning my action to a 4-5 year old version that results in people getting an unsupported version that doesn't have trusted publishing implemented). Fixing it is as challenging.
So this issue I filed is a part of my effort to present people who land here from Google with better defaults..
@webknjaz does #8 (comment) look good to you? If so I'll duplicate it in my other cookiecutter templates - click-app and datasette-plugin and sqlite-utils-plugin and llm-plugin.
GitHub has this thing called Deployments API. It's meant to represent project deployments into different environments. It's integrated with webhooks, and you can add the deployment information via API.
GitHub Environments is one of the UI presentations of that, quite poorly named if you ask me.
They are visible in the repository settings, where you can also configure additional protection rules. You can also have environment-scoped secrets, which are better security-wise, compared to the org- or repo-global ones. I used to use them all the time in the pre-trusted publishing world.
Nowadays, I use the required reviewers feature, which allows having a pause in the release pipeline. If you apply an environment to a job @ GHA CI/CD, it will not start said job until the conditions are met. So with the required reviewers, you don't have to babysit the pipeline. You can trigger it, and it'll safely go through the unconditional jobs. Once it reaches the publishing job, it'll pause and send notifications to all required reviewers. Those notifications will lead back to the workflow run page, presenting a button on the UI that they need to click to approve the run. This is the last gate and a chance to do some final manual checks or have some extra wait if needed. If you choose to opt out of using the protection rules, it'll just start the job right away.
When configuring the trust on the PyPI side, you're asked to type in the environment name. If entered, PyPI will match it on upload and reject any attempts not originating from jobs with that environment selected. So this is another security aspect related to the feature.
To summarize, the GitHub Environments represent deployment targets — places where you put your dists. So it's only logical to name them for what they are. This is why I'm pushing for pypi and testpypi as they represent the publishing targets. Environments don't represent processes (like release). That's a misconception I'm seeing in many places.
If you were to use GitHub Pages with the modern GHA-based publishing method, they auto-create an environment called github-pages. Heroku also creates them. Here, I've found a few in your repositories:
N.B. Technically, you don't have to create these environments via GH UI in the repo settings. They are being auto-created on first use @ GHA. However, if you were to apply the protection settings I mentioned above, that would need access to the settings of repositories where you do this. Having them configured in the repository template will not complicate things for the users. If they skip configuring the environments, it'll still work.
In addition to what I explained above, I also noticed something else that is not strictly required but is usually nice to have — in your example, building the dists happens after testing. This means that your tests are running against something else.
I like building before testing so that the tests could pick up the artifacts that are prepared for uploading and run against them instead (I even have a thing to help with that: https://github.com/re-actors/checkout-python-sdist). Though, this is a separate effort. There is no reason for making building depend on testing. Even if you don't integrate them into the testing jobs, you can still smoke-test builds in parallel with testing and the publishing job would depend on both build and tests.
Another thing to run continuously is twine check --strict. My action does that under the hood, so it's typically nicer to get alerted about problems in the dist metadata and not when publishing fails.
Activity
simonw commentedon Aug 20, 2024
I'm having trouble figuring out the recommended pattern here. Is this the best way to do. it? https://docs.github.com/en/actions/security-for-github-actions/security-hardening-your-deployments/configuring-openid-connect-in-pypi
In that example the build step happens in an entirely different job, which uploads the built asset as an artifact. Then a separate job with
id-token: writedownloads that artifact and posts it to PyPI.Is there a simple pattern for this? I'm happy to go with that if it's the right thing to do.
simonw commentedon Aug 20, 2024
Looks like that's the approach described here too: https://packaging.python.org/en/latest/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/#checking-out-the-project-and-building-distributions
webknjaz commentedon Aug 20, 2024
Yes, the PyPUG guide is mine. So it already shows what I believe is best: separate jobs, environments etc.
The GH doc was authored by @woodruffw and we're trying to get more updates into it but their processes are very slow so I only hope it'll get merged this year..
GitHub's starter workflows also have a lot of outdated stuff (like pinning my action to a 4-5 year old version that results in people getting an unsupported version that doesn't have trusted publishing implemented). Fixing it is as challenging.
So this issue I filed is a part of my effort to present people who land here from Google with better defaults..
Safer publishing mechanism, refs #8
simonw commentedon Aug 20, 2024
Here's the new workflow:
I tested it by creating a new package called
python-lib-issue-8in a private repo and publishing it to PyPI: https://pypi.org/project/python-lib-issue-8/I'm going to delete that package from PyPI now though since it's effectively useless.
simonw commentedon Aug 20, 2024
Before I delete the PyPI package this works:
simonw commentedon Aug 20, 2024
Project deleted:
simonw commentedon Aug 20, 2024
@webknjaz does #8 (comment) look good to you? If so I'll duplicate it in my other cookiecutter templates -
click-appanddatasette-pluginandsqlite-utils-pluginandllm-plugin.simonw commentedon Aug 20, 2024
Is the
if: startsWith(github.ref, 'refs/tags/')line there necessary? Mypublish.ymlworkflow already only triggers on this:simonw commentedon Aug 20, 2024
What does including the
urlpart in there actually do?I have to admit I am very confused by GitHub Actions "environments" - I don't understand what they do, or what they are for.
webknjaz commentedon Aug 20, 2024
@simonw so I recommend using specifically
pypiin the name and rendering theurlas shown. The URL is displayed in a few places on GH UI. For example, on the workflow run graph page. See the box with the publishing jobs @ https://github.com/cherrypy/cheroot/actions/runs/8786461123. Another place is the in-repo deployment pages, for example: https://github.com/cherrypy/cheroot/deployments/pypi (here you can also see that they are linked to commits and logs/CI runs).GitHub has this thing called Deployments API. It's meant to represent project deployments into different environments. It's integrated with webhooks, and you can add the deployment information via API.
GitHub Environments is one of the UI presentations of that, quite poorly named if you ask me.
They are visible in the repository settings, where you can also configure additional protection rules. You can also have environment-scoped secrets, which are better security-wise, compared to the org- or repo-global ones. I used to use them all the time in the pre-trusted publishing world.
Nowadays, I use the required reviewers feature, which allows having a pause in the release pipeline. If you apply an environment to a job @ GHA CI/CD, it will not start said job until the conditions are met. So with the required reviewers, you don't have to babysit the pipeline. You can trigger it, and it'll safely go through the unconditional jobs. Once it reaches the publishing job, it'll pause and send notifications to all required reviewers. Those notifications will lead back to the workflow run page, presenting a button on the UI that they need to click to approve the run. This is the last gate and a chance to do some final manual checks or have some extra wait if needed. If you choose to opt out of using the protection rules, it'll just start the job right away.
When configuring the trust on the PyPI side, you're asked to type in the environment name. If entered, PyPI will match it on upload and reject any attempts not originating from jobs with that environment selected. So this is another security aspect related to the feature.
To summarize, the GitHub Environments represent deployment targets — places where you put your dists. So it's only logical to name them for what they are. This is why I'm pushing for
pypiandtestpypias they represent the publishing targets. Environments don't represent processes (likerelease). That's a misconception I'm seeing in many places.If you were to use GitHub Pages with the modern GHA-based publishing method, they auto-create an environment called
github-pages. Heroku also creates them. Here, I've found a few in your repositories:N.B. Technically, you don't have to create these environments via GH UI in the repo settings. They are being auto-created on first use @ GHA. However, if you were to apply the protection settings I mentioned above, that would need access to the settings of repositories where you do this. Having them configured in the repository template will not complicate things for the users. If they skip configuring the environments, it'll still work.
webknjaz commentedon Aug 20, 2024
In addition to what I explained above, I also noticed something else that is not strictly required but is usually nice to have — in your example, building the dists happens after testing. This means that your tests are running against something else.
I like building before testing so that the tests could pick up the artifacts that are prepared for uploading and run against them instead (I even have a thing to help with that: https://github.com/re-actors/checkout-python-sdist). Though, this is a separate effort. There is no reason for making building depend on testing. Even if you don't integrate them into the testing jobs, you can still smoke-test builds in parallel with testing and the publishing job would depend on both build and tests.
Another thing to run continuously is
twine check --strict. My action does that under the hood, so it's typically nicer to get alerted about problems in the dist metadata and not when publishing fails.