feat: support unmaintained and unstable Ubuntu releases #238

letFunny · 2025-08-07T08:10:36Z

Have you signed the CLA?

This commits adds maintenance status to a release as the top-level field maintenance in chisel.yaml. The dates there specify the lifecycle of the release in regards to its support status, with and without Ubuntu Pro subscriptions. This enables Chisel to fail if the release and its combination of archives is no longer supported officially and it may lead to security problems. For example, interim releases like mantic and no longer officially supported, and LTSs like focal are only maintained when using ESM which needs an Ubuntu Pro subscription. If no maintained archive is available Chisel will fail, but the user can continue execution by passing --ignore=unmaintained as a CLI flag.

When the dates are into the future it indicates instead that the release is unstable. The user can choose to continue the execution by passing --ignore=unstable as a CLI flag.

Lastly, this commits also allows Chisel to work with interim releases by using the dates described above to change the archive's URL from archive.ubuntu.com to old-releases.ubuntu.com when needed.

Compatibility-wise, Chisel has a list of releases with their default dates so it continues to work with existing chisel.yaml. There will be a window when maintenance is optional for these releases and chisel-releases will be updated. Then, the field will be mandatory.

IMPORTANT: This new version of Chisel will fail to cut Ubuntu 20.04 when Ubuntu Pro is not enabled because the standard archive is unmaintained. Users need to add --ignore=unmaintained or move to a more recent release or use the Ubuntu Pro archives.

github-actions · 2025-08-07T08:14:11Z

Command	Mean [s]	Min [s]	Max [s]	Relative
`BASE`	8.782 ± 0.049	8.709	8.862	1.00
`HEAD`	8.813 ± 0.040	8.740	8.865	1.00 ± 0.01

cmd/chisel/cmd_cut.go

spread.yaml

niemeyer

The PR looks very good, thanks. There are a few UX related issues mainly, but the logic is clear and clean.

cmd/chisel/cmd_cut.go

internal/archive/archive.go

spread.yaml

letFunny · 2025-08-25T08:53:52Z

Tests are only failing because the new code expects maintenance to be set in the release but the upstream chisel-releases repository is yet to be updated.

EDIT: Discussed offline with Cris the spec, maintenance is "optional".

This reverts commit f0f4f06.

letFunny · 2025-08-28T10:56:18Z

As discussed offline and according to the new version of the spec,

We are using maintenance with dates that conform to the format in the spec.
We are using a default for existing chisel-releases based on the official dates. In the future we will remove the default and make the map mandatory. This is partly to ease the transition and partly to remove the edge cases that are needed to handle in the code (i.e. empty maintenance).
--ignore= is now used. At the moment only "umaintained" is a valid value.
Tests have been added to cover all the new changes.

letFunny · 2025-08-28T11:03:00Z

cmd/chisel/cmd_debug_check_release_archives_test.go

-			test-key:
-				id: ` + archiveKey.ID + `
-				armor: |` + "\n" + testutil.PrefixEachLine(archiveKey.PubKeyArmor, "\t\t\t\t\t\t"))
+	rawChiselYaml := testutil.Reindent(testutil.DefaultChiselYaml)


[Comment for reviewer]: I have taken the opportunity to finally unify all the defaultYaml strings which were copies of each other but needed to be updated individually every time we wanted to make a change.

Per https://git.launchpad.net/ubuntu/+source/distro-info-data/plain/ubuntu.csv

niemeyer

Looking good, thank you. Another pass, with mostly simple bits.

cmd/chisel/cmd_cut.go

niemeyer · 2025-08-29T15:43:46Z

cmd/chisel/cmd_cut.go

 		return err
 	}

+	unmaintained := release.Maintenance.EndOfLife.Before(time.Now())


Logic on the positive tends to be much easier to work with:

maintained := release.Maintenance.EndOfLife.After(time.Now())

Because we now need to check for both unstable and unmaintained I think its easier if I reverse the check instead of making it positive: time.Now().After(release.Maintenance.EndOfLife).

cmd/chisel/cmd_cut.go

niemeyer · 2025-08-29T15:51:45Z

cmd/chisel/cmd_cut.go

+			Pro:          archiveInfo.Pro,
+			CacheDir:     cache.DefaultDir("chisel"),
+			PubKeys:      archiveInfo.PubKeys,
+			Unmaintained: unmaintained,


Again should be on the positive, Maintained.

With that said, hadn't we agreed to take into account the other stages as well, as being in "legacy" but having no access to the archive is similar to being "end-of-life"?

I suppose we need Maintenance here instead, with proper enum values.

With that said, hadn't we agreed to take into account the other stages as well, as being in "legacy" but having no access to the archive is similar to being "end-of-life"?

It is my understanding the agreement was that we would add only add them as values for now, the only ones that are used are end-of-life and standard (@cjdcordeiro).

In short, we have decided to have 4 different maintenance values to reflect 5 different maintenance states (before reaching the standard date, the release is in devel).

At the moment, in https://github.com/canonical/chisel-releases/, we have the releases in the following maintenance states:

devel

standard

expanded

end-of-life

The CLI option introduced in this PR and the initial scope were aiming to address releases that have reached end-of-life (which in our cases it's only Interim releases at the moment). So this PR should at least address that situation. If it's a PR scoping issue, bare in mind that we know which maintenance fields we want to support (so an enum makes sense), but we still need to discuss the UX for the maintenance states != unmaintained

This is not even the selection we agreed, though (IIRC, it was: standard, expanded, legacy, end-of-life)?

Given the looseness of the conversation, can we please tune this PR to:

Use the terms agreed

Implement logic that we won't have backwards compatibility with

I'm guessing that this can be done without extensive changes, which would be unfortunate as this has been reviewed many times already. If that's not true, let's please sync on it before moving on.

This was a misunderstanding in the words used. We discussed it offline on Tuesday, to summarize it, we have four dates: standard, expanded, legacy and end-of-life. These four dates create different maintenance status for the release:

"devel" / "unstable": if today is before standard. It uses the same archive as "standard".

"standard": if today is after standard and before end-of-life.

"expanded" / "legacy" which are different tiers of support that require different archives, one for ESM and for legacy [1].

"unmaintained": if today is after end-of-life.

In Chisel we already have support for ESM working but that does not include the new archives for "legacy". We do not have support for the pro key nor the archive. Furthermore, based on the releases we support right now, the oldest one is 20.04 which will be available in legacy in April 2030. For all these reasons, I think we should treat it as a different feature request different from this PR. This PR will add support for "unmaintained" and "unstable" and only those.

[1]: Confirmed with the pro team.

niemeyer · 2025-08-29T15:57:56Z

internal/archive/archive.go

 	CacheDir   string
 	PubKeys    []*packet.PublicKey
+	// This archive belongs to a release that is no longer actively maintained.
+	Unmaintained bool


See earlier comments. I think we want an enum here that points out the actual status of the archive, and we need to verify that we have access to whatever "maintained" means for that specific status.

internal/setup/yaml.go

tests/use-a-custom-chisel-release/task.yaml

tests/basic/task.yaml

niemeyer

Pending sorting out the maintenance flags, mainly.

letFunny · 2025-09-10T13:59:33Z

@niemeyer ready for the final review. I summarized the discussion we had on Tuesday in this comment which defines the scope of the PR.

niemeyer

This looks very close now. Some remarks:

cmd/chisel/cmd_cut.go

internal/archive/archive.go

cmd/chisel/cmd_debug_check_release_archives.go

cmd/chisel/cmd_cut.go

letFunny · 2025-09-23T11:06:15Z

Per my offline discussion with Gustavo, we will finally separate the concerns. Muddling them up is what has been making this PR hard to follow. There are two distinct ideas:

Whether a release and an archive combination is officially maintained. For example, mantic is no longer officially maintained, but the same goes for focal if ESM is not enabled in the system. In order to know whether an archive combination is officially maintained we have to take into account the release's dates and also the pro archives and if they are enabled. We will fail by default is the user attempts to use archives which are not maintained and could lead to security problems, the user will have to use --ignore=unmaintained or use the pro archives.
Whether the release's ubuntu archive is at archive.ubuntu.com or in old-releases.ubuntu.com. The move only happens after a release's end-of-life and it is not connected to whether there is still support or not. This is needed to support interim releases like mantic after end-of-life and was the initial scope of the PR.

See also the updated PR description and the "IMPORTANT" disclaimer.

niemeyer

Thanks for the long ride. :)

Let's see how this works out in practice.

feat: support Ubuntu interim releases after end of maintenance

4622fc0

letFunny commented Aug 7, 2025

View reviewed changes

cmd/chisel/cmd_cut.go Outdated Show resolved Hide resolved

spread.yaml Outdated Show resolved Hide resolved

letFunny added 2 commits August 7, 2025 10:20

pro archive spread is manual, restore

71ec1f5

docs

ec13945

letFunny requested a review from niemeyer August 7, 2025 08:24

letFunny added the Priority Look at me first label Aug 7, 2025

cjdcordeiro mentioned this pull request Aug 7, 2025

[PROTOTYPE]: Interim bases canonical/rockcraft#909

Closed

3 tasks

niemeyer requested changes Aug 11, 2025

View reviewed changes

cmd/chisel/cmd_cut.go Outdated Show resolved Hide resolved

internal/archive/archive.go Outdated Show resolved Hide resolved

spread.yaml Outdated Show resolved Hide resolved

spread.yaml Outdated Show resolved Hide resolved

letFunny added 5 commits August 12, 2025 12:01

restore spread folders

afbb8c3

correct spread.yaml

db59cb9

change ack flag for ignore=<condition>

3ecbf38

use "maintenance" instead of a boolean

a41e51d

change chisel.yaml in existing tests

f0f4f06

letFunny added 6 commits August 25, 2025 13:53

make optional

53b7085

Revert "change chisel.yaml in existing tests"

a0e7cf8

This reverts commit f0f4f06.

add maintenance support for tests

71892d5

default chisel yaml common to all tests

a65228e

fix spread

43d1c9e

default dates

6fd6203

letFunny added 2 commits August 28, 2025 13:00

typo

7e60de3

spread, enable mantic

c8b7ee0

letFunny commented Aug 28, 2025

View reviewed changes

letFunny added 2 commits August 28, 2025 13:13

fix spread

5782703

correct dates

e922c52

Per https://git.launchpad.net/ubuntu/+source/distro-info-data/plain/ubuntu.csv

letFunny requested a review from niemeyer August 28, 2025 15:18

niemeyer requested changes Aug 29, 2025

View reviewed changes

spread, extract option into variable

d489010

change TODO to match agreement

0bcde22

niemeyer requested changes Sep 8, 2025

View reviewed changes

letFunny mentioned this pull request Sep 9, 2025

feat: add maintenance to the reference canonical/chisel-docs#22

Draft

use unset instead of empty

cc6b74c

letFunny changed the title ~~feat: support Ubuntu interim releases after end of maintenance~~ feat: support unmaintained and unstable Ubuntu releases Sep 9, 2025

letFunny added 5 commits September 9, 2025 12:43

add support for unstable

e369edb

make maintenance an enum for archives

17be3dd

change error messages to include urls

02deb1d

git forgot to add spread test

686e857

final scope of the PR

c2681a3

letFunny requested a review from niemeyer September 10, 2025 13:59

niemeyer requested changes Sep 15, 2025

View reviewed changes

cmd/chisel/cmd_cut.go Outdated Show resolved Hide resolved

internal/archive/archive.go Outdated Show resolved Hide resolved

cmd/chisel/cmd_debug_check_release_archives.go Outdated Show resolved Hide resolved

cmd/chisel/cmd_cut.go Outdated Show resolved Hide resolved

full stop

6a255a9

lengau mentioned this pull request Sep 16, 2025

Add support for EOL bases with chisel canonical/craft-parts#1159

Closed

letFunny added 3 commits September 22, 2025 14:03

archive maintenance different than release

7783156

use different flags for maintenance vs oldRelease

4f77c93

leftover from unmaintained

b31ea6c

niemeyer approved these changes Sep 29, 2025

View reviewed changes

niemeyer merged commit ecd8b8a into canonical:main Sep 29, 2025
18 checks passed

feat: support unmaintained and unstable Ubuntu releases #238

feat: support unmaintained and unstable Ubuntu releases #238

Uh oh!

Conversation

letFunny commented Aug 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

niemeyer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

letFunny commented Aug 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

letFunny commented Aug 28, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

niemeyer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

letFunny Sep 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

niemeyer left a comment

Choose a reason for hiding this comment

Uh oh!

letFunny commented Sep 10, 2025

Uh oh!

niemeyer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

letFunny commented Sep 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

niemeyer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

letFunny commented Aug 7, 2025 •

edited

Loading

github-actions bot commented Aug 7, 2025 •

edited

Loading

letFunny commented Aug 25, 2025 •

edited

Loading

letFunny Sep 9, 2025 •

edited

Loading

letFunny commented Sep 23, 2025 •

edited

Loading