Skip to content

Conversation

@letFunny
Copy link
Collaborator

@letFunny letFunny commented Jun 4, 2024

  • Have you signed the CLA?

Streamline slicer.go moving all of the non essential functionality to self-contained function / abstractions and adding comments where necessary. Most importantly, combine all maps that recorded information about paths into one (knownPaths, pathInfos and pathUntil) and record the information in the same place instead of during several stages.

@letFunny letFunny force-pushed the refactor-slicer-steps branch from 5169ba3 to e207761 Compare June 4, 2024 09:29
@letFunny letFunny mentioned this pull request Jun 4, 2024
1 task
@letFunny letFunny force-pushed the refactor-slicer-steps branch from ad0bb03 to 9905560 Compare June 4, 2024 10:20
@letFunny letFunny force-pushed the refactor-slicer-steps branch from 9905560 to 6b60e86 Compare June 4, 2024 10:31
targetDirAbs = filepath.Join(dir, targetDir)
}

packages, err := fetchPackages(options)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Comment for reviewer]: Before, we were doing package fetching in two steps, checking if the package existed and then fetching it.

It makes sense for the refactor to consolidate both for the sake of readability but, apart from that reason, we also need this function for the chisel.db and for the FIPS PRs. The reason for the former is that we need a single place were we have all the data about packages and archives in order to include that information in the db. The reason for the latter is that we have to do more logic for archive selection and it is far simpler if that logic is all in one place.

}
sort.Slice(extractInfos, func(i, j int) bool {
return strings.Compare(extractInfos[i].Path, extractInfos[j].Path) < 0
return extractInfos[i].Path < extractInfos[j].Path
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Comment for reviewer]: Per the go compile source code:

	// NOTE(rsc): This function does NOT call the runtime cmpstring function,
	// because we do not want to provide any performance justification for
	// using strings.Compare. Basically no one should use strings.Compare.

extract[slice.Package] = extractPackage
}
arch := archives[slice.Package].Options().Arch
archiveName := options.Selection.Release.Packages[slice.Package].Archive
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Comment for reviewer]: The archive map from archive name to archive was redundant and was only used to get the architecture. When we add support for FIPS we will return the architecture as part of the package data but for now I prefer to keep it like this.

@letFunny letFunny added Simple Nice for a quick look on a minute or two Priority Look at me first labels Jun 4, 2024
@cjdcordeiro cjdcordeiro requested a review from niemeyer June 10, 2024 08:03
}
done[relPath] = true
addKnownPath(relPath)
addParents(knownPaths, relPath)
Copy link
Contributor

@niemeyer niemeyer Jun 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been looking at this PR for a while, but I have to say I'm quite uncomfortable with the diff as is. It's not that it is wrong, but rather that it is difficult to understand if it is wrong or not. Major pieces are being moved around, which was the plan, but also once again we have semantic changes that were not supposed to be bundled with a major move, as it makes reviewing hard. For example, here we used to add a known path and all its parents, but now we're simply adding parents. Why? Elsewhere we have a reviewer note claiming we now do not trust parents to have been added because maybe they weren't, but that was an impossible situation on the current design because we always add known paths together with their own parents, thus every time you hit a parent you know their preceding parents have also been added.

Can we please take a step back and do as we discussed before: major code reorganizations should be done independently from semantic changes, otherwise we can easily introduce hard to track bugs as has happened recently elsewhere.

Then, there are minor-looking changes that are actually bundled here for no good reason. For example, it modifies where the architecture setting is looked at. Why? How's that related to splitting slicer steps?

We need to take a step back here. If this is about moving major parts of some complex logic around, let's please not touch anything else while doing this, and let's try to preserve what is being done as much as possible, so our brains can process this in "same thing but elsewhere" mode. If you want to change other details later, that's fine of course, but then we'll have a chance of reviewing the changes as actual changes.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per the discussion we had today I will remove all the changes to the logic, namely:

  • Combine adding the parent paths to known paths and the path data into one function f4dbed4.
  • Remove the fetchArchives function and do it in a separate PR later 25d52ac.
  • Bring back the archives map to get the arch 6ea272c.

@letFunny letFunny requested a review from niemeyer June 18, 2024 09:25
Copy link
Contributor

@niemeyer niemeyer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you. Only one rename and we can merge it.


// removeUntilMutate removes entries marked with until: mutate. A path is marked
// only when all slices that refer to the path mark it with until: mutate.
func removeUntilMutate(rootDir string, knownPaths map[string]pathData) error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/removeUntilMutate/removeAfterMutate/

At the moment it's reading reversed: it's actually removing now that files have already been mutated, instead of before (and until) they are mutated.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 7c4efa2.

CheckWrite: checkWrite,
CheckRead: checkRead,
CheckWrite: checker.checkMutable,
CheckRead: checker.checkKnown,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure we want to break away from the original names, but we can roll with that for now and see how far we get.

@niemeyer niemeyer merged commit da3152c into canonical:main Jun 18, 2024
@letFunny letFunny deleted the refactor-slicer-steps branch October 17, 2024 08:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Priority Look at me first Simple Nice for a quick look on a minute or two

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants