Skip to content

Smarter make#3270

Merged
kritzcreek merged 9 commits intopurescript:masterfrom
kritzcreek:smarter-make
Apr 6, 2018
Merged

Smarter make#3270
kritzcreek merged 9 commits intopurescript:masterfrom
kritzcreek:smarter-make

Conversation

@kritzcreek
Copy link
Copy Markdown
Member

This significantly improves startup time on incremental builds with lots of already built modules.

I'll still move things around and document what's happening here, but this change can be backported to 0.11.7 as is, and I'd like people to try that and see what kind of numbers they get, and if their projects still build properly. Thanks!

@kritzcreek kritzcreek changed the title [WIP] Smarter make Smarter make Mar 7, 2018
@kritzcreek
Copy link
Copy Markdown
Member Author

I timed some builds and wrote down my results:

https://gist.github.com/kRITZCREEK/c2070885d70cdcb047866ed395a06cb7

It's a pretty clear improvement.

@kritzcreek
Copy link
Copy Markdown
Member Author

Who can I ask to take a look at this? :) I need this to go in before I can start working on only parsing module headers to build the module graph.

@garyb
Copy link
Copy Markdown
Member

garyb commented Mar 24, 2018

Unfortunately the diff is pretty much useless, I'm inclined to say :shipit: since you've been testing it for a while without any trouble (right? 😉).

else return Nothing

-- | A set of make actions that read and write modules from the given directory.
buildMakeActions
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything downwards from here moved into Make.Actions

return externs

-- | A monad for running make actions
newtype Make a = Make
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was moved into Make.Monad


(sorted, graph) <- sortModules ms

barriers <- zip (map getModuleName sorted) <$> replicateM (length ms) ((,) <$> C.newEmptyMVar <*> C.newEmptyMVar)
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The real change in this PR is this section, which is now broken down between Make.BuildPlan and here.

Instead of checking the timestamps of the dependencies of a module before building it, we now check all the timestamps once, and make a plan of which modules need to be rebuilt (BuildPlan.construct).

@kritzcreek
Copy link
Copy Markdown
Member Author

I've successfully used this for a while now, so I'm confident it works. I've added a few comments to differentiate noise from actual change in the PR. Any remaining questions?

@garyb
Copy link
Copy Markdown
Member

garyb commented Apr 5, 2018

Sorry, I meant to comment back here - I did actually read it a bit after you added the comments, and yeah it seems reasonable to me!

@kritzcreek
Copy link
Copy Markdown
Member Author

Thanks, I'll merge it then.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants