Skip to content

Load externs concurrently#3609

Merged
garyb merged 3 commits intopurescript:masterfrom
natefaubion:concurrent-extern-loading
May 6, 2019
Merged

Load externs concurrently#3609
garyb merged 3 commits intopurescript:masterfrom
natefaubion:concurrent-extern-loading

Conversation

@natefaubion
Copy link
Copy Markdown
Contributor

@natefaubion natefaubion commented Apr 16, 2019

Previously externs were loaded serially with foldM. This means that if a change is detected farther upstream, it won't try to read downstream externs. This PR loads and parses them all concurrently, which will result in downstream results that are thrown away if there are upstream changes. However, I think this is a decent tradeoff since during development, changes generally happen downstream.

On my machine, with 551 modules compiled and a noop call to purs compile, v0.12.5 takes ~4s, while this PR takes ~2.6s, and with the parser changes as well in #3608 it takes ~1.6s.

@natefaubion
Copy link
Copy Markdown
Contributor Author

This appears to break TestUtils code that provides an exception for the output timestamp:

return (if exists then Just (P.internalError "getOutputTimestamp: read timestamp") else Nothing)

@hdgarrood
Copy link
Copy Markdown
Contributor

However, I think this is a decent tradeoff since during development, changes generally happen downstream.

I agree. I think another point in favour of this approach is that if there does happen to be a change very far upstream, the compiler will have a lot of work to do rebuilding all of the downstream modules, so you're probably going to be waiting a little while anyway.

@natefaubion
Copy link
Copy Markdown
Contributor Author

natefaubion commented Apr 20, 2019

@hdgarrood Does this look correct? Tests are passing for me at least. Another option is to give RebuildNever builds a timestamp in the past. If this were in PS I would have just used bottom 😄.

Copy link
Copy Markdown
Contributor

@hdgarrood hdgarrood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me :) I think it might be worth getting @kritzcreek to weigh in too, as he has actually worked on this code before, unlike me

@natefaubion natefaubion force-pushed the concurrent-extern-loading branch from 3d8b8f0 to 76f5a05 Compare May 5, 2019 17:26
@hdgarrood
Copy link
Copy Markdown
Contributor

Shall we merge this one now too? The same reasoning applies here as for the CST one I think - having it in master for a bit before the next release is probably the best way of testing it properly now. I don't want to put pressure on Christoph to review since he's very busy at the moment.

@garyb garyb merged commit ba37ff2 into purescript:master May 6, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants