Skip to content

expand pip install#1075

Merged
djherbis merged 7 commits into
Kaggle:mainfrom
Borda:pip/install-default
Jan 26, 2023
Merged

expand pip install#1075
djherbis merged 7 commits into
Kaggle:mainfrom
Borda:pip/install-default

Conversation

@Borda

@Borda Borda commented Sep 20, 2021

Copy link
Copy Markdown
Contributor

Description

this allows pip to resolve eventual version conflicts within particular layers
as it was now, there was no guarantee that the next installed package would break the previous installation (for example, min NumPy version)

Follow-up

I would suggest also doing it in the remaining dockers.
Moreover, reducing duplication may simplify the readability, aka define required packages in extra files and install them within the docker build...
(if agreed, I am happy to make both suggestions 🐰 )

@Borda

Borda commented Sep 20, 2021

Copy link
Copy Markdown
Contributor Author

is there a way how to check why the build is failing?
maybe #1077 could help here 🐰

Comment thread Dockerfile Outdated
@rosbo

rosbo commented Sep 23, 2021

Copy link
Copy Markdown
Contributor

I'm currently moving our build to a templatized build and will incorporate this idea after I merge #1078

Comment thread Dockerfile Outdated
@Borda

Borda commented Sep 23, 2021

Copy link
Copy Markdown
Contributor Author

I'm currently moving our build to a templatized build and will incorporate this idea after I merge #1078

@rosbo I a not sure why it would not be compatible, the idea is to decouple the packages/version from the Docker structure so it will be lighter and easier to compare differences...

@Borda Borda requested a review from rosbo August 26, 2022 19:20
@rosbo rosbo requested review from djherbis and removed request for rosbo August 26, 2022 19:21
@Borda

Borda commented Jan 8, 2023

Copy link
Copy Markdown
Contributor Author

@rosbo is there an interest in finishing this one, if yes, I am happy to resolve conflicts and continue... 🐰

@rosbo

rosbo commented Jan 24, 2023

Copy link
Copy Markdown
Contributor

Looping in @djherbis who is now responsible to manage this docker image / repo.

@djherbis

Copy link
Copy Markdown
Contributor

@Borda I think this is a good change, though I think the main branch has drifted a decent amount from where this is.
If you can resolve the conflicts, feel free to ping me and I'll review.
I think this is a good step, I think the only concern I might have is if it might slow installation time if it has to do complex version matching but happy to try.

@Borda

Borda commented Jan 25, 2023

Copy link
Copy Markdown
Contributor Author

If you can resolve the conflicts, feel free to ping me and I'll review.

yes, happy to do it (that was my concern if there is interest in this direction so the work wont land flat 🐿️ )

@Borda Borda force-pushed the pip/install-default branch from f9fced9 to fc7b613 Compare January 25, 2023 11:26
@Borda Borda requested review from rosbo and removed request for djherbis January 25, 2023 11:26
Comment thread Dockerfile.tmpl Outdated
@Borda

Borda commented Jan 25, 2023

Copy link
Copy Markdown
Contributor Author

@djherbis just cleaned out the PR - reset to head and add all changes from the ground, so it shall be fine now... just curious is there a CI to verify these changes? as discussed in #1077 and #1078 🐿️

Comment thread Dockerfile.tmpl Outdated
@djherbis

Copy link
Copy Markdown
Contributor

@Borda There is a CI, however it's private so only Kaggle sees the output.
I'll let you know when/if CI passes.

We should eventually try to get the build to work on a public CI as well.

@Borda

Borda commented Jan 25, 2023

Copy link
Copy Markdown
Contributor Author

I'll let you know when/if CI passes.

cool, thanks 🐰

@Borda Borda requested review from djherbis and removed request for rosbo January 25, 2023 17:00
Comment thread Dockerfile.tmpl Outdated
@Borda Borda requested a review from rosbo January 26, 2023 09:18
@Borda Borda requested review from djherbis and rosbo and removed request for djherbis and rosbo January 26, 2023 09:18
@djherbis djherbis merged commit eca5485 into Kaggle:main Jan 26, 2023
@djherbis

Copy link
Copy Markdown
Contributor

Thanks for all your hard work @Borda everything passed :) great work!

@Borda Borda deleted the pip/install-default branch January 26, 2023 18:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants