Skip to content

Use volatile referneces to prevent double creation of Ryuk containers - #2929

Closed
aguibert wants to merge 1 commit into
testcontainers:masterfrom
aguibert:ryuk-client-sync
Closed

Use volatile referneces to prevent double creation of Ryuk containers#2929
aguibert wants to merge 1 commit into
testcontainers:masterfrom
aguibert:ryuk-client-sync

Conversation

@aguibert

Copy link
Copy Markdown
Contributor

I believe this fixes #2395

According to my logs for single-container usage, the docker client is being obtained twice:

[06/21/2020 19:07:33:948 UTC] 001 DockerClientFactory            client                         I Docker host IP address is <remote machine>
[06/21/2020 19:07:33:989 UTC] 001 DockerClientFactory            client                         I Connected to docker: 
  Server Version: 19.03.8
  API Version: 1.40
  Operating System: Ubuntu 18.04.1 LTS
  Total Memory: 16039 MB
[06/21/2020 19:07:34:022 UTC] 001 RegistryAuthLocator            lookupUncachedAuthConfig       W Failure when attempting to lookup auth config (dockerImageName: quay.io/testcontainers/ryuk:0.2.3, configFile: /home/jazz_build/.docker/config.json. Falling back to docker-java default behaviour. Exception message: /home/jazz_build/.docker/config.json (No such file or directory)
[06/21/2020 19:07:34:164 UTC] 001 DockerClientFactory            client                         I Docker host IP address is <remote machine>
[06/21/2020 19:07:34:186 UTC] 001 DockerClientFactory            client                         I Connected to docker: 
  Server Version: 19.03.8
  API Version: 1.40
  Operating System: Ubuntu 18.04.1 LTS
  Total Memory: 16039 MB

@rnorth

rnorth commented Jun 24, 2020

Copy link
Copy Markdown
Member

Ah, this would make sense. I wonder if @bsideup's #2882, which is also touching DockerClientFactory, will solve this. I think Lombok's lazy Getter should take care of race conditions on the dockerClient field by wrapping it in an atomic reference, so perhaps this problem goes away?


@Synchronized
private DockerClientProviderStrategy getOrInitializeStrategy() {
private synchronized DockerClientProviderStrategy getOrInitializeStrategy() {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Already @Synchronized

Suggested change
private synchronized DockerClientProviderStrategy getOrInitializeStrategy() {
private DockerClientProviderStrategy getOrInitializeStrategy() {

@bsideup

bsideup commented Jun 24, 2020

Copy link
Copy Markdown
Member

Since we synchronize our access to the fields, volatile should be unnecessary:
https://shipilev.net/blog/2016/close-encounters-of-jmm-kind/#myth-sync-vs-volatiles

@aguibert do you think you can provide a reproducer (e.g. by starting multiple threads and triggering the resolve)?

@aguibert

Copy link
Copy Markdown
Contributor Author

I can't easily create a reproducer for this -- it happens pretty intermittently.

Thanks for pointing out the article, I did a bit more research on volatile vs. synchronized and realized that synchrnozed also synchronizes the thread memory and main memory for all variables, in addition to locking. Given that, I'll go ahead and close this PR as it will not fix the problem I was trying to solve

@aguibert aguibert closed this Jun 24, 2020
@bsideup

bsideup commented Jun 24, 2020

Copy link
Copy Markdown
Member

@aguibert I will try to run it with JCStress then. Thanks for confirming that it is happening on your environment! At least we will be able to verify whether the fix worked or not :)

@aguibert

Copy link
Copy Markdown
Contributor Author

yw, and thanks for being so helpful as always @bsideup! Just to confirm I've got this right, do you think that #2930 will fix the issue? If so I will watch for that release and upgrade my env to use it

@bsideup

bsideup commented Jun 25, 2020

Copy link
Copy Markdown
Member

@aguibert I am afraid it won't fix the double Ryuk issue, but I am currently trying to reproduce it, so that we can finally fix the problem (although without any luck so far :( )

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ryuk container name already in use

3 participants