"docker-java-stream" thread stuck at com.github.dockerjava.okhttp.UnixDomainSocket.recv() #1475
Description
After IntelliJ IDEs had switched from Netty to OkHttp transport PyCharm (being one of them) started to hang periodically during the "Launching skeleton generator" activity started for Docker-image based Python interpreters (see the issue https://youtrack.jetbrains.com/issue/PY-44994).
Investigation of the problem led to the point where docker-java-stream thread is being blocked during the attempt to read new bytes from UNIX domain socket:
"docker-java-stream--1353262400@39058" daemon prio=4 tid=0xdb nid=NA runnable
java.lang.Thread.State: RUNNABLE
at com.github.dockerjava.okhttp.UnixDomainSocket.recv(UnixDomainSocket.java:-1)
at com.github.dockerjava.okhttp.UnixDomainSocket$UnixSocketInputStream.read(UnixDomainSocket.java:240)
at java.io.FilterInputStream.read(FilterInputStream.java:133)
at com.github.dockerjava.okhttp.UnixSocketFactory$1$1.read(UnixSocketFactory.java:45)
at okio.Okio$2.read(Okio.java:140)
at okio.AsyncTimeout$2.read(AsyncTimeout.java:237)
at okio.RealBufferedSource.indexOf(RealBufferedSource.java:358)
at okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.java:230)
at okhttp3.internal.http1.Http1ExchangeCodec.readHeaderLine(Http1ExchangeCodec.java:242)
at okhttp3.internal.http1.Http1ExchangeCodec.readHeaders(Http1ExchangeCodec.java:251)
at okhttp3.internal.http1.Http1ExchangeCodec.readResponseHeaders(Http1ExchangeCodec.java:219)
at okhttp3.internal.connection.Exchange.readResponseHeaders(Exchange.java:115)
at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:94)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
at com.github.dockerjava.okhttp.HijackingInterceptor.intercept(HijackingInterceptor.java:20)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:43)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:94)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:88)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:229)
at okhttp3.RealCall.execute(RealCall.java:81)
at com.github.dockerjava.okhttp.OkDockerHttpClient$OkResponse.<init>(OkDockerHttpClient.java:251)
at com.github.dockerjava.okhttp.OkDockerHttpClient.execute(OkDockerHttpClient.java:225)
at com.github.dockerjava.core.DefaultInvocationBuilder.execute(DefaultInvocationBuilder.java:228)
at com.github.dockerjava.core.DefaultInvocationBuilder.lambda$executeAndStream$1(DefaultInvocationBuilder.java:269)
at com.github.dockerjava.core.DefaultInvocationBuilder$$Lambda$2717.1081146579.run(Unknown Source:-1)
at java.lang.Thread.run(Thread.java:834)
The closer look at com.github.dockerjava.okhttp.UnixDomainSocket.UnixSocketInputStream.read(byte[], int, int) method revealed the cause of the problem. There are two branches based on off > 0 condition. recv() method is executed in a loop in the first one. As recv() method is blocking after reading all available data it waits for the new one. But at this point, all data might have been already sent by Docker daemon. In this case the waiting becomes endless.