Skip to content

OpenSSL sockets spin indefinitely on timeout in handshake #549

@robinmessage

Description

@robinmessage

OpenSSL sockets spin in a variety of ways (generally in an epoll wrapper) if there is a timeout during the handshake (might be other causes too but that seemed to cause the problem reliably.)

The culprit seems to be line 416 of SSLSocket.java, which is the only place the reads the return value of flushData. flushData returns true if there is no data left in the buffer - however, in the default case of an empty buffer, as far as I can see, this is going to cause an infinite loop.

I patched this on our production system by reversing the return value from flushData - so it returns true if there is still data in the buffer. This completely fixed our problem.

I suggest just swapping the return values from flushData, but not sure if that fits the bigger picture correctly.

diff --git a/src/org/jruby/ext/openssl/SSLSocket.java b/src/org/jruby/ext/openssl/SSLSocket.java
index b692528..579ee5c 100644
--- a/src/org/jruby/ext/openssl/SSLSocket.java
+++ b/src/org/jruby/ext/openssl/SSLSocket.java
@@ -448,9 +448,9 @@ public class SSLSocket extends RubyObject {
             throw ioe;
         }
         if (netData.hasRemaining()) {
-            return false;
-        }  else {
             return true;
+        }  else {
+            return false;
         }
     }

-- 

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions