You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Improve error message when SSH connection drops #1369
On a spotty network that resulted in a dropped connection between client and v2-provisioned EC2 instance, the error returned was: wait: remote command exited without exit status or exit signal
this message is a bit confusing and does not clearly explain that the connection was dropped.
Definition of Done
Check for this error and translate this to "Lost connection to agent."
Test for lost connection produces this error
I did some investigation into how the golang.org/x/crypto/ssh package handles unexpectedly dropped connections:
The library's example code (using a direct TCP connection) returns the same remote command exited without exit status or exit signal message, which as @ketang pointed out is the string representation of an ssh.ExitMissingError. It's possible to retrieve the underlying I/O error (e.g. a timeout) by calling client.Close(); client.Wait().
With our WebRTC backend, client.Wait() returns an io.EOF error regardless of whether the connection was closed cleanly or uncleanly. I haven't figured out yet what's causing this difference in behavior.
With our Wireguard backend, I couldn't reproduce the issue. Even disabling the physical network interface doesn't seem to drop the SSH connection -- it just hangs and comes back when the connection is available again.
So in the case of WebRTC, I don't see a way to do much better than just replacing the error message with a better one (#3131).
I think the Wireguard behavior of keeping the connection alive is better overall. But it might still be worth trying to give some indication (maybe a desktop notification?) while it's interrupted, especially if we can determine that the problem is with the user's internet connection rather than the remote end.
I suspect the desktop notification could easily become a pest in certain conditions. I think we should kick that one down the road until we get into the system tray business.
Depends on your interpretation. If the goal of this ticket is to improve that one specific error message, then yes. If the goal is to reliably give a useful message when an SSH connection drops, then it's not really done, because #3131 only affects the behavior with WebRTC, so it'll become moot when #2779 lands and we switch to Tailscale for client connections.
I figured it made sense to keep this ticket open in order to not split the discussion, but I don't mind closing it if others feel differently.
Depends on your interpretation. If the goal of this ticket is to improve that one specific error message, then yes. If the goal is to reliably give a useful message when an SSH connection drops, then it's not really done, because #3131 only affects the behavior with WebRTC, so it'll become moot when #2779 lands and we switch to Tailscale for client connections.
I figured it made sense to keep this ticket open in order to not split the discussion, but I don't mind closing it if others feel differently.
I would suggest that we close this ticket as the original requirements of this ticket are met by improving that specific error message.
But we should also have an epic or something to track the general improvements to SSH drop messages.
Activity
tjcran commentedon May 16, 2022
@kylecarbs what is returning this error message? Is this being returned from
coderdto the CLI?tjcran commentedon May 16, 2022
This is confusing and makes it harder to troubleshoot, but is not a breaking experience. It's a
nice to havefor the community launch right now.2 remaining items
ketang commentedon Jun 7, 2022
For reference in case it helps:
https://go.googlesource.com/crypto/+/master/ssh/session.go#476
dwahler commentedon Jul 22, 2022
I did some investigation into how the
golang.org/x/crypto/sshpackage handles unexpectedly dropped connections:remote command exited without exit status or exit signalmessage, which as @ketang pointed out is the string representation of anssh.ExitMissingError. It's possible to retrieve the underlying I/O error (e.g. a timeout) by callingclient.Close(); client.Wait().client.Wait()returns anio.EOFerror regardless of whether the connection was closed cleanly or uncleanly. I haven't figured out yet what's causing this difference in behavior.So in the case of WebRTC, I don't see a way to do much better than just replacing the error message with a better one (#3131).
I think the Wireguard behavior of keeping the connection alive is better overall. But it might still be worth trying to give some indication (maybe a desktop notification?) while it's interrupted, especially if we can determine that the problem is with the user's internet connection rather than the remote end.
ketang commentedon Jul 31, 2022
I suspect the desktop notification could easily become a pest in certain conditions. I think we should kick that one down the road until we get into the system tray business.
bpmct commentedon Aug 1, 2022
Once #2779 is merged, will we likely see a different behavior/error pattern?
bpmct commentedon Aug 8, 2022
@dwahler is this fixed by #3131?
dwahler commentedon Aug 9, 2022
Depends on your interpretation. If the goal of this ticket is to improve that one specific error message, then yes. If the goal is to reliably give a useful message when an SSH connection drops, then it's not really done, because #3131 only affects the behavior with WebRTC, so it'll become moot when #2779 lands and we switch to Tailscale for client connections.
I figured it made sense to keep this ticket open in order to not split the discussion, but I don't mind closing it if others feel differently.
tjcran commentedon Aug 10, 2022
I would suggest that we close this ticket as the original requirements of this ticket are met by improving that specific error message.
But we should also have an epic or something to track the general improvements to SSH drop messages.