It'd be nice if we could use SSH PSRemote sessions with our configured session ssh-agent-compatible daemon to get the required security information/

For instance, I use gpg-agent to store my SSH private key securely on my Yubikey 4, as a result my key doesn't actually exist on my laptop.

From looking at the output of ps while New-PSSession is run, it appears you guys are shelling out to ssh anyway, and the ssh client supports the ssh-agent by default, so I assume something is being done to disable it for the invocation.