For jruby-9.0.1.0 (discovered on 9.0.0.0). I don't know if there are other characters that trigger this.

E.g.
jruby-9.0.1.0 :001 > require 'pathname'
=> true
jruby-9.0.1.0 :002 > x = "joe"
=> "joe"
jruby-9.0.1.0 :003 > y = "joe/pete!/bob"
=> "joe/pete!/bob"
jruby-9.0.1.0 :004 > Pathname.new(y).relative_path_from(Pathname.new(x))
ArgumentError: different prefix: "joe/pete!/" and "joe"
from /Users/hding2/.rvm/rubies/jruby-9.0.1.0/lib/ruby/stdlib/pathname.rb:508:in relative_path_from' from (irb):4:in'
from org/jruby/RubyKernel.java:978:in eval' from org/jruby/RubyKernel.java:1291:inloop'
from org/jruby/RubyKernel.java:1098:in catch' from org/jruby/RubyKernel.java:1098:incatch'
from /Users/hding2/.rvm/rubies/jruby-9.0.1.0/bin/irb:13:in `'

In ruby-2.2.3:
2.2.3 :001 > require 'pathname'
=> true
2.2.3 :002 > x = "joe"
=> "joe"
2.2.3 :003 > y = "joe/pete!/bob"
=> "joe/pete!/bob"
2.2.3 :004 > Pathname.new(y).relative_path_from(Pathname.new(x))
=> #Pathname:pete!/bob