Skip to content

Ripper error "lex_p < tokp" caused by multi-byte character #1417

@DavidEGrayson

Description

@DavidEGrayson

While I was using YARD I found another bug in Ripper. The following code runs fine with ruby 2.0.0p0 in Windows:

# coding: UTF-8
require 'ripper'
Ripper.parse "[:µ]\n\n"

However, if I run it with JRuby 1.7.10 then I get a cryptic error. In the shell output below you can see the exact version of JRuby I am using and the error:

$ jruby -v && jruby jruby_bug.rb
jruby 1.7.10 (1.9.3p392) 2014-01-09 c4ecd6b on Java HotSpot(TM) 64-Bit Server VM 1.7.0_45-b18 [Windows 8-amd64]
RuntimeError: lex_p < tokp
   parse at org/jruby/ext/ripper/RubyRipper.java:306
   parse at c:/jruby-1.7.10/lib/ruby/1.9/ripper/core.rb:23
  (root) at jruby_bug.rb:3

This is probably a bug in Ripper because eval can happily process the same string that Ripper.parse chokes on.

(this was copied from issue #789)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions