JRuby 9.x slower than 1.7.21 when running Brakeman #3735
Description
Per https://twitter.com/headius/status/709447899902578689
Environment
Brakeman 3.2.1
Linux 4.1.15-desktop-2.mga5 SMP Wed Jan 20 17:05:51 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
These Ruby versions:
- jruby 1.7.21 (1.9.3p551) 2015-07-07 a741a82 on Java HotSpot(TM) 64-Bit Server VM 1.8.0_65-b17 +jit [linux-amd64]
- jruby 9.0.5.0 (2.2.3) 2016-01-26 7bee00d Java HotSpot(TM) 64-Bit Server VM 25.65-b01 on 1.8.0_65-b17 +jit [linux-amd64]
- jruby 9.1.0.0-SNAPSHOT (2.3.0) 2016-03-14 79cfa58 Java HotSpot(TM) 64-Bit Server VM 25.65-b01 on 1.8.0_65-b17 +jit [linux-amd64]
- MRI ruby 2.3.0p0 (2015-12-25 revision 53290) [x86_64-linux]
Expected Behavior
I expect (and hope) performance will improve with each release, or at least stay the same.
Actual Behavior
Scan times for Brakeman (and Brakeman Pro) have gotten slower.
I tested by scanning Redmine, which is decent size but not gigantic. Here I am reporting scan times (wall time) as reported by Brakeman, which excludes startup and report generation. This helps JRuby's numbers.
Command (after cloning Redmine):
brakeman -q --summary redmine
I averaged over five runs (they were pretty consistent).
| Version | Time (seconds) | % slower than MRI |
|---|---|---|
| 9.1 | 36.823 | 80.2% |
| 9.0.5.0 | 35.303 | 72.8% |
| 1.7.21 | 28.570 | 39.8% |
| MRI 2.3.0 | 20.430 | 0 |
Some things to consider:
- Brakeman tends to use a lot of memory
- Brakeman allocates a lot of small objects
- GC time is usually significant % of run time
- There is some I/O going on
I have played with different GCs to no real effect.