|
| 1 | +[](https://travis-ci.org/fuzzitdev/example-java) |
| 2 | +[](https://fuzzit.dev) |
| 3 | +[](https://slack.fuzzit.dev) |
| 4 | + |
| 5 | +# Continuous Fuzzing for Java Example |
| 6 | + |
| 7 | +This is an example of how to integrate your [JQF](https://github.com/rohanpadhye/jqf) targets with the |
| 8 | +[Fuzzit](https://fuzzit.dev) Continuous Fuzzing Platform (Java support is currently in Beta). |
| 9 | + |
| 10 | +This example will show the following steps: |
| 11 | +* [Building and running a simple JQF target locally](#building-jqf-target) |
| 12 | +* [Integrate the JQF target with Fuzzit via Travis-CI](#integrating-with-fuzzit-from-ci) |
| 13 | + |
| 14 | +Result: |
| 15 | +* Fuzzit will run the fuzz targets continuously on a daily basis with the latest release. |
| 16 | +* Fuzzit will run regression tests on every pull-request with the generated corpus and crashes to catch bugs early on. |
| 17 | + |
| 18 | +Coverage Guided Structure Aware Fuzzing for Java can help find both complex bugs, as well as correctness bugs. |
| 19 | +Java is a safe language so memory corruption bugs are very unlikely to happen, but some bugs can still have security |
| 20 | +implications. |
| 21 | + |
| 22 | +This tutorial focuses less on how to build JQF targets and more on how to integrate the targets with Fuzzit. A lot of |
| 23 | +great information is available at the [JQF Wiki](https://github.com/rohanpadhye/jqf/wiki/Fuzzing-with-Zest). |
| 24 | + |
| 25 | +## Building JQF Target |
| 26 | + |
| 27 | +The targets that are currently supported on Fuzzit are targets that utilize the |
| 28 | +[JQF+Zest](https://github.com/rohanpadhye/jqf/wiki/Fuzzing-with-Zest) engine. |
| 29 | + |
| 30 | +### Understanding the bug |
| 31 | + |
| 32 | +The bug is located at `ParseComplex.Java` in the following code |
| 33 | + |
| 34 | +```java |
| 35 | +package dev.fuzzit.examplejava; |
| 36 | + |
| 37 | +public class ParseComplex { |
| 38 | + public static boolean parse(String data) { |
| 39 | + if (data.length() > 4) { |
| 40 | + return false; |
| 41 | + } |
| 42 | + return data.charAt(0) == 'F' && |
| 43 | + data.charAt(1) == 'U' && |
| 44 | + data.charAt(2) == 'Z' && |
| 45 | + data.charAt(3) == 'Z'; |
| 46 | + } |
| 47 | +} |
| 48 | +``` |
| 49 | + |
| 50 | +This is a VERY simple case for the sake of the example the author made a mistake. |
| 51 | +and Instead of `data.length() > 4 ` the correct code should be `data.length() < 4`. |
| 52 | + |
| 53 | +### Understanding the fuzzer |
| 54 | + |
| 55 | +the fuzzer is located at `ParseComplexFuzz.Java` in the following code: |
| 56 | + |
| 57 | +```java |
| 58 | +import org.junit.runner.RunWith; |
| 59 | +import edu.berkeley.cs.jqf.fuzz.Fuzz; |
| 60 | +import edu.berkeley.cs.jqf.fuzz.JQF; |
| 61 | + |
| 62 | +@RunWith(JQF.class) |
| 63 | +public class ParseComplexFuzz { |
| 64 | + |
| 65 | + @Fuzz |
| 66 | + public void fuzz(String data) { |
| 67 | + ParseComplex.parse(data); |
| 68 | + } |
| 69 | +} |
| 70 | +``` |
| 71 | + |
| 72 | +This is pretty straight forward the fuzzer will generate the psudo random string via data according to |
| 73 | +the coverage feedback |
| 74 | + |
| 75 | +### Building & Running the fuzzer |
| 76 | + |
| 77 | +```bash |
| 78 | +git clone https://github.com/fuzzitdev/example-java |
| 79 | +docker run -v `pwd`:/app -it maven:3.6.1-jdk-12 /bin/bash |
| 80 | +cd /app |
| 81 | +# Change to maven repo once 1.3 is out |
| 82 | +curl -o zest-cli.jar https://storage.googleapis.com/public-fuzzit/jqf-fuzz-1.3-SNAPSHOT-zest-cli.jar |
| 83 | +mvn package |
| 84 | +java -jar zest-cli.jar -e ./target/example-java-1.0-SNAPSHOT-fat-tests.jar dev.fuzzit.examplejava.ParseComplexTest fuzz |
| 85 | +``` |
| 86 | + |
| 87 | +Will print the following output and stacktrace: |
| 88 | + |
| 89 | +```text |
| 90 | +Semantic Fuzzing with Zest |
| 91 | +-------------------------- |
| 92 | +
|
| 93 | +Test name: dev.fuzzit.examplejava.ParseComplexTest#fuzz |
| 94 | +Results directory: /app/target/fuzz-results/dev.fuzzit.examplejava.ParseComplexTest/fuzz |
| 95 | +Elapsed time: 2s (no time limit) |
| 96 | +Number of executions: 582 |
| 97 | +Valid inputs: 562 (96.56%) |
| 98 | +Cycles completed: 0 |
| 99 | +Unique failures: 1 |
| 100 | +Queue size: 2 (0 favored last cycle) |
| 101 | +Current parent input: 0 (favored) {581/800 mutations} |
| 102 | +Execution speed: 263/sec now | 221/sec overall |
| 103 | +Total coverage: 5 branches (0.01% of map) |
| 104 | +``` |
| 105 | + |
| 106 | +You can see 1 crash is instantly found. Results are saved by default to target/fuzz-results |
| 107 | + |
| 108 | + |
| 109 | +```text |
| 110 | +.id_000000: FAILURE (java.lang.StringIndexOutOfBoundsException) |
| 111 | +E |
| 112 | +Time: 0.079 |
| 113 | +There was 1 failure: |
| 114 | +1) fuzz(dev.fuzzit.examplejava.ParseComplexTest) |
| 115 | +java.lang.StringIndexOutOfBoundsException: String index out of range: 0 |
| 116 | + at java.base/java.lang.StringLatin1.charAt(StringLatin1.java:47) |
| 117 | + at java.base/java.lang.String.charAt(String.java:702) |
| 118 | + at dev.fuzzit.examplejava.ParseComplex.parse(ParseComplex.java) |
| 119 | + at dev.fuzzit.examplejava.ParseComplexTest.fuzz(ParseComplexFuzz.java) |
| 120 | + at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) |
| 121 | + at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) |
| 122 | + at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) |
| 123 | + at java.base/java.lang.reflect.Method.invoke(Method.java:567) |
| 124 | + at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) |
| 125 | + at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) |
| 126 | + at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) |
| 127 | + at edu.berkeley.cs.jqf.fuzz.junit.TrialRunner$1.evaluate(TrialRunner.java:59) |
| 128 | + at edu.berkeley.cs.jqf.fuzz.junit.TrialRunner.run(TrialRunner.java:65) |
| 129 | + at edu.berkeley.cs.jqf.fuzz.junit.quickcheck.FuzzStatement.evaluate(FuzzStatement.java:165) |
| 130 | + at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) |
| 131 | + at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) |
| 132 | + at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) |
| 133 | + at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) |
| 134 | + at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) |
| 135 | + at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) |
| 136 | + at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) |
| 137 | + at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) |
| 138 | + at org.junit.runners.ParentRunner.run(ParentRunner.java:363) |
| 139 | + at org.junit.runner.JUnitCore.run(JUnitCore.java:137) |
| 140 | + at edu.berkeley.cs.jqf.fuzz.junit.GuidedFuzzing.run(GuidedFuzzing.java:184) |
| 141 | + at edu.berkeley.cs.jqf.fuzz.junit.GuidedFuzzing.run(GuidedFuzzing.java:126) |
| 142 | + at edu.berkeley.cs.jqf.plugin.ReproGoal.execute(ReproGoal.java:184) |
| 143 | + at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137) |
| 144 | + at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:210) |
| 145 | + at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:156) |
| 146 | + at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:148) |
| 147 | + at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117) |
| 148 | + at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81) |
| 149 | + at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56) |
| 150 | + at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128) |
| 151 | + at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:305) |
| 152 | + at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192) |
| 153 | + at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105) |
| 154 | + at org.apache.maven.cli.MavenCli.execute(MavenCli.java:956) |
| 155 | + at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288) |
| 156 | + at org.apache.maven.cli.MavenCli.main(MavenCli.java:192) |
| 157 | + at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) |
| 158 | + at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) |
| 159 | + at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) |
| 160 | + at java.base/java.lang.reflect.Method.invoke(Method.java:567) |
| 161 | + at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:282) |
| 162 | + at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:225) |
| 163 | + at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:406) |
| 164 | + at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:347) |
| 165 | +
|
| 166 | +FAILURES!!! |
| 167 | +Tests run: 1, Failures: 1 |
| 168 | +
|
| 169 | +``` |
| 170 | + |
| 171 | +For the possible command-lines for ZestCLI you can run `java -jar zest-cli.jar --help` |
| 172 | + |
| 173 | +## Integrating with Fuzzit from CI |
| 174 | + |
| 175 | +The best way to integrate with Fuzzit is by adding a two stages in your Continuous Build system |
| 176 | +(like Travis, CircleCI, Github Actions or any other CI). |
| 177 | + |
| 178 | +Fuzzing stage: |
| 179 | + |
| 180 | +* Build a package containing the fuzz targets together with all dependencies. This can be done using the assembly plugin like in this repository |
| 181 | +* Download `fuzzit` cli |
| 182 | +* Authenticate via passing `FUZZIT_API_KEY` environment variable |
| 183 | +* Create a fuzzing job by uploading the fuzzing target |
| 184 | + |
| 185 | +Regression stage (This stage in Java is currently in Alpha and will be rolled out to Public beta in the upcoming week) |
| 186 | +* Build a fuzzing target |
| 187 | +* Download `fuzzit` cli |
| 188 | +* Authenticate via passing `FUZZIT_API_KEY` environment variable OR defining the corpus as public. This way |
| 189 | +No authentication would be require and regression can be used for [forked PRs](https://docs.travis-ci.com/user/pull-requests#pull-requests-and-security-restrictions) as well |
| 190 | +* Create a local regression fuzzing job - This will pull all the generated corpuses and run them through |
| 191 | +the fuzzing binary. If new bugs are introduced this will fail the CI and alert |
| 192 | + |
| 193 | +Here is the relevant snippet from the [fuzzit.sh](https://github.com/fuzzitdev/example-java/blob/master/fuzzit.sh) |
| 194 | +which is being run by [.travis.yml](https://github.com/fuzzitdev/example-java/blob/master/.travis.yml) |
| 195 | + |
| 196 | +```bash |
| 197 | +wget -q -O fuzzit https://github.com/fuzzitdev/fuzzit/releases/download/v2.4.35/fuzzit_Linux_x86_64 |
| 198 | +chmod a+x fuzzit |
| 199 | + |
| 200 | +## upload fuzz target for long fuzz testing on fuzzit.dev server or run locally for regression |
| 201 | +./fuzzit create job --engine jqf --type ${1} --args "dev.fuzzit.examplejava.ParseComplexTest fuzz" fuzzitdev/parse-complex ./target/example-java-1.0-SNAPSHOT-fat-tests.jar |
| 202 | +``` |
| 203 | + |
| 204 | +In production it is advised to download a pinned version of the [CLI](https://github.com/fuzzitdev/fuzzit) |
| 205 | +like in the example. In development you can use the latest version: |
| 206 | +https://github.com/fuzzitdev/fuzzit/releases/latest/download/fuzzit_${OS}_${ARCH}. |
| 207 | +Valid values for `${OS}` are: `Linux`, `Darwin`, `Windows`. |
| 208 | +Valid values for `${ARCH}` are: `x86_64` and `i386`. |
| 209 | + |
| 210 | + |
0 commit comments