Skip to content
This repository was archived by the owner on Aug 4, 2020. It is now read-only.

Commit 41d4239

Browse files
author
Yevgeny Pats
committed
Beta Release for Java JQF support
0 parents  commit 41d4239

9 files changed

Lines changed: 379 additions & 0 deletions

File tree

.gitignore

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
.idea
2+
fuzz-results
3+
.DS_Store
4+
target
5+
*.iml

.travis.yml

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
dist: bionic
2+
language: java
3+
services:
4+
- docker
5+
6+
env:
7+
global:
8+
# This is FUZZIT_API_KEY. see https://docs.travis-ci.com/user/environment-variables/#encrypting-environment-variables for how to create secrets in Travis. you can also create this environment variable in travis dashboard
9+
secure: "vuAoiozUO/335kehSHLR/biDYJnPgB0setWuk2GHxCL9cFsbllBllB2TRgY+LqXnb8jV5cK74Xd6nw/nJytjJeeNDYB7MoCPBkQ3uFdzx79KV4OTHy03YDoW4hNp/49N6OwjEoQhdQzI+nuHyIYKmlmvW6r0M5tlO/RMsuD4LDA8PRLcPG3MTZ1/XQTlpX528YpcO0ADixJs8wvxrTpC06LTo4ieOZQtsYuxJJaVHs14no+qmvF90sLzG2A+SPgdkFmxaUbVp11tswuxo05DeA7epveYsLG1CT12ZLWPBG5FPFVJaL47bz66SciTQKZP+z6SHtIp5Q+bI91IwitOC5r69OlcpcUXKlmQ3GDlP5RRMVef4ZFRJGGEaWU9EXUMD11SbH8RGkf9PMCE8tc/TlWm+MAuww1hVLWW5dM4ttLb9jOBt4OvsiSC+mIEVDi2k9964mZ69YeiYKjPtf5vpx76uS8gR2R6LhsQruPKEwoy0A2/a3L33RtlXBlKURt5YJQuV535KjsVZx5aQUEkDcbNuQHO0HrmrS/rXWUWHEu4xb33nqFypCaB7qgaID2aqKa1TGv6g3aWRmkpqA+ohNSJ80w8EMDBhWwxvHqV2jS4x+XQDaaGvz7Bi3H56keYk4HXhT4Ex8/8jUizzs8Yr5j+glwRhsFrs8WKehpVYt0="
10+
11+
jobs:
12+
include:
13+
- stage: Build, Unit-Tests & Fuzz-Tests
14+
go: 1.12.x
15+
script:
16+
- ./fuzzit.sh fuzzing
17+
18+
# - stage: Fuzzit (Fuzzing)
19+
# if: branch = master AND type IN (push)
20+
# go: 1.12.x
21+
# script:
22+
# - ./fuzzit.sh fuzzing

README.md

Lines changed: 210 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,210 @@
1+
[![Build Status](https://travis-ci.org/fuzzitdev/example-java.svg?branch=master)](https://travis-ci.org/fuzzitdev/example-java)
2+
[![fuzzit](https://app.fuzzit.dev/badge?org_id=hP8u8bNAda91Cnj0mKPX&branch=master)](https://fuzzit.dev)
3+
[![Join the chat at https://slack.fuzzit.dev](https://app.fuzzit.dev/static/slack-join.svg)](https://slack.fuzzit.dev)
4+
5+
# Continuous Fuzzing for Java Example
6+
7+
This is an example of how to integrate your [JQF](https://github.com/rohanpadhye/jqf) targets with the
8+
[Fuzzit](https://fuzzit.dev) Continuous Fuzzing Platform (Java support is currently in Beta).
9+
10+
This example will show the following steps:
11+
* [Building and running a simple JQF target locally](#building-jqf-target)
12+
* [Integrate the JQF target with Fuzzit via Travis-CI](#integrating-with-fuzzit-from-ci)
13+
14+
Result:
15+
* Fuzzit will run the fuzz targets continuously on a daily basis with the latest release.
16+
* Fuzzit will run regression tests on every pull-request with the generated corpus and crashes to catch bugs early on.
17+
18+
Coverage Guided Structure Aware Fuzzing for Java can help find both complex bugs, as well as correctness bugs.
19+
Java is a safe language so memory corruption bugs are very unlikely to happen, but some bugs can still have security
20+
implications.
21+
22+
This tutorial focuses less on how to build JQF targets and more on how to integrate the targets with Fuzzit. A lot of
23+
great information is available at the [JQF Wiki](https://github.com/rohanpadhye/jqf/wiki/Fuzzing-with-Zest).
24+
25+
## Building JQF Target
26+
27+
The targets that are currently supported on Fuzzit are targets that utilize the
28+
[JQF+Zest](https://github.com/rohanpadhye/jqf/wiki/Fuzzing-with-Zest) engine.
29+
30+
### Understanding the bug
31+
32+
The bug is located at `ParseComplex.Java` in the following code
33+
34+
```java
35+
package dev.fuzzit.examplejava;
36+
37+
public class ParseComplex {
38+
public static boolean parse(String data) {
39+
if (data.length() > 4) {
40+
return false;
41+
}
42+
return data.charAt(0) == 'F' &&
43+
data.charAt(1) == 'U' &&
44+
data.charAt(2) == 'Z' &&
45+
data.charAt(3) == 'Z';
46+
}
47+
}
48+
```
49+
50+
This is a VERY simple case for the sake of the example the author made a mistake.
51+
and Instead of `data.length() > 4 ` the correct code should be `data.length() < 4`.
52+
53+
### Understanding the fuzzer
54+
55+
the fuzzer is located at `ParseComplexFuzz.Java` in the following code:
56+
57+
```java
58+
import org.junit.runner.RunWith;
59+
import edu.berkeley.cs.jqf.fuzz.Fuzz;
60+
import edu.berkeley.cs.jqf.fuzz.JQF;
61+
62+
@RunWith(JQF.class)
63+
public class ParseComplexFuzz {
64+
65+
@Fuzz
66+
public void fuzz(String data) {
67+
ParseComplex.parse(data);
68+
}
69+
}
70+
```
71+
72+
This is pretty straight forward the fuzzer will generate the psudo random string via data according to
73+
the coverage feedback
74+
75+
### Building & Running the fuzzer
76+
77+
```bash
78+
git clone https://github.com/fuzzitdev/example-java
79+
docker run -v `pwd`:/app -it maven:3.6.1-jdk-12 /bin/bash
80+
cd /app
81+
# Change to maven repo once 1.3 is out
82+
curl -o zest-cli.jar https://storage.googleapis.com/public-fuzzit/jqf-fuzz-1.3-SNAPSHOT-zest-cli.jar
83+
mvn package
84+
java -jar zest-cli.jar -e ./target/example-java-1.0-SNAPSHOT-fat-tests.jar dev.fuzzit.examplejava.ParseComplexTest fuzz
85+
```
86+
87+
Will print the following output and stacktrace:
88+
89+
```text
90+
Semantic Fuzzing with Zest
91+
--------------------------
92+
93+
Test name: dev.fuzzit.examplejava.ParseComplexTest#fuzz
94+
Results directory: /app/target/fuzz-results/dev.fuzzit.examplejava.ParseComplexTest/fuzz
95+
Elapsed time: 2s (no time limit)
96+
Number of executions: 582
97+
Valid inputs: 562 (96.56%)
98+
Cycles completed: 0
99+
Unique failures: 1
100+
Queue size: 2 (0 favored last cycle)
101+
Current parent input: 0 (favored) {581/800 mutations}
102+
Execution speed: 263/sec now | 221/sec overall
103+
Total coverage: 5 branches (0.01% of map)
104+
```
105+
106+
You can see 1 crash is instantly found. Results are saved by default to target/fuzz-results
107+
108+
109+
```text
110+
.id_000000: FAILURE (java.lang.StringIndexOutOfBoundsException)
111+
E
112+
Time: 0.079
113+
There was 1 failure:
114+
1) fuzz(dev.fuzzit.examplejava.ParseComplexTest)
115+
java.lang.StringIndexOutOfBoundsException: String index out of range: 0
116+
at java.base/java.lang.StringLatin1.charAt(StringLatin1.java:47)
117+
at java.base/java.lang.String.charAt(String.java:702)
118+
at dev.fuzzit.examplejava.ParseComplex.parse(ParseComplex.java)
119+
at dev.fuzzit.examplejava.ParseComplexTest.fuzz(ParseComplexFuzz.java)
120+
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
121+
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
122+
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
123+
at java.base/java.lang.reflect.Method.invoke(Method.java:567)
124+
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
125+
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
126+
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
127+
at edu.berkeley.cs.jqf.fuzz.junit.TrialRunner$1.evaluate(TrialRunner.java:59)
128+
at edu.berkeley.cs.jqf.fuzz.junit.TrialRunner.run(TrialRunner.java:65)
129+
at edu.berkeley.cs.jqf.fuzz.junit.quickcheck.FuzzStatement.evaluate(FuzzStatement.java:165)
130+
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
131+
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
132+
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
133+
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
134+
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
135+
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
136+
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
137+
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
138+
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
139+
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
140+
at edu.berkeley.cs.jqf.fuzz.junit.GuidedFuzzing.run(GuidedFuzzing.java:184)
141+
at edu.berkeley.cs.jqf.fuzz.junit.GuidedFuzzing.run(GuidedFuzzing.java:126)
142+
at edu.berkeley.cs.jqf.plugin.ReproGoal.execute(ReproGoal.java:184)
143+
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
144+
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:210)
145+
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:156)
146+
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:148)
147+
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
148+
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
149+
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56)
150+
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
151+
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:305)
152+
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192)
153+
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105)
154+
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:956)
155+
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
156+
at org.apache.maven.cli.MavenCli.main(MavenCli.java:192)
157+
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
158+
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
159+
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
160+
at java.base/java.lang.reflect.Method.invoke(Method.java:567)
161+
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:282)
162+
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:225)
163+
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:406)
164+
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:347)
165+
166+
FAILURES!!!
167+
Tests run: 1, Failures: 1
168+
169+
```
170+
171+
For the possible command-lines for ZestCLI you can run `java -jar zest-cli.jar --help`
172+
173+
## Integrating with Fuzzit from CI
174+
175+
The best way to integrate with Fuzzit is by adding a two stages in your Continuous Build system
176+
(like Travis, CircleCI, Github Actions or any other CI).
177+
178+
Fuzzing stage:
179+
180+
* Build a package containing the fuzz targets together with all dependencies. This can be done using the assembly plugin like in this repository
181+
* Download `fuzzit` cli
182+
* Authenticate via passing `FUZZIT_API_KEY` environment variable
183+
* Create a fuzzing job by uploading the fuzzing target
184+
185+
Regression stage (This stage in Java is currently in Alpha and will be rolled out to Public beta in the upcoming week)
186+
* Build a fuzzing target
187+
* Download `fuzzit` cli
188+
* Authenticate via passing `FUZZIT_API_KEY` environment variable OR defining the corpus as public. This way
189+
No authentication would be require and regression can be used for [forked PRs](https://docs.travis-ci.com/user/pull-requests#pull-requests-and-security-restrictions) as well
190+
* Create a local regression fuzzing job - This will pull all the generated corpuses and run them through
191+
the fuzzing binary. If new bugs are introduced this will fail the CI and alert
192+
193+
Here is the relevant snippet from the [fuzzit.sh](https://github.com/fuzzitdev/example-java/blob/master/fuzzit.sh)
194+
which is being run by [.travis.yml](https://github.com/fuzzitdev/example-java/blob/master/.travis.yml)
195+
196+
```bash
197+
wget -q -O fuzzit https://github.com/fuzzitdev/fuzzit/releases/download/v2.4.35/fuzzit_Linux_x86_64
198+
chmod a+x fuzzit
199+
200+
## upload fuzz target for long fuzz testing on fuzzit.dev server or run locally for regression
201+
./fuzzit create job --engine jqf --type ${1} --args "dev.fuzzit.examplejava.ParseComplexTest fuzz" fuzzitdev/parse-complex ./target/example-java-1.0-SNAPSHOT-fat-tests.jar
202+
```
203+
204+
In production it is advised to download a pinned version of the [CLI](https://github.com/fuzzitdev/fuzzit)
205+
like in the example. In development you can use the latest version:
206+
https://github.com/fuzzitdev/fuzzit/releases/latest/download/fuzzit_${OS}_${ARCH}.
207+
Valid values for `${OS}` are: `Linux`, `Darwin`, `Windows`.
208+
Valid values for `${ARCH}` are: `x86_64` and `i386`.
209+
210+

fuzzit.sh

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
mvn package
2+
3+
## Install fuzzit specific version for production or latest version for development :
4+
# https://github.com/fuzzitdev/fuzzit/releases/latest/download/fuzzit_Linux_x86_64
5+
wget -q -O fuzzit https://github.com/fuzzitdev/fuzzit/releases/download/v2.4.51/fuzzit_Linux_x86_64
6+
chmod a+x fuzzit
7+
8+
## upload fuzz target for long fuzz testing on fuzzit.dev server or run locally for regression
9+
./fuzzit create job --engine jqf --type ${1} --args "dev.fuzzit.examplejava.ParseComplexTest fuzz" fuzzitdev/parse-complex ./target/example-java-1.0-SNAPSHOT-fat-tests.jar

pom.xml

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
2+
<project xmlns="http://maven.apache.org/POM/4.0.0"
3+
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
4+
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
5+
<modelVersion>4.0.0</modelVersion>
6+
7+
<groupId>dev.fuzzit.example-java</groupId>
8+
<artifactId>example-java</artifactId>
9+
<version>1.0-SNAPSHOT</version>
10+
<dependencies>
11+
<dependency>
12+
<groupId>junit</groupId>
13+
<artifactId>junit</artifactId>
14+
<version>4.12</version>
15+
</dependency>
16+
<dependency>
17+
<groupId>com.pholser</groupId>
18+
<artifactId>junit-quickcheck-generators</artifactId>
19+
<version>0.8</version>
20+
</dependency>
21+
<dependency>
22+
<groupId>com.pholser</groupId>
23+
<artifactId>junit-quickcheck-core</artifactId>
24+
<version>0.8</version>
25+
</dependency>
26+
<dependency>
27+
<groupId>edu.berkeley.cs.jqf</groupId>
28+
<artifactId>jqf-fuzz</artifactId>
29+
<version>1.2</version>
30+
</dependency>
31+
</dependencies>
32+
33+
<properties>
34+
<maven.compiler.source>1.7</maven.compiler.source>
35+
<maven.compiler.target>1.7</maven.compiler.target>
36+
</properties>
37+
38+
<build>
39+
<plugins>
40+
<plugin>
41+
<groupId>edu.berkeley.cs.jqf</groupId>
42+
<artifactId>jqf-maven-plugin</artifactId>
43+
<version>1.2</version>
44+
</plugin>
45+
46+
<plugin>
47+
<groupId>org.apache.maven.plugins</groupId>
48+
<artifactId>maven-surefire-plugin</artifactId>
49+
<version>2.19.1</version>
50+
<configuration>
51+
<skipTests>true</skipTests>
52+
</configuration>
53+
</plugin>
54+
<plugin>
55+
<artifactId>maven-assembly-plugin</artifactId>
56+
<version>3.1.1</version>
57+
<configuration>
58+
<descriptors>
59+
<descriptor>src/main/assembly/assembly.xml</descriptor>
60+
</descriptors>
61+
</configuration>
62+
<executions>
63+
<execution>
64+
<id>make-assembly</id>
65+
<phase>package</phase>
66+
<goals>
67+
<goal>single</goal>
68+
</goals>
69+
</execution>
70+
</executions>
71+
</plugin>
72+
</plugins>
73+
</build>
74+
</project>

src/main/assembly/assembly.xml

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
<assembly
2+
xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.3"
3+
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
4+
xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.3 http://maven.apache.org/xsd/assembly-1.1.3.xsd">
5+
<id>fat-tests</id>
6+
<formats>
7+
<format>jar</format>
8+
</formats>
9+
<includeBaseDirectory>false</includeBaseDirectory>
10+
<dependencySets>
11+
<dependencySet>
12+
<outputDirectory>/</outputDirectory>
13+
<useProjectArtifact>true</useProjectArtifact>
14+
<unpack>true</unpack>
15+
<scope>test</scope>
16+
</dependencySet>
17+
</dependencySets>
18+
<fileSets>
19+
<fileSet>
20+
<directory>${project.build.directory}/test-classes</directory>
21+
<outputDirectory>/</outputDirectory>
22+
</fileSet>
23+
</fileSets>
24+
</assembly>
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
package dev.fuzzit.examplejava;
2+
3+
public class ParseComplex {
4+
public static boolean parse(String data) {
5+
if (data.length() > 4) {
6+
return false;
7+
}
8+
return data.charAt(0) == 'F' &&
9+
data.charAt(1) == 'U' &&
10+
data.charAt(2) == 'Z' &&
11+
data.charAt(3) == 'Z';
12+
}
13+
}
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
package dev.fuzzit.examplejava;
2+
3+
public class ParseMain {
4+
public static void main(String [] args) {
5+
System.out.println("hello world");
6+
}
7+
}
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
package dev.fuzzit.examplejava;
2+
3+
import org.junit.runner.RunWith;
4+
import edu.berkeley.cs.jqf.fuzz.Fuzz;
5+
import edu.berkeley.cs.jqf.fuzz.JQF;
6+
7+
@RunWith(JQF.class)
8+
public class ParseComplexTest {
9+
10+
@Fuzz
11+
public void fuzz(String data)
12+
{
13+
ParseComplex.parse(data);
14+
}
15+
}

0 commit comments

Comments
 (0)