\section{Testing}

\label{Testing}

To comfort us in the idea that our software is functionally correct, we used a thorough testing methodology. Because of the obvious connection between our \texttt{diff} and \texttt{patch} implementations, the two most important qualities to strive for are consistency and robustness. In other words, our software \textit{must} produce the same output if it is fed the same input multiple times.

\paragraph{Unit Testing}

We wrote our unit tests with \texttt{TestNG}~\cite{testng}, mainly due to previous exposure. We also used the \texttt{hamcrest}~\cite{hamcrest} library. This allowed us to write powerful assertions very easily, as illustrated in the following code snippet.

\begin{lstlisting}[caption={\texttt{diffr.suffixtree.impl.MatcherImplTest}}]

@Test(expectedExceptions = IllegalStateException.class)

public void testMatchNextElementMatcherFinished() {

final SuffixTree<Character> suffixTree =

SuffixTreeImpl.newSuffixTree(Lists.charactersOf("mississippi"));

final Matcher<Character> matcher = suffixTree.matcher();

for (Character c : Lists.charactersOf("issipp")) {

assertThat(matcher.matchNext(c), is(Matched.YES));

}

assertThat(matcher.matchNext('p'), is(Matched.NO));

assertThat(matcher.isFinished(), is(true));

matcher.matchNext('b');

}

\end{lstlisting}

Our tests cover over 90\% of the codebase. We are confident that our code is robust and thoroughly tested. Both \texttt{diffr} and \texttt{patchr} are tested individually on manually derived test files, designed to thoroughly test corner cases.

\paragraph{Integration Testing}

\label{IntegrationTesting}

Due to the unique nature of our software (i.e. there is no reference implementation to compare results with), we had to come up with files to perform integration testing. We have concatenated all the source files from the \texttt{kernel/} directory in linux kernel, versions \texttt{2.6.27.62} and \texttt{3.2.13}, and the entire kernel version \texttt{0.1} and run \texttt{diffr} on all possible pairs (\texttt{original file}, \texttt{new file}) and \texttt{patchr} on the \texttt{original file} and patch file generated using \texttt{diffr} to see if we can get back the \texttt{new file}.

This approach is allows us to test the integration between the two tools quite thoroughly and definitely, as we test the entire loop of generating the patch file and applying it to the original file. The integration tests give us great confidence in the correctness of our implementation.

\paragraph{Performance Testing}

We used \texttt{caliper}~\cite{caliper}, Google's open-source framework for writing, running and viewing the results of Java Microbenchmarks. It allowed us to fine-tune the Suffix Tree implementation to guarantee high performance.