Currently, we add it no matter what. But it would be better to add is specifically for when the tests directory actually exists.

This is debatable since the user might want to manually create tests directories. But this is related to the thinking in #42 and I lean toward implementing the functionality in this ticket