This release is PHP 8+ only, so we are back on track with modern PHP. We are dropping support for PHPUnit < 9, and are technically ready for PHPUnit 10. And we also support Symfony 6 without dropping support of previous Symfony versions. As always, we did our best to keep backward compatibility so if you can update your dependencies, all tests should be working for you.

So let’s take a look at some new features:

New Directory Structure

Codeception 5 will follow the PSR-12 coding standard. So all tests and classes will have their own namespace Tests. The directory structure was updated accordingly:

tests/
  _output
  Acceptance
  Functional
  Support/
    _generated/
    Data/
    Helper/
  Unit/

All suite names have their own namespace, as well as actor and helper classes:

<?php

namespace Tests\Acceptance;

use \Tests\Support\AcceptanceTester;

class LoginCest
{
    public function tryToTest(AcceptanceTester $I)
    {
        $I->amOnPage('/');
    }
}

This new directory structure will be generated by running codecept bootstrap.

Upgrading from Codeception 4

Codeception 5 is compatible with the Codeception 4 directory structure. So if you don’t want to change your file locations now, just keep your existing codeception.yml, and everything’s fine.

However, if you want to upgrade to the new Codeception 5 directory structure (recommended), here’s the new default codeception.yml:

namespace: Tests
support_namespace: Support
paths:
    tests: tests
    output: tests/_output
    data: tests/Support/Data
    support: tests/Support
    envs: tests/_envs
actor_suffix: Tester
extensions:
    enabled:
        - Codeception\Extension\RunFailed

Next steps:

1. Capitalize your suite configuration files:
   • acceptance.suite.yml => Acceptance.suite.yml
   • functional.suite.yml => Functional.suite.yml
   • unit.suite.yml => Unit.suite.yml
2. Inside those configuration files, update to the new namespace:

    modules:
        enabled:
            - Tests\Support\Helper\Unit
   

3. In your composer.json, update to the new namespace:

    "autoload-dev": {
        "psr-4": {
            "Tests\\": "tests/"
        }
    },
   

4. In your tests/Support/Acceptance|Functional|UnitTester.php files, update to the new namespace Tests\Support
5. If you have tests/Support/Helper/Acceptance|Functional|Unit.php files, update their namespace to Tests\Support\Helper
6. Run vendor/bin/codecept build to create the files in tests/Support/_generated
7. Modify the namespaces in all your test/cest files
8. Run the tests with capitalized suite names: vendor/bin/codecept run Unit

Attributes

Annotations were an essential part of the Codeception testing framework. Even though they were not native language constructs, they proved to be quite good to separate a test from its metadata. We believe that test should not include code that doesn’t belong to the test scenario.

So we were glad that native attributes have landed in the PHP world. In this release we encourage you to start using them:

#[Group('important')]
#[Group('api')]
#[Examples('GET', '/users')]
#[Examples('GET', '/posts')]
#[Env('staging-alpha')]
#[Env('staging-beta')]
#[Env('production')]
#[Prepare('startServices')]
public function testApiRequests(ApiTester $I, Example $e)
{
  $I->send($e[0], $e[1]);
  $I->seeResponseCodeIsSuccessful();
  $I->seeResponseIsJson();
}

As you see, attributes decouple all preparation steps, keeping the test scenario minimal. We also keep supporting annotations, so an urgent upgrade is not needed. Attributes can’t do anything that traditional annotations can’t, they are just a modern alternative.

List of available attributes (all under Codeception\Attribute namespace):

• Before - specifies a method that should be executed before each test
• After - specifies a method that should be executed after each test
• Group - sets the group for the test
• Skip - skips the current test
• Incomplete - marks test as incomplete
• Depends - sets the test that must be executed before the current one
• Prepare - sets a method to execute to initialize the environment (launch server, browser, etc)
• DataProvider - specifies a method that provides data for data-driven tests
• Examples - sets data for data-driven tests inside the annotation
• Env - sets environment value for the current test
• Given, When, Then - marks a method as BDD step

Debugging

Do you remember, Hoa\Console? Unfortunately, this library was deprecated and we were looking for a modern alternative that could power codecept console and $I->pause();. We switched to PsySH, a PHP Read-Eval-Print Loop.

An interactive console is used to pause a test in the given state. While in pause you can try different Codeception commands, and check variable values. Instead of fixing tests blindly, you can start an interactive session. This is quite a similar effect you can get with a real debugger like XDebug but focused on Codeception commands. This is especially helpful for acceptance tests as the test scenario can be planned while executing a test. So the basic scenario can be written as:

$I->amOnPage('/');
$I->pause();

After opening a page you will be able to try commands in a browser. If a command succeeds you can use it in your tests.

Also new functions were added:

• codecept_pause() - starts interactive pause anywhere in debug mode
• codecept_debug() - prints a variable to console using Symfony VarDumper

Upgrading from Codeception 4

Just remove hoa/console from your composer.json.

Sharding

The Parallel Execution guide has been rewritten and focused on a new feature: sharding. It is the simplest way to run slow tests (think of acceptance tests first) in parallel on multiple agents.

In this case, you specify the batch of tests that should be executed independently and each job picks up its own not intersecting group of tests to run them.

# first job
./vendor/bin/codecept run --shard 1/3

# second job
./vendor/bin/codecept run --shard 2/3

# third job
./vendor/bin/codecept run --shard 3/3

This feature reduces the need for complex configuration and usage of robo task runner to split tests.

It is recommended to use sharding to parallelize tests between multiple jobs as the simplest approach. Unfortunately, PHP doesn’t have native multi-threading for test parallelization, and even if it had, it doesn’t solve the problem of running slow browser tests that are interacting with the entire application. So only horizontal scaling by jobs can be suggested as a long-running approach. The more build agents you add to your Continuous Integration server, the faster tests will run. That’s it!

Grep and Filter

New options --grep and --filter were introduced to select tests by part of their name. Actually, it is the same option and an alias. --grep is a common way to select tests to execute in NodeJS test runners, so we ported it to Codeception. But as usual, specific tests can also be executed by group or specifying a test signature.

./vendor/bin/codecept run --grep "user"

Other Changes

Please go through the list of changes introduced to see if they don’t affect your codebase:

• Requires PHP 8.0 or higher
• Compatible with PHPUnit 9 and ready for PHPUnit 10
• Compatible with Symfony 4.4 - 6.0
• Stricter check for phpdotenv v5 (older versions are not supported)
• Throw exception if actor setting is missing in suite configuration
• Removed generate:cept command (Cept format is deprecated)
• Removed settings disallow_test_output and log_incomplete_skipped.
• Removed setting paths.log (replaced by paths.output in Codeception 2.3)
• Removed suite setting class_name (replaced by actor in Codeception 2.3)
• Removed global setting actor (replaced by actor_prefix in Codeception 2.3)
• Removed Configuration::logDir method (replaced by Configuration::outputDir in 2.0)
• Moved XmlBuilder class to SOAP module
• Decoupled test execution and reporting from PHPUnit
• Custom reporters implementing TestListener are no longer supported and must be converted to Extensions
• Added optional value to fail-fast option (#6275) by #Verest
• Removed JSON and TAP loggers
• Removed code coverage blacklist functionality
• Removed deprecated class aliases
  • Codeception\TestCase\Test
  • Codeception\Platform\Group
  • Codeception\Platform\Group
  • Codeception\TestCase
• Introduced strict types in the code base.

Complete Changelog

We are really happy that we are finally here with Codeception 5. This release was crafted during the war that happens in Ukraine. It is mentally and morally hard to work on tech products knowing that this peaceful virtual life can end at any moment by a random missile. Codeception was created in 2011 by Michael Bodnarchuk in Kyiv, and today in 2022 he also stays there writing this post. If you want to support Codeception, all the Ukrainian PHP community, and all our brave nation who stands for democracy against this barbaric Russian invasion, consider donating to Ukrainian charities. Not a single time. Every month until the war ends. Every time you travel or enjoy tasty food in a restaurant think of people who are forced to defend their land, or who fled their homes. Glory to Ukraine!

This release wouldn’t be possible without the hard work of Gintautas Miselis who keeps constant work on modernizing internals and keeping Codeception up to date. Also we are really thankful to Gustavo Nieves who did a lot of work transitioning Codeception to Symfony 6 and more! Thanks to our maintainers! If you want to support our work we have OpenCollective!

Codeception 4.0 is released and this means a lot for us!

After long months on splitting core into modules, publishing new packages and setting up build pipelines, we are proud to show you the result.

As you know - Codeception 4 changes the way it is installed. From now on, Codeception package won’t include any modules by default and you will need to install them manually. This can be a bit annoying at first but in the end, this will allow you to upgrade each package individually and not relying on the core.

In Codeception 4 we also added support for Symfony 5, so if you plan to use the latest Symfony components - you need to upgrade today.

Upgrade to Codeception 4

To start upgrading to Codeception 4 bump a version in composer.json and run composer update.

"codeception/codeception": "^4.0"

To simplify the upgrading process we introduced an upgrade script which will go through your config file and scan for all included modules. Corresponding packages will be added to composer.json, so you could install all Codeception modules you need.

Run upgrade script on current project:

php vendor/bin/codecept init upgrade4

The script will upgrade your composer.json file and it is recommended to run composer update immediately, so new modules will be installed.

If you use several Codeception configs for multiple bundles you will need to execute upgrade script for each of them:

php vendor/bin/codecept init upgrade4 -c src/app/frontend/

Congratulations! An upgrade is done. Now you will be able to use each module individually.

Thanks to Gintautas Miselis @Naktibalda for splitting the core and adding Symfony 5 support

Phar Distribution

Codeception 4.0 will provide two phar files with selected modules included. One file will target PHP 7.2+ and include Symfony 4.4, PHPUnit 8 and newer versions of other libraries. Another file will target PHP 5.6 - 7.1 and include Symfony 3.4 and PHPUnit 5.7.

Target audience of phar files is acceptance testers, so framework modules won’t be included. Here is the full list of bundled modules:

• Amqp
• Apc
• Asserts
• Cli
• Db
• FileSystem
• FTP
• Memcache
• Mongodb
• PhpBrowser
• Queue
• Redis
• Rest
• Sequence
• Soap
• Webdriver

Phar files will no longer be built on release of codeception/codeception, but on more ad-hoc basis either monthly or after important changes to core or some bundled module.

What’s new in 4.0

Since 4.0 our longread changelogs are gone. So to track changes in modules you will need to go to corresponding repositories and read changelogs. For instance, here is the changelog for Db module.

However, we will still publish changelogs for core changes here. And one of the most interesting features coming to Codeception 4 is command stashing in interactive pause.

Codeception 3.0 introduced interactive pause for better writing and debugging tests. By adding $I->pause() to your tests you could try all Codeception commands while a test will be interrupted. This helps for browser testing and framework testing, so you could try commands in action before leaving a test.

In Codeception 4.0 this instrument was improved so you could automatically stash all successful commands:

Use hotkeys to save successful commands into a file. So you no longer need to keep in mind which commands you need to take into your test. Just copy all successful commands and paste them into a test.

Stashing was brought to you by Poh Nean

Thank you for using Codeception, and thank you for staying with us these years. It is December 2019, so it means that Codeception turns 8. Through these years we reached 15Mln installations on Packagist, we are used in some well-known companies and we became a valuable part of PHP ecosystem.

We accept congratulations in Twitter or on OpenCollective. If your business gets some value from Codeception it is ok to support its maintainers. Especially, today, when we have to maintain not one repository but 32! So, if possible, talk to your boss and ask for sponsoring Codeception. This means a lot to us, Codeception Team, and it will motivate us to work more on Codeception.

We say “thank you” to all our current and previous sponsors:

• Thanks to ThemeIsle for becoming our first regular financial contributor
• Thanks to Sami Greenbury, Lars Moelleken, Dan Untenzu for investing their own money into open-source
• Thanks to Rebilly and 2Amigos who fully supported the development since 2013 till 2016
• Thanks to Seravo for sponsoring Codeception 3.0 release.

That’s all. Upgrade to Codeception 4 and improve your testing!

As you know, we are very proud of our policy of keeping backward compatible releases. We know that tests should be stable and reliable at first. If a business is still powered by PHP 5.6, this is a business issue, and we can’t force a company to upgrade all their servers and processes just to support some more testing.

Codeception 4.0 is coming, 5.0 is on the way. Be prepared!

However, keeping backward compatibility is harder and harder because of our dependencies - PHPUnit & Symfony. They all go forward, break API and require newer versions of PHP and to follow them we need to do the same.

But can’t leave you without upgrades even if you don’t plan to switch to the latest Symfony or PHPUnit. That’s why we announce decoupling of Codeception.

That’s right! Since Codeception 4.0 the core will be separated from Codeception modules and extensions. This allows you to get the latest updates for the modules you use while keeping running Codeception on PHP 5.6+ with your set of dependencies.

See all modules decoupled at Codeception organization

After Symfony 5 is released we will release Codeception 5.0, with a direct dependency on Symfony 5 components. If you never plan to upgrade to Symfony 5 you can keep using Codeception 4, as (again) all modules are independent of the core. Codeception 5 (and upcoming versions) will support only the latest major versions of Symfony & PHPUnit.

Transforming into a new release takes time and requires hard work from the Codeception team. Many thanks go to Gintautas Miselis @Naktibalda for doing the hardest work on splitting modules into repositories. Maintaining many repositories instead of one also requires more involvement from us. It would be easier for us just to release major versions every year and ask you to upgrade. But we didn’t do that. We care about you.

That’s why we ask you to sponsor development of Codeception. Please talk to your boss, to your CTO, to your marketing team, to your CEO. Tell that reliable software comes with a cost and if tests save your life, it’s a good time to give back!

Now you donate in a few clicks on OpenCollective!

Sponsor Codeception

Sponsors receive priority support from our team, can ask for consulting, and add their logo on our website! Over 15K PHP developers visit our website monthly. If your brand needs to be recognizable in the PHP community - sponsoring Codeception is a very good opportunity for you!

From your side, we will prepare an upgrade script from version 3.x to 4.x. So once Codeception 4.0 is released you will be able to run a smooth upgrade.

We won’t accept patches to 3.1 branch, so please send your pull requests to the corresponding module repositories or to 4.0 branch of the core.

A few notes on a new release process:

• Each module will be released independently. You won’t need to wait for a month for a patch.
• Documentation for modules on codeception.com will be updated each month.
• Phar will contain only framework-agnostic modules (PhpBrowser, WebDriver, REST, Db) and will have a rolling weekly release.

Official 4.0 release announcement coming soon as we need to prepare our website site and update documentation. Stay tuned and consider donating Codeception. We require your support these days!

Maintaining such project that tries to embrace all kind of frameworks and CMSes, from Symfony to WordPress, from Laravel to Magento, is challenging, but what we see that people from those communities send us patches regularly. Probably, our philosophy - separate tests from the framework, share similar solutions via modules, shows the way. And our awesome community continue to improve this project. Hey, just look into our changelog. Even patch releases contain HUGE list of improvements!

Ok, so what about Codeception 3.0?

Breaking Changes

PHPUnit 8 Support

In 3.0 release we didn’t break a lot of stuff. We tried, but probably we just leave that for next major release. So we do not bump supported PHP version. We are still PHP 5.6+ compatible, just because testing should be available for everyone. We still support all major frameworks and we keep PHPUnit 6, and PHPUnit 7 compatibility.

However, keeping all that parts together is hard. So we assume, in 3.0 you can get possible breaking change, as we bring PHPUnit 8 support. For everything else, the upgrade should be smooth for you. If you face issues upgrading, go and change phpunit version to 6 or 7 in your composer.json:

"phpunit/phpunit": "^7.0"

We say thank you to our core contributor @Naktibalda to bring PHPUnit 8 support without breaking compatibilitiy. That was huge job, and if you look into our codeception/phpunit-wrapper project you will understand why.

Modules Removed

We also decided to drop some of old modules, which we hope no one uses. Here they are

• AngularJS - it worked only for Angular 1
• ZF1 - framework outdated
• Yii1 - framework outdated
• Silex - framework outdated
• Facebook - module not maintained
• XMLRPC - module not maintained

If you need them, just copy their code from the 2.5 branch and create a custom helper. However, we won’t support them any longer.

Changed Defaults

If you use multi-session testing and $I->haveFriend commands, you will see your tests fail. Friend methods no longer included into Codeception\Actor class, so you should add them manually. In your AcceptanceTester class (or other class which uses multi session testing) include Codeception\Lib\Actor\Shared trait:

<?php
class AcceptanceTester extends \Codeception\Actor
{
    use _generated\AcceptanceTesterActions;
    use \Codeception\Lib\Actor\Shared\Friend;
}    

We also disabled conditional assertions for new setups (as people regularly misuse them), so if you want to use canSee methods you will need to enable them. We will take a look on that in next sections of this post.

Features

M-m-m… Now the most tasty thing. Yes, we wouldn’t do the major release without new features. Thanks to sponsorship from Seravo we could finish some features we had in our roadmap.

Improved Interactive Shell

This feature was backported from our friendly project CodeceptJS. It brings a new full featured REPL interface into a test, so you could pause test execution and fire different commands into console.

This feature absolutely changes the way you write your functional and acceptance tests. Instead of blindly trying different commands and restart tests all the time, you could write:

$I->amOnPage('/');
$I->pause();

And start typing commands one by one, writing a test step by step. You copy successful commands into your tests, and in the end you get a fully working test. If you use WebDriver, you could write a complete acceptance test using one browser session.

Unlike, previous interactive shell implementation, this one based on hoa/console, so you can use Left, Right keys to edit your input. Better to show it.

Learn more about using interactive shell in our updated Getting Started guide

Try & Retry

Those features were introduced to make browser testing less painful. In the world full of JavaScript and Single Page Applications, you can no longer rely on single click, or fillField commands. Sometimes you need to retry action few times to make perform it.

Now you can do that via retry* actions which can help to stabilize flaky steps in your tests:

<?php
// use
$I->retryClick('Create');

// instead of just
$I->click('Create');

This feature was also ported from CodeceptJS but implemented a bit differently. Learn how to use it in updated Acceptance Testing guide.

What if your site behaves differently in different environments? Like showing the “Cookie” badge depending on region? And you need to accept cookies notifications if the badge is shown? You can do it via tryTo* actions:

<?php
if ($I->tryToSeeElement('.alert.cookie')) {
  $I->click('Accept', '.cookie');
}

Thanks to Iain Poulson from Delicious Brains for showing this use case

Learn how to use it in updated Acceptance Testing guide.

Try and Retry were implemented as Step Decorators. As you know, actor classes contain code generated parts, and step decorators allow to generate additional actions by wrapping methods of modules. Conditional Assertions (canSee* methods) were refactored as step decorators too. So you need to enable them explicitly on new setups.

Artifacts in Output

Failed tests now contain list of all available artifcats. For REST API test each test will contain a body of last response to simplify debugging on CI servers.

Install or Upgrade

Codeception 3.0 is as easy to upgrade as bumping version to ^3.0 in composer.json:

"codeception/codeception": "^3.0"

Also please read the changelog for more details.

Final Notes

Thanks for everyone who have been with us all this years and who helps making Codeception better every day. Again, thanks to Naktibalda for taking care of maintaining minor releases and building compatibility bridge.

Thanks to Otto Kekäläinen from Seravo for generous donation.

And a few words from Michael Bodnarchuk, who started Codeception 8 years ago:

As a person who is interested in quality and testing I was interested in answering the question: what needs to be tested at first place. And the best answer I found so far is to get that information from the user feedback. Sometimes, a bug can be found only by a regular user, without causing a stack trace. And what I’d love to see, that users of web site or web application could submit such bugreports instantly. That’s why I built Bugira Bugtracker, my new SaaS product to get bugreports from your users. If you like my projects like Codeception or CodeceptJS, please try Bugira as well. It will help you to establish better communication with you users and build a better products!

Snapshots

That’s the new feature, which might be useful if you are tired of harcoding your data into fixtures. For instance, when you test the search engine you don’t know the exact list of items to check, however, you are interested that the list would be the same every time for a search term. What about API responses? You don’t want to hardcode it fully but you may want to check that response is the same as it was before.

So now Codeception can do what php-vcr was doing all the time. Saving data into the file (called snapshot), and comparing with it on next runs.

This is nice feature for testing REST APIs.

Let’s assume you have such JSON response.

{
  "firstName": "John",
  "lastName" : "doe",
  "phoneNumbers": [
    {
      "type"  : "iPhone",
      "number": "0123-4567-8888"
    },
    {
      "type"  : "home",
      "number": "0123-4567-8910"
    }
  ]
}

And you want to check that phone numbers are the same on each run. For this we can use a snapshot.

namespace Snapshot;

class PhoneNumberSnapshot extends Codeception\Snapshot {
  
    /** @var \ApiTester */
    protected $i;

    public function __construct(\ApiTester $I)
    {
        $this->i = $I;
    }

    protected function fetchData()
    {
        // return an array of phone numbers
        return $this->i->grabDataFromResponseByJsonPath('$.phoneNumbers[*].number');
    }  

}

Then in test we can check if data matches the snapshot by calling the snapshot class:

public function testUsers(ApiTester $I, \Snapshot\PhoneNumberSnapshot $shapsnot)
{  
  $I->sendGET('/api/users');
  $snapshot->assert();
}

If the data changes, snapshot is easy to update. Just run the test in --debug mode. The snapshot will be overwritten with a new data.

So, the good thing about snapshots:

• you don’t keep flaky data in your code
• you don’t need to hardcode data values
• data can be easily updated on change

Use them!

There are also some other minor changes:

• Db module now supports multiple database connections. If you use few databases we got you covered! That was a long awaited feature and finally we have a very nice implementation from @eXorus.
• --seed parameter added to run so you could rerun the tests in same order. This feature works only if you enabled shuffle mode.
• Possible breaking: seeLink behavior changed.
  • Previous: $I->seeLink('Delete','/post/1'); matches <a href="https://codeception.com/post/199">Delete</a>
  • Now: $I->seeLink('Delete','/post/1'); does NOT match <a href="https://codeception.com/post/199">Delete</a>

See changelog for the complete list of fixes and improvements.

Thanks to all our contributors who made this release happen!

Call to Sponsors!

We try to keep Codeception going and bring more releases to you. If your company uses this framework and you’d like to give back please consider sponsoring Codeception. That’s right. We are asking to invest into open source, to get the features you expect and to give back to open source.

For instance, how would you like improving stability of WebDriver tests? It would be cool to automatically retry failed steps and rerun failed tests. These could be a very cool features but they can’t be made without your help.

If you are interested consider sponsoring us:

Sponsor Codeception

Yes, we also provide enterprise support and trainings. This is another way you can support the development. Thank you!

This follows up the PHPUnit release and mostly contains compatibility fixes for PHPUnit 7. Internal usage of PHPUnit in Codeception was switched to new namespaced class names. This doesn’t break any public APIs but can break some setups. That’s why this is a minor release with a proper announcement. Please upgrade and unless you have some PHPUnit version conflict you should not experience any issues.

Upgrade Notice: If you have tests extending PHPUnit_Framework_TestCase (old-style names) you have two options:

• Lock version for PHPUnit in composer.json: "phpunit/phpunit":"^5.0.0"
• Update your codebase to extend tests from Codeception\Test\Unit (which won’t be changed in future).

Codeception 2.4.0 is a stability release. We dropped unmaintained PHP 5.4 and PHP 5.5. But we keep compatibility with PHP 5.6 so if your company still uses PHP 5 (and we understand business reasons) you can install this update too. Codeception is still going to be maintained with PHP 5.6 compatibility in mind. We don’t break API just for the sake of clean code. We try to make things reliable.

In the same Codeception 2.3.9 is released, so if you want to get the latest changes and you still use old PHP, upgrade to this version.

In Codeception 2.4 you will also find new hooks for Cest files:

• _passed is called when a test inside Cest is successful
• _failed is called for unsuccessful tests
• _after is called at the end of a test for both cases.

Previously _after hook worked as _passed and that behavior was confusing. This was changed so it can be potentially a breaking change as well.

See changelog for the complete list of fixes and improvements.

Thanks to all our contributors who made this release happen!

We also launched Codeception Slack to talk about testing and Codeception.

Join Slack

We’d also love to see more contributors there. We need more talented developers to help to bring new features and fix the issues. It’s opensource, after all. We face constant challenges: new APIs, new libraries, new approaches, etc. And Codeception as a really big project would benefit from any help. We are open to accepting new maintainers, to discuss trends, to build a better product! As you know, we are merging 90% of submitted Pull Requests so if you are not contributing, please go ahead!

We try to keep Codeception going and bring more releases to you. If your company uses this framework and you’d like to give back please consider sponsoring Codeception. That’s right. We are asking to invest into open source, to get the features you expect and to give back to open source.

Sponsor Codeception

Please show this form to your company bosses. If you want to have stable reliable product it’s ok to pay for that. Be our first official sponsors!

Yes, we also provide enterprise support and trainings. This is another way you can support the development. Thank you!

Before writing any line of test code we should think of how meaningful this test would be. The best code is the code you never write, you know. So if you ever begin to think of which parts of the application you should cover with tests.

• Part 1: Expectation vs Implementation
• Part 2: Obtaining Specification

Let’s start with a heart of your application, the Business Logic.

Layered Architecture

Depending on architecture Business logic could be in Controller (ough), in Model (ough-ough), or in a special Service layer, which is the most preferred way. Business Logic responsible for taking decisions: which entities are valid and which are not, who can create entities, what happens when the entity is stored. Service layer should delegate the database operations to the infrastructure level.

Infrastructure level is purely technical. It just takes orders from Service layer and executes them. It doesn’t make any decisions on its own. In most cases, ORM should be at the infrastructure level.

While models or repositories do not contain any decision making logic it is not that important what ORM type to use: ActiveRecord or DataMapper.

CommandBus and Domain Events

One of the solution to separate business logic from everything would be the CommandBus architecture.

CommandBus is well-known in PHP community. Learn more about it from

• A wave of command buses blogpost by Matthias Noback
• Laracats
• Tactician library by Ross Tuck

If the application implements CommandBus pattern it is easy to find the business logic. It is located in command handlers. So let’s test them!

Testing CommandHandler

CommandHandler takes a command, calls the infrastructure services, and sends the domain events. In theory, CommandHandlers can be tested in isolation, with unit tests. Infrastructure services can be mocked and domain events can be caught. However, those tests may have a little sense. And here is why/

“Trust me, I will save this to database” - may say the test with mocked infrastructure. But there is no trust to it. To make the command handler reliable and reusable we need to ensure it does what it is expected to do.

Also, mocks comes with their price. While integration test is really similar to the actual business code, unit test is bloated with mock definitions. It will get hard to maintain and support it really soon:

<?php
// is that a business logic test?
// but no business inside of it
$eventDispatcher = $this->createMock(EventDispatcher::class);
$busDispatcher = $this->getMockBuilder(BusDispatcher::class)
   ->setMethods(['dispatch'])
   ->getMock();

$busDispatcher->expects($this->exactly(2))
   ->method('dispatch')
   ->withConsecutive(
       [$this->isInstanceOf(CreateAccount::class)],
       [$this->isInstanceOf(CreateCompany::class)]
   );

$handler = new RegisterUserHandler($eventDispatcher, $busDispatcher);
// ... 

Here we also mix implementation with a specification, which is a pure sin. What if dispatch method will be renamed? What if we fire more than 2 commands in a call? How is this related to business?

Even you can mock services you shouldn’t always do it.

In most cases, business logic should be tested with an integration test. Because the database is an essential part of your application. You can’t deliver an app without a database. The same way you can’t make sure domain logic works as expected when nothing is stored in the database.

Testing Flarum Forum

As an example let’s pick Flarum project which is a forum built on top of Illuminate components (Laravel) and Symfony Components. What is most important that it has the Commands and CommandHandlers. By looking at one directory we can learn what Flarum does. That’s awesome!

Looks like the #1 priority is to get all those Command Handlers tested. We can use StartDiscussionHandler to start.

For integration tests, we need to initialize Application with its Dependency Injection Container. Then, we fetch StartDiscussionHandler out of it:

<?php
protected function _before()
{
    // initialize Flarum app
    $server = new \Flarum\Forum\Server(codecept_root_dir());
    $app = $server->getApp();
    // get StartDiscussionHandler from container
    $this->handler = $app->make(StartDiscussionHandler::class);
}

When the handler is prepared we can write the first basic test:

<?php
public function testAdminCanStartADiscussion()
{
    // ARRANGE: create command object with all required params
    $admin = User::find(1); // User #1 is admin
    $command = new StartDiscussion($admin, [ // create command
        'attributes' => [
            'title' => 'How about some tests?',
            'content' => 'We discuss testing in this thread'
        ]
    ], '127.0.0.1');
    // ACT: proceed with handler
    $discussion = $this->handler->handle($command);
    // ASSERT: check response
    $this->assertNotNull($discussion);
    $this->assertEquals('How about some tests?', $discussion->title);
}

How do you like this test? This test so tiny and so easy to read. Sure, we should add more assertions to find out that all the business rules are applied, but for now it’s ok to try the very basic scenario. Maintaining and extending this test will be a pleasure.

A bit of Stubs and Mocks

Integration test represents a part of a system, a working component. Once it triggers the outer service, it should be mocked. In layered architecture, a class from a layer should have access to its neighbors, and to classes of lower levels. So in our case, CommandHandler of business logic can access other command handlers but should be banned from accessing other services or other command buses.

For sure, mailers, queues, and other asynchronous services should be replaced with stubs.

Conclusion

Your application has a heart. Don’t make it die from a heart attack. Write tests. Write meaningful stable tests that will last.

Written by Michael Bodnarchuk

We provide consulting services and trainings on Codeception and automated testing in general.

So what was added in the last release?

RunProcess

RunProcess extension was introduced. Use it to easily start/stop Selenium Server, ChromeDriver, or other required services for a test suite.

Enable it in suite config or in environment:

# acceptance.suite.yml
extensions:
    enabled:
        - Codeception\Extension\RunProcess:
            - php -S 127.0.0.1:8000
            - java -jar selenium-server.jar

# or inside environment config
# in this case, run tests as
#
# codecept run --env selenium
env:
  selenium:
    extensions:
        enabled:
            - Codeception\Extension\RunProcess:
                - php -S 127.0.0.1:8000
                - java -jar selenium-server.jar           

Depending on the environment (dev host, CI server) you can easily switch setups if you use environment configs.

SmartWaits

This is the new unique feature of Codeception which incorporates implicit waits. By itself, the implicit wait was available in Codeception with wait: option in WebDriver config. However, it was not usable, because by design it slowed down test execution. In this release, we introduce the SmartWait concept. Implicit waits are used only when they are really needed and disabled for all other cases. This makes tests extremely stable and fast.

Thus, a test won’t fail if expected element didn’t yet appear on a page but waits for it a few seconds more. Just set wait: 5 to WebDriver config to try it and read the documentation.

Customization

Codeception took to much of browser control. Let give it back to you. With start: false option you can disable autostart browser before a test, and create a browser session manually. Codeception doesn’t provide actions to start browser session inside a test (because it is supposed you already have one in a test), but you can write a custom method in a helper:

<?php
// support/Helper/Acceptance.php:
// 
public function startBrowser()
{
    $this->getModule('WebDriver')->_initializeSession();
}

public function changeBrowser($browserName)
{
    $this->getModule('WebDriver')->_restart(['browser' => $browser]);
}

public function closeBrowser()
{
    $this->getModule('WebDriver')->_closeSession();
}

Then these methods can be used inside a test.

<?php
// I prepare something for a test
$I->startBrowser();
// I do something inside a browser
$I->closeBrowser();

If you use BrowserStack you can use this features to set a test name dynamically to capabilities. Here is how can you do it in Acceptance Helper:

<?php
// inside Helper\Acceptance
// 
public function _before(TestInterface $test)
{
    $webDriver = $this->getModule('WebDriver');
    $name = $test->getMetadata()->getName();
    $webDriver->_capabilities(function($currentCapabilities) use ($name) {
        $currentCapabilities['name'] = $name;
        return new DesiredCapabilities($currentCapabilities);
    });    
    $webDriver->_initializeSession();
}

Please don’t forget to set start: false in config option, so browser wouldn’t be started twice!

@prepare

What if you need to change the configuration for a specific test? Let’s say you want to run all tests in Firefox but this specific one in Chrome? Or something like that.

We added new annotation @prepare which can be used in Cest and Test formats. It will execute a method which can change the configuration before the module is called.

<?php
/**
 * @prepare switchToFirefox
 */
public function testSomethingInFirefox(AcceptanceTester $I)
{
    $I->amOnPage('/');
    $I->click('Detect Browser');
    $I->see('Firefox');
}

protected function switchToFirefox(\Codeception\Module\WebDriver $webDriver)
{
    $webDriver->_reconfigure(['browser' => 'firefox']);
}

This @prepare can be used not only for browser tests but everywhere. Use it wisely!

Is that all? Almost.

We also updated WebDriver docs to include more options for Local Testing, like ChromeDriver. We also published a reference on running Chrome in Headless mode.

Update to the latest 2.3.4 version to try all the new features.

We worked hard to bring all this stuff to you. Now is your turn: please help to spread the word and encourage more of your colleagues to use PHP and Codeception for web testing. Setting up Codeception for web tests nowadays is as simple as running:

codecept init acceptance

Happy testing!

At first, we need to say “thank you” to Luis Montealegre for bringing PHPUnit 6.0 support to Codeception. This is done by using class aliases so this doesn’t break old PHPUnit versions, so as usual, Codeception can be used with PHPUnit 4.8 and higher. However, If you run PHP7+ and you experience PHPUnit issues in this release we recommend you to set phpunit/phpunit package to ^5.7.0 in your composer.json.

Supporting 3 major versions of PHPUnit may have its issues, so in next release, which is going to be 3.0 we will probably drop support for old PHPUnits and PHP < 7.

Ok, let’s talk about other changes in this release!

Installation Templates

Codeception is a wide-range testing framework and sometimes it is hard to get it configured for a specific case. Nevertheless, you need only acceptance testing Codeception will still install all 3 test suites for you. In 2.3 we prepared installation templates: customized setup wizards that can configure tests for your needs.

To setup Codeception for acceptance testing with browser only you can run this command

codecept init acceptance

After answering a few questions, you will be able to execute first browser test

The setup wizard will prepare only acceptance test suite with tests located in tests directory. Suite configuration will be stored in codeception.yml.

For QA engineers codecept init acceptance is a new convenient way to start end-to-end testing in PHP.

We also included API and Unit templates for faster setup of REST API or unit tests:

codecept init api
codecept init unit

But the best thing about installation templates that they can be created by anyone! They can be placed inside a separate package and loaded by init command. A template class should be placed into Codeception\Template namespace and then it can be autoloaded. Installation templates are pretty simple, learn how to build your own by taking a look at Acceptance template as an example.

We have added Installation templates to Customization guide and added a reference to Codeception\InitTemplate class.

Configuration Improvements

Suites Inside Main Config

Suites can be defined inside main config:

actor_suffix: Tester
paths:
    tests: .
    log: _output
    data: _data
    support: _support
suites:
    unit:
        path: .
        actor: UnitTester
        modules:
            enabled:
                - Asserts

A good option if you have a single suite.

Suite Paths

The suite can have its custom path (specified by path). From config above expects unit tests to be placed into the root directory, where codeception.yml is located.

Suites With No Actor

Suite can be defined without an actor, which is useful for unit testing

paths:
    tests: tests
    log: tests/_output
    data: tests/_data
    support: tests/_support
    envs: tests/_envs
suites:
    unit:
        path: .
        modules:
            enabled:
                - Asserts

In this case, UnitTester won’t be created, as well as _generated/UnitActions. However, such suites won’t be able to use Cest and Cept files.

Naming Changes

class_name suite in suite config replaced with actor:

class_name: UnitTester => actor: UnitTester

actor from global config is replaced with actor_suffix config option (which makes more sense).

All these changes are backward compatible, so old values in config will work.

Extensions

Dynamical loading of extensions was already with --override option but was not very usable. Now extensions can be loaded with --ext option:

codecept run --ext Recorder

or by providing a full class name

codecept run --ext "Codeception\Extension\Recorder"

This can be used to enable a custom reporter. For this reason, the new DotReporter has been added:

codecept run --ext DotReporter

Db Populator

From the early days of Codeception we had the Db module which was trying to do its best to populate database and clean up it between tests. However, parsing all possible SQL dialects and running them through PHP was not very effective. What if you could use native Database tools to import data instead of doing it from PHP? Why not!

In Codeception 2.3 we recommend to specify a command to load a database in populator option of Db module:

 modules:
    enabled:
       - Db:
          dsn: 'mysql:host=localhost;dbname=testdb'
          user: 'root'
          password: ''
          cleanup: true # run populator before each test
          populate: true # run populator before all test
          populator: 'mysql -u $user $dbname < tests/_data/dump.sql'

This approach is system-dependent, you can’t use the same config on Windows and Nix systems, but is much faster. Thanks to Mauro Asprea @brutuscat for this feature.

Db module defaults

Important notice: we changed defaults for Db module, so cleanup and populate options are disabled by default. They were quite dangerous in use, so we decided that you need to set them explicitly in Db module config.

Codeception 2.2.12 has been released as well. See complete changelog for all notable changes.

P.S. Codeception is seeking for Symfony module maintainer. If you use Codeception with Symfony and you’d like to improve it, please contact us at team@codeception.com. Maintainer responsibilities are: review issues, pull requests, update symfony demo app and so on. Take a part in project development and make open source brighter!

Today we’d love to announce a minor but very nice change on our website. We updated the Addons page so community-built modules and extensions would look nice there. The new Addons page looks as slick as a marketplace but everything there is for free!

If you have a helper or an extension that might be useful to others, don’t hesitate to share it! New “cards” format for extensions and modules will make them more visible to others. You can also attach your pictures and set colors for cards to be even more visible (If you think your extensions worth it).

The philosophy of Codeception is: don’t reinvent the wheel, share testing solutions. Spend more time writing tests and not test support code. Get more powerful tools and make testing fun!

There are some tools you definitely should not miss:

• Codeception Extension for Chrome created by Marcel Pociot.
• WebCeption a web-based Codeception runner.
• Recorder an official extension to capture screenshots after each step.
• VisualCeption and CSSRegression to test visual regressions on site.

and many many others! If you don’t have a module or extension but you still have some experience to share, write a few lines to the Cookbook.

Thamks for reading. Stay tuned for next cool updates!

Part 1: Expectation vs Implementation

It is always thought that you should have several acceptance tests, a dozen of functional, and a lot of unit or integration tests. However, If you develop a media portal it is more important to you, and for your business to have a good-looking UI. Same story if you develop new To-Do application focused on perfect UX. It is not much business logic for a To-Do app, but you need to ensure that everything user sees is slick. Even you have some internal bugs it is more important to you to assure that all buttons are visible, they can be easily clicked and so on.

Like in this app:

That’s right for this type of applications you should revert your testing pyramid. You should have lots of acceptance tests and few unit/integration.

If UI/UX is a part of your business, concentrate on acceptance (browser) tests. But if you run banking application, ecommerce solution, ERP or CRM, UI is not that important. Those applications include complex domain logic and internal quality is much more important than visual part.

What if user by a bug creates 3 extra todos in To-Do app? Well, it doesn’t look nice but not a big deal. But what if user by a bug withdraws from ATM 3 times more money they own? That’s a real problem. In case, you deal with money or real things it is critically important to test all the business processes. Test all possible scenarios, all edge cases, make the domain to be 100% covered with tests.

That brings us to key idea of this post. There are two kinds of IT products:

• software that automates business
• software that is a business by itself

And depending on the kind of software you work on you have different priorities. This also answers a question: “do I need BDD for my project”. In case, you are automating traditional business, you need to translate all business processes into a code, and you need ubiquitous languages for that. In case, you are building a new business, a startup, you probably don’t need that layer of complexity as presumably founders already speak the IT language.

So learn what is important in your business. What brings you money. What are the risks. Cover that part at first and think from business side and not from technical. This is how a business would understand the real value of automated tests.

Written by Michael Bodnarchuk

We provide consulting services on Codeception and automated testing in general.

What makes a meaningful test? This question should always be asked. No matter we write your tests first or tests after, they may stay in a project for years and it is pretty important them to test the things that really matters. Badly written tests can slow us down by constantly failing on each implementation change, and such tests can be written no matter you follow TDD or not. The idea of a test is to ensure that software works and not to freeze it at specific point.

Such situation happens when a test is bound to implementation details. We are choosing to rely on parts which we may consider to be stable but they are not. Whenever we come to unit testing and we are writing a mock we should understand that we change the reality, and how that artificial reality will survive the test of time. Consider using such test written in XIX century as an example:

Given I am drunk 
And I am in a pub
And I want to get home
When I order a cab
Then I expect to see 2 horses and carriage
And they bring me home

By ordering a cab nowadays you probably won’t expect to see 2 horses but you probably will get home even faster. In XXI century we still have pubs and we still need cabs to get home, that’s something stable in our world. The point is: you should not be worried of HOW you will be brought home: by horses, by car, by flying dragons or by teleport. That’s implementation details.

But how to understand what is stable and what is not? We need to use interfaces. Not that one which is written as interface keyword in PHP but a general term: User Interface, API. And that’s what makes unit testing and browser testing similar: we always need to rely on public interfaces for a test. If break this rule and we start to test private methods or raw values in database we are producing a synthetic tests which will lead us the very tricky path.

An opposite to follow public APIs is to depend on implementation.

But back to software development. Here is an example of magento generators written in PhpSpec. Not saying it is bad or not, but we use it to illustrate the point.

<?php
class ModuleSpec extends ObjectBehavior
{
  function let(\Symfony\Component\Filesystem\Filesystem $filesystem)
  {
      $this->beConstructedWith($filesystem);
  }

  function it_should_create_a_folder_given_a_valid_path($filesystem)
  {
      $path = 'app/code/test/test';
      $filesystem->exists($path)->willReturn(false);
      $filesystem->mkdir($path, 0700)->willReturn(true);
      $this->make($path)->shouldReturn(true);
  }
}

The test is very descriptive: there is a class that depends on Filesystem and can be used to create a given folder. However, one line is questionable:

$filesystem->mkdir($path, 0700)->willReturn(true);

This is the mock that expects mkdir to be called with a specific parameters and to return an expected result. But is that stable to changes? What if at some point mkdir gets deprecated in favor of mkdirPlus, what if a method signature changes?

Well, in this current case we can be sure that this method won’t be changed. The reason is simple: it’s Symfony, it’s LTS, and its API is stable. But can you say that about internal classes of your application? That they are 100% documented and they won’t change their behavior in future.

When we change the implementation of Module->make() it still fits expected specification (‘it should create a folder given a valid path’), but the test fails. This happens because the test pretend to know too much. In similar manner a strict master doesn’t just ask apprentice to do the job. He thinks he knows better how to do it and provides him with a detailed instructions. He doesn’t care of the outcome but apprentice should understand the basics: disobedience will be prosecuted.

But what if the actual result is not important to us? What if we want to ensure that tested method took the right path. That what happens in a test above: the implementation is verified. Indeed, If you are master (senior developer) and you have an apprentice (junior), this works pretty well: you ask them to implement the method just the way you see it.

This makes a difference between testing behavior or testing result. Probably in most cases you want to test the real outcome, a result, but in some important areas you may need to test the behavior as well.

To test a result we should rely on public interfaces only. We can validate class only by calling public method of this class and its collaborators In this case we need to call $filesystem->exists('app/code/test/test'), and make a class with in-memory filesystem. We can even make this class without using any mocking framework, just with anonymous classes.

By making a test bound to implementation we are copying parts of that implementation into a test. This breaks the DRY principle. At some point test and code can be unsynchronized and this leads to fragility. If you change class, the test using a mock of that class may still pass, but actual code using it will fail, and this false positiveness is dangerous.

Overusing mocks also makes tests longer. Lines of test code you write doesn’t always convert to software quality. Actually, from the software developer experience the true is quite the contrary: the more code you write the more bugs you may introduce. And your tests are just the same as production code: they will evolve with your software and they needs to be maintained.

So the actionable advice here is to think what is important to you: a behavior or a result (no matter how it is achieved). Try to discover what is stable and what is not, rely on stable parts, and use public interfaces. Thanks for reading.

Written by Michael Bodnarchuk @davert.

Thanks to Nick Palamarchuk for the review and ideas.

Despite its growth and feature richness Codeception is still the most simple tool for test automation. We updated documentation to keep up to date with latest changes and improved the learning experiences. Also we prepared some Case Study pages which will help you to setup Codeception for your application. Now it is easier to start with Laravel and Symfony apps.

If you have questions about Codeception you can ask them on our community forum which is PhpTestClub. We plan to build a sustainable community, so If you use Codeception for a long please share your experience there, help others to get into the world of PHP testing.

How To Upgrade

We encourage you to UPGRADE to 2.2. Please take a look into changes marked as breaking.

• The most important change is introduction of module conflicts. If modules share the same interface they probably should not be used together except the cases when the modules are loaded partially. This way you can’t use Laravel5 module with WebDriver but you can use Laravel5 with part: ORM so only ORM actions to be loaded. This change is important in order to avoid confusion in your functional or acceptance tests. If you use 2 similar modules you can’t be sure which one is executed when you call $I->amOnPage. Learn more from previous blogpost.
• Please also note that Dbh and Laravel4 module were removed from core package and moved into separate packages.
• Codeception\TestCase was replaced with Codeception\TestInterface to it is a good idea to replace its usages in your Helper files.
• Cept and Cest formats are no longer extend PHPUnit_Framework_TestCase, so they don’t have expectException, getMock, etc.

Please read changelog to learn more.

But wait, you may ask, why there are breaking changes in minor release? Does Codeception follow semver? Not right now but future releases will do. Next major version is going to be Codeception 3.0 which sure will introduce breaking changes for good reason. Also we plan to decouple most of modules from core package to make faster release cycles for them. Codeception 2.2 is also marked as minor that its still support PHP 5.4. However, we encourage you to upgrade for future versions.

New Features Described

• BDD and Gherkin
• Examples
• Params
• Module Conflicts
• DataFactory
• AngularJS
• Custom Commands
• New Redis module
• COMPLETE CHANGELOG

If you are using 2.1 and you won’t plan to upgrade in nearest future, you can still use releases from 2.2 branch. Minor fixes and improvements will still be accepted there. Also the site will contain documentation for 2.2 and 2.2 versions.

Try Codeception 2.2 today by installing it via Composer:

composer require "codeception/codeception: ~2.2" --dev

We already announced lots of changes in Codeception:

• Test Dependencies, Params, and Conflicts
• DataFactory, AngularJS, Examples, Custom Commands, …

but we forgot the most important one. It is Gherkin Support, which allows you to combine business requirements with functional testing. Yes, *.feature files are now part of a family with Cest, Cept and Tests. Codeception is a tool for running all kind of tests and in this release we significantly improved test internal architecture and test formats.

But back to Gherkin.

Feature: ToDo items
  In order to get my tasks done
  As a motivated person
  I need to be able to organize my todo list

  Scenario: show current todos
    Given there are todo items:
      | Task             | State   |
      | make Gherkin PR  | opened  |
      | update docs      | opened  |
      | create examples  | closed  |
    When I open my todos
    Then I should see 2 todo items

Complete Guide on BDD with Codeception is not ready yest, but you can start with generating first feature file:

codecept g:feature <suiteName> <featureName>

We recommend to have a special features folder in acceptance or functional suite, so it could be symlinked to the features dir in root of your project. This way non-technical users can esaily access feature files, without need to examine actual tests.

Next thing to do is to describe feature with scenarios. When you are done, prepare scenario steps for implementation be running

codecept gherkin:snippets <suiteName>

You will get a list of methods which should be included into your actor class (let’s say AcceptanceTester). Then you should have it implemented. In theory, you can use any method of any class annotated with @Given, @When, @Then to be the step definition. So don’t worry you will end up with everything to put in one context, you will have option to use multiple contexts depending on role, tags, etc. More about it in BDD guide coming in next weeks.

For those of you, who set your dependencies as “codeception/codecepton:*” and (with no settings of "minimum-stability": "stable" you will probably have some wonderful time once you get back from holidays. This release is considered minor, so it won’t break everything, but still has some breaking changes. We notified of breaking changes in earlier versions by “deprecation warnings”, and we actually removed lots of deprecated stuff. The most important breaking change is proper implementation of Conflicts API. Please make sure you are not using modules of the same kinds in your configs.

Codeception 2.2-beta is available for installation only via Composer:

composer require --dev "codeception/codeception:2.2.0-beta"

Next steps to do:

• Read the changelog. Check for all breaking changes. Probably you will need to update your Helper classes.
• Refer to updated module docs for this beta release in master branch.

Stable release will come in next week(s). The release date depends on reported issues and on progress of updating documentation.

Have nice weekends and testing time! We, Codeception team, hope you will love our product.

P.S. Reported issues should include a version. Pull Requests should be sent to master branch.

As you know, in Codeception we try to help developers and testers share their experience by providing set of shared pieces, and not to reinvent the wheel. Modules play exactly that role: you just include modules to fit your project, and write your tests, concentrating on its business logic, and not on implementation. 90% test steps in functional and acceptance tests are covered with out modules. So what do we have prepared for 2.2?

DataFactory

This module should solve problem of generating and populating data for tests. Right now we have Db module, MongoDb with really limited functionality, methods like haveRecord, seeRecord in various frameworks, and Sequence module for generating unique keys for data. However, they didn’t provide any way to generate data needed exactly for a particular test.

DataFactory module should solve this with fixture generators, called factories. The original idea take from popular Ruby gem FactoryGirl and implemented in php in FactoryMuffin library, which we use. Laravel 5 users are already aware of using factories. Now you can use them with all PHP frameworks with ActiveRecord pattern and in Doctrine.

This is how it works. You define rules to generate models:

<?php
use League\FactoryMuffin\Faker\Facade as Faker;

$fm->define(User::class)->setDefinitions([
 'name'   => Faker::name(),
    // generate email
 'email'  => Faker::email(),
 'body'   => Faker::text(),

 // generate a profile and return its Id
 'profile_id' => 'factory|Profile'
 ]);

and DataFactory creates them using underlying ORM, inserts them to database once you call

<?php
$I->have(User::class);

and deletes them after the test. As we said that will work for Doctrine as well, if you are familiar with Nelmio’s Alice, you might find the same idea but with easier syntax to use.

AngularJS

AngularJS is probably the most popular framework for building single page web applications. It provides its own tool for acceptance testing - Protractor but what if you already use Codeception and you are not so passionate to switch to NodeJS? Well if you actually do, please check out our side-project CodeceptJS which brings Codeception concepts to JavaScript world. However, in case of Angular testing there is less reasons to switch, as we brought you Protractor experience in new AngularJS module.

It simply wraps WebDriver module, and adds an asynchronous script between steps to ensure that all client-side operations finished before proceed. This way no new actions are taken before Angular finishes rendering. Also you’ve got new strict locator to use:

<?php
$I->see('davert', ['model' => 'user']);

to match elements by their ng-model.

But enough with modules, we have something more than that!

Examples

In PHPUnit you could have one test to be executed several times with different data, using the data provider mechanism. You can do the same in Codeception inside unit tests. But what about functional and acceptance testing? What if you need to run same scenario but passing different values into it? To be honest, the data provider didn’t looked like an elegant way to define a test data. Data was stored in additional method, often few pages below the original test, so it was hard to see the picture in a whole. We introduce a concept of Example, similar to what you might have seen in BDD frameworks. Using the @example annotation you can define data in test annotation and receive from Codeception\Example instance, injected into test:

<?php
  /**
   * @example(path=".", file="scenario.suite.yml")
   * @example(path=".", file="dummy.suite.yml")
   * @example(path=".", file="unit.suite.yml")
   */
  public function filesExistsAnnotation(FileTester $I, \Codeception\Example $example)
  {
      $I->amInPath($example['path']);
      $I->seeFileFound($example['file']);
  }
?>

For REST API testing this might look like:

<?php
 /**
  * @example ['/api/', 200]
  * @example ['/api/protected', 401]
  * @example ['/api/not-found-url', 404]
  * @example ['/api/faulty', 500]
  */
  public function checkEndpoints(ApiTester $I, \Codeception\Example $example)
  {
    $I->sendGET($example[0]);
    $I->seeResponseCodeIs($example[1]);
  }
?>

Data in @example annotation can be defined using JSON objects, JSON-arrays, or Symfony-style annotation. And yes, examples work only in Cests.

Custom Commands

Long requested feature that finally was implemented by Tobias Matthaiou allows you to register custom commands to Codeception runner.

If you ever created Symfony Console commands you will be familiar in creating custom commands for Codeception. You migth probably use to have your own template generators, perform data migrations, etc. You can register one as simple as you do it for extension:

extensions:
    commands: [MyApp\MyCustomCommand]

Getting current browser and capabilities in tests

The last one, simple yet useful thing that might improve your acceptance testing experience. If you want to have different behavior of tests for different browsers, you can get current browser name from a $scenario->current value:

<?php
if ($scenario->current('browser') == 'phantomjs') {
  // emulate popups for PhantomJS
  $I->executeScript('window.alert = function(){return true;}'); 
}
?>

in a similar manner you have access to all browser capabilities:

<?php
$capabilities = $scenario->current('capabilities');
if (isset($capabilities['platform'])) {
  if ($capabilities['platform'] == 'Windows') {
    // windows specific hooks
  }
}
?>

That’s all for today, but not for Codeception 2.2 The most important and most impressive feature is waiting for you in next post. Subscribe to our Twitter to not miss it. Stay tuned!

We don’t plan any major breaking changes in this release. However, we will remove deprecations, as you may know we display deprecation warnings in output, so this break should be at least predictable.

PHPUnit5 support

Yes, Codeception is finally will unlock it’s dependency on PHPUnit 4.8 so you could receive the latest stable versions of PHPUnit. As you may know PHPUnit 5.x has much better support of PHP7, so anyone who develop using the latest version of PHP should use it. But wait, it’s not the feature of Codeception 2.2 itself! Codeception 2.1.7 will be released with PHPUnit 5.x support as well.

Test Dependencies

Finally @depends annotation works as you might expected it to. The hardest thing about dependencies is that PHPUnit does nothing to organize dependent tests in a right order:

PHPUnit supports the declaration of explicit dependencies between test methods. Such dependencies do not define the order in which the test methods are to be executed but they allow the returning of an instance of the test fixture by a producer and passing it to the dependent consumers.

So their usage in Codeception was useless, especially if you were executing tests in random order with shuffle: true configuration. Some test frameworks, like minitest from Ruby run tests in shuffle by default. On the contrary PHPUnit restricts that, test with @depends annotation must be be executed after the test it relies on. It actually requires you to declare dependent tests in the same file and use the exact order of test execution.

Codeception takes the exact @depends annotation, reorders tests so dependent tests always will executed after the tests they rely on. This will work for Cest and Test formats both. And you can actually declare dependencies for tests in other files. Be sure, they will be taken, executed in the right order, and if a test fails its dependencies will be skipped. This is likely improve acceptance testing. If login test doesn’t pass, you shouldn’t even try to launch all other tests with authorization required.

Here is the annotation format to use to declare dependencies. It should be {fullClassName}:{testName} or {testName} for the test in the same class.

@depends login
@depends UserCest:login
@depends App\Service\UserTest:create

Unlike PHPUnit, you can’t pass data between dependent tests. Alternatively you can use Helpers or Codeception\Util\Fixtures class to store shared data.

Params

This lines are written just in the same time as corresponding pull request is merged:

Codeception 2.2 will allow you to use parameter in configuration files populated from environment variables, yaml configs or .env files. You can use parametrized vars in suite configs, by wrapping them in % placeholder, and adding one or several parameter providers using the params key in global config:

params: [env] # get prameters from environment vars

You can use Symfony-style .yml files, Laravel-style .env, or .ini files to receive params.

This feature would be useful if you want to share database configuration with application and tests. This will especially be useful if you use some sort of credentials passed via environment vars. For instance, if you use cloud testing services, you can set login and password with params and get the real values from environment:

    modules:
       enabled:
          - WebDriver:
             url: http://mysite.com
             host: '%SAUCE_USER%:%SAUCE_KEY%@ondemand.saucelabs.com'
             port: 80
             browser: chrome
             capabilities:
                 platform: 'Windows 10'

Conflicts

This is possibly a breaking feature which was announced in Codeception 2.1 but never worked the way it supposed to be. So let’s talk about it once again. A friendly reminder, if you use config like this:

modules:
    enabled: [PhpBrowser, Laravel5, WebDriver]

you will have problems in Codeception 2.2, if you didn’t have them earlier. Basically the issue here, that those modules provide pretty the same API but different backends for test executions. So it’s hard to tell from config, will this tests be executed as Laravel functional tests, as acceptance tests in real browser, or HTML-scrapper called PhpBrowser. We have lots of issues, when developers misuse those modules, trying to include everything in one config, and we can’t help them besides the good advice to choose one module of one kind.

To avoid this confusion we introduced Conflicts API. Module can declare conflict for modules of the same API, and ConfigurationException will be thrown if you try to use them in group.

But what if you really need to use Laravel5 module along with, let’s say, WebDriver? Don’t worry, partially loaded modules won’t get into conflict. The most common use case is using Laravel ORM methods to create-delete data for acceptance test. In this case you don’t need to load all actions of Laravel5 module but only those related to ORM:

modules:
    enabled:
        - WebDriver:
            url: http://localhost
            browser: firefox
        - Laravel5:
            part: orm

this way you receive haveRecord, seeRecord, grabRecord, have, haveMultiple methods running though Eloquent but amOnPage, see, etc will be executed through WebDriver.

If you were mixing WebDriver with PhpBrowser in order to use REST API inside acceptance tests, you can still have it. Don’t enable PhpBrowser but set it as dependent module for REST:

modules:
    enabled:
        - WebDriver:
            url: http://localhost/
            browser: firefox
        - REST:
            url: http://localhost/api
            depends: PhpBrowser

This way you can use API to create test data for application:

<?php
$I = new AcceptanceTester($scenario);
// let's create a post
$I->sendPOST('/posts', ['title' => 'Hello, Codeception 2.2', 'body' => 'Almost here']);

// check it is actually there
$I->amOnPage('/posts');
$I->see('Hello, Codeception', 'h2');

…and much more to come

Sure, that was not everything what is coming in 2.2! Stay tuned for new cool announcements, which will be posted soon. We will describe new modules, new test formats, and new features! Stay tuned.

Why would you someone ever choose Codeception if Symfony already has mature testing infrastructure. Well, here are our reasons. Codeception tests are:

• fast, as each functional/integration test is wrapped into transaction using Doctrine ORM
• scenario-driven, it means that tests are linear, described in easy to get PHP DSL
• can be used for testing complex interactions inside functional tests.
• easy to write, as Codeception already provides bundled actions and assertions for most popular use cases.
• combine all testing levels (acceptance, functional, unit) in one tool.

Today we will write how Codeception can be installed into a Symfony project and fit with its structure: we will put functional and unit tests to corresponding bundles, write acceptance tests for complete application, and use one runner to execute them all at once.

We will use official symfony-demo application in this example. Once you get it installed you should add Codeception testing framework to dev dependencies list in composer.json:

composer require --dev "codeception/codeception:~2.1"

As you may know Codeception has bootstrap command which create common file structure for acceptance, functional, and unit tests. However, as we decided we will follow Symfony way and skip creating global functional and unit tests. So we start with bootstrapping empty project without predefined suites.

php bin/codecept bootstrap --empty

As you may see, we got new tests directory and codeception.yml file created. Let’s have acceptance tests there:

php bin/codecept g:suite acceptance

Acceptance tests are expected to test the site from an end-user’s perspective. No matter how many unit tests you have in your projects you can’t get without acceptance testing. What if you see a blank page even all unit tests passed. How could this happen? Maybe you rendered wrong template, maybe some scripts or styles were not loaded. Those things can’t be handled with internal: unit or functional testing, however with acceptance tests you may be confident that UI is available for customers.

That’s why we recommend to have tests with real browser interaction. You can learn more about acceptance testing from our guides.

But what about unit and functional tests? As we decided we will put them into bundles. Symfony-demo has only AppBundle included, so we will create new Codeception setup in src/AppBundle. Take a note that we want those tests to be placed in their own namespace:

php bin/codecept bootstrap --empty src/AppBundle --namespace AppBundle

We will also create unit and functional suites there:

php bin/codecept g:suite functional -c src/AppBundle
php bin/codecept g:suite unit -c src/AppBundle

As you noticed we specify path to different Codeception setup with -c or --config option.

Unit tests of Codeception are not quite different from regular PHPUnit tests. You can even copy your old PHPUnit tests to src/AppBundle/tests/unit and have Codeception run them. It is much more interesting to use Codeception to have functional tests replacing ones extending Symfony\Bundle\FrameworkBundle\Test\WebTestCase class.

Let’s have a test that will check that there is specific number of posts on a page. Symfony-demo app has the similar test included:

 
<?php
namespace AppBundle\Tests\Controller;
use Symfony\Bundle\FrameworkBundle\Test\WebTestCase;
use AppBundle\Entity\Post;

class BlogControllerTest extends WebTestCase
{
  public function testIndex()
  {
      $client = static::createClient();
      $crawler = $client->request('GET', '/en/blog/');
      $this->assertCount(
          Post::NUM_ITEMS,
          $crawler->filter('article.post'),
          'The homepage displays the right number of posts.'
      );
  }
}

We will rewrite it in Codeception manner. At first we are generating new empty test case for it. We use scenario-driven test format called Cest:

php bin/codecept g:cest functional BlogCest -c src/AppBundle

And here goes the test:

 
<?php
namespace AppBundle;
use AppBundle\Entity\Post;

class BlogCest 
{
    public function postsOnIndexPage(FunctionalTester $I)
    {
        $I->amOnPage('/en/blog/');
        $I->seeNumberOfElements('article.post', Post::NUM_ITEMS);
    }
}
?>

As you see, Codeception test is shorter. It is simple, clean, and can be easily extended for more complex interactions. However, we are not ready to run it yet. We need to prepare Codeception to run functional tests inside Symfony context. For this we need to edit src/AppBundle/tests/functional.yml configuration file to enable modules: Symfony2 and Doctrine2 to use:

class_name: FunctionalTester
modules:
    enabled:
        - Symfony2:
            app_path: '../../app'
            var_path: '../../app'
        - Doctrine2:
            depends: Symfony2
        - \AcmeBundle\Helper\Functional

The most important thing here is to provide valid app and var paths for Symfony. Also we are specifying that Doctrine’s EntityManager should be taken from Symfony DIC. Let’s run functional tests of AppBundle:

php bin/codecept run functional -c src/AppBundle

In this case you will see following output:

Codeception PHP Testing Framework v2.1.2
Powered by PHPUnit 4.8.2 by Sebastian Bergmann and contributors.

AcmeBundle.functional Tests (1) ------------------------
Posts on index page (BlogCest::postsOnIndexPage)     Ok
--------------------------------------------------------


Time: 274 ms, Memory: 36.00Mb

OK (1 test, 1 assertion)

But you can do much more with functional testing. You can insert/assert data with Doctrine by using prepared methods like haveInRepository, seeInRepository of Doctrine2 module. You can perform complex web interactions like filling forms, clicking links, following redirects and much more with methods of Symfony2 module. Those modules are combined together and their methods are available in FunctionalTester class you are supposed to use for writing functional tests. If you are interested to see more complex Codeception tests, we’ve got them for you.

Btw, you can use Symfony2 and Doctrine2 module for writing your unit and integration tests as well.

But how can we run acceptance tests of a project with tests from AppBundle together? We need to edit codeception.yml configuration file in project root to make it. Let’s add those lines there:

include:
    - src/*Bundle

That’s it. For now Codeception will include all installations stored in Bundles on run. If you execute:

php bin/codecept run

you will probably see that BlogCest of AppBundle was executed as it was expected to.

The most complex thing in starting using Codeception with Symfony is have it configured. Despite Codeception is auto-connecting to Symfony framework and Doctrine you still have to do some changes to follow Symfony structure. Please take a detailed look into our forked version of symfony-demo project which we configured in the manner we described in this post. Please use similar configuration for all your Symfony projects.

Start using Codeception and discover how complex things can be tested in really simple manner. And once again, even functional and integration tests are really fast, as we start transaction before each test and rollback it afterwards. Write them as many as you need to, do not rely on unit tests only!

P.S. Symfony2 and Doctrine2 module is seeking for an active maintainer. If you work with Symfony and Codeception please contact us to join Codeception team!

If you didn’t track for the changes in master we will list all the new features here:

• We added Recorder extension, which is probably the most fancy feature you may try. Using it you can record test execution history by saving a screenshot of each step. This is handy for running tests on CI, debugging tests executed via PhantomJS or showing nice reports to your boss.

• Updated to Guzzle 6. Codeception can now work both with Guzzle v5 and Guzzle v6. PhpBrowser chooses right connector depending on Guzzle version installed.
• PSR-4: all support classes moved to tests/_support by default. Actors, Helpers, PageObjects, StepObjects, GroupObjects to follow PSR-4 naming style. New AcceptanceTester, FunctionalTester, UnitTester classes are expected to be extended with methods of common behaviors there. For instance, it is a good idea to place login method into the actor class:

<?php
class AcceptanceTester extends \Codeception\Actor
{
    use _generated\AcceptanceTesterActions;

    public function login()
    {
       $this->amOnPage('/login');
       $this->fillField('name', 'jon');
       $this->fillField('password', '123345');
       $this->click('Login');
    }
}
?>

• Dependency Injection: support classes can be injected into tests. Support classes can be injected into each other too.

<?php
class UserCest 
{
  // inject page objects into Cests
  function updatePassword(\Page\User $page, AcceptanceTester $I)
  {
      $page->openProfile();
      $page->editProfile();
      $I->fillField($page->oldPasswordField, '123456');
      $I->fillField($page->newPasswordField, '654321');
      $I->fillField($page->passwordFieldRepeat, '654321');
      $I->click($page->saveBtn);
      $I->see('Password has been updated');
  }

  // inject step object into Cest
  function adminUpdatePassword(\Step\Admin $I)
  {
      $I->enterAdminArea();
      $I->changePassword('654321');
      $I->see('Password has been updated');
  }
}
?>

• Module config simplified: Modules can be configured in enabled section of suite config. Take a look of this sample declaration of Api suite, there is no config section inside modules.

modules:
    enabled:
        - WebDriver:
            url: https://codeception.com
            browser: firefox
        - \Helper\Acceptance

• Dependencies: module can explicitly define dependencies and expect their injection. REST, SOAP and Doctrine2 modules rely on another module which should be explicitly set via depends config param.

class_name: ApiTester
modules:
    enabled:
        - REST:
            url: https://api.github.com/
            depends: PhpBrowser           
        - \Helper\Api

As you can see, you don’t need to specify PhpBrowser in enabled section, you can set it only via depends.

• Conflicts: module can define conflicts with each other. Modules that share similar interfaces like WebDriver, PhpBrowser, and framework modules won’t run together. You should avoid enabling more than one of those modules in suite config to avoid confusion. If you enable Laravel5 and WebDriver and execute $I->amOnPage('/') you can’t be sure how this command is exected - will it open a browser window using WebDriver protocol, or it will be handled by Laravel framework directly.

• Current modules, environment, and test name can be received in scenario.

<?php
$scenario->current('name'); // returns current test name
$scenario->current('modules'); // returns current modules
$scenario->current('env'); // returns environment

// can be used like this
if ($scenario->current('env') == 'firefox') {
  // do something for firefox only
}
// naming screenshot
$I->makeScreenshot($scenario->current('name').'_screenshot.png');
?>

• Environment Matrix: You can run tests combining several environments by separating their names by comma:

codecept run --env dev,phantom --env dev,chrome --env dev,firefox

Environments can now be stored in separate configuration files in tests/_envs dir created with generate:environment command:

tests/_envs
|── chrome.yml
|── phantom.yml
|── firefox.yml
|── dev.yml

• Cept files should avoid setting their metadata via $scenario methods. Instead of calling $scenario->skip(), $scenario->group('firefox'), etc, it is recommended to set scenario settings with annotations // @skip, // @group firefox. However, you can use $scenario->skip whenever you need to do it on some condition, like

<?php
if (substr(PHP_OS, 0, 3) == 'Win') $scenario->skip()
?>

• Improved HTML reports

• Modules can be partially loaded. If you need only some actions to be included into tester objects. For instance, you want to have REST API tests with Laravel5 module. In this case you probably don’t want methods from Laravel module like amOnPage or see to be included into the ApiTester as they interact with HTML pages, which we are not supposed to use. But you still need Laravel ORM methods like seeRecord to verify that changes were saved to database. In this case you can enable only ORM methods of Laravel5 module.

modules:
    enabled: 
        - Laravel5:
            part: ORM
        - REST:
            part: Json
            depends: Laravel5

As for REST module you can load methods only for API format you are using. You can choose either XML or Json, so only methods for Json or XML will be loaded.

• Whenever you have steps grouped inside actor class, pageobject, or stepobject, you can the step executed with its substeps in console or HTML report.
• RunFailed extension is enabled by default for all new projects. It means that you can rerun your failed tests by executing codecept run -g failed when this extension is enabled. 1

And lots of other notable improvements which you can see in changelog.

Starting from Codeception 2.1 we recommend using Cest as a default format for all scenario-driven acceptance and functional, or api tests. Guides were rewritten to reflect new improved approaches to testing that you should practice by using Codeception.

Upgrading

• It is pretty easy to upgrade from 2.0. The only thing you should start with is to rebuild your actor classes by running codecept build command. The old actor classes (*Tester or *Guy) in suite directories should be deleted manually.
• REST, SOAP, Doctrine2 module will need to be configured to use a dependent module as it was shown above. You will get a detailed exception with configuration example once you execute tests with those modules.
• Helpers, PageObjects, StepObjects are expected to follow PSR-4 standard and use namespaces. It is recommended to rename classes to replace suffixes with namespaces, like UserPage => Page\User. However, those changes are not required, so you can keep your support objects without changes.

If you are using 2.0 and you won’t plan to upgrade in nearest future, you can still use releases from 2.0 branch. Minor fixes and improvements will still be accepted there. Also the site will contain documentation for 2.0 and 2.1 versions.

Try Codeception 2.1 today by installing it via Composer:

composer require "codeception/codeception:*" --dev

or by downloading it as Phar archive

And provide a feedback!

• REST, SOAP and Doctrine2 modules rely on another module which should be explicitly set via depends config param. For instance, you should pass Laravel5 module if you do API testing with REST module

modules:
    enabled: [REST, Laravel5]
    config:
        REST:
            depends: Laravel5

• Actor classes are moved to _support directory. Please delete all previously generated actor classes which reside in suites directories. New AcceptanceTester, FunctionalTester, UnitTester classes are expected to be extended with custom methods.
• Modules that share similar interfaces like WebDriver, PhpBrowser, and framework modules won’t run together. You should avoid enabling more than one of those modules in suite config to avoid confusion. If you enable Laravel5 and WebDriver and execute $I->amOnPage('/') you can’t be sure how this command is exected - will it open a browser window using WebDriver protocol, or it will be handled by Laravel framework directly.
• Cept files should avoid setting their metadata via $scenario methods. Instead of calling $scenario->skip(), $scenario->group('firefox'), etc, it is recommended to set scenario settings with annotations // @skip, // @group firefox. With this change you can now skip tests based on a condition: if (substr(PHP_OS, 0, 3) == 'Win') $scenario->skip().
• Kohana, Symfony1, Doctrine1 modules were removed and they reside in separate repositories in Codeception organization on GitHub.

That should be enough to have everything upgraded. We already prepared updated guides (they didn’t change from 2.1.0-beta). Please try new Codeception and tell us your opinion in comments or on GitHub.

Codeception 2.1 can be used with latest Guzzle 6 (as well as previous Guzzle releases). That’s the most important change since the beta version. See the changelog for more info. We are preparing site updates and demo apps to reflect new changes. Stay tuned!

Download 2.1.0-rc phar

via composer.json:

"codeception/codeception": "2.1.0-rc1"

Take a look what is landed in Codeception 2.1.

Full list of features is available in changelog. Documentation was updated accordingly.

You can install Codeception 2.1 via composer by requiring 2.1.0-beta version, or by downloading phar.

Give new Codeception a try and send us feedback. As always use GitHub issues for that. Thanks for being with us all that time! We hope you will love this release.

Stable 2.1.0 is expected by the middle-end of May

Upgrade Notes

• If you are upgrading from Codeception 2.0 delete actor classes in tests/acceptance, tests/functional, etc, and rebuild actor classes. They will be recreated in tests/_support.
• Try to execute new tests. If there are unmet dependencies, or module conflicts, you will be notified by exception with an advice how to resolve it.
• It is recommended to recreate Helpers per suite with g:helper generator, like codecept g:helper Acceptance. New helpers will be store in tests/_support/Helper.
• Same for PageObjects and StepObjects, they should be moved to _support.

A frequent question that appear from time to time is How to set up Codeception with Jenkins. Despite the growth of popularity of various Continuous Integration solutions Jenkins is still the most popular open-source solution on market. It is easy to setup and is easy to customize by applying various 3rd-party plugins. In this post we will take a look on how to setup testing process of PHP project with Jenkins CI and Codeception.

Preparing Jenkins

We will start from an empty Jenkins Server executed on localhost. Let’s create a new Job for project tested with Codeception and name it PHPCodeceptionProject. Before we proceed to configuring it, lets install required plugins:

• Git Plugin - for building tests for Git repo
• Green Balls - to display success results in green.
• xUnit Plugin, jUnit Plugin - to process and display Codeception XML reports
• HTML Publisher Plugin - to process Codeception HTML reports
• AnsiColor - to show colorized console output.

Basic Setup

Once everything is installed lets get back to configuring our newly created PHPCodeceptionProject. At first we will set up Git to checkout code and build. Depending on your needs you can set up periodical build or trigger build once the change is pushed to GitHub (you will need GitHub plugin for that). You may also set up Jenkins as alternative to Travis CI to build your pull requests with GitHub pull request builder plugin.

We will enable colors in console using Ansi Color

Next and the most important part is to define build step. We won’t create any sofisticated environment settings like setting up database and services. We assume that all we need to execute tests is to inctall composer dependencies and run codeception tests:

composer install
./vendor/bin/codecept run

And that’s the absolute minimum we need to execute tests in Jenkins. We can save project and execute the job. Jenkins will clone repository from Git, install composer dependencies and run Codeception tests. If tests fail you can review the console output to discover what went wrong.

XML Reports

But we don’t want to analyze console output for each failing build. Especially If Jenkins can collect and display the results inside its web UI. Codeception can export its results using JUnit XML format. To generate XML report on each build we will need to append --xml option to Codeception execution command. Codeception will print result.xml file containing information about test status with steps and stack traces for failing tests. Unfortunately Jenkins can’t process such special like Codeception steps. So we will need to configure to print data only which strictly applies to xUnit xml format. This should be done in codeception.yml

settings:
    strict_xml: true

Now let’s update our build step to generate xml:

composer install
./vendor/bin/codecept run --xml

and ask Jenkins to collect resulted XML. This can be done as part of Post-build actions. Let’s add Publish xUnit test result report action and configure it to use with PHPUnit reports.

Now we should specify path to PHPUnit style XML reports. In case of standard Codeception setup we should specify tests/_output/*.xml as a pattern for matching resulted XMLs. Now we save the project and rebuild it.

Now for all builds we will see results trend graph that shows us percentage of passing and failing tests. We also will see a Latest Test Result link which will lead to to the page where all executed tests and their stats listed in a table.

HTML Reports

To get more details on steps executed you can ask Codeception to generate HTML report. Jenkins can display them as well.

composer install
./vendor/bin/codecept run --html

Now we need HTML Publisher plugin configured to display generated HTML files. It should be added as post-build action similar way we did it for XML reports.

Jenkins should locate report.html located at tests/_output/. Now Jenkins will display HTML reports for each build.

That’s pretty much everything you need to get a simple Jenkins setup with Codeception.

This post summarizes some recent work that was done for creating portable testing environments. The idea was to reduce dependencies required by acceptance testing. As you know, testing of web pages require Selenium or PhanromJS servers, which allow to interact with web page, and execute user scripts on page. However, installing those dependencies, can be a pain. Especially Selenium. If you are not running it inside desktop environment you probably need to have Firefox or Chromium, and they can’t be run without virtual framebuffer, as they have some display server to run on. Setting up those packages can take liters of sweat and bunch of nerves.

That’s why we decided to pack Selenium and PhantomJS into portable Docker containers. Did you hear about Docker? If not - you probably should take some time to look at it. Docker allows you to create virtual environments on Linux hosts without real virtualization. Docker creates an isolated environment inside the host machine without recreating operating system.

Docker containers are lightweight and easy to use. If you have Linux. Probably all your servers use Linux, so you might find docker images of Selenium and PhantomJS pretty useful!

SeleniumEnv: Selenium, Xvfb, Firefox, and Chromium Inside Container

OK, lets assume you are using Linux and Docker >=1.2. From now on running Selenium won’t take from you anything more then running those commands:

docker run -i -t -p 4444:4444 davert/selenium-env

This will start virtual display server (Xvfb) and Selenium inside a container. Please note, you don’t need anything of those installed on your host. Not even Java, required to run Selenium. Only Docker is needed and you can run Selenium in one line!

By running this command you will see this output:

Running Selenium Env: Selenium Server, and Xvfb with Firefox and Chromium
Host IP: 172.17.42.1
[dix] Could not init font path element /usr/share/fonts/X11/cyrillic, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/100dpi/:unscaled, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/75dpi/:unscaled, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/Type1, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/100dpi, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/75dpi, removing from list!
[dix] Could not init font path element /var/lib/defoma/x-ttcidfont-conf.d/dirs/TrueType, removing from list!
02:02:30.641 INFO - Launching a standalone server
02:02:30.670 INFO - Java: Oracle Corporation 24.60-b09
02:02:30.670 INFO - OS: Linux 3.16.0-24-generic amd64
02:02:30.684 INFO - v2.44.0, with Core v2.44.0. Built from revision 76d78cf
02:02:30.735 INFO - Default driver org.openqa.selenium.ie.InternetExplorerDriver registration is skipped: registration capabilities Capabilities [{platform=WINDOWS, ensureCleanSession=true, browserName=internet explorer, version=}] does not match with current platform: LINUX
02:02:30.790 INFO - RemoteWebDriver instances should connect to: http://127.0.0.1:4444/wd/hub
02:02:30.791 INFO - Version Jetty/5.1.x
02:02:30.792 INFO - Started HttpContext[/selenium-server/driver,/selenium-server/driver]
02:02:30.792 INFO - Started HttpContext[/selenium-server,/selenium-server]
02:02:30.793 INFO - Started HttpContext[/,/]
02:02:30.806 INFO - Started org.openqa.jetty.jetty.servlet.ServletHandler@1a966bcb
02:02:30.807 INFO - Started HttpContext[/wd,/wd]
02:02:30.809 INFO - Started SocketListener on 0.0.0.0:4444
02:02:30.809 INFO - Started org.openqa.jetty.jetty.Server@2b6ea258

By specifying -p 4444:4444 option in docker run we mapped container’s port 4444, which is default for Selenium to the same port on our host machine. So, any process can connect to Selenium on its standard port, even Selenium is completely isolated from other processes.

And here comes a tricky question. If Selenium is completely isolated, how can we test applications running on a local web server? We have a solution for that. If your application is served by nginx or any other web server, you can probably access it by hostname (other than localhost). What we can do is to pass hostname inside a container.

docker run -i -t -p 4444:4444 -e APP_HOST=myhost davert/selenium-env

This will bind myhost to your host machine IP. So, container, and Firefox, Chromium, can get access to your local web site from inside container.

Similar idea is used for application executed on specific port. If you use PHP built-in server, you can start it at specific port, and pass it into container as well. Please note, that to make server accessible from container, it should be started on 0.0.0.0 IP:

php -S 0.0.0.0:8000
# in new terminal window
docker run -i -t -p 4444:4444 -e APP_PORT=8000 davert/selenium-env 
# in new terminal window
codecept run acceptance

PhantomJsEnv: PhantomJS Inside Container

Instructions are the same for running PhantomJS. It can be started with

docker run -i -t -p 4444:4444 davert/phantomjs-env

Everything else is pretty similar. Also you can easily switch from Selenium to PhantomJS by starting or stopping those two containers. Isn’t that useful?

Images and Source Code

SeleniumEnv and PhantomJsEnv are created from Travis CI Cookbooks and packed into containers. They are free to use and free to modify for custom needs.

If you need to have Selenium Server from version other than 2.44.0 (which is latest for today), you should update Docker file and build an image from it.

SeleniumEnv and PhantomJsEnv may simplify testing for all Linux users. They will definitely simplify setting up Continuous Integration servers. You can even recreate a complete testing environment inside a container. Take a look into RoboCI project for this. It allows you to build Travis CI-like service on your own host!

Use them and enjoy :)

P.S. Feedback, Pull Requests, and Issues, are welcome )

But it looks like factories finally came to PHP! The League of Extraordinary Packages produced a great package, self-claimed as factory_girl for PHP. Meet Factory Muffin. It allows simple generation of database records for integration and functional testing. Let’s see how it can be used with Codeception.

Setup in Codeception

At first lets add league/factory-muffin": "~2.0 to composer.json:

"require-dev": {
    "codeception/codeception": "~2.0",
    "league/factory-muffin": "~2.0"
}

Then we execute composer install.

Loading composer repositories with package information
Updating dependencies (including require-dev)
  - Installing fzaninotto/faker (v1.4.0)
    Downloading: 100%         

  - Installing league/factory-muffin (v2.1.1)
    Downloading: 100%         

Writing lock file
Generating autoload files
Generating optimized class loader

As you may noticed, FactoryMuffin uses awesome Faker library for generating random data for models.

For using factories inside tests we will create FactoryHelper:

php vendor/bin/codecept g:helper Factory
Helper /home/davert/project/tests/_support/FactoryHelper.php created

We will define factories in the _initialize method of FactoryHelper. Let’s say we have two models, Post and User in application. This is how we specify rules for generating new objects of these models.

<?php
class FactoryHelper extends \Codeception\Module
{
    /**
     * @var \League\FactoryMuffin\Factory
     */
    protected $factory;
    
    public function _initialize()
    {
        $this->factory = new \League\FactoryMuffin\Factory;
        $this->factory->define('Post', array(
            'title'    => 'sentence|5', // title with random 5 words
            'body'   => 'text', // random text
        ));

        $this->factory->define('User', array(
            'email' => 'unique:email', // random unique email
        ));
    }
}
?>

This is how generation of Post and User is defined. From the box, Muffin is designed to work with ActiveRecord models, and with Eloquent in particular. So you can use it in Yii, Phalcon, yet we will demonstrate its usage in Laravel application. FactoryMuffin can also be customized to work with Doctrine as well.

Using Factories in Laravel

To use FactoryMuffin in Laravel functional tests we need to create additional methods in FactoryHelper for generating and saving records: havePosts and haveUsers methods. They will populate database with number of records specified. We will need to clean up those records at the end of a test.

<?php
class FactoryHelper extends \Codeception\Module
{
    /**
     * @var \League\FactoryMuffin\Factory
     */
    protected $factory;
    
    public function _initialize()
    {
        $this->factory = new \League\FactoryMuffin\Factory;
        $this->factory->define('Post', array(
            'title'    => 'sentence|5', // title with random 5 words
            'body'   => 'text', // random text
        ));

        $this->factory->define('User', array(
            'email' => 'unique:email', // random unique email
        ));
    }

    public function havePosts($num)
    {
        $this->factory->seed($num, 'Post');
    }

    public function haveUsers($num)
    {
        $this->factory->seed($num, 'Post');
    }

    public function _after(\Codeception\TestCase $test)
    {
        // actually this is not needed
        // if you use cleanup: true option 
        // in Laravel4 module
        $this->factory->deleteSaved();
    }
}
?>

By including FactoryHelper to the functional suite config, we can use it inside the actor ($I) object.

This allows us to test features like pagination. For instance, we can check that only 20 posts are listed on a page:

<?php
$I = new FunctionalTester($scenario);
$I->wantTo('check that only 20 posts are listed');
$I->havePosts(40);
$I->amOnPage('/posts');
$I->seeNumberOfElements('.post', 20);
?>

Source code of this example is on GitHub.

Factories in Unit Tests

Factories can also be used in unit testing. Let’s say user can create posts, and in order to optimize queries we are saving number of user posts in num_posts column of users table. We are going to test that this column is updated each time a new post by user is created.

We will need one more method added into FactoryHelper class:

<?php
public function produce($model, $attributes = array())
{
    return $this->factory->create($model, $attributes);
}
?>

After we include FactoryHelper into unit suite we can use its methods in tests:

<?php
class UserTest extends \Codeception\TestCase\Test
{
    /**
     * @var UnitTester
     */
    protected $tester;

    function testCounterCache()
    {
        $user = $this->tester->produce('User');
        $this->assertEquals(0, $user->num_posts);

        $this->tester->produce('Post', ['user_id' => $user->id]);
        $this->assertEquals(1, User::find($user->id)->num_posts);

        $this->tester->produce('Post', ['user_id' => $user->id]);
        $this->assertEquals(2, User::find($user->id)->num_posts);
    }
}
?>

Conclusion

As you see, factories make your tests clean and expressive. Forget about managing test data manually, forget about fixtures. Use FactoryMuffin.

Update FactoryHelper for Symfony2 and Doctrine modules

In order to maintain a predictable release cycle we plan to release a new bugfix version each month and deliver a new major version each 6 months. We tend to follow this plan for future, yet we are not ready to follow it strictly.

Today we have a few more announcements to sum up all recent work on Codeception and its support projects.

CodeCoverage

Latest updates to c3 brought full compatibility with Codeception 2.0. Also c3 can be installed and updated via Composer.

"require-dev": {
    "codeception/c3": "2.*"
},
"scripts": {
    "post-install-cmd": [
        "Codeception\\c3\\Installer::copyC3ToRoot"
    ],
    "post-update-cmd": [
        "Codeception\\c3\\Installer::copyC3ToRoot"
    ]
}

This allows to follow updates of code coverage collector (c3) easily. Also please note, that c3 issues were moved to main Codeception repo.

AspectMock

AspectMock was recently updated to version 0.5.0. Latest update is powered by Go Aop framework version 0.5. In new AspectMock fixed passing parameters by reference, and added ability to mock PHP functions.

AspectMock allows to replace PHP function in desired namespace with user-defined closure or a return single return value. For instance, you can stub file operations by mocking file_get_contents or file_put_contents. You can also mock time functions, in order to make tests stable to time changes.

<?php
namespace App\Cache;

$func = test::func('App\Cache', 'file_put_contents');
$cache = new FileCache;
$cache->store('hello'); // calls file_put_contents
$func->verifyInvoked(); // verifies file_put_content was called
test::cleanup(); // restore original function
?>

Same can be done for time functions:

<?php
namespace demo;
test::func('demo', 'time', 'now');
$this->assertEquals('now', time());
?>

You can install latest AspectMock via Composer:

composer require "codeception/aspect-mock:~0.5"

Codeception 2.1

Next major release of Codeception will include new features, breaking changes, with ability to keep backwards compatibility. Here are the most interesting features you should wait for:

• All support classes like actors, steps, pages, etc, will be moved to tests/_support directory.
• Support classes will use PSR-4 standard.
• Dependency Injection. All support and test classes can be included one into another: Pages can be injected into Helpers, Helpers can be injected into Cest classes, etc.
• Module Conflicts. Lots of issues are happening when users try ot include modules with same apis into one suite. Like PhpBrowser + Laravel4 + WebDriver. Those module is possible to be used together, but possibly that was not really intended to be used this way. Call to $I->amOnPage may be executed by any provided modules depending on the order of their declaration. This leads to unpredictable errors, and is the source of confusion. In 2.1 Codeception will allow to define module conflicts, and throw exceptions if modules are not expected to be used together.
• …and more

Global Installation

One of recently discussed topics for Codeception, was global installation of Codeception. From now on you can install Codeception globally using Composer:

composer global require 'codeception/codeception=2.0.*'

This will be recommended installation method since Codeception 2.1.

RoboCI

This project that desires its own post in this blog. It is open-source reproduction of Travis CI using Docker and Travis Cookbooks. RoboCI can be use to execute Travis CI tests locally or on remote server. It does not require VirtualBox nor Vagrant installed. It was executed with Codeception internal tests, with Selenium + Firefox, MySQL, PostgreSQL, MongoDb, and other services. You can read more about RoboCI and try it in your projects. In future we plan to provide consulting services on using RoboCI with Codeception or any other PHP testing framework. Stay in touch for updates!

So let’s not to concentrate on terms. Let’s concentrate on testing.

Testing Faster With Transactions

What drawbacks we receive if unit tests are connecting to database? Sure it will be dramatic slowdown. We will need to connect to database, insert data, and perform a cleanup afterwards. The most simple way of cleaning things up is to repopulate database completely. Yet, this is the slowest way. But there is a much better way of doing things. Especially for unit (and functional) tests. We can use transactions.

That’s right. We can cover a test into transaction, and rollback it after the test is finished. This is the fastest and the most stable way of working with database in unit tests. The only issue with it is that not all databases support nested transactions. For instance, in MySQL you can’t begin transaction and begin next transaction as there can be only one at one point. Yet, those nested transactions can be emulated. And here are good news: if you use one of popular PHP Frameworks: Laravel, Yii2, Phalcon, or Doctrine ORM, your framework already can do that! Thus, in unit test you can begin transaction in setup, and finish it in teardown of a test. Sure, you should use the framework’s database layer for that.

In Codeception we already use this feature for speeding up functional tests. But it was not that obvious that you can use the same practice for unit tests.

Let’s connect our framework to the unit.suite.yml. In current example we will use Laravel, yet very similar steps can be done for Yii2, Phalcon, and Symfony+Doctrine.

# Codeception Test Suite Configuration

# suite for unit (internal) tests.
class_name: UnitTester
modules:
    enabled: [Laravel4, UnitHelper, Asserts]

That’s all. This line of Laravel module will start the transaction before each test in suite.

Using ActiveRecord Helpers

Laravel, Phalcon, and Yii are frameworks that use ActiveRecord pattern for working with database. Codeception Framework modules have built-in helpers, which share the standard interface to work with database records.

Let’s write a sample Laravel test which demonstrates usage of Codeception helpers. In this example we will test activate method of User model. It activates user when provided activation key matches predefined one.

<?php
class UserTest extends \Codeception\TestCase\Test
{
    /**
     * @var UnitTester
     */
    protected $tester;

    protected $user_id;

    function _before()
    {
        // preparing a user, inserting user record to database
        $this->user_id = $this->tester->haveRecord('users', [
            'username' => 'John',
            'email' => 'john@snow.com',
            'activation_key' => '123456',
            'is_active' => 0

        ]);
    }

    function testUserCanBeActivatedWithValidKey()
    {
        // lookup for user with Eloquent API
        $user = User::find($this->user_id);
        // executing action
        $isActivated = $user->activate('123456'));
        // performing assertion
        $this->assertTrue($isActivated);
        // checking that data was actually saved into database
        $this->tester->seeRecord('users', [
            'id' => $this->user_id,
            'is_active' => 1
         ]);
    }

    function testUserNotActivatedWithInvalidKey()
    {
        $user = User::find($this->user_id);
        $this->assertFalse($user->activate('00000'));
        $this->tester->dontSeeRecord('users', [
            'id' => $this->user_id,
            'is_active' => 1
         ]);
    }
}
?>

As it was mentioned, similar test can be written for other mentioned frameworks. Depending on framework API ActiveRecord helper methods

• grabRecord
• seeRecord
• haveRecord
• doneSeeRecord

may take table name (Laravel) or model name (Phalcon, Yii2).

Who is that Tester?

$this->tester object used here is instance of Actor class and contain all methods of modules and helpers used in unit suite. It would be a good idea to put any shared code into helpers, and reuse across test cases by accessing $this->tester. This object is injected into any testcase that extends Codeception\TestCase\Test.

Conclusion

Sure, we could implement similar test using just Eloquent API without using Codeception helpers. Still, they are quite useful. Especially assertion methods like seeRecord which queries database for a record with given attributes.

The next notable change is see method of WebDriver module. Now it checks for a visible text on a page. If you need to check if text is present inside HTML source of a page you can use newly added seeInPageSource method for that. Thanks artyfarty for this bugfix.

Update

redownload your codeception.phar for update:

php codecept.phar self-update

for composer version

$ php composer.phar update codeception/codeception

Changelog Page

From now on there will be no blogposts for minor releases. There is new changelog page which displays changes in latest releases. You should refer to it in order to follow the updates. Important changes, features, bugfixes will be mentioned there.

That’s it. Codeception does not have 2-phases of execution and runs your tests as native PHP scenarios. It also does not require you to rebuild actor classes on configuration change - it is done automatically. And at last we don’t have Guys anymore. They were renamed in more neutral Tester objects. But it is up to you how will you call your actors: Testers, Guys, or even ninjas, those names can be configured!

Naming Conventions

We decided to change some naming conventions in Codeception.

• _log directory was renamed to _output. It was logical decision, as Codeception does not print any logs by default, but constantly uses _log directory for printing debug information and reports.
• _helpers was renamed to _support. Now it is more obvious that this directory may contain not only helpers but any kind of support code required for tests.

As it was said, Guys were renamed to Testers. Actor classes and helpers are named by suite. For instance, acceptance test will start with this line:

<?php
$I = new AcceptanceTester($scenario);
// and uses AcceptanceHelper
?>

In unit tests we simplified access to actor classes by just using $this->tester property (actor title). Looks a bit more readable.

<?php
function testSavingUser()
{
    $user = new User();
    $user->setName('Miles Davis');
    $user->save();
    $this->tester->seeInDatabase('users',array('name' => 'Miles', 'surname' => 'Davis'));
}
?>

If you are upgrading from 1.x you can still use codeGuy as it was left for compatibility.

Listed naming changes are recommended defaults. You don’t need to follow them if you plan to upgrade from 1.x.

Changes

Let’s briefly name some important changes added in Codeception 2.0

• PHP >= 5.4
• Upgraded to PHPUnit 4.x
• Upgraded to Guzzle 4.x
• Removed Mink (and Selenium, Selenium2, ZombieJS modules)
• Removed Goutte
• Logger moved to extension and disabled by default
• Naming conventions changed
• One-time execution for all tests
• Autorebuild of actor classes
• MultiSession testing with Friends
• Strict Locators introduced
• Fail fast option -f added to run command
• Coverage options changed behavior in run command
• Bootstrap test are loaded suite, not before each test
• Parallel Execution (!!!)

Guides

Guides were reviewed and updated for 2.0 release. Unit Testing guide was completely rewritten, and new chapter Parallel Execution was added. Please note that parallel running tests is advanced feature and will require some time for you for set everything up correctly. But If your tests run longer then 30 minutes, you will get a real benefit out of it.

The Most Important Bug Fixed

The Most Important Bug was reported by many of our users:

@codeception Nice website footer: © 2011 - <?php date('Y') ?> You should create a test for that ;-)

— Bret R. Zaun (@bretrzaun) November 15, 2013

Hey, @codeception you have unparsed PHP in your footer.

— Phil Kershaw (@PhilKershaw) April 10, 2014

Oh @codeception !!!!!! Should _really_ fix this. lol. <3<3<3 pic.twitter.com/Wi3szPMKMN

— Darren Nolan (@DarrenNolan_) May 23, 2014

We decided to fix it for 2.0 release. Now the date in footer is displayed correctly according to PHP rules (using <?=).

Upgrading

Codeception 2.0 was publishing pre-releases since February. Lots of developers already gave it a try and reported their issues. We hope the upgrade will go smoothly for you. Basically what you need is to:

• run build command
• remove any internal usage of Mink or Goutte
• remove any usage of $scenario->prepare() as it always return false now
• optionally remove $scenario->running() calls and $I->execute calls. They are not required anymore.
• take a look into PhpBrowser module which is uses Guzzle4 directly. You may have issues with using it (as it was completely rewritten). Please report those issues to GitHub.
• Selenium2 module should be replaced with WebDriver (as you already didn’t do so).
• if you use variables in _bootstrap.php, rename it to _fixtures.php and load it manually into your tests.

If you have issues during upgrade send issues to GitHub and we will make upgrade more smoother.

wget https://codeception.com/codecept.phar

Via Composer:

composer require --dev "codeception/codeception:2.0.0" 

P.S. Codeception 1.8.6 was released as well. List of its changes will go tomorrow.

As you know PHPBroswer of Codeception used Mink, Goutte, and Guzzle to emulate visiting pages, filling forms, and other user-like web interaction. Recently Guzzle got a new major release, so we had to upgrade Codeception as well. We also removed Mink and Goutte and Codeception uses Guzzle directly. This will allow us faster to hunt down and fix the issues which may appear in future.

Guzzle changed their package name and namespace in latest release. If you were using Guzzle 3 and you want to upgrade to Codeception 2 you won’t get any conflicts in dependencies. But if you were using Guzzle\Http\Client class in your tests or helpers you will to change it to GuzzleHttp\Client.

PHPBrowser additional configuration

Guzzle4 got very flexible configuration. You may omit CURL_ constants and use request parameters to set headers, auth, cookies, and SSL verification. Codeception now accepts all request parameters of Guzzle in module configuration part:

 
modules:
   enabled: [PhpBrowser]
   config:
      PhpBrowser:
         url: 'http://localhost'
         auth: ['admin', '123345]
         headers:
            'User-Agent': 'Codeception'
            Accept: 'application/json'
         curl:
             CURLOPT_RETURNTRANSFER: true

And yes, curl configuration section is left for more advanced options and for backwards compatibility. BTW, SSL settings should be also defined via new configuration using verify, cert, ssl_key request options.

Try it

As usual we need your feedback. Guzzle upgrade is very dramatic change and we recommend to try it on your test suites. If you have issues report them and we will try to fix them before the final release, which is schenduled for next week.

Download:

wget https://codeception.com/releases/2.0.0-RC2/codecept.phar

Via Composer:

composer require --dev "codeception/codeception:2.0.0-RC2" 

Improvements

Strict Locators

In order to provide better element locations you can now specify locators explicitly like this: ['css' => 'input.name']. This will work for PHPBrowser, Frameworks modules, and WebDriver.

<?php
$I->fillField(['css' => 'input.name'], 'jon');
$I->click(['link' => 'Submit']);
$I->seeElement(['xpath' => './/div[@data-name="jon"]']);
?>

Allowed locator mechanisms are:

• id
• name
• css
• xpath
• link
• class

You can use strict locators in your Page Object classes as well.

See Element with Attributes

To improve checking element with expected attributes seeElement now takes second parameter:

<?php
$I->amOnPage('/form/field');
$I->seeElement('input[name=login]');
$I->seeElement('input', ['name' => 'login']); // same as above
?>

dontSeeElement works in corresponding manner.

grabAttributeFrom method added

To fetch value of attribute of element:

<?php
$I->amOnPage('/search');
$attr = $I->grabAttributeFrom('form', 'method');
$attr == 'GET'
?>

Also makeScreenshot method of WebDriver was simplified. Filename of saved screenshot does not include a test name.

Try it

Download:

wget https://codeception.com/releases/2.0.0-RC/codecept.phar

Via Composer:

composer require --dev "codeception/codeception:2.0.0-RC" 

Ecosystem

But before we proceed you should check out the new Addons section on this site. We had only extensions listed there, but now you can find new modules and even applications there. And let’s name at least two projects you should definitely be aware of:

• WebCeption - web interface for Codeception.
• VisualCeption - module for visual regression tests.

Thank you for all extensions and modules developers. If you developed a module, helper, or extension, share your work with others by adding your project to addons page (click Edit link in the bottom).

Bugfixes

• [WebDriver] facebook/webdriver version was set strictly to 0.3. And this won’t be changed in future for 1.8 branch. We accidentaly commited ~0.3 into the 1.8.4 which made composer to install 0.4 version of facebook/webdriver. There was API break in 0.4, so WebDriver module didn’t work properly.
• [Phalcon] Phalcon 1.3 compatibility added #944
• [Laravel] Service providers are now correcly loaded by ShawnMcCool.
• [Laravel] findRecord fixed by andkom.
• [ZF2] Fixed handling of query parameters by piccagliani.
• [Frameworks] Fixed problem with submitting form with input[type=image] by piccagliani.

Features

• [Webdriver][Frameworks] input field can now be located by its name.
• [Webdriver][Frameworks] element can be clicked by name too.

Update

redownload your codeception.phar for update:

1.8.5

php codecept.phar self-update

for composer version

$ php composer.phar update codeception/codeception

• [WebDriver] Correctly return to main page when switchToIFrame() is called by ** n8whnp**.
• [WebDriver] fixed screenshot capture by FnTm.
• [Frameworks] More details in debug output by Ragazzo.
• Fixed problem with encoding in $I->wantTo() by vgoodvin.
• Add clone and unset capabilites to Maybe by brutuscat.
• [WebDriver] Fixes to submitForm when typing password by pcairns.
• [Phalcon1] haveRecord return id even when id is not public by xavier-rodet.
• Modules to be loaded from global context by Ragazzo. Now you can pass long name of module into module configuration:

class_name: WebGuy
modules:
    enabled: [\My\Custom\Module, WebHelper]

Really helpful feature if you have multiple Codeception inits in one project and you want to have shared module.

All those features were included in 2.0-beta. Thanks to all contributors!

Update

redownload your codeception.phar for update:

1.8.4

php codecept.phar self-update

for composer version

$ php composer.phar update codeception/codeception

Changes

• Upgraded to PHPUnit 4.0
• Upgraded to facebook/webdriver 0.4
• RunFailed extension added. To rerun tests on fail include Codeception\Platform\RunFailed extension, and call codecept run -g failed.
• Logger disabled by default and moved to Extension. You will need to install Monolog by yourself, and enable Codeception\Platform\Logger as extension.
• Methods _before/_after of Cest can use $I object.
• Destination for xml/html/reports can be customized. For instance

codecept run --xml myreport.xml
codecept run --xml /home/davert/report.xml
codecept run -g database --xml database.xml
```

• --coverage + --xml or --html won’t produce xml or html codecoverage output. Use new options coverage-xml and coverage-html instead. They were added so you could specify custom destination for codecoverage reports as well.
• you can get current environment in a test by accessing $this->env (even in Cept)
• shortcut functions added: codecept_debug to print custom output in debug mode, codecept_root_dir, codecept_log_dir, codecept_data_dir for retrieving Codeception paths.
• extensions can change global config before processing.

Breaking Changes

In Codeception 1.x bootstrap files were used differently for different types of tests. They were introduced to pass variables (and fixtures) into Cept scenarios. They were almost useless in Cests, and absolutely useless for Test files. They are not removed at all, but now they are loaded only once before the suite. If you were using bootstrap files for setting fixtures, you still can use them, but you will need to require them manually.

<?php
require __DIR__ . '/_bootstrap.php'

$I = new WebGuy($scenario);
?>

The same way you can load bootstrap files into Cests. But we recommend to create _fixtures.php file, and use bootstrap file for suite initialization.

Dynamic Groups

One significant internal feature is dynamic groups. You can save test names into file and run them inside a group. That’s how the RunFailed extension works: it saves names of failed tests into file tests/_log/failed, then execute codecept run -g failed to run these tests.

groups:
  # add 2 tests to db group
  db: [tests/unit/PersistTest.php, tests/unit/DataTest.php]

  # add list of tests to slow group
  slow: tests/_log/slow

For example, you can create the list of the most slow tests, and run them inside their own group. Pretty interesting and powerful feature that also can be used to run tests in parallel.

Parallel Test Execution

Not yet! Codeception does not support parallel test execution, nor will be provide it out of the box. That’s because there is no solution that will fit for everyone. You may want to run parallel tests locally in threads via pthreads extension, or run them on different hosts via SSH, AMQP, etc. But we already have everything to implement parallel testing.

Here is the algorithm:

• split all tests into groups (with dynamic groups)
• run each groups separately
• merge results

And we will write guides on implementing this algorithm, as well as we plan to release a sample tool that will run tests in parallel.

Try it

As usual, we need your feedback to get 2.0 released as stable.

Download:

wget https://codeception.com/releases/2.0.0-beta/codecept.phar

Via Composer:

composer require --dev "codeception/codeception:2.0.0-beta" 

P.S. Our sponsors 2Amigos updated their site and logo. Check it out!

Codeception: Not Just For Guys

Before 2.0 there were only Guys in it. That is not fair! We wanted to make Codeception a tool for everyone: for guys, girls, developers, test engineers, ninjas and even wookiees… (nope, wookiees should wait for 3.0).

That’s why you can now choose the desired actor from a list during the bootstrap process.

Before proceed you can choose default actor:

$I = new [ACTOR]
  Select an actor. Default: Guy  
  [0] Guy
  [1] Girl
  [2] Person
  [3] Engineer
  [4] Ninja
  [5] Dev

so you all your test actors (that’s how we call guy classes now) will be in form like TestGirl, WebNinja, CodeDev, etc. Pretty flexible.

Codeception: Not Alone

Before Codeception 2.0 guys (or should we say actors now) were left to themselves. You know, it is so sad to see that there is only $I in the test, forever alone, like on a desert island, or in space… But in 2.0 you can invite some friends into your tests. Let’s say…

<?php
$I = new WebGuy($scenario);
$nick = $I->haveFriend('nick');
?>

So we can write multi-session tests that can be executed in two browser windows. You may try to run this test on github.com to see how it works:

<?php
$I = new WebGuy($scenario);
$I->wantTo('surf Github with Nick');
$I->amOnPage('/');
$I->submitForm('#top_search_form', array('q' => 'php-webdriver'));
$nick = $I->haveFriend('nick');
$nick->does(function(WebGuy $I) {
    $I->amOnPage('/Codeception/Codeception');
    $I->click('Issues');
    $I->canSeeInTitle('Issues');
});
$I->click('li.public:nth-child(1) > h3:nth-child(3) > a:nth-child(1) > em:nth-child(2)');
$I->seeInTitle("php-webdriver");
$nick->does(function(WebGuy $I) {
    $I->click('Milestones');
    $I->canSeeInTitle('Milestones');
});
$I->seeCurrentUrlEquals('/facebook/php-webdriver');
$I->click('Issues');
$I->canSeeInTitle('Issues');
?>

As you see, everything in does closure is executed in separate session. This way you can test user-to-user interactions on your site. For example Alice writes a private message to Bob and Bob responds back. Similarly you can have multiple REST sessions in a test.

Such scenario cases can be implemented. That’s what friends are for.

Notable Changes

As it was announced earlier, the main goals for Codeception 2.0 was internal refactoring.

• Mink (and its drivers) was removed completely. Instead you can use WebDriver module to do Selenium testing, and PhpBrowser (which uses Goutte) for browser-emulation. PhpBrowser module is now more compatible with frameworks modules, they use the same methods and acts in the same manner. If you were using Selenium or Selenium2 module you should switch to WebDriver, for PHPBrowser everything should (crossing fingers) work smoothly.
• 2-phases test execution with tricky magic including usage of Maybe class was removed. Tests are now executed one time, like any regular PHP file. So you can use any PHP code in your tests, and appearance of Maybe object would not confuse you anymore.

<?php
$card = $I->grabTextFrom('#credit_card')
var_dump($card); // was showing `Maybe` instead of real value
?>

• Codeception 2.0 require PHP 5.4 and higher. Time changes, PHP 5.3 is getting harder and harder to support, thus we decided to move to 5.4 and keep our code base up to date. And yes, we wanted to use short array syntax. We are tired to keeping write all those nasty array() stuff.
• Actor classes (initial Guy classes) are now rebuilt automatically. Thus, you want get exception when change suite configuration, or add methods to helper. Rebuilds are not required anymore.
• Added Assert module that can be used to write common asserts in your tests. You can now use seeEquals, seeContains and other actions inside your Cepts or Cests.
• Experimental: added Silex module. We need your Feedback on using it.

Minor Internal Changes

Refactoring, Refactoring, Refactoring. We use PSR-2 now. We rewrote CodeCoverage. We have better directory structure… More new files. What else? Oh, lets admit it, these are not the changes you would actually notice. But internals are now more clean and easy to understand (Except the parts which heavily rely on PHPUnit).

Upgrading

We’d like to ask you to try Codeception 2.0 on your projects. Before the release is ready we need to collect feedback and fix all encountered issues. You know where GitHub issues are.

Download:

wget https://codeception.com/releases/2.0.0-alpha/codecept.phar

Via Composer:

composer require --dev "codeception/codeception:2.0.0-alpha" 

Development of 2.0 now happens in master branch, so keep track on changes and send your pull requests.

Some Upgrading Notes

• Run build command
• Replace Selenium2 to WebDriver module
• Check you don’t use PHPBrowser->session property anywhere (it was Mink part)
• CodeCoverage with c3 will require new version of c3.

What’s next?

We need your feedback, and meanwhile we will work on updating documentation parts. 1.8.x will be maintained, but new features will be added to 2.x branch.

Laravel, Yii2, and Phalcon frameworks implement ActiveRecord pattern. That’s why all database actions in this modules look and work in a very similar manner.

Laravel 4.1

<?php
$user_id = $I->haveRecord('users', array('name' => 'Davert'));
$I->seeRecord('users', array('name' => 'davert'));
$I->dontSeeRecord('users', array('name' => 'davert'));
$user = $I->grabRecord('users', array('name' => 'davert'));
?>

This methods will work for Laravel 4 as well, but Laravel 4.1 supports nested transactions and allows us to wrap functional test into one database transaction. This is really useful, as we can rollback any database changes we do in functional tests. Tests also run really fast, as nothing is written in database. A new cleanup config option was introduced to Laravel4 module, and by default it is turned on.

Now it is really simple to use database in your functional tests. Don’t hesitate and try *Record methods in action!

Also nice seeSessionErrorMessage was added by elijan to perform validation error assertions.

Yii2

Yii2 is in very active development, and its official basic application is tested with Codeception, and uses Specify and Verify libraries. Yii2 module is tested on Travis as in official Yii2 repo and in Codeception repo as well.

<?php
$user_id = $I->haveRecord('app\model\users', array('name' => 'Davert'));
$I->seeRecord('app\model\users', array('name' => 'davert'));
$I->dontSeeRecord('app\model\users', array('name' => 'davert'));
$user = $I->grabRecord('app\model\users', array('name' => 'davert'));
?>

ActiveRecord methods work in very similar manner. We expect that Yii2 will have nested transactions support before the release.

Thanks to Ragazzo for Yii2 module contributions and bughunting.

Bugfixes

• CodeCoverage was improved. Remote codecoverage with WebDriver or PhpBrowser opens page, sends authorization cookie before the test.
• WebDriver cookies (and sessions) are destroyed after the test. If you have troubles when session is not restared in WebDriver - toggle restart option.

Update

redownload your codeception.phar for update:

1.8.3

php codecept.phar self-update

for composer version

$ php composer.phar update codeception/codeception

P.S. Yeah, yeah, Codeception 2.0 is on its way.

Changes:

• [REST] match similar elements in JSON arrays fixed in #837 by blacknoir.
• generate:pageobject now takes -c option correctly #809
• [REST] Fixed setting Content-Type header #827
• [REST] Headers are uppercased according to BrowserKit standard.
• [Db] records inserted with haveInDatabase now are cleaned in _after event by dirk-helbert #761.
• [Laravel] Fixed usage of Redirect::back in tests
• Fixed collecting CodeCoverage using WebDriver/Selenium2 modules.
• [REST] Fixed “Call to undefined method Symfony\Component\HttpFoundation\Request::getCookies() in codeception/codeception/src/Codeception/Module/REST.php line 352” by casconed #814
• Fixed: tests run twice if you use .dist.yml config by tomtomsen #582
• Environments: Test classes was loaded only once in multi-environment mode. Fixed by ayastreb and his nice tokenizer solition #812
• Excluding running abstract classes in addCest by filipgorny #792
• [PhpBrowser] Fixed setting cookies from headers
• [Framework] Form data on page was not cleaned after form submit. So when sending the same form twice, cached data was submitted. This is now fixed.
• and others…

Also we’ve got a Mockery module for using powerful Mockery framework with Codeception. It is not included in Codeception itself but can be installed via Composer. (Thanks to Jáchym Toušek)

Thanks to all contributors, thanks for getting bug fixed and reported. We are trying to be better with each release.

Update

redownload your codeception.phar for update:

1.8.2

php codecept.phar self-update

for composer version

$ php composer.phar update codeception/codeception

What’s Next

We are moving to Codeception 2.0. It’s first alpha is expected on next week. Stay tuned and get ready for new features!

While developing Codeception 2.0 we released a lightweight task runner Robo. We needed it to run routine tasks for Codeception - building phar archives, merging releases, etc. You may use it as lightweight alternative for Phing or PHP alternative for shell scripts. It’s not very documented, it doesn’t include list of all required tasks, but it will evolve. If you have ideas on improvement, or you want to add more tasks into it - please send Pull Requests.

And thanks to @pfaocle for the feedback on using Robo!

I've been playing w/ @davert's Robo this evening, converting a rickety shell script for building #Drupal. Very nice. pic.twitter.com/gGtJu0EWSC

— Paul Byrne (@pfaocle) January 29, 2014

For both questions we have an answer. There are two awesome services that have been developed to help developers in dealing with email hell. They are Mailtrap and Mailcatcher. Both services run an SMTP server that does not deliver emails, but stores them locally. They both have a web interface in which you can review all the outgoing emails. The difference between these services are: mailtrap runs as a web service, and mailcatcher is a ruby gem that can be installed locally.

It’s up to you which one to use. Definitely they will simplify your life while developing a web application. Do they have something to offer for testing? Sure! We can access all handled emails via REST API and verify our assertions.

In this post we will marry Mailcatcher with the PHPUnit testing framework. We’ve chosen Mailcatcher so we do not have to rely on a 3rd-party web service and have all the tests run locally. We will write methods for both PHPUnit and Codeception in order to provide different solutions and compare them.

Before we start we need to make sure that Mailcatcher is installed and running. When done you can access it’s web interface on port 1080 and use port 1025 for the fake SMTP server. Configure your web application to use exactly that port when running in test environment.

Testing emails in PHPUnit

Mailcatcher has a really simple REST API that is used for email access. Here is a quote from their official site.

A fairly RESTful URL schema means you can download a list of messages in JSON from /messages, each message’s metadata with /messages/:id.json, and then the pertinent parts with /messages/:id.html and /messages/:id.plain for the default HTML and plain text version, /messages/:id/:cid for individual attachments by CID, or the whole message with /messages/:id.source.

What was not mentioned was that you can also clear all emails by sending DELETE request to /messages. The most complete documentation on API is its code. Even if you don’t know Ruby, it is quite easy.

Thus, we will need to send GET and DELETE requests and parse the json response. To send them we will use the Guzzle framework. PHPUnit and Guzzle can be easily installed via Composer:

{
    "require-dev": {
    	"phpunit/phpunit": "*",
    	"guzzle/guzzle": "~3.7"
    }
}

Let’s create EmailTestCase file and place MailCatcher API calls into it.

<?php
class EmailTestCase extends PHPUnit_Framework_TestCase {

    /**
     * @var \Guzzle\Http\Client
     */
    private $mailcatcher;

    public function setUp()
    {
        $this->mailcatcher = new \Guzzle\Http\Client('http://127.0.0.1:1080');

        // clean emails between tests
        $this->cleanMessages();
    }

    // api calls
    public function cleanMessages()
    {
        $this->mailcatcher->delete('/messages')->send();
    }

    public function getLastMessage()
    {
        $messages = $this->getMessages();
        if (empty($messages)) {
            $this->fail("No messages received");
        }
        // messages are in descending order
        return reset($messages);
    }

    public function getMessages()
    {
        $jsonResponse = $this->mailcatcher->get('/messages')->send();
        return json_decode($jsonResponse->getBody());
    }
?>    

That’s enough to fetch a list of all delivered emails. All the emails will be cleaned between tests, so each test will be executed in isolation. Let’s implement some assertion methods to check the sender, recipient, subject and body of the email.

<?php
    // assertions
    public function assertEmailIsSent($description = '')
    {
        $this->assertNotEmpty($this->getMessages(), $description);
    }
    
    public function assertEmailSubjectContains($needle, $email, $description = '')
    {
        $this->assertContains($needle, $email->subject, $description);
    }

    public function assertEmailSubjectEquals($expected, $email, $description = '')
    {
        $this->assertContains($expected, $email->subject, $description);
    }

    public function assertEmailHtmlContains($needle, $email, $description = '')
    {
        $response = $this->mailcatcher->get("/messages/{$email->id}.html")->send();
        $this->assertContains($needle, (string)$response->getBody(), $description);
    }

    public function assertEmailTextContains($needle, $email, $description = '')
    {
        $response = $this->mailcatcher->get("/messages/{$email->id}.plain")->send();
        $this->assertContains($needle, (string)$response->getBody(), $description);
    }

    public function assertEmailSenderEquals($expected, $email, $description = '')
    {
        $response = $this->mailcatcher->get("/messages/{$email->id}.json")->send();
        $email = json_decode($response->getBody());
        $this->assertEquals($expected, $email->sender, $description);
    }

    public function assertEmailRecipientsContain($needle, $email, $description = '')
    {
        $response = $this->mailcatcher->get("/messages/{$email->id}.json")->send();
        $email = json_decode($response->getBody());
        $this->assertContains($needle, $email->recipients, $description);
    }
?>

The complete code listing is published as gist.

Example

How might a test using this EmailTestCase look?

<?php		
	function testNotificationIsSent()
	{
		// ... trigger notifications

        $email = $this->getLastMessage();
        $this->assertEmailSenderEquals('<bugira@bugira.com>', $email);
        $this->assertEmailRecipientsContain('<davert@ukr.net>', $email);
        $this->assertEmailSubjectEquals('[Bugira] Ticket #2 has been closed', $email);
        $this->assertEmailSubjectContains('Ticket #2', $email);
        $this->assertEmailHtmlContains('#2 integer pede justo lacinia eget tincidunt', $email);
    }
?>    

We now have a really simple class for testing emails from your application. Ok, that’s not a unit test. For unit testing you should use a mocking framework to fake the delivery in your PHP code. But you can use this class in acceptance tests (with Selenium) or integration tests. It is much simpler to test emails this way, than to dig into the internals of your email sending library and define mocks. The drawbacks here are the usage of standalone daemon, and reconfiguring your application to use its SMTP server.

It looks like this post is long enough to be published. We will continue email testing next time with Codeception framework. We will develop EmailHelper class for scenario-driven tests of Codeception.

EmailTestCase Source

Small yet important change in WebDriver module. Default wait parameter is set to 0. This was done because Selenium implicit waits didn’t work as expected - this parameter slowed down test execution. Browser waited for element even if it was already on a page. Please notice this on upgrading.

Changes

• upgraded to php-webdriver 0.3
• added general sendAjaxRequest() method to all framework/phpbrowser modules to send ajax requests of any types by gchaincl.
• fixed URI construction in Yii1 module by kop(also in 1.7)
• Fixed Yii2 statusCode by cebe
• fixed: Placeholder\Registry::unsetRegistry() should only be used with < 2.2.0 by Bittarman
• waitForElementChange() was fixed in WebDriver module
• fixed seeLink and dontSeeLink methods in framework modules by enumag.
• added seeHttpHeaderOnce to REST module for checking if headers appear only once.
• setUpBeforeClass/tearDownAfterClass method will work as they are expected in PHPUnit
• Debug::debug can output any variable in console.
• fixed: “WebDriver makeScreenshot doesn’t create directories” by joksnet
• fixed grabValueFrom method in framework modules (also in 1.7)
• fixed: “Disable remote coverage not work” by tiger-seo (also in 1.6, 1.7)
• self-update command added to phar by frqnck

Dependencies were updated, thus Symfony components were updated to 2.4. No changes to composer.json was made, so this release is compatible with Symfony 2.3 as well.

Update

redownload your codeception.phar for update:

1.8.1

wget https://codeception.com/codecept.phar -O codecept.phar

1.7.4

wget https://codeception.com/releases/1.7.4/codecept.phar -O codecept.phar

for composer version

$ php composer.phar update

Installation

Depending on your preferences you can install Codeception either by downloading codecept.phar archive from the site, or alternatively by using composer.

{
    "require-dev": {
        "codeception/codeception": "*",
    }
}

With composer you will need to execute:

php composer.phar install

In previous tutorial we did installation using Composer, so in current examples we will be using it as well.

Sure, we also need Selenium server executable as well. You need Java installed in order to run the Selenium server. You can launch it by running this:

java -jar selenium-server-standalone-2.37.0.jar

When all installation steps are done, we can continue with creating Codeception bootstrap.

Bootstrap

Unlike the phpunit Codeception requires a small bootstrap step. Codeception helps you to organize tests into 3 categories: acceptance, functional, and unit tests. To create all tests and support directories, you will need to run the bootstrap command.

vendor/bin/codecept bootstrap

Selenium tests are acceptance tests. So let’s create a skeleton for the basic acceptance test:

vendor/bin/codecept generate:cept acceptance GitHub

This will create new file in tests/acceptance/GitHubCept.php. But we need some additional configuration to be done before proceeding. Instead of creating webdriver session in tests manually, we delegate this to Codeception. Codeception will take care for creating session before each test, and closing it after. That’s why Selenium configuration should be written into tests/acceptance.suite.yml.

class_name: WebGuy
modules:
    enabled:
        - WebDriver
        - WebHelper
    config:
        WebDriver:
            url: 'https://github.com'
            browser: 'firefox'

Each time you change configuration in Codeception you should run the build command.

vendor/bin/codecept build

Writing a Test

Let’s start with something very basic. We will open the Github page and we will make sure that GitHub word is within the page title.

<?php
$I = new WebGuy($scenario);
$I->wantTo('see GitHub word in title ');
$I->amOnPage('/');
$I->seeInTitle('GitHub');
?>

We are using the wantTo command just to give a test clean description. amOnPage command opens browser on the github home page, and all the commands that start with see are assertions you can use. There are lots of commands in WebGuy class you can use writing the test. All them are taking from corresponging modules, in our case it is WebDriver module. If you use IDE you can check them all with autocompletion.

But let’s execute a test with run command:

vendor/bin/codecept run

And you will see this output:

Codeception PHP Testing Framework v1.8.0
Powered by PHPUnit 3.7.28 by Sebastian Bergmann.

Functional Tests (0) ------------------------
---------------------------------------------

Acceptance Tests (1) -------------------------------------------
Trying to see github word in title  (GitHubCept.php)       Ok
----------------------------------------------------------------

Unit Tests (0) ------------------------------
---------------------------------------------


Time: 21.62 seconds, Memory: 5.00Mb

OK (1 test, 1 assertion)

You may have noticed that Codeception is itself powered by PHPUnit. Basically you can execute native PHPUnit tests inside Codeception, but the main idea of Codeception is scenario driven tests written from an eye of a tester. Each test should descibe user’s actions in simple words: I see, I click, etc. Let’s see how using just this simple terms with no OOP complexity we can write a bit more complex test.

Our test should open github in a browser and use the search form to get to “facebook/php-webdriver” library page. Before writing the test we did some research of GitHub page to find selectors we can use to match elements on page. This was described in previous tutorial.

<?php
$I = new WebGuy($scenario);
$I->wantTo('find facebook/php-webdriver on GitHub');
$I->amOnPage('/');
$I->fillField('#js-command-bar-field','php-webdriver');
$I->pressKey('#js-command-bar-field', WebDriverKeys::ENTER);
$I->click('li.public:nth-child(1) > h3:nth-child(3) > a:nth-child(1) > em:nth-child(2)');
$I->seeInTitle("php-webdriver");
$I->seeCurrentUrlEquals('/facebook/php-webdriver');
?>

You can execute this test and see it passes. Unlike the classical PHPUnit test, this code does not require comments. It is very clean and easy to understand and edit by anyone a team with basic PHP/HTML/CSS skills.

Tips and Tricks

In this section we will share some ideas, that you can use in your tests. Sure that would not be the same tips we have in PHPUnit. But in PHPUnit test we wrote a helper function to pause the execution. This helped us to search for required locators. In Codeception (starting from 1.8) we can use $I->pauseExecution(); method for this. The test will stop the scenario in that place and wait for Enter to be pressed. Worth to mention, that pauseExecution works only in debug mode (sure, you don’t want to freeze when running tests on CI). So you should pass --debug option when using it.

But even Codeception provides us with dozen of predefined commands, we might want to access webdriver session by your own. For this we have a executeInSelenium command:

<?
$I->executeInSelenium(function(RemoteWebDriver $webDriver) {
	$webDriver->getKeyboard()->pressKey(WebDriverKeys::ENTER);
});
?>

But that doesn’t make our test more readable. What can we do to simplify it a bit? Well, let’s replace this double search for ‘#js-command-bar-field’ input field. We can use submitForm command that works for forms on any kind, and you can pass array of field values into it. Current form id is top_search_form, so we can rewrite our test like:

<?php
$I = new WebGuy($scenario);
$I->wantTo('find facebook/php-webdriver on GitHub');
$I->amOnPage('/');
$I->submitForm('#top_search_form', array('q' => 'php-webdriver'));
$I->click('li.public:nth-child(1) > h3:nth-child(3) > a:nth-child(1) > em:nth-child(2)');
$I->seeInTitle("php-webdriver");
$I->seeCurrentUrlEquals('/facebook/php-webdriver');
?>

When running WebDriver tests with Codeception they may look a bit slower then the same tests run with PHPUnit. That’s because Codeception has default delay of 5 seconds for each test action on page. This delay can be set to 0 in order to run test faster. This is configured in acceptance.suite.yml

class_name: WebGuy
modules:
    enabled:
        - WebDriver
        - WebHelper
    config:
        WebDriver:
            url: 'https://github.com'
            browser: 'firefox'
            wait: 0

This should execute the same test 30% faster then before. But if your page use JavaScript and Ajax, you should increase the wait time or use various waitFor commands of WebDriver.

Conclusion

At first sight setting up Codeception Selenium test may look a bit harder then with PHPUnit. At first sight it’s hard to see real benefits in it. But the key idea of Codeception is to separate the test code and support code. So you should write only tests, and most of helper methods was already written for you. Thus, the test is kept clean and readable. It is easy to change and easy to manage. If you want to switch a browser, you should not edit a test, but change the configuration. You want to execute test in 2 browsers one by one? No problems, just change a configuration.

Codeception is very flexible framework that you can use to write your Selenium tests. After all it’s really simple and fun.

Source file of this tutorial are available on GitHub.

Phalcon Framework Support

Phalcon is the fastest PHP framework in the world. And that’s not just a marketing slogan. Phalcon was developed as C extension, so it is already precompiled and loads into memory only once. Phalcon is modern framework with DI container, ORM (inspired by Doctrine), and templating engine (inspired by Twig). It is one of the most innovative projects in PHP world, so you should at least check it out.

If you already work with Phalcon, here is a good news for you. Codeception got Phalcon1 module, that allows you to write functional tests with minimum setup. Besides standard framework interface this module provides actions for session and database access. It also wraps all tests into nested transactions, and rollbacks them in the end. Thus, all your database tests run superfast.

Is there any Phalcon project tested with Codeception? Yes, it is the official Phalcon’s forum.

This module was developed by cujo and improved by davert. We hope you like it.

Environments

This is something you might not expect in the form it was produced, but, probably, that was long awaited. This is to run tests multiple times over different environments. The most common issue is running acceptance tests over different browsers: firefox, phantomjs, chrome. Now you can create 3 different configurations and you can get tests will be repeated 3 times: in firefox, in phantomjs, and in chrome. The second usecase is run tests over different databases.

Feature is pretty straightforward in use. You define the name of environment below the env key, and then you redefine any of configuration values you need.

``` yaml
class_name: WebGuy
modules:
    enabled:
        - WebDriver
        - WebHelper
    config:
        WebDriver:
            url: 'http://127.0.0.1:8000/'
            browser: 'firefox'

env:
    phantom:
         modules:
            config:
                WebDriver:
                    browser: 'phantomjs'

Advanced Usage chapter was updated.

DataProviders

You can use PHPUnit dataproviders in Codeception\TestCase\Test files. Yep. Finally.

Probably DataProviders are not really readable, as you need always to refer into data sets, which may be defined in the different part of a testcase. You can consider using examples of Codeception\Specify library, as for alternative for dataproviders.

Is there a way you can use data providers in scenario driven test? Not exactly, but you can emulate them with loops and conditional asserts:

<?php
foreach ($posts as $post) {
	$I->canSee($post->title,'.post h2');
}
?>

Depends

Declare depending tests in Cest and Test files. Works just as the original @depends of PHPUnit. In Cests you can combine this with @before annotation. More information in Advanced Usage.

Debug

Debug output was refactored, and moved out to Codeception\Util\Debug class. This class can be used globally, i.e in tests, helpers, - wherever you want. To print debug information you should call:

<?php
use Codeception\Util\Debug;
Debug::debug("This is working");
?>

This change dramatically improves debug output. You can also pause execution with pause static method of this class. Useful for debugging and tests development, implemented in WebDriver module as pauseExecution action.

Bugfixes and Minor Changes

• WebDriver module got pauseExecution method which pauses running test in debug mode.
• Generated PageObject file URL const was changed to static variable.

• waitForElementChange() callback return value was not being used by wheelsandcogs

  also in 1.7

• bugfix for making screenshots with WebDriver by Michael Wang.

  also in 1.7

• Doctrine2: handle NULL value in seeInRepository param array by imanzuk

  also in 1.7

• REST: fix adding parameters to url to non-GET HTTP methods by sheershoff
• REST: Added sendOPTIONS() and sendHEAD() requests for CORS testing by elazar
• Symfony2: fixed usage of profiler

• Strict declaratin error fixes for framework constraints

  also in 1.6, 1.7

Update

Warning. Module Unit was deprecated in 1.6 and was removed in 1.8. Please disable it in unit.suite.yml if you were using it. If you see this error:

Codeception PHP Testing Framework v1.8.0
Powered by PHPUnit 3.7.28 by Sebastian Bergmann.



  [Codeception\Exception\Configuration]
  Unit could not be found and loaded

Just disable Unit module. Thanks

If you prefer stability over features you can stay on 1.7 or 1.6 releases. We’ve got them updated too.

redownload your codeception.phar for update:

1.8.0

wget https://codeception.com/codecept.phar -O codecept.phar

1.7.3

wget https://codeception.com/releases/1.7.3/codecept.phar -O codecept.phar

1.6.12

wget https://codeception.com/releases/1.6.12/codecept.phar -O codecept.phar

for composer version

$ php composer.phar update

What to expect from 2.0

It is almost 2 years of Codeception, and we are planning to release 2.0 version as a next major to celebrate that. It is a major change, thus we can add few BC breaks. We are planning to:

• move to PHP 5.4. Not really necessary, yet it is getting hard to support 5.3.
• remove Mink entirely in favor of WebDriver for browser-testing and Goutte for headless testing.
• remove 2-times test execution (finally!).
• ??? (proposed by you)

Selenium allows us to record user actions that we do inside a browser and then automate them. PHPUnit will be used to do various assertions and check them for fallacy. And php-webdriver is used to connect PHP with Selenium, in order to do browser manipulation in PHP.

Probably you know, that PHPUnit itself can do Selenium manipulations via PHPUnit. There is PHPUnit_Selenium extension you may use. We will use php-webdriver instead because this implementation is modern and its API is more clean and looks much the same way as original Java Selenium API. That’s why it is easier to learn, and much powerful, then the PHPUnit’s one. For example, it allows you use send native touch events, which is important in era of mobile-ready web applications.

Grab the tools

Let’s install all the required tools using Composer. For this we will need to have composer.json file created:

{
    "require-dev": {
        "phpunit/phpunit": "*",
        "facebook/webdriver": "dev-master"
    }
}

We won’t develop any application, thus we are ok, with require-dev section only. Let’s run

php composer.phar install

and grab the latest versions of both libraries. We also will need Selenium server executable as well. You need Java installed in order to run the Selenium server. You can launch it by running this:

java -jar selenium-server-standalone-2.37.0.jar

And when the tools are prepared, let’s write some tests.

PHPUnit Test

So let’s try to test something in the web. Let’s start with something simple and well known, like Github. So, let’s create a file and call it GitHubTest.

<?php
class GitHubTests extends PHPUnit_Framework_TestCase {
}
?>

As any PHPUnit test it should extend PHPUnit_Framework_TestCase (as it was mentioned, we are not using PHPUnit_Extensions_Selenium2TestCase here). For every test we will need to launch a browser, or in other words, we are starting a Selenium session. This is done by creating RemoteWebDriver instance:

<?php
class GitHubTests extends PHPUnit_Framework_TestCase {

    /**
     * @var \RemoteWebDriver
     */
    protected $webDriver;

	public function setUp()
    {
        $capabilities = array(\WebDriverCapabilityType::BROWSER_NAME => 'firefox');
        $this->webDriver = RemoteWebDriver::create('http://localhost:4444/wd/hub', $capabilities);
    }

}
?>

This initialization part is taken from php-webdriver README file. We don’t have any test yet, so let’s write something a very basic. Something like: “If I open https://github.com, page title should contain GitHub”.

<?php
class GitHubTest extends PHPUnit_Framework_TestCase {

    /**
     * @var \RemoteWebDriver
     */
    protected $webDriver;

	public function setUp()
    {
        $capabilities = array(\WebDriverCapabilityType::BROWSER_NAME => 'firefox');
        $this->webDriver = RemoteWebDriver::create('http://localhost:4444/wd/hub', $capabilities);
    }

    protected $url = 'https://github.com';

    public function testGitHubHome()
    {
        $this->webDriver->get($this->url);
        // checking that page title contains word 'GitHub'
        $this->assertContains('GitHub', $this->webDriver->getTitle());
    }    

}
?>

Now we execute our first test with phpunit

vendor/bin/phpunit GitHubTest.php

and in a few seconds we should see a Firefox window with Github Page in it

please notice the WebDriver text in the status bar, this tells you that this browser window is controlled by WebDriver.

In a console we will see this output:

PHPUnit 3.7.28 by Sebastian Bergmann.


.

Time: 19.44 seconds, Memory: 1.75Mb

OK (1 test, 1 assertion)

We will see that test has finished, but the browser window stays opened. That is because we did not implement a tearDown method, that should be used to close the webdriver session:

<?php
class GitHubTest extends PHPUnit_Framework_TestCase {

    /**
     * @var \RemoteWebDriver
     */
    protected $webDriver;

	public function setUp()
    {
        $capabilities = array(\WebDriverCapabilityType::BROWSER_NAME => 'firefox');
        $this->webDriver = RemoteWebDriver::create('http://localhost:4444/wd/hub', $capabilities);
    }

    public function tearDown()
    {
        $this->webDriver->quit();
    }
}
?>

Advanced Test

We didn’t touch any of page elements in a previous test. We just opened the page and checked its title. But the power of webdriver reveals when you want to click elements, fill forms, drag and drop elements, etc. That’s why we will write a test that demonstrates some of this features.

But how can control the browser? Should we move the mouse in order to click on element? Well, not exactly. WebDriver allows us to locate element on page by its ID, class name, element name, CSS, or XPath. Let’s list all possible locator types, taken from WebDriverBy class:

• WebDriverBy::className() - searches for element by its CSS class.
• WebDriverBy::cssSelector() - searches for element by its CSS selector (like jQuery).
• WebDriverBy::id() - searches for element by its id.
• WebDriverBy::linkText() - searches for a link whose visible text equals to the value provided.
• WebDriverBy::partialLinkText() - same as above, but link partly contain the value.
• WebDriverBy::tagName() - search for element by its tag name.
• WebDriverBy::xpath() - search for element by xpath. The most complex, yet, most powerful way for element location.

To find element we should use webDriver->findElement method, with locator specified with WebDriverBy.

After the matched element is found we can click on it. Like this:

<?php
class GitHubTest extends PHPUnit_Framework_TestCase {

    /**
     * @var \RemoteWebDriver
     */
    protected $webDriver;

	public function setUp()
    {
        $capabilities = array(\WebDriverCapabilityType::BROWSER_NAME => 'firefox');
        $this->webDriver = RemoteWebDriver::create('http://localhost:4444/wd/hub', $capabilities);
    }

    public function tearDown()
    {
        $this->webDriver->close();
    }

    public function testSearch()
    {
        $this->webDriver->get($this->url);
        // find search field by its id
        $search = $this->webDriver->findElement(WebDriverBy::id('js-command-bar-field'));
        $search->click();
	}    
}
?>

We are clicking the GitHub global search field, located in top menu bar, matched by its id. By the way, how did we get the element’s id? That’s a good question. Searching for element locators is the most important task in acceptance testing. For every test we need to get the elements that are involved in it. Let’s show some simple tricks that will definitely help you in writing complex acceptance tests.

Locating Elements: Tips & Tricks

The first thing we can do is to pause the test execution. While browser window is still open, we can use it to find the locator. To pause the test execution lets write this helper method inside a test class:

<?php
    protected function waitForUserInput()
    {
        if(trim(fgets(fopen("php://stdin","r"))) != chr(13)) return;
    }
?>

If we use it somewhere in our tests, PHPUnit will wait for Enter key pressed in console before going on.

<?php
    public function testSearch()
    {
        $this->webDriver->get($this->url);
		$this->waitForUserInput(); // paused       
    }    
?>

Now when the browser window is opened we are free to search for required element with no hurry. We are using Firfox Developer Tools for that. With the Element Inspector within we can point to element and get its unique CSS locator.

That is how we got search field id: #js-command-bar-field. Doing the sample steps, let’s continue writing our test and find php-webdriver repository on GitHub.

<?php
class GitHubTest extends PHPUnit_Framework_TestCase {

    protected $url = 'https://github.com';
    /**
     * @var \RemoteWebDriver
     */
    protected $webDriver;

	public function setUp()
    {
        $capabilities = array(\WebDriverCapabilityType::BROWSER_NAME => 'firefox');
        $this->webDriver = RemoteWebDriver::create('http://localhost:4444/wd/hub', $capabilities);
    }

    public function tearDown()
    {
        $this->webDriver->close();
    }

    public function testSearch()
    {
        $this->webDriver->get($this->url);
        
        // find search field by its id
        $search = $this->webDriver->findElement(WebDriverBy::id('js-command-bar-field'));
        $search->click();
        
        // typing into field
        $this->webDriver->getKeyboard()->sendKeys('php-webdriver');
        
        // pressing "Enter"
        $this->webDriver->getKeyboard()->pressKey(WebDriverKeys::ENTER);
        
        $firstResult = $this->webDriver->findElement(
            // some CSS selectors can be very long:
            WebDriverBy::cssSelector('li.public:nth-child(1) > h3:nth-child(3) > a:nth-child(1) > em:nth-child(2)')
        );
        
        $firstResult->click();
        
        // we expect that facebook/php-webdriver was the first result
        $this->assertContains("php-webdriver",$this->webDriver->getTitle());
        
        // checking current url
        $this->assertEquals(
        	'https://github.org/facebook/php-webdriver', 
        	$this->webDriver->getCurrentURL()
        );
    }


    protected function waitForUserInput()
    {
        if(trim(fgets(fopen("php://stdin","r"))) != chr(13)) return;
    }

}
?>

If we run this test we will see that it is failing on the last step:

That’s because we forgot that GitHub uses https by default, and GitHub is a company and not a non-profit organization (as we used to think of it, he-he). Though let’s change the expected url to ‘https://github.com/facebook/php-webdriver’ and see the test is passing.

Element Not Found

Probably we will also want to check if element is located on a page. If we were using Selenium2TestCase of PHPUnit, we would have several nice assertion that we can use just for that. In case of php-webdriver library we will need to implement them on our own. But that’s pretty easy. Php-Webdriver throws various exceptions, which we can handle and transform into PHPUnit’s assertions:

<?php
    protected function assertElementNotFound($by)
    {
        $els = $this->webDriver->findElements($by);
        if (count($els)) {
            $this->fail("Unexpectedly element was found");
        }
        // increment assertion counter
        $this->assertTrue(true);        
    }

?>

You can create similar assertion just in the same manner.

We will use newly created assertElementNotFound method to check that there is no user avatar on “facebook/php-webdriver” page.

<?php
class GitHubTest extends PHPUnit_Framework_TestCase {

    protected $url = 'https://github.com';
    /**
     * @var \RemoteWebDriver
     */
    protected $webDriver;

	public function setUp()
    {
        $capabilities = array(\WebDriverCapabilityType::BROWSER_NAME => 'firefox');
        $this->webDriver = RemoteWebDriver::create('http://localhost:4444/wd/hub', $capabilities);
    }

    public function tearDown()
    {
        $this->webDriver->close();
    }

    public function testGitHubHome()
    {
        $this->webDriver->get($this->url);
        // checking that page title contains word 'GitHub'
        $this->assertContains('GitHub', $this->webDriver->getTitle());
    }

    public function testSearch()
    {
        $this->webDriver->get($this->url);

        // find search field by its id
        $search = $this->webDriver->findElement(WebDriverBy::id('js-command-bar-field'));
        $search->click();

        // typing into field
        $this->webDriver->getKeyboard()->sendKeys('php-webdriver');

        // pressing "Enter"
        $this->webDriver->getKeyboard()->pressKey(WebDriverKeys::ENTER);

        $firstResult = $this->webDriver->findElement(
            // some CSS selectors can be very long:
            WebDriverBy::cssSelector('li.public:nth-child(1) > h3:nth-child(3) > a:nth-child(1) > em:nth-child(2)')
        );

        $firstResult->click();

        // we expect that facebook/php-webdriver was the first result
        $this->assertContains("php-webdriver",$this->webDriver->getTitle());

        $this->assertEquals('https://github.com/facebook/php-webdriver', $this->webDriver->getCurrentURL());

        $this->assertElementNotFound(WebDriverBy::className('avatar'));

    }

    protected function waitForUserInput()
    {
        if(trim(fgets(fopen("php://stdin","r"))) != chr(13)) return;
    }

    protected function assertElementNotFound($by)
    {
        $els = $this->webDriver->findElements($by);
        if (count($els)) {
            $this->fail("Unexpectedly element was found");
        }
        // increment assertion counter
        $this->assertTrue(true);
        
    }
}
?>

Refactoring

To clean up some things we will separate test methods and support methods. It is a good idea to move custom assertions into trait: WebDriverAssertions. And the pause switcher waitForUserInput can be moved into WebDriverDevelop trait. We can enable this trait in a test class, when we develop a test, and turn it off once we finished.

The complete demo project, after this basic refactoring, you can find on GitHub.

And what about Codeception?

So you did notice that this is Codeception blog. But we didn’t use Codeception framework at all in this tutorial. Sure, we need to mention, that Codeception includes php-webdriver library and WebDriver module out of the box starting from version 1.7. In Codeception you can perform all the web manipulations in a much simpler manner using the WebGuy APIs. If you use Codeception you don’t need to implement your own WebDriver assertions nor write boilerplate code.

Conclusions

No matter you are using Codeception or not it is a good idea to understand how to perform browser based acceptance testing using just the php-webdriver by itself. Php-webdriver library provides very clean and flexible APIs you will enjoy working with.

PHP is a language popular for web development, but not for web testing. Test automation engineers prefer Java and Ruby over it. And there is a serious reason for that. There is no such thing like “official Selenium bindings” for PHP, i.e. there is no Selenium client library for PHP created by Selenium team. Developers of php-webdriver get very close to the official Selenium client APIs. That’s why you should use php-webdriver - it really feels and works like native and official Selenium libraries. That is especially important if you have an experience writing acceptance tests in Java or Ruby. Moving to PHP is not that hard, when all the APIs are the same.

Bugfixes:

• Updated to be able to use seeOptionIsSelected and dontSeeOptionIsSelected by label in framework modules by piccagliani
• Update to appendField function to handle checkbox value or label by allmyitjason
• Removed default delay WebDriver allmyitjason
• better error description whenever config file could not be found by tiger-seo.
• Added appendField function into WebDriver by allmyitjason
• switched to use getInternalResponse in frameworks modules. Fixes “PHP Fatal error: Call to undefined method **\Response::getStatus() “ in Symfony2, Laravel4.
• Fixed PhpBrowser module persisting HTTP authentication between tests by elazar
• Hardcoded ‘localhost’ removed for Yii1 by kop
• CodeCoverage works for WebDriver module as well by allmyitjason
• Improved output formatting for unit tests by davert

Some of this fixes were also added into 1.6.

Update

redownload your codeception.phar for update:

1.7.2

wget https://codeception.com/codecept.phar -O codecept.phar

1.6.11

wget https://codeception.com/releases/1.6.10/codecept.phar -O codecept.phar

for composer version

$ php composer.phar update

Bugfixes in 1.7.1

• error and failures are now displayed correctly with improved stack traces.
• fix for module before/after hooks in Codeception\TestCase\Test.
• select option in WebDriver throws readable message
• wait in WebDriver throws exception when receives 1000 seconds.

Bugfixes in 1.6.10 (also in 1.7)

• Fix the problem when the method getPort() return 443 80 or null by thbourlove.
• Notifies about CURL not installed.

Update

redownload your codeception.phar for update:

1.7.1

wget https://codeception.com/codecept.phar -O codecept.phar

1.6.10

wget https://codeception.com/releases/1.6.10/codecept.phar -O codecept.phar

for composer version

$ php composer.phar update

WebDriver

WebDriver module is pretty new, yet you may want to switch to it from Selenium2. It uses just the same interface, so migration should come smoothly. If you have issues, then no hurry, stay with Selenium2 for a while. Let’s list a few features that WebDriver module has:

Implicit waits

<?php
$I->waitForText('foo', 30); // secs
$I->waitForElement('#agree_button', 30);
$I->waitForElementChange('#menu', function(\WebDriverElement $el) {
    return $el->isDisplayed();
}, 100);
$I->waitForJS("return $.active == 0;", 60);
?>

Better Keyboard Manipoluation

<?php
// <input id="page" value="old" />
$I->pressKey('#page','a'); // => olda
$I->pressKey('#page',array('ctrl','a'),'new'); //=> new
$I->pressKey('#page',array('shift','111'),'1','x'); //=> old!!!1x
$I->pressKey('descendant-or-self::*[@id='page']','u'); //=> old!!!1xu
$I->pressKey('#name', array('ctrl', 'a'), WebDriverKeys::DELETE); //=>''
?>

submitForm

A submit form method was implemented in WebDriver.

<?php
$I->submitForm('#login', array('login' => 'davert', 'password' => '123456'));
?>

Common Selenium API

Thus If you got any Selenium questions any answers on StackOverflow will help you. PHP implementation is so close to Java that you can use any answer in PHP.

You can invoke WebDriver methods directly with executeInSelenium

<?php
$I->executeInSelenium(function(\WebDriver $webdriver) {
  $webdriver->get('https://google.com');
});
?>

Symfony Output

You will notice a better output formatting in your console. That’s because we migrated to Symfony Console output. As you may know, Symfony console has 3 levels of verbosity, that can set via -v option. Codeception now support them. If you want to get all available information about test, run with -vvv option. The --debug option is now equivalent for running -vv. That’s right, you can get even more information with vvv. The output will be improved during the development of 1.7 branch. We hope to get completely different output depending on level of verbosity set.

Cest Annotations

Cests are now much smarter then they were before. If you were using StepObject you might wondered how can you pass step object class into it. That was not really obvious, as by default you get Guy class defined in config. But now you can use guy annotation to specify which guy class to use.

<?php
/**
 * @guy WebGuy\AdminSteps
 */
class AdminCest {

	function banUser(WebGuy\AdminSteps $I)
	{
		// ...
	}

}
?>

Alternatively you can use guy annotation for the method itself.

Also you can now use before and after annotations to define whic methods of Cest class should be executed before the current one. Thus you can move similar actions into protected methods and invoke them via annotations.

<?php
class ModeratorCest {

	protected function login(WebGuy $I)
	{
		$I->amOnPage('/login');
		// ...
	}

	/**
	 * @before login
	 */
	function banUser(WebGuy $I)
	{
		$I->see('Logout');
		// ...
	}

}
?>

Just by using annotations you can control the invokations of methods of the Cest class. Sure, you should define your support methods with protected, so they won’t be executed as tests themselves. Another thing worth to mention, that callbacks defined in after annotation will be called even the main test has failed, thus it makes them useful for clean ups.

We still maintain and bugfix 1.6 branch and there will be 1.6 bugfix releases. The old release 1.6.9 (yep, 1.6.9 was released to with minor bugfixes) can be downloaded from https://codeception.com/releases/1.6.9/codecept.phar.

Removed Dependencies

Optional dependencies were removed from the core, so if you use Composer version and one of the following packages:

    "behat/mink-selenium-driver": "1.1.*",
    "facebook/php-sdk": "3.*",
    "behat/mink-zombie-driver": "1.1.*",
    "videlalvaro/php-amqplib": "*"

you should include them manually in your composer.json. If you use phar version - nothing is changed for you.

Update

redownload your codeception.phar for update:

wget https://codeception.com/codecept.phar -O codecept.phar

for composer version

$ php composer.phar update

In 1.7 you will see new Selenium WebDriver module, better output formatter moved to Symfony Components, and other useful features. It will come shortly during next week(s). But for now lets list bugfixes contributed by our community members. Thank you all!

Bugfixes

• ZF2 module fixes by Marcelo Araújo.
• Cleanup for custom enum types in PostgreSQL module by korotovsky.
• Disabled fields are sent by submitForm method no more. Thx to TrustNik.
• seeResponseIsXml added to REST module by FnTm. Expect more REST/Xml features in 1.7
• WebDriver module usage fixed.
• Fixed unit testing results display when used with CodeGuy object
• seeElement and dontSeeElement work correctly now.
• Conditional Asserts work as expected now.
• seeCookie can be used with different domain set. Thx vkn.
• fixed submitForm for forms that don’t have submit buttons in PhpBrowser by vkn.

Translations

It’s good for you to know, that Russian and Portuguese translations of Codeception Guides were started. If you know either of this langueges (or even both, he-he) please, help us get translations done! By now several chapters are already translated, but you know, there are still lot of work,

• Russian Guides. Editor: xomaa
• Portuguese Guides. Editor: Giuseppe

Update

redownload your codeception.phar for update:

wget https://codeception.com/codecept.phar -O codecept.phar

for composer version

$ php composer.phar update

Does this test checks something important? Maybe specification has changed and the test should be rewritten? The team who worked on tests that early days wrote them only to make them pass. Team is not sure what was the purpose of test.

In such situations it would have been nice if a developer who wrote the test at least has left some comments. But not. Usually no one documents tests. Test passes and developer is completely satisfied with that fact.

Proper test structure and readability is the only way to maintainable tests. Tests must not turn to a legacy code. Is there a way to write better tests?

The rule, dictated by BDD, is quite simple: write tests for specifications. Do not test just methods, test the behavior. As you know, there are plenty of BDD frameworks that replace classical unit testing with specification testing. You may have heard of RSpec in Ruby, Jasmine, Mocha and others in JavaScript.

If you ever did testing in JavaScript you know how popular mentioned BDD frameworks are. Why can’t we have something similar in PHP? We got PHPSpec which is nice, but looks much different from mentioned frameworks. What if we want something more usual? Something like Jasmine in PHP?

Even if we had such BDD framework none will ever adopt it as we have PHPUnit for all kind of testing in PHP. We won’t switch PHPUnit in favor of some geeky BDD tool. But actually to write BDD-styled tests, inspired by Jasmine we don’t need to do dramatic changes.

We can use Specify, a simple trait inserted into your PHPUnit’s TestCase that allows you to store several specifications in a test and write them in BDD way.

At first we will write down some specifications in a body of typical PHPUnit’s test:

<?php
// this is just a PHPUnit's testcase
class PostTest extends PHPUnit_Framework_TestCase {

	use Codeception\Specify;

	// just a regular test declaration
	public function testPublication()
	{
		$this->specify('post can be published');
		$this->specify('post should contain a title');
		$this->specify('post should contain a body');
		$this->specify('author of post should not be banned');		
	}
}
?>

Pretty sweet, we started with describing things before the test. But can’t we do the same with comments? specify method is much better then comments as it introduces code blocks into PHPUnit.

To see it in action, let’s write the tests.

<?php
// this is just a PHPUnit's testcase
class PostTest extends PHPUnit_Framework_TestCase {

	use Codeception\Specify;

	// just a regular test declaration
	public function testPublication()
	{
		$this->post = new Post;
		$this->post->setAuthor(new User());

		$this->specify('post can be published', function() {
			$this->post->setTitle('Testing is Fun!');
			$this->post->setBody('Thats for sure');
			$this->assertTrue($this->post->publish());
		});

		$this->specify('post should contain a title', function() {
			$this->assertFalse($this->post->publish());
			$this->assertArrayHasKey('title', $this->post->errors());		
		});

		$this->specify('post should contain a body', function() {
			$this->assertFalse($this->post->publish());
			$this->assertArrayHasKey('body', $this->post->errors());		
		});

		$this->specify('author of post should not be banned', function() {			
			$this->post->getAuthor()->setIsBanned(true);

			$this->post->setTitle('Testing is Fun!');
			$this->post->setBody('Thats for sure');			

			$this->assertFalse($this->post->publish());
			$this->assertArrayHasKey('author', $this->post->errors());
		});		
	}
}
?>