Last year I participated in a project, which goal was to prepare a platform to automate acceptance testing. For such task, Codeception emerges as the best solution, but it’s only a component of a bigger platform. This article contains my experience gained during creation of such platform for one of the biggest European eCommerce. It doesn’t cover much details though, it’s more of an overview than a tutorial, but hopefully it could help you to get oriented if you ever would like to try preparing such platform yourself. Tested application is the biggest German online furniture shop, available in 7 European countries. The vast number of articles, processing payments, and various layouts and options for different types of products made testing this project quite challenging.

Project structure

Writing acceptance tests for such extensive application requires a well thought concept. Approach recommended by Codeception is to split tests into suites that cover particular subpages, like cart page, or more widely – like the whole process of placing order. Those suites are called Cepts in Codeception. From technical point of view Codeption’s Cept is just a set of test classes. Furthermore, those classes also have their special name in Codeception – they’re called Cest files.

A good example to represent that idea could be “My Account” Cept, which covers part of the application related to user settings, billing and order history. Cept classes in this case are:

  • ManageMyAddressCest.php
  • ManageMyOrdersCest.php
  • ManageMyWishlistCest.php
  • UserAddressValidationCest.php
  • and so on

Story becomes code

Let’s assume, we want to test whether default billing and shipping addresses are the same as the ones given during registration. So, our story is:

  • As an unsigned user I want to register a new account
  • As a signed user I navigate to the settings page (successful registration signs user in)
  • Default billing address should be the same as the one given during registration
  • Default shipping address should be the same as the one given during registration

According to the behavior-driven development (BDD)
code of this simple test might look like this:

That’s all! Each line of the scenario is represented by, more or less, one line of code. It can’t be much simpler than that.

Ingredients

Where are all those famous “I see”, “I click”, “I don’t see”? They are all moved to adequate Page Object Models. LoginPage and ProfilePage are the two examples of those in action. In fact well prepared Page Object Model does all the job for a tester. So what is it?

Page Object Model

It’s a class which provides an interface for tested page. In particular, it “knows” all important CSS selectors and provides methods to manipulate or inspect any element of a concerned page. LoginPage page model for example, provides methods like checkRememberLoginCheckbox, submitForgottenPasswordPopup or assertLastNameGetsTruncatedWhenLongerThanAllowed. PHPdoc and self explanatory names make them really easy to use for person writing tests.

To make finding required controls or page elements easier, many elements got additional classes, prepended with ac- prefix, like ac-login-input-email. Those classes had no styles defined. Thanks to that, even if a page structure gets changed or element’s classes change, the model is still valid.

Well prepared Page Model allows avoiding code duplication and make tests look really easy to understand, even for a person with very little technical knowledge.

Modules

AccountTester is Codeception’s actor – a class that wraps all methods scattered amongst different modules. What are those modules and what are they for?

First of all, a tester needs an ability to select any possible product to test all quirks related to it. For example: the scenario requires to test if on a page of a product which is a part of some set, all remaining set’s elements are also displayed, or if user selects two products with different shipping times, but from the same warehouse – will the shipping time be calculated correctly? For such scenarios, products with precisely restricted parameters are required. The hardest thing was to provide such products, having in mind that tests were run parallelly and products had limited quantity. Moreover, required data were scattered across MySQL database and Solr. Atop on that, some search terms required comparing data from multiple sources to decide if given product suits all needs. This issue however is beyond the scope of this article.

There is plenty of other modules too, which are responsible for managing test users’ accounts, events logging or serving details related to current environment.

WebDriver

Acceptance testing would be crippled without possibility of testing within all popular browsers. What if there was a common API for all modern browsers? Working out of the box with all tests? Is Selenium with WebDriver such a remedy? Partly yes – after long hours of cursing on how different WebDriver implementations work with different browsers, you may eventually come to the point where you get no false positives. Meaning you may run a given test in any modern browser, expecting it to finish without failing to find some element on a page, while there really is one, or failing to perform some actions, only because WebDriver doesn’t wait until page finishes loading.

One of the modules was responsible for providing consistent API to work with different browsers’ WebDriver. For example, using Safari’s WebDriver fillField method triggers no event, which is really bad. To fix this problem, we provided overwritten method which works in all required browsers:

Running tests

Running all tests required enormous amounts of time. Testing all locales in one browser only, like Chrome or Safari, would take about 2 days to finish. Running tests in parallel decreases that time drastically. For that purpose Robo was used.

Using Selenium, each test is started with opening a desired web browser, which itself consumes quite a lot of time. The time increases when connecting to remote services, like Browser Stack or Sauce Labs – but that’s the only reasonable way of running 100 parallel tests. To deal with that, default Codeception’s browser was used – phpBrowser. With some improvements, it was able to properly process asynchronous requests, but it still couldn’t execute JavaScript. Its biggest advantage is its speed though – it’s much faster than any other browser we used. If test failed on phpBrowser, we knew it wouldn’t pass on any other browser.

Another option was to use an own grid of virtual machines equipped with all necessary browsers and operating systems, but that idea was abandoned as in the end such solution is much more expensive than using any of aforementioned services.

To simplify the process of running tests, multiple configuration were created and stored within Jenkins. To distinct which test should be run in a particular browser – locale combination, we used code annotations.

Software development: acceptance tests process diagram

Summary

In the end we’ve achieved what we went for. The platform itself is a reusable solution, which can be utilized not only in that particular eCommerce, but in any big project. Of course specific Page Object Models and helpers are required, but having our testing platform ready, we can deliver new acceptance tests really quickly, allowing us to greatly increase code coverage. Having high code coverage with unit tests only, doesn’t guarantee that business logic is implemented properly. In conjunction with acceptance tests, we can be almost sure, that the application works in the way we’ve planned.

PHP Developer

Software engineer at Codete with over 10 years of experience in professional software development. Dedicated fan of PHP ecosystem and clean code.