News-Feeds:
Testing a TYPO3 extension on travis
Having tests for your application is obligatory if you care about code quality and wanna sleep well. The amount of environments is growing steadily and it is impossible to run every test in every different environment.
One tool which can help you achieving this is travis-ci [1] which is absolutly free for projects hosted on GitHub.
This article is about my successful story about using travis for my extension "news".
Using Travis
Travis lets you build complete environments using various languages (Java, ObjectiveC, Ruby, PHP, Erlang, Scala, Go,...) and systems.
By creating a yml file it is possible to configure how those environments are built.
When looking at the travis.yml file of EXT:news [2] you will notice this section at the beginning
language: php
php:
- 5.3
- 5.4
env:
- DB=mysql TYPO3=master INTEGRATION=master
- DB=mysql TYPO3=TYPO3_6-1 INTEGRATION=master
- DB=mysql TYPO3=TYPO3_6-0 INTEGRATION=master
- DB=mysql TYPO3=TYPO3_4-7 INTEGRATION=TYPO3_4-7
- DB=mysql TYPO3=TYPO3_4-6 INTEGRATION=TYPO3_4-7
- DB=mysql TYPO3=TYPO3_4-5 INTEGRATION=TYPO3_4-7
As TYPO3 is using PHP, the language is set to this one too. However various versions of PHP are currently supported and need to be considered for the tests.
While writing this article, 5 TYPO3 CMS branches are maintained which makes it even harder to test an extension in every version.
If this configuration is used, travis will create virtual machines with every combination of this matrix, which means having a virtual machine for TYPO3 master with PHP 5.3 and PHP 5.4, same for 6.1, 6.0, 4.7, 4.6 (just because it is only one line, even though this version is outdated) and 4.5.
By using an additional setting
matrix:
include:
- php: 5.5
env: DB=mysql TYPO3=master INTEGRATION=master
PHP 5.5 is included for current master (the upcoming TYPO3 6.2) only.
This makes 13 machines, isn't that awesome?
Someone at T3DD13 asked me why I am testing so many combinations as if everything is fine with PHP 5.4, everything would be fine with PHP 5.3 as well. My response was just "How can you be sure?" and this is all testing is about. It is not about having more tests available than your colleque but to be able to know and to tell your customer: Yes I am sure that this application will deliver the defined results in all supported environments. This is impossible without testing or lying.
Firing the tests
If a TYPO3 extension should be tested, it will need a basic TYPO3 running as well.
The awesome guys of the TYPO3 Server Team did this work already. TYPO3 CMS is already using travis since a while for firing their tests [3].
Therefore I had a great base to start.
Basically the travis configuration file lets you write bash scripts as seen below
before_script:
# Install build dependencies
- cd ..
- git clone --single-branch --branch $INTEGRATION --depth 1 git://github.com/georgringer/TYPO3-Travis-Integration.git
build-environment
- git clone --single-branch --branch $TYPO3 --depth 1 https://github.com/TYPO3/TYPO3.CMS.git core
- source build-environment/install-helper.sh
[...]
This will download the complete TYPO3 CMS core, the introduction page, fill the database with some needed records and so on.
In the end, all it takes are 2 lines which will call the phpunit extension (which has been cloned as well) to start all provided tests.
script:
- php $PWD/typo3/cli_dispatch.phpsh phpunit -c typo3conf/ext/news/Tests/Build/UnitTests.xml
The result
You will get a totally cool green bar feeling when looking at the results [4] as you are welcomed by the enormous amount of 13 green bars (one for every environment).
The next steps
- Currently there is no automatic sync to the GitHub repository, therefore the tests are not running after every commit but only when I mirror the repositories. This will change in the near future.
- More tests: As there is now a platform to run all those tests (Unit tests and functional tests), I want to provide more test cases to make use of travis.
Testing other extensions
For the moment you can reuse my code if you want to start testing your extension by travis.
Many changes inside TYPO3 6.2 will allow an easier setup of a TYPO3 installation with some given files and records soon.
About this post
Writing this posting and attending the T3DD13 has been sponsored by the TYPO3 Agency Cyberhouse.
---------------------
comments
July 18, 2013
09:42
yes of course I know that, that is exactly why I want more tests, especially those functional ones which test if the correct records are returend from a repository, ....
July 19, 2013
00:26
seems we work in the same direction. You might want to check https://github.com/tolleiv/ext-alwaysgreen/blob/master/.travis.yml - I miss the DB integration atm.
Maybe we can join forces?
Cheers.
Buzz is the official blog resource of the TYPO3 project. You can be part of Buzz too. If you want to blog about the achievements of your team or your personal involvement please check out our HELP section.
thanks for the article. But keep in mind that if all tests pass it does not automatically guarantee that there will be no errors (»Who guarantee that we have tests for all cases?« :X …) But I am sure that this is a huge step and improves quality a lot, keep the good work going!