TESTING

CiviCRM is a complex piece of software which is constantly evolving through the contributions of many people. As you can imagine, making sure everything is working correctly (i.e. making sure everything is bug free) is not a trivial task. Tests are one method that CiviCRM uses to ensure that things are working as expected.  

There are already a great number of tests in CiviCRM, and the number of tests is growing all the time.  All tests are collected together in a test suite which is run periodically on the development version of the code base. The results of tests are available at http://tests.dev.civicrm.org/.

During CiviCRM development, if someone introduces a bug in the code for which there is a test, that test will fail. The bug can then be fixed before the next stable release.

Writing tests

All developers and users of CiviCRM are strongly encouraged to write tests to which have a direct impact on the quality of CiviCRM.

There are two types of test in CiviCRM: code tests, and web tests, and we use two different tools for each type of test.

To ensure consistency all tests should be carried out using the standard CiviCRM example data.

Code tests

Code tests ensure that CiviCRM code is working as expected, for example that a contact is created when you call the 'contact create' API method, or a contribution type is retrieved as expected when you call the contribution type BAO. Code tests are written with PHP Unit which is a PHP based testing framework. 

PHP Unit tests help you understand your own code better and force you to think about how your code will be used before you write any of it. If you submit unit tests with patches you guarantee that your patch will continue to work as expected and you will make the core team very happy.  

There is good documentation on writing PHP unit tests at http://www.phpunit.de/manual/3.2/en/database.html and the wiki has a growing page of example code at  http://wiki.civicrm.org/confluence/display/CRM/Writing+a+PHPUnit+testcase+HOWTO  

Web tests

Web tests are designed to ensure the interface is working as expected, for example, that the right things happen when you click on the right buttons. Examples of web tests include that the event confirmation screen is displayed when I hit the register for an event button, or that all 23 contacts are displayed when I search for contacts that live in France (in an instance of CiviCRM that has 23 contacts living in France).

You can record tests using the Selenium IDE which you can download from the Selenium website http://seleniumhq.org/. Web tests should be recorded using an instance of CiviCRM that has standard sample data. The demo (running the latest stable version of CiviCRM) and sandbox (running the latest development version) are good candidates to use for writing selenium tests because they have the demo data installed.

Setting up a local test environment

If you want to run tests locally, you need to set up a testing environment. If you want to submit PHP unit tests you will need to code these locally first. Note that you don't need a local testing environment to record selenium test.

To create a test environment, start with a local development environment and then follow the instructions here: http://wiki.civicrm.org/confluence/display/CRM/Setting+up+your+personal+testing+sandbox+HOWTO