Testing for Developers¶
If you are installing CKAN from source, or developing extensions, then you need to know how to run CKAN tests.
This section describes testing topics for developers, including basic tests, migration testing and testing against PostgreSQL.
After completing your source installation of CKAN, you should check that tests pass. You should also check this before checking in changes to CKAN code.
Make sure you’ve created a config file at pyenv/ckan/development.ini. Then activate the Python environment:
Install nose into your virtual environment:
pip install --ignore-installed nose
At this point you will need to deactivate and then re-activate your virtual environment to ensure that all the scripts point to the correct locations:
deactivate . pyenv/bin/activate
Then run the quick development tests:
cd pyenv/src/ckan nosetests ckan/tests --ckan
You must run the tests from the CKAN directory as shown above, otherwise the --ckan plugin won’t work correctly.
By default, the test run is ‘quick and dirty’ - only good enough as an initial check.
Testing against PostgreSQL¶
The default way to run tests is defined in test.ini (which is the default config file for nose - change it with option --with-pylons). This specifies using SQLite and sets faster_db_test_hacks, which are compromises.
cd pyenv/src/ckan nosetests ckan/tests --ckan
Although SQLite is useful for testing a large proportion of CKAN, actually in deployment, CKAN must run with PostgreSQL.
Running the tests against PostgreSQL is slower but more thorough for two reasons:
- You test subtleties of PostgreSQL
- CKAN’s default search relies on PostgreSQL’s custom full-text search, so these (100 or so) tests are skipped when running against SQLite.
So when making changes to anything involved with search or closely related to the database, it is wise to test against PostgreSQL.
To test against PostgreSQL:
- Edit your local development.ini to specify a PostgreSQL database with the sqlalchemy.url parameter.
- Tell nose to use test-core.ini (which imports settings from development.ini)
nosetests ckan/tests --ckan --with-pylons=test-core.ini
The test suite takes a long time to run against standard PostgreSQL (approx. 15 minutes, or close to an hour on Ubuntu/10.04 Lucid).
This can be improved to between 5 and 15 minutes by running PostgreSQL in memory and turning off durability, as described in the PostgreSQL documentation.
If your changes require a model change, you’ll need to write a migration script. To ensure this is tested as well, you should instead run the tests this way:
nosetests ckan/tests --ckan --ckan-migrate --with-pylons=test-core.ini
By default, tests are run using the model defined in ckan/model, but by using the --ckan-migrate option the tests will run using a database that has been created using the migration scripts, which is the way the database is created and upgraded in production. These tests are the most thorough and will take around 20 minutes.
Ordinarily, you should set development.ini to specify a PostgreSQL database so these also get used when running test-core.ini, since test-core.ini inherits from development.ini. If you were to change the sqlalchemy.url option in your development.ini file to use SQLite, the command above would actually test SQLite rather than PostgreSQL, so always check the setting in development.ini to ensure you are running the full tests.
A common error when wanting to run tests against a particular database is to change sqlalchemy.url in test.ini or test-core.ini. The problem is that these are versioned files and people have checked in these by mistake, creating problems for other developers and the CKAN buildbot. This is easily avoided by only changing sqlalchemy.url in your local development.ini and testing --with-pylons=test-core.ini.
Common problems running tests¶
nose.config.ConfigError: Error reading config file ‘setup.cfg’: no such option ‘with-pylons’
This error can result when you run nosetests for two reasons:
Pylons nose plugin failed to run. If this is the case, then within a couple of lines of running nosetests you’ll see this warning: Unable to load plugin pylons followed by an error message. Fix the error here first.
The Python module ‘Pylons’ is not installed into you Python environment. Confirm this with:
python -c "import pylons"
OperationalError: (OperationalError) no such function: plainto_tsquery ...
This error usually results from running a test which involves search functionality, which requires using a PostgreSQL database, but another (such as SQLite) is configured. The particular test is either missing a @search_related decorator or there is a mixup with the test configuration files leading to the wrong database being used.