Testing CKAN

If you’re a CKAN developer, if you’re developing an extension for CKAN, or if you’re just installing CKAN from source, you should make sure that CKAN’s tests pass for your copy of CKAN. This section explains how to run CKAN’s tests.

CKAN’s testsuite contains automated tests for both the back-end (Python) and the front-end (JavaScript). In addition, the correct functionality of the complete front-end (HTML, CSS, JavaScript) on all supported browsers should be tested manually.

Back-end tests

Most of CKAN’s testsuite is for the backend Python code. You can run the code in a dockerized environment that replicates circleci, or you can use a virtual environment based testing.

Dockerized Tests

The test-infrastructure directory contains a configuration using docker compose replicating the circleci test process on the local machine.

Set up the testing environment

cd test-infrastructure
./setup.sh

This starts a docker compose environment with the supporting postgres, redis, and solr containers from the circleci test environment. The databases are initialized, and the current ckan is installed into a python container.

Run the tests

./execute.sh

Or, if you wish to run a specific test, for example test_get_translated in test_helpers.py:

docker compose exec ckan pytest --ckan-ini=test-core-circle-ci.ini ckan/tests/lib/test_helpers.py::test_get_translated

Teardown

./teardown.sh

Virtual Environment based tests

Install additional dependencies

Some additional dependencies are needed to run the tests. Make sure you’ve created a config file at /etc/ckan/default/ckan.ini, then activate your virtual environment:

. /usr/lib/ckan/default/bin/activate

Install pytest and other test-specific CKAN dependencies into your virtual environment:

pip install -r /usr/lib/ckan/default/src/ckan/dev-requirements.txt

Set up the test databases

Create test databases:

sudo -u postgres createdb -O ckan_default ckan_test -E utf-8
sudo -u postgres createdb -O ckan_default datastore_test -E utf-8

Set the permissions:

ckan -c test-core.ini datastore set-permissions | sudo -u postgres psql

When the tests run they will use these databases, because in test-core.ini they are specified in the sqlalchemy.url and ckan.datastore.write_url connection strings.

You should also make sure that the Redis database configured in test-core.ini is different from your production database.

Configure Solr Multi-core

The tests assume that Solr is configured ‘multi-core’, whereas the default Solr set-up is often ‘single-core’. You can ask Solr for its cores status:

curl -s 'http://127.0.0.1:8983/solr/admin/cores?action=STATUS' |python -c 'import sys;import xml.dom.minidom;s=sys.stdin.read();print(xml.dom.minidom.parseString(s).toprettyxml())'

Each core will be within a child from the <lst name="status" element, and contain a <str name="instanceDir"> element.

You can also tell from your ckan config (assuming ckan is working):

grep solr_url |ckan.ini|
# single-core: solr_url = http://127.0.0.1:8983/solr
# multi-core:  solr_url = http://127.0.0.1:8983/solr/ckan

To enable multi-core:

Find the instanceDir of the existing Solr core. It is found in the output of the curl command above.

e.g. /usr/share/solr/ or /opt/solr/example/solr/collection1

Make a copy of that core’s directory e.g.:

sudo cp -r /usr/share/solr/ /etc/solr/ckan

Find your solr.xml. It is in the Solr Home directory given by this command:
```
curl -s 'http://127.0.0.1:8983/solr/admin/' | grep SolrHome
```

Configure Solr with the new core by editing solr.xml. The ‘cores’ section will have one ‘core’ in it already and needs the second one ‘ckan’ added so it looks like this:

<cores adminPath="/admin/cores" defaultCoreName="collection1">
  <core name="collection1" instanceDir="." />
  <core name="ckan" instanceDir="/etc/solr/ckan" />
</cores>

Restart Solr by restarting Jetty (or Tomcat):
```
sudo service jetty restart
```
Edit your main ckan config (e.g. /etc/ckan/default/ckan.ini) and adjust the solr_url to match:
```
solr_url = http://127.0.0.1:8983/solr/ckan
```

Run the tests

To run CKAN’s tests using PostgreSQL as the database, you have to give the --ckan-ini=test-core.ini option on the command line. This command will run the tests for CKAN core and for the core extensions:

pytest --ckan-ini=test-core.ini ckan/ ckanext/

The speed of the PostgreSQL tests can be improved by running PostgreSQL in memory and turning off durability, as described in the PostgreSQL documentation.

Common error messages

OperationalError

OperationalError: (OperationalError) no such function: plainto_tsquery ...: This error usually results from running a test which involves search functionality, which requires using a PostgreSQL database, but another (such as SQLite) is configured. The particular test is either missing a @search_related decorator or there is a mixup with the test configuration files leading to the wrong database being used.

SolrError

SolrError: Solr responded with an error (HTTP 404): [Reason: None]
<html><head><meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type" /><title>Error 404 NOT_FOUND</title></head><body><h2>HTTP ERROR 404</h2><p>Problem accessing /solr/ckan/select/. Reason:<pre>    NOT_FOUND</pre></p><hr /><i><small>Powered by Jetty://</small></i>``

This means your solr_url is not corresponding with your SOLR. When running tests, it is usually easiest to change your set-up to match the default solr_url in test-core.ini. Often this means switching to multi-core - see Configure Solr Multi-core.

Front-end tests

Front-end testing consists of both automated tests (for the JavaScript code) and manual tests (for the complete front-end consisting of HTML, CSS and JavaScript).

Automated JavaScript tests

The JS tests are written using the Cypress test framework. First you need to install the necessary packages:

sudo apt-get install npm nodejs-legacy
sudo npm install

To run the tests, make sure that a test server is running:

. /usr/lib/ckan/default/bin/activate
ckan -c |ckan.ini| run

Once the test server is running switch to another terminal and execute the tests:

npx cypress run

Manual tests

All new CKAN features should be coded so that they work in the following browsers:

Internet Explorer: 11, 10, 9 & 8
Firefox: Latest + previous version
Chrome: Latest + previous version

Install browser virtual machines

In order to test in all the needed browsers you’ll need access to all the above browser versions. Firefox and Chrome should be easy whatever platform you are on. Internet Explorer is a little trickier. You’ll need Virtual Machines.

We suggest you use https://github.com/xdissent/ievms to get your Internet Explorer virtual machines.

Testing methodology

Firstly we have a primer page. If you’ve touched any of the core front-end code you’ll need to check if the primer is rendering correctly. The primer is located at: http://localhost:5000/testing/primer

Secondly whilst writing a new feature you should endeavour to test in at least in your core browser and an alternative browser as often as you can.

Thirdly you should fully test all new features that have a front-end element in all browsers before making your pull request into CKAN master.

Common front-end pitfalls & their fixes

Here’s a few of the most common front end bugs and a list of their fixes.

Reserved JS keywords

Since IE has a stricter language definition in JS it really doesn’t like you using JS reserved keywords method names, variables, etc… This is a good list of keywords not to use in your JavaScript:

https://developer.mozilla.org/en-US/docs/JavaScript/Reference/Reserved_Words

/* These are bad */
var a = {
  default: 1,
  delete: function() {}
};

/* These are good */
var a = {
  default_value: 1,
  remove: function() {}
};

Unclosed JS arrays / objects

Internet Explorer doesn’t like it’s JS to have unclosed JS objects and arrays. For example:

/* These are bad */
var a = {
  b: 'c',
};
var a = ['b', 'c', ];

/* These are good */
var a = {
  c: 'c'
};
var a = ['b', 'c'];