Installing CKAN from source¶
This section describes how to install CKAN from source. Although Installing CKAN from package is simpler, it requires Ubuntu 12.04 64-bit. Installing CKAN from source works with other versions of Ubuntu and with other operating systems (e.g. RedHat, Fedora, CentOS, OS X). If you install CKAN from source on your own operating system, please share your experiences on our How to Install CKAN wiki page.
From source is also the right installation method for developers who want to work on CKAN.
1. Install the required packages¶
If you’re using a Debian-based operating system (such as Ubuntu) install the required packages with this command:
sudo apt-get install python-dev postgresql libpq-dev python-pip python-virtualenv git-core solr-jetty openjdk-6-jdk
If you’re not using a Debian-based operating system, find the best way to install the following packages on your operating system (see our How to Install CKAN wiki page for help):
|Python||The Python programming language, v2.6 or 2.7|
|PostgreSQL||The PostgreSQL database system, v8.4 or newer|
|libpq||The C programmer’s interface to PostgreSQL|
|pip||A tool for installing and managing Python packages|
|virtualenv||The virtual Python environment builder|
|Git||A distributed version control system|
|Apache Solr||A search platform|
|Jetty||An HTTP server (used for Solr).|
|OpenJDK 6 JDK||The Java Development Kit|
2. Install CKAN into a Python virtual environment¶
If you’re installing CKAN for development and want it to be installed in your home directory, you can symlink the directories used in this documentation to your home directory. This way, you can copy-paste the example commands from this documentation without having to modify them, and still have CKAN installed in your home directory:
mkdir -p ~/ckan/lib sudo ln -s ~/ckan/lib /usr/lib/ckan mkdir -p ~/ckan/etc sudo ln -s ~/ckan/etc /etc/ckan
Create a Python virtual environment (virtualenv) to install CKAN into, and activate it:
sudo mkdir -p /usr/lib/ckan/default sudo chown `whoami` /usr/lib/ckan/default virtualenv --no-site-packages /usr/lib/ckan/default . /usr/lib/ckan/default/bin/activate
The final command above activates your virtualenv. The virtualenv has to remain active for the rest of the installation and deployment process, or commands will fail. You can tell when the virtualenv is active because its name appears in front of your shell prompt, something like this:
(default) $ _
For example, if you logout and login again, or if you close your terminal window and open it again, your virtualenv will no longer be activated. You can always reactivate the virtualenv with this command:
Install the CKAN source code into your virtualenv. To install the latest stable release of CKAN (CKAN 2.7.1), run:
pip install -e 'git+https://firstname.lastname@example.org#egg=ckan'
If you’re installing CKAN for development, you may want to install the latest development version (the most recent commit on the master branch of the CKAN git repository). In that case, run this command instead:
pip install -e 'git+https://github.com/ckan/ckan.git#egg=ckan'
The development version may contain bugs and should not be used for production websites! Only install this version if you’re doing CKAN development.
Install the Python modules that CKAN requires into your virtualenv:
Changed in version 2.1: In CKAN 2.0 and earlier the requirement file was called pip-requirements.txt not requirements.txt as below.
pip install -r /usr/lib/ckan/default/src/ckan/requirements.txt
Deactivate and reactivate your virtualenv, to make sure you’re using the virtualenv’s copies of commands like paster rather than any system-wide installed copies:
deactivate . /usr/lib/ckan/default/bin/activate
3. Setup a PostgreSQL database¶
List existing databases:
sudo -u postgres psql -l
Check that the encoding of databases is UTF8, if not internationalisation may be a problem. Since changing the encoding of PostgreSQL may mean deleting existing databases, it is suggested that this is fixed before continuing with the CKAN install.
Next you’ll need to create a database user if one doesn’t already exist. Create a new PostgreSQL database user called ckan_default, and enter a password for the user when prompted. You’ll need this password later:
sudo -u postgres createuser -S -D -R -P ckan_default
Create a new PostgreSQL database, called ckan_default, owned by the database user you just created:
sudo -u postgres createdb -O ckan_default ckan_default -E utf-8
If PostgreSQL is run on a separate server, you will need to edit postgresql.conf and pg_hba.conf. For PostgreSQL 9.1 on Ubuntu, these files are located in etc/postgresql/9.1/main.
Uncomment the listen_addresses parameter and specify a comma-separated list of IP addresses of the network interfaces PostgreSQL should listen on or ‘*’ to listen on all interfaces. For example,
listen_addresses = 'localhost,192.168.1.21'
Add a line similar to the line below to the bottom of pg_hba.conf to allow the machine running Apache to connect to PostgreSQL. Please change the IP address as desired according to your network settings.
host all all 192.168.1.22/32 md5
4. Create a CKAN config file¶
Create a directory to contain the site’s config files:
sudo mkdir -p /etc/ckan/default sudo chown -R `whoami` /etc/ckan/
Change to the ckan directory and create a CKAN config file:
cd /usr/lib/ckan/default/src/ckan paster make-config ckan /etc/ckan/default/development.ini
Edit the development.ini file in a text editor, changing the following options:
This should refer to the database we created in 3. Setup a PostgreSQL database above:
sqlalchemy.url = postgresql://ckan_default:pass@localhost/ckan_default
Replace pass with the password that you created in 3. Setup a PostgreSQL database above.
If you’re using a remote host with password authentication rather than SSL authentication, use:
sqlalchemy.url = postgresql://ckan_default:pass@<remotehost>/ckan_default?sslmode=disable
Each CKAN site should have a unique site_id, for example:
ckan.site_id = default
Provide the site’s URL (used when putting links to the site into the FileStore, notification emails etc). For example:
ckan.site_url = http://demo.ckan.org
Do not add a trailing slash to the URL.
5. Setup Solr¶
CKAN uses Solr as its search platform, and uses a customized Solr schema file that takes into account CKAN’s specific search needs. Now that we have CKAN installed, we need to install and configure Solr.
These instructions explain how to setup Solr with a single core. If you want multiple applications, or multiple instances of CKAN, to share the same Solr server then you probably want a multi-core Solr setup instead. See Multicore Solr setup.
These instructions explain how to deploy Solr using the Jetty web server, but CKAN doesn’t require Jetty - you can deploy Solr to another web server, such as Tomcat, if that’s convenient on your operating system.
Edit the Jetty configuration file (/etc/default/jetty) and change the following variables:
NO_START=0 # (line 4) JETTY_HOST=127.0.0.1 # (line 15) JETTY_PORT=8983 # (line 18)
Start the Jetty server:
sudo service jetty start
You should now see a welcome page from Solr if you open http://localhost:8983/solr/ in your web browser (replace localhost with your server address if needed).
If you get the message Could not start Jetty servlet engine because no Java Development Kit (JDK) was found. then you will have to edit the JAVA_HOME setting in /etc/default/jetty to point to your machine’s JDK install location. For example:
Replace the default schema.xml file with a symlink to the CKAN schema file included in the sources.
sudo mv /etc/solr/conf/schema.xml /etc/solr/conf/schema.xml.bak sudo ln -s /usr/lib/ckan/default/src/ckan/ckan/config/solr/schema.xml /etc/solr/conf/schema.xml
Now restart Solr:
sudo service jetty restart
and check that Solr is running by opening http://localhost:8983/solr/.
Finally, change the solr_url setting in your CKAN config file to point to your Solr server, for example:
6. Create database tables¶
Now that you have a configuration file that has the correct settings for your database, you can create the database tables:
cd /usr/lib/ckan/default/src/ckan paster db init -c /etc/ckan/default/development.ini
You should see Initialising DB: SUCCESS.
If the command prompts for a password it is likely you haven’t set up the sqlalchemy.url option in your CKAN configuration file properly. See 4. Create a CKAN config file.
7. Set up the DataStore¶
Setting up the DataStore is optional. However, if you do skip this step, the DataStore features will not be available and the DataStore tests will fail.
Follow the instructions in DataStore extension to create the required databases and users, set the right permissions and set the appropriate values in your CKAN config file.
9. You’re done!¶
You can now use the Paste development server to serve CKAN from the command-line. This is a simple and lightweight way to serve CKAN that is useful for development and testing:
cd /usr/lib/ckan/default/src/ckan paster serve /etc/ckan/default/development.ini
Open http://127.0.0.1:5000/ in a web browser, and you should see the CKAN front page.
Now that you’ve installed CKAN, you should:
- Run CKAN’s tests to make sure that everything’s working, see Testing CKAN.
- If you want to use your CKAN site as a production site, not just for testing or development purposes, then deploy CKAN using a production web server such as Apache or Nginx. See Deploying a source install.
- Begin using and customizing your site, see Getting started.
The default authorization settings on a new install are deliberately restrictive. Regular users won’t be able to create datasets or organizations. You should check the Organizations and authorization documentation, configure CKAN accordingly and grant other users the relevant permissions using the sysadmin account.
Source install troubleshooting¶
Solr setup troubleshooting¶
Solr requests and errors are logged in the web server log files.
For Jetty servers, the log files are:
For Tomcat servers, they’re:
Unable to find a javac compiler¶
If when running Solr it says:
Unable to find a javac compiler; com.sun.tools.javac.Main is not on the classpath. Perhaps JAVA_HOME does not point to the JDK.
See the note in 5. Setup Solr about JAVA_HOME. Alternatively you may not have installed the JDK. Check by seeing if javac is installed:
If javac isn’t installed, do:
sudo apt-get install openjdk-6-jdk
and then restart Solr:
sudo service jetty restart
AttributeError: ‘module’ object has no attribute ‘css/main.debug.css’¶
This error is likely to show up when debug is set to True. To fix this error, install frontend dependencies. See Frontend development guidelines.
After installing the dependencies, run bin/less and then start paster server again.
If you do not want to compile CSS, you can also copy the main.css to main.debug.css to get CKAN running.
cp /usr/lib/ckan/default/src/ckan/ckan/public/base/css/main.css /usr/lib/ckan/default/src/ckan/ckan/public/base/css/main.debug.css