.. This tocdepth stops Sphinx from putting every subsection title in this file into the master table of contents. :tocdepth: 1 --------- Changelog --------- .. towncrier release notes start v.2.12.0 2026-08-26 =================== Overview -------- - CKAN 2.12 supports Python 3.10 and later - This version requires a requirements upgrade on source installations - This version requires a database upgrade with ``ckan db upgrade``. - This version requires the DataStore database to be updated by running the SQL script produced by ``ckan datastore set-permissions``. - Make sure to check the :ref:`migration-notes-2.12` Major features -------------- New **Midnight Blue** theme available. This theme will become the default in 3.0. - ``midnight_blue`` templates and public folders for new UI can be enabled in the config with two settings (`#8420 `_): .. code:: ini ckan.base_public_folder = public-midnight-blue ckan.base_templates_folder = templates-midnight-blue Files are now **first-class entities** and can be uploaded and managed separately from resources. Multiple file storage options are available and may be used simultaneously. - Added support of file storages: :ref:`using-configured-storages`. When either ``default`` storage, or individual storages for resource, group, user and admin uploads are configured, :py:class:`~ckan.lib.uploader.Upload` and :py:class:`~ckan.lib.uploader.ResourceUpload` are replaced by :py:class:`~ckan.lib.uploader.FKUpload` and :py:class:`~ckan.lib.uploader.FKResourceUpload`. These new classes use configurable storages and serve the role of the bridge between classic file management system in CKAN and the new one. (`#8745 `_) - New `file management API actions `_ (`#9026 `_) New DataStore **keyset pagination** and **advanced filtering** are powerful tools for working with large tabular datasets. - Advanced DataStore filters + fast ``datastore_search`` pagination - ``datastore_search`` and ``datastore_delete`` ``filters`` now accept range values and nested AND and OR operations - ``datastore_search`` now returns a ``next_page`` value with filters that can be used for fast keyset pagination when ``"include_next_page": true`` is passed and the records are sorted by their ``_id`` field - datastore dump endpoint now uses fast keyset pagination to more quickly stream large datasets as CSV, JSON, etc. (`#9027 `_) CKAN UI, updates and queries are **much faster** and dataset and DataStore tables use **less storage** in this release. - Dataset search page facets, sorting and pagination is now faster and no longer requires page reloads. (`#8914 `_) - Applying filters and paginating through activities now avoids refreshing the page and is much faster. (`#8129 `_) - API tokens are now managed faster, without requiring page reloads (`#8728 `_) - Updating dataset metadata is now significantly faster, especially for datasets with many resources, because unchanged resources are no longer revalidated. (`#5713 `_) - Package Group and Organization models now store extra fields as JSONB for faster updates and reduced storage. (`#8273 `_) - Simplify package indexing by updating index from logic layer instead of an ``IDomainObjectModification`` plugin. Also makes ``package_update`` faster (`#8395 `_) - Calculating the exact number of rows for large tables dominates the time required to return results from ``datastore_search``. The counts of rows is now computed and cached in the background after every modification so that it can be relied on for much faster search results. (`#9234 `_) - DataStore full text field index field types may now be configured with :ref:`ckan.datastore.default_fts_index_field_types`. The default is an empty list which avoids creating separate full text indexes for any individual columns. The whole-row full text index still exists for all tables. Use the ``ckan datastore fts-index`` command to remove existing column indexes and reclaim database space. (`#5847 `_) **Scheduled jobs** let CKAN work smarter and skip repeated updates. - Faster dataset metadata updates by detecting changes and only updating resource metadata when dataset fields defined by ``IDatasetForm.resource_validation_dependencies`` have changed (default: None). - Activities are created and ``metadata_modified`` updated only if there is a real change. - ``metadata_modified`` may now be set by sysadmins which is useful for harvesting or mirroring. - ``ckan jobs worker`` CLI now runs a scheduler by default to enqueue scheduled jobs. Use ``--no-scheduler`` to disable the scheduler on this worker. Only one worker can run as a scheduler for each queue so this option may be used on secondary workers. ``datastore_create``, ``datastore_upsert`` and ``datastore_delete`` now schedule a job to patch the corresponding resource's ``last_modified`` value for datastore-first resources. A scheduled job is used to reduce duplicated metadata updates that would slow down these operations. (`#8980 `_) Richer APIs report back the **actual changes** made enabling tighter integration with other systems. - ``package_update`` and actions that call it now report whether there was a real change by adding the package id to a new changed_entities context value or changed_entities envelope value for API calls. (`#8407 `_) - ``datastore_create``, ``datastore_upsert``, and ``datastore_delete`` now accept a ``"include_records": true`` parameter to return the *actual* data inserted, updated or deleted in the response including the ``_id`` values and any transformations. (default: ``false``) ``datastore_upsert`` no longer includes a copy of the passed ``records`` in the response. (`#8684 `_) Minor changes ------------- - Saving a draft dataset is now called "Publish" and an "Unpublish" button is available to move an active dataset back to the draft state. (`#8308 `_) - Publishing a dataset now does an extra error check, so validators that only apply rules to fields for published datasets (e.g. required-only-when-published) will be properly displayed. (`#8308 `_) - The internal ``allow_partial_update`` context parameter has been removed. Now normal API users may call ``package_update`` without passing resources and the existing resources will remain untouched instead of being deleted. - Sysadmins can no longer demote themselves or the system user. (`#8155 `_) - Activity plugin now adds an activity tab to the sysadmin panel to purge old activities (predefined options: older than a day/month/year, or delete all). Additionally there is a new CLI command ``ckan clean activities`` with multiple options and an ``activity_delete`` action. (`#8189 `_) - New interface ``INotifier`` allows notifications beyond simple emails. (`#8200 `_) - Add ``--max-idle-time`` to the ``ckan jobs worker`` command. (`#8240 `_) - Adds ``argmode`` to the ``datastore_function_create`` action, allowing custom DataStore SQL functions to support ``inout`` and ``output`` parameters. (`#8279 `_) - Added a schema and validators for the Perform Password Reset form. (`#8292 `_) - DataTables view column visibility modal now has ``Show All`` and ``Hide All`` buttons and fixes for responsive issues. (`#8341 `_) - DataStore field names/IDs are now validated at the schema level. Control characters are no longer allowed in DataStore field names/IDs. (`#8342 `_) - New CKAN logo is used by ``midnight-blue`` theme. - New helpers ``endpoint_from_url`` and ``page_is_active``. (`#8492 `_) - ``datastore_info``: new ``include_meta`` and ``include_fields_schema`` parameters that may be set to ``False`` to avoid computing table size, index size, row count, aliases or per-field index, unique and notnull status (`#8589 `_) - Allow plugins to set user IDs during user creation. (`#8609 `_) - Allow plugins to change the user names during ``user_patch`` or ``user_update``, and retrieve restricted properties like ``email`` or ``apikey`` from ``user_show``. (`#8622 `_) - :ref:`ckan.user.unique_email_states` setting can be used to specify statuses of user accounts that are used for checking uniqueness of the email during registration. After changing the value of the option, run ``ckan db duplicate_emails`` CLI command to verify that all existing emails are still unique. (`#8626 `_) - ``datatablesview`` plugin now uses the default DataTables responsive view (hidden columns folded below each row) instead of a custom modal dialog. For the old responsive modal behaviour set: ``ckan.datatables.responsive_modal = true`` (`#8754 `_) - ``IDatasetForm`` and ``IGroupForm`` plugins may now override the new ``search_template_htmx`` and ``read_template_htmx`` methods to control which parts of the page are updated when new search results are displayed. (`#8914 `_) - New reusable ``ckan.toast`` module that displays Bootstrap 5 toast notifications across CKAN (`#9016 `_) - New reusable ``ckan.confirm`` module that displays Bootstrap 5 confirmation modal windows across CKAN (`#9018 `_) - New helper to display Datastore action buttons based on theme being used (i.e. not displaying buttons on "Midnight blue" theme, displaying on default theme). (`#9041 `_) - CKAN config options are not shared with Flask application by default. To pass option into Flask application(for example, when configuring Flask extension), add ``flask: true`` to the declaration of the config option. (`#9068 `_) - New resource icons for common AI/ML and geospatial formats, including: parquet, md, tif, pmtiles, atx, dbf, fgb, gpkg, py, gz, shp, shx, and ipynb. (`#9170 `_) - DataStore's global public tables, previously only ``_table_metadata``, are now configurable with :ref:`ckan.datastore.public_table_search` (`#9298 `_) - Configuration options can be declared with ``nullable`` flag, which set their value to ``None`` when option is missing from the config file. (`#9308 `_) - New CLI command that reports discrepancies between the real DB schema and model definition: ``ckan db check`` (`#9355 `_) - New config option :ref:`ckan.webassets.debug` that controls aggregation logic of webassets. When it's disabled(default), webassets are aggregated and served from :ref:`ckan.webassets.path`. When it's enabled, every source file of asset is served separately, allowing easier front-end debugging but substantially increasing the number of additional requests. (`#9379 `_) - Login by email is now case-insensitive. (`#8626 `_) - The ``include_users`` parameter to ``organization_show`` and ``group_show`` actions now default to False regardless of the :ref:`ckan.auth.public_user_details` configuration setting. (`#9232 `_) - Improved ``organization_list`` / ``group_list`` API performance with ``"all_fields": true``, resulting in faster load times for Organization/Group List Pages. (`#9278 `_) - DataStore now returns milliseconds from timestamp columns by default. To restore the old behavior set :ref:`ckan.datastore.ms_in_timestamp` to ``false``. (`#8202 `_) - New config option :ref:`ckan.uploads_enabled` that can be used to disable all uploads. (`#8977 `_) - Logging configuration no longer needs to be defined in the configuration ini file when extending a base configuration file. Base logging settings will be used unless the overridden by a higher level configuration file. (`#9002 `_) - New ``IAuthenticator.identify_user`` method can be used to assign a User object to the current session. (`#8636 `_) - ``job_list`` action returns 200 jobs by default. Use ``limit`` or ``ckan.jobs.default_list_limit`` config option to modify this limit. (`#8070 `_) - ``user_with_token`` / ``sysadmin_with_token`` factory fixtures added to test suite. (`#7631 `_) - Update settings and readme for running cypress tests locally with the test-infrastructure docker compose configuration (`#8724 `_) - Reintroduced ``email_is_unique`` validator. (`#7723 `_) - Add the site user ``id`` to the ``get_site_user`` function response (`#8386 `_) - Add ``--create`` option to ``ckan sysadmin add`` command to automatically create the user if it doesn't exist (`#8673 `_) - Register per-test plugin using ``provide_plugin`` fixture or ``ckan_plugin`` mark. (`#8786 `_) - Extension generator now generates linting workflow using ruff. (`#8969 `_) - Replace usage of ``pkg_resources.iter_entry_points``, update exception message (`#8992 `_) - Expose ``NotFound`` exception in plugins toolkit for consistency with core extensions (it's the same exception as the existing ``ObjectNotFound``) (`#8996 `_) - Update Twitter to X in CKAN templates. (`#9036 `_) - Added ``remove_locale_from_url()`` and ``remove_root_path_from_url()`` template helpers (`#9093 `_) - Follow and unfollow actions now call their respective authentication methods (`#9229 `_) - datatablesview's ``lastView`` local storage key is now suffixed with the view ID, allowing each DataTable view to respect its given ``responsive-flag``. (`#8209 `_) - Improved JS translations documentation (`#8927 `_) - Use pglast instead of sqlparse to sanitize datastore_search_sql input (`#9475 `_) Bugfixes -------- - `GHSA-8frv-ccr7-4m2p (CVE pending) `_: Stored XSS via resource fields in Text view - `GHSA-5r6j-4c43-7mx6 (CVE pending) `_: Unauthenticated nested Solr QParser allowlist bypass in ``package_search`` - `GHSA-73fv-x47v-f4j5 (CVE pending) `_: Authenticated stacked SQL injection in ``datastore_create`` - `GHSA-8hw7-23gj-5599 (CVE pending) `_: datastore_search_sql Authorization Bypass - `GHSA-3g5q-3wf6-p8rc (CVE pending) `_: ``markdown_extract()`` Stored XSS on Dataset Listings - `GHSA-6499-jgj4-2wpf (CVE pending) `_: Session Fixation in Registration View - `GHSA-jgwg-vp4m-5xw5 (CVE pending) `_: Exposure of private metadata via follow API actions - `GHSA-p5rh-49m9-56vx (CVE pending) `_: Stored XSS vector via resource name in DataTables view - Fix "Option lang is not declared" warning at application startup. (`#8777 `_) - Logout link changed to a POST submit so it interacts properly with caching and CSRF (`#7892 `_) - Activity performance fix replaces unions with ``in(list)`` and activity outer join to packages with subselects. (`#7903 `_) - DataTable columns now automatically re-fit after sorting. (`#8065 `_) - Only show the count of an org's datasets on the dashboard organization list (not harvesters, showcases, etc.). (`#8082 `_) - Fix 403 error when a user removes themselves from a group (`#8256 `_) - ``ckan db upgrade`` can now be used after ``ckan db create-from-model`` (`#8336 `_) - DataStore dump XML format now conforms field names to XML element name conventions. (`#8340 `_) - ``ckan db clean`` and ``ckan search-index rebuild`` now remove orphaned entries from search index (`#8347 `_) - Fix error message when creating a new user with an existing user name. (`#8388 `_) - Fix display of social media links on resource when ckanext-scheming is in use. (`#8454 `_) - Add public folder to extension template (`#8565 `_) - Fix language code in ``html`` tag ``lang`` attribute to use BCP-47 with a new ``unix_locale_to_bcp47`` template helper function. (`#8698 `_) - Fix argument parsing inside ``ckan search-index clear-orphans`` CLI command. (`#8750 `_) - Fixed slightly too large slug edit button, as ``.btn-xs`` was dropped from Bootstrap. (`#8835 `_) - Fix for historical versions of a custom dataset types using the ``IDatasetForm`` interface. (`#8875 `_) - Prevent layout shifting in header when logo loads (`#8905 `_) - Fix for ``datastore_search_sql`` and multithreaded environments. (`#8929 `_) - Fix server error if session data was not properly encoded (`#8939 `_) - Remove user data from ``stats`` if the :ref:`ckan.auth.public_user_details` setting is set to ``False``. (`#9030 `_) - Fix context passed to Table Designer from ``resource_create`` and ``resource_update`` (`#9057 `_) - Fix handling of plugin order for ``ITemplateHelpers`` to align with ``add_template_directory`` precedence again (first plugin wins). (`#9069 `_) - Fix the font of the sort indicator in datatablesview (`#9078 `_) - Fix Preview for resource view create/update pages (`#9128 `_) - Add handling for ``ObjectNotFound`` to Datapusher: delete-datastore route (`#9130 `_) - Fix auth check for ``resource_view_reorder`` (`#9131 `_) - Fix performance issue with DataStore dump startup time affecting large tables (`#9144 `_) - Enable using helpers that were overridden by plugins in ``model_dictize.py`` (`#9155 `_) - Fix for markdown_extract tag removal bug (`#9162 `_) - Fix ``organization_purge`` reporting deleted datasets. (`#9167 `_) - Fix SMTP TLS error with embedded port (`#9186 `_) - Display correct errors and leave resource form empty when dataset validation errors are triggered by publishing a dataset (setting state=active) (`#9202 `_) - Use ``h2`` in sidebar instead of ``h1`` as pages should only have one ``h1`` (`#9219 `_) - Fix an issue with Resources not uploading new files during ``package_update`` (`#9265 `_) - Fix ``get_param_int`` helper to return ``0`` instead of ``10`` when the given parameter is not available or cannot be parsed as an integer. (`#9281 `_) - Capture the ``undefined_table`` psql error if a DataStore resource table was removed. (`#9286 `_) - Fix error when creating an extension by calling cookiecutter directly. (`#9296 `_) - Fixed duplicated ``root_path`` in webassets when assets directory is public directory and debug mode is used or ``cssrewrite`` is present in filters. (`#9300 `_) - Fix ``organization_delete`` incorrectly failing for organizations with no datasets when ``ckan.auth.create_unowned_dataset`` is disabled. (`#9311 `_) - Fix DataStore distinct queries with limit/offset (`#9365 `_) - Fix organization admins not being allowed to invite new users when public registration is closed (`#9369 `_) - Use case-insensitive email search when requesting password reset. (`#9370 `_) - Add the missing CSRF token (``h.csrf_input()``) to the Table Designer add-row, edit-row and delete-rows forms (`#9372 `_) - Return a 404 response instead of a 500 error when opening a resource edit page for a dataset that does not exist. (`#9375 `_) - Reindex datasets after ``member_create`` and ``member_delete`` change package group memberships, ensuring group pages show the correct datasets. (`#9381 `_) - Fix resource form allowing creation of completely empty resources. (`#9397 `_) - Always return dataset with the current ``site_id`` when searching by dataset id (`#9422 `_) - DataStore auth function fix (`#9021 `_) - Clear CSRF token from session when logging out programmatically. (`#9049 `_) - Pass ``about_formatted`` to user info snippet to display this value in the sidebar. (`#9148 `_) - Fix missing string translation for ``Searching...`` in the Select2 JS autocomplete module. (`#9184 `_) - Fix server errors on badly formatted requests (`#9201 `_) - Remove unused ``ckan.static_max_age`` config option (`#9005 `_) - Avoid exceptions in tracking extension when dataset fields are missing (`#8499 `_) - Renamed ``.btn-default`` to ``.btn-secondary`` due to Bootstrap conventions. (`#8611 `_) - Allow users to follow and unfollow groups and organizations when ``ckan.auth.public_user_details`` is disabled. (`#9394 `_) .. _migration-notes-2.12: Migration notes --------------- - The ``migrate_package_activity.py`` script and revision tables have been removed. Migrate your revision data to activities before upgrading to this version or the revision history will be lost. (`#8319 `_) - The ``ckan datastore set-permissions`` SQL must be run against an existing datastore database to define the new ``fast_table_row_count`` function. (`#9234 `_) - Extensions must now include the :ref:`CSRF protection ` snippet in their forms. The ``ckan.csrf_protection.ignore_extensions`` config option has been removed. (`#8918 `_) - The minimum Python version supported is Python 3.10 (`#8998 `_) - The unique user email index is now case insensitive. This will prevent duplicate user emails that use different cases. Use the ``ckan db duplicate_emails`` command to confirm any duplicate users you may have in your site. (`#9178 `_) Removals and deprecations ------------------------- - Updating dataset metadata no longer revalidates unchanged resources. This change will affect custom validation rules that access resource metadata from dataset or other resource validators: - only changed resources are passed to validation - flattened data the validators receive won't include unchanged resources This change does not affect validation of resource fields that depend on package fields. Package metadata fields are always available in flattened data. (`#5713 `_) - The ``form_to_db_*`` and ``db_to_form_*`` methods of the ``IGroupForm`` interface have been replaced by ``create_group_schema()``, ``update_group_schema()`` and ``show_group_schema()``. (`#9050 `_) - ``PackageExtra`` and ``GroupExtra`` tables have been removed. Code that accesses these models directly will need to be updated to use the Package.extras and Group.extras dicts for updating and JSON queries like ``query(Package, Package.extras['name'] == '"value"')`` (`#8273 `_) - The ``page_primary_action``, ``form`` and ``package_search_results_list`` blocks have been moved from ``templates/package/search.html`` to ``templates/package/snippets/search_results.html`` so that dataset search results may be updated without rendering the whole page. Extensions that override these blocks will need to be updated. (`#8914 `_) - The following deprecated functions and properties have been removed: - ``h.truncate()`` (use the builtin jinja2 filter instead) - ``h.get_site_statistics()`` - ``common.is_flask_request()`` - ``Request.args`` - ``views.user.set_repoze_user()`` - ``Tag.search_by_name()`` - ``PackageTag.by_name()`` - ``Package.is_private`` - ``IDomainObjectModification.notify_after_commit()`` - Removed ``activity-stream.js`` module (`#8129 `_) - Removed ``/api/1/snippet``, ``templates/ajax_snippet``, ``getTemplate`` feature: normal views with parameter validation and permission checking should be used instead, possibly combined with HTMX. (`#7619 `_) - Removed ``_alerts.scss`` and ``alert-error`` class due to deprecated ``alert-variant()`` (`#8427 `_) - Removed unused jQuery truncator plugin (`#8481 `_) - ``context["model"]`` is deprecated in favor of ``ckan.model`` module. (`#8702 `_) - Removed Organization facet from individual organization pages. Removed Groups facet from individual groups pages. (`#8923 `_) - Tests no longer support authentication by ``REMOTE_USER`` environment variable. Use ``app.set_session_user`` or ``Authorization`` header with user's API Token instead. (`#9068 `_) - Removed old and unused ``ckan.lib.cli`` module (`#8558 `_) - Removed the ``HostHeaderMiddleware`` flask middleware. (`#9123 `_) - Global variable ``search_facets_limits`` is no longer available on dataset search pages:: # before g.search_facets_limits.get("facet") # after h.get_param_int(f"_{facet}_limit", config["search.facets.default"]) (`#9281 `_) - ``data-site-root`` and ``data-locale-root`` attributes have been moved to ``site-root``/``locale-root`` meta elements inside ``head``. Front-end code like ``$('body').data('site-root');`` must be replaced with ``$('meta[name=site-root]').attr('content');`` or ``document.head.querySelector(['meta[name=site-root]'])?.content`` (`#9284 `_) - ``templates/package/new_package_form.html`` content has been merged into its parent template ``templates/package/snippets/package_form.html``. ``templates/package/new_package_form.html`` is now marked as deprecated and will be removed in a future release. (`#8308 `_) v.2.11.4 2025-10-29 =================== Migration notes --------------- - This version requires a requirements upgrade on source installations - Restore handling of plugin order for ``ITemplateHelpers`` to align with ``add_template_directory`` precedence again (first plugin wins). You might have to change the order of a plugin in ``ckan.plugins`` if you relied on overriding a template helper from another plugin. (`#9069 `_) - A new config option :ref:`ckan.uploads_enabled` was added to prevent a critical error being shown in the logs about a missing :ref:`ckan.storage_path` setting. This is not required and existing sites should work as before. When the ``ckan.uploads_enabled`` is not set, uploads will be shown in the UI if ``ckan.storage_path`` is defined or there is an ``IUploader`` plugin configured. (`#8977 `_) Minor changes ------------- - Replaced lint tool flake8 with ruff. (`#8964 `_) - Remove unused `ckan.static_max_age` config option (`#9005 `_) - Include CHANGELOG.rst in MANIFEST.in. (`#9083 `_) Bugfixes -------- - `CVE-2025-64100 `_: Rotate session identifiers to prevent Session Cookie Fixation . - `CVE-2025-54384 `_: Stored XSS vector in Markdown description fields - Add handling for ``ObjectNotFound`` to Datapusher ``delete-datastore`` route (`#9130 `_) - Fix auth check for ``resource_view_reorder`` (`#9131 `_) - Rewrite query to fetch one row instead of whole table in tracking code (`#8757 `_) - Fix ``ckan datastore upgrade`` command with columns containing definition with ``%`` characters (`#9115 `_) - Fix: document js translations generation (`#8927 `_) - Ensure ``UnicodeDecodeError`` is caught if session data was not improperly encoded (`#8939 `_) - Pass proper context to tabledesigner ``_create_table_and_view`` to preserve the original context from ``resource_create`` and ``resource_update`` (`#9057 `_) - Use non-zero exit codes when search-index CLI fails (`#9011 `_) - Hide users data from `stats` if the `ckan.auth.public_user_details` setting is set to `False`. This ensures that user details are not exposed in the statistics when public user details are disabled. (`#9030 `_) - Fixes the font of the sort indicator in datatablesview by properly closing the opening `_) - Historical versions of a custom dataset type ignores base template specified by the ``IDatasetForm`` interface. (`#8875 `_) - Fix ``EXPLAIN JSON`` output sometimes being auto-decoded in multithreaded environments by setting json_deserializer at engine level. (`#8929 `_) - Catch ``NotAuthorized`` exception raised by ``datastore_search`` (`#8989 `_) - Remove override of group_dict['package_count'] to display the correct dataset count (`#8252 `_ v.2.11.3 2025-05-07 =================== Migration notes --------------- - This releases includes a migration to fix existing activities created before a 2.11 migration not showing up in ``package_activity_list`` calls. If you are using the ``activity`` plugin, apply it with (`#8784 `_):: ckan db upgrade -p activity - A new session serialization strategy is used to fix errors when the ``flash_success`` and ``flash_error`` helpers called with the ``html`` flag. Existing sessions stored inside Redis cannot be deserialized with this new strategy and must be removed using the command:: redis-cli keys "session:*" | xargs redis-cli del Non-redis session backends are not affected by this change. (`#8704 `_) Minor changes ------------- - Re-add `get _site_statistics()` helper that was removed without mention in the changelog (`#8522 `_) - Register pytest plugins as entrypoints to make them available to all extensions (`#8507 `_) - Restore activity API documentation. (`#8780 `_) - Switch 'datastore_info' to use 'resource_id' as input argument (`#8907 `_) - Update release process docs (`#8586 `_) - Migrate CI checks to GitHub Actions (`#8909 `_) - Upgrade Jinja2 requirement to address CVE-2025-27516 Bugfixes -------- - Set license model ``od_conformance`` and ``osd_conformance`` attributes' default values to `False` to prevent errors. (`#8268 `_) - Restore usage of ``follow_button`` snippet so it can be overridden and customised (`#8651 `_) - Don't encode binary font files when building the frontend (`#8666 `_) - Fix exception in ``recently_changed_packages_activity_list`` action (`#8677 `_) - Fix error in datastore upgrade: don't process datastore column comments for sqlalchemy bind parameters (`#8693 `_) - Fix database revision 105 downgrade. Ensure ``resource.package_id -> package.id`` foreign key constraint is dropped. (`#8707 `_) - You can now use non-string values in ``datastore_search`` and ``datastore_delete`` filters for text datatype fields. (`#8729 `_) - ``datastore_search``: return records as ``LazyJSONObject`` only when called from api view. (`#8739 `_) - Fixed an issue with using ``filters`` in ``datastore_search`` with CSV/TSV records format. (`#8741 `_) - Return ``resource_id`` from ``resource_view_delete`` so that the activity plugin does not fail when recording the deleted view. (`#8760 `_) - Fix tabledesigner integration with datatables and the way datatables work with the i18n files (`#8782 `_) - Default to a long CSRF token timeout to fix "The CSRF token is invalid." errors (`#8803 `_) - Fix search button styling by removing deprecated wrapper (`#8737 `_) - Catch an error in datastore to avoid 500 error in POSTs to `datatables/ajax/` (`#8149 `_) - Invalidate cached pages and load fresh ones if cookies change (`#6955 `_) - Remove unsupported legacy API keys from documentation (`#8195 `_) - Ensure ``session["last_active"]`` is stored as an iso string instead of a `datetime` so that it can be serialized to JSON (e.g. in cookies). (`#8379 `_) - Fix ``check_access`` order for resource create view (`#8588 `_) - Fix auth check for datastore data dictionary view (`#8639 `_) - Add missing boolean_validator to sysadmin field in user schema (`#8674 `_) - ``authz.has_user_permission_for_some_org`` returns True for sysadmins. (`#8680 `_) - Apply ``humanize_entity_type`` helper consistently across the group listing page (`#8682 `_) - ``datastore_search``: fix for sort on array column types (`#8709 `_) - Fix some ``.btn-default`` classes that were mistakenly changed to ``.btn-light``. (`#8828 `_) v.2.11.2 2025-02-05 =================== Migration notes --------------- * Going forward, if both ``ckan.upload.[type].mimetypes`` and ``ckan.upload.[type].types`` are empty, no uploads will be allowed for this object type (e.g. ``user`` or ``group``). It previously meant that all file types were allowed. To keep the old behaviour use the string ``*`` as value in both options (this is dangerous and **not** recommended). Minor changes ------------- - Adapt login failure message if reCAPTCHA is enabled (`#8627 `_) - Update release process docs (`#8586 `_) Bugfixes -------- - `CVE-2025-24372 `_: Fix potential XSS vector through user and group/organization images. - Invalidate cached pages and load fresh ones if cookies change (`#6955 `_) - Fix `check_access` order for resource create view (`#8588 `_) - Fix CSV export error by ensuring BOM is written correctly as a string for Excel compatibility. (`#8635 `_) - Fix auth check for datastore data dictionary view (`#8639 `_) v.2.11.1 2024-12-11 =================== Migration notes --------------- - This version requires a requirements upgrade on source installations Minor changes ------------- - Allow configuring datastore full text field indexes with new `ckan.datastore.default_fts_index_field_types` config option. The default is "text tsvector" but this can be changed to "" to avoiding automatically creating separate full text indexes for any individual columns. This will result in a significant reduction in storage space. The whole-row full text index still exists for all tables. After upgrading to CKAN 2.12 the default changes to "". Use the `ckan datastore fts-index` command to remove existing column indexes to reclaim database space. (`#5847 `_) - Allow SECRET_KEY to fall back to beaker.session.secret for easier upgrades (`#7853 `_) - `datastore_info` action method now has `side_effect_free`, allowing it to be available via GET requests in the API. (`#8457 `_) - Remove unnecessary beaker.session.secret warning (`#8468 `_) - Upgrade requirements with security issues (`#8505 `_) - Register pytest plugins as entrypoints to make them available to all extensions (`#8507 `_) - Don't add author email to pyproject.toml if empty when creating an extension (`#8519 `_) - Add id attribute to AnonymousUser (`#8571 `_) - Automate publishing CKAN package to PyPI (`#8520 `_) - Automate creation of GitHub release (`#8570 `_) Bugfixes -------- - fix Page view tracking of datasets is not working if ckan is running at a subpath (`#5468 `_) - Load the right i18n files for Chinese locales in DataTables View. (`#8432 `_) - Fix exception in `ckan generate extension` command (`#8437 `_) - Template helper `member_count` will return 0 for unauthorized users. (`#8438 `_) - Fix `tracking` extension to use ORM models and comply with new `ckan.model` models (`#8447 `_) - Fix internal server error when viewing a deleted user. (`#8482 `_) - Fix display of user organizations page if user belongs to no organizations. (`#8483 `_) - Fix error when viewing history of a deleted resource or its package before the deletion date. (`#8501 `_) - Fix showing '0 members' for all groups on a dataset page. (`#8537 `_) - Include ``public`` folder in MANIFEST.in (`#8565 `_) - Fix 403 error when a user removes itself from a group (`#8256 `_) v.2.11.0 2024-08-21 =================== Overview -------- - CKAN 2.11 supports Python 3.9 to 3.12 - This version requires a requirements upgrade on source installations - This version requires a database upgrade. The minimum version required is PostgreSQL 12. - This version does not require a Solr schema upgrade if you are already using the 2.10 schema, but it is recommended to upgrade to the 2.11 Solr schema. Users of the `official Docker images `_ can use the ``ckan/ckan-solr:2.11-solr9`` tag. - Make sure to check the :ref:`migration-notes-2.11` Major features -------------- - Added support for **Python** 3.11 and 3.12 (`#8357 `_) - **Table Designer** is a form-builder for CKAN DataStore tables with enforced data validation. Use the :doc:`maintaining/table-designer` on the resource url/upload control for (`#6118 `_): - automatic creation of DataTable view for new Table Designer resources - add/delete columns and edit schema via Data Dictionary page - primary keys and required columns fully supported - add individual rows with an auto-generated form based on the schema - data validation enforced by PostgreSQL triggers, rendered as friendly errors in forms - extended DataTables view with "edit row" and "delete rows" buttons for managing data - automatic API documentation for create/upsert/delete with examples from real data when available - Increased **performance**: - Render snippets faster through better use of existing jinja2 tags. Use ``{% snippet 'path/to/snippet.html', arg1=test %}`` instead of ``{{ h.snippet('path/to/snippet.html', arg1=test) }}`` in templates for better performance. (`#6146 `_) - Improved start-up performance (`#8219 `_) - :py:class:`~ckanext.datastore.interfaces.IDataDictionaryForm` interface for extending and validating new keys in the ``fields`` dicts of the DataStore API actions. Unlike the ``info`` free-form dict, these new keys are possible to tightly control with a schema. The schema is built by combining schemas from from all plugins implementing this interface so plugins implementing different features may all contribute to the same schema. The underlying storage for data dictionary fields has changed. Use: ``ckan datastore upgrade`` after upgrading to this release. (`#7971 `_) - Start using **htmx** (`htmx.org `_) to modernize the CKAN frontend. For more information check :doc:`theming/htmx`. (`#7685 `_) - Enabled saving of **activities on private datasets**. Added filtering of dataset activities based on user permission labels. (`#5772 `_) Minor changes ------------- - Added user, group, and organization view functions and templates to make organization/group membership more public. Group and Organization lists now show the number of members. Group and Organization lists on a user's profile and dashboard now display the role for the group. Refactor: renamed ``.members`` to ``.manage_members``. ``.members`` is no longer an admin page. New: ``read_groups`` and ``read_organization`` view functions and templates for users. Adds group and organization tabs to a user profile to list the groups they belong to. New: ``member_dump`` view function. Downloads group/organization members into a CSV file with headers [Username,Email,Name,Role] (`#7007 `_) - :py:class:`~ckan.plugins.toolkit.BaseModel` class for declarative SQLAlchemy models added to :py:mod:`ckan.plugins.toolkit`. Models extending ``BaseModel`` class are attached to the SQLAlchemy's metadata object automatically (`#7351 `_):: from ckan.plugins import toolkit class ExtModel(toolkit.BaseModel): __tablename__ = "ext_model" id = Column(String(50), primary_key=True) ... - The PyUtilib dependency has been removed. All the primitives for the plugin system are now defined in CKAN. (`#7976 `_) - Allow sysadmins to change usernames of other accounts (`#4193 `_) - ``date_str_to_datetime`` helper accepts values with timezone information. (`#8305 `_) - ``follow_*`` and ``unfollow_*`` APIs will no longer return an error if the user is already following or not following the entity. (`#7685 `_) - JS translations are no longer generated on each server restart. The are built when starting the development server with `ckan run` or explicitly with `ckan translations js` (`#8219 `_) - Added support for :ref:`ckan.download_proxy` to the `resourceproxy` plugin (`#8354 `_) - The ``datastore_rw_resource_url_types`` helper can be overridden to define additional resource url_type values that can be modified without force=True (`#7617 `_) - ``datastore_create`` now allows removing fields when passing a new list of ``fields`` and ``delete_fields=True`` (`#7622 `_) (`#7919 `_) - New ``reset_redis`` and ``clean_redis`` test fixtures for removing data from Redis. (`#7630 `_) - ``ckan generate fake-data`` accepts ``--user`` option that is used as ``context["user"]``. Some factories(``api-token`` for example), have a special meaning for the ``user`` parameter and do not pass it to context. (`#7635 `_) - Add tooltips when links are truncated, to show the full text. (`#7742 `_) - ``datastore_create``, ``datastore_upsert`` now include a ``records_row`` number when an error occurs while inserting, upserting or updating records (`#7748 `_) - Added processing and pre-processing indicators to Datatables Views. (`#7900 `_) - Adds button to delete a Resource's datastore table in ckanext-datapusher (`#7902 `_) - ``datastore_create``: Add a ``delete_fields`` flag that must be set to True to delete any existing fields not passed in the fields list - Introducing a new parameter to the ``user_create`` action ``with_apitoken``. When set, this parameter triggers the creation of an API token for the user. (`#7932 `_) - ``ckan db upgrade`` CLI command automatically applies migrations from plugins. Use ``ckan db upgrade --skip-plugins`` if this behavior does not fit into your deployment process. (`#7961 `_) - Added ``bytes`` property to the test CKANResponse class which returns bytes from the response data. (`#7982 `_) - Activity plugin now tracks new, changed, and deleted resource views. (`#8043 `_) - ``datastore_records_delete`` action now calls the ``datastore_delete`` action via the toolkit for better frameworking. (`#8101 `_) - Use a definition list for the Data Dictionary view on resource pages to allow extra information for each field. Update ``example_idatadictionaryform`` plugin to display extra information. (`#8110 `_) - Add reCAPTCHA protection on login and password reset (`#8121 `_) - Resource view list items now have an additional ``view-item`` class. (`#8154 `_) - Add ``ckan.logic.schema.validator_args`` and ``ckan.logic.validate`` decorators to toolkit. (`#8215 `_) - fix profile cli, add ``--cold`` and ``--best-of`` options. By default cli profile will now run the request once (cold), then give the best of the next 3 (hot) runs. Use ``--cold --best-of=1`` for the old cli profile behavior. (`#8223 `_) - Sysadmins can now search by ``email`` in the ``user_autocomplete`` component. (`#8228 `_) - add ``ckan generate migration --autogenerate`` option, sync models with migrations (`#8238 `_) - Integrate flask-multistatic extension into the CKAN code base and remove it from requirements. (`#7244 `_) - Added ``--disable-debugger`` option to CKAN cli ``run`` command. (`#7278 `_) - Added new ``datastore_records_delete`` action. Functions the same as ``datastore_delete`` action, but will never drop the database table. (`#7341 `_) - ``datastore_search`` ``sort`` parameters now support ``nulls first`` and ``nulls last`` (`#7356 `_) - ``datastore_upsert``: Treat empty strings as null for non-text types (`#7358 `_) - Add a new optional parameter to the ``datastore_dictionary`` helper that filters the columns returned and fix a datatablesview show-columns bug with it (`#7387 `_) - update documentation for CKAN SHELL command. (`#7402 `_) - Improve CKAN Data API dialog with syntax highlighting, multiple client languages and jinja2 blocks for expansion (`#7573 `_) - Added ``ckan.datatables.null_label`` config option and ``h.datatablesview_null_label`` helper. Datatables Views will now show blank cells for NoneType field values by default. (`#7574 `_) - faster navigation between dataset and resource edit pages (`#7586 `_) - ``user_logged_in`` and ``user_logged_out`` signals added to the ``ckan`` namespace (`#7608 `_) - Store JS translation files in the storage folder rather than the source, to avoid permission problems (`#7585 `_) - Hide full helpers dict to tidy flask debug template listing (`#7668 `_) - Because of a new version of Sphinx, the command to rebuild the documentation is now ``sphinx-build doc build/sphinx`` (`#7808 `_) - Hide `Add new resource` button in the resource list while viewing activity history. (`#7814 `_) - Serve i18n js faster with LazyJSONObject. Generate compact json instead of pretty-printed json to send less data (`#7852 `_) - Show existing resource navigation on new resource page (`#7889 `_) - Use object-group icon for Embed button (`#7890 `_) - Note that md5 use in tracking is not a security context (`#7906 `_) - Remove mentions of username change in documentation (`#8000 `_) - Fix an old remainder in the documentation about permanent deletion of organizations and groups (`#8022 `_) - Allow preventing users from changing their passwords by hiding the ``password1`` and ``password2`` fields in the user edit form. (`#8208 `_) - ``ckan db init`` is now alias of ``ckan db upgrade``, which provides better support for includuing plugin migrations (`#8339 `_) - Use case sensitive email unique validator (`#7934 `_) - It is now possible to extend interface classes directly when implementing plugins, which provides better integration with development tools, e.g. (`#7976 `_):: class Plugin(p.SingletonPlugin, IClick): pass This is equivalent to:: class Plugin(p.SingletonPlugin): p.implements(p.IClick, inherit=True) - New ``ckan config docs`` command, support for config options Markdown documentation (`#8397 `_) Bug fixes --------- - `CVE-2024-43371 `_: SSRF prevention mechanisms. Added support for the :ref:`ckan.download_proxy` setting in the `Resource Proxy `_ plugin. - `CVE-2024-41674 `_: fixed Solr credentials leak via error message in ``package_search`` action. - `CVE-2024-41675 `_: fixed XSS vector in DataTables view. - Add support for custom resource_view auth in view templates (`#5909 `_) - datastore_search_sql returns correct numeric data (`#5753 `_) - Use ``resource_delete`` auth function in ``views.resource.DeleteView``. (`#7131 `_) - Fix ``member_list`` action to exclude deleted user(when state deleted is not updated in member table) (`#7170 `_) - Fixes a bug causing ``ckan.datasets_per_page`` config not being used. ``limit`` parameter in group/organization view has been removed in favor of the config. (`#7254 `_) - Create user using one line command. (`#7343 `_) - Fixes ``datastore_active`` flagging during the ``datastore_delete`` action when an empty ``filters`` dict is passed. (`#7345 `_) - Fix 500 error caused from passing null to a field using the ``ckanext.datastore.logic.schema.json_validator`` in its schema (`#7346 `_) - Create user reference added in Installing CKAN from source (`#7366 `_) - Fixed links and labels on dashboard/organization page. (`#7432 `_) - Fix exception in ``license_list`` action (`#7454 `_) - In tests, templates from ``ckan.plugins`` set by the config file are used even if these plugins are disabled for the test via ``pytest.mark.ckan_config("ckan.plugins", "")`` (`#7483 `_) - Fix usage of ``defer_commit`` in context in create actions for users, datasets, organizations and groups. ``model.Dashboard.get()`` no longer creates a dashboard object under the hood if it does not exist in the database (`#7487 `_) - "Groups" link in the header is not translated. (`#7500 `_) - Remove unnecessary use of add_public_directory from core extensions. Standardize on assets directory as the convention for extension web assets. (`#7504 `_) - Redirect dashboard news feed to login page if not logged in (`#7507 `_) - Fixed context in ``set_datastore_active_flag`` to solve possible solr errors during ``index_package`` (`#7571 `_) - ``ckan generate fake-data --factory-class x.y.z:Factory`` does not accept field values. (`#7607 `_) - Source files for webassets with identical names loaded from the wrong path. (`#7610 `_) - Context requires type-casting when ``model`` passed explicitly. (`#7611 `_) - POST request to GET-only endpoint causes 500 error (`#7616 `_) - Plugins randomly change their order during test session and sometimes they work even without ``with_plugins`` fixture. (`#7638 `_) - datastore_upsert method=insert: prevent 500 on invalid data datastore_create datastore_create: invalid data errors now reported against records value (not "message") (`#7683 `_) - Don't rely on stable ordering from unstable ``model.Package.resources list`` (`#7749 `_) - Updated the ``ckan.plugins.toolkit.check_ckan_version()`` to use packaging.version for version comparison/testing, Remove ``ckan.plugins.toolkit._version_str_2_list()`` method because of no use. (`#7777 `_) - Use current CKAN version in cookiecutter tests runner template (`#7938 `_) - URLs in activities always points to ``/organization/*`` but custom org types requeres ``/custom-organization/*`` URLs. This fixes those links. (`#7943 `_) - Fixed issues with the ``ckan views create`` CLI sub-command. (`#7944 `_) - Add missing translations to aria-label attributes (`#7945 `_) - libmagic error when CKAN 2.10.3 is installed from source (`#7986 `_) - Populate email notification checkbox from the profile it's on, not from the logged-in user (`#8124 `_) - ``use_default_schema`` in ``package_show`` is now evaluated as boolean. (`#8130 `_) - Allow using ``.`` in Solr local parser parameters (`#8138 `_) - Hide invite user form if the user can't create users (`#8141 `_) - Add error notification when rebuilding the search index via the cli when the requested package can't be found. (`#8148 `_) - Correct package_patch docstring re: updating resources (`#8179 `_) - Fix exception in ``group_list`` / ``organization_list`` when passing the ``groups`` / ``organizations`` parameters (`#8210 `_) - Set license model `od_conformance` and `osd_conformance` attributes' default values to `False` to prevent errors. (`#8268 `_) - Prevent exception in Datatables view when the size field is missing (`#8284 `_) - Remove mutable global state usage in group blueprint (`#8359 `_) - Added back ``header_extra`` and ``body_extra`` template blocks (`#8264 `_) .. _migration-notes-2.11: Migration notes --------------- - Starting from CKAN 2.11, the :ref:`SECRET_KEY` configuration option is required to start CKAN. This is the secret token that is used by security related tasks by CKAN and its extensions. Previous CKAN versions relied on the ``beaker.session.secret`` config option for this. The ``ckan generate config`` command generates a unique value for this option each time it generates a config file. Alternatively, you can generate one manually with the following command:: python -c "import secrets; print(secrets.token_urlsafe(20))" Note that all the following secret configuration options will fallback to the ``SECRET_KEY`` value if not defined in your ini file (`#7781 `_): * :ref:`WTF_CSRF_SECRET_KEY` * :ref:`api_token.jwt.encode.secret` * :ref:`api_token.jwt.decode.secret` - The sessions handling has been refactored, dropping the Beaker library in favour of `Flask-Session `_. Note that the default session backend for new sites remains the client-side browser cookie based. See :ref:`SESSION_TYPE` for alternative backends available. The following configuration options need to be updated (`#7893 `_) : ================================= ============================================== Old configuration key New configuration key ================================= ============================================== ``beaker.session.type`` :ref:`SESSION_TYPE` ``beaker.session.key`` :ref:`SESSION_COOKIE_NAME` ``beaker.session.cookie_expires`` :ref:`SESSION_PERMANENT` (with opposite value) ``beaker.session.timeout`` :ref:`PERMANENT_SESSION_LIFETIME` ``beaker.session.cookie_domain`` :ref:`SESSION_COOKIE_DOMAIN` ``beaker.session.secure`` :ref:`SESSION_COOKIE_SECURE` ``beaker.session.httponly`` :ref:`SESSION_COOKIE_HTTPONLY` ``beaker.session.samesite`` :ref:`SESSION_COOKIE_SAMESITE` ================================= ============================================== - When parsing the configuration file, the default behaviour starting from CKAN 2.11 is the old ``strict`` mode, where CKAN will not start unless **all** config options are valid according to the validators defined in the :ref:`configuration declaration `. For every invalid config option, an error will be printed to the output stream. (`#7776 `_) - If using the DataStore, the underlying storage for data dictionary fields has changed. Use ``ckan datastore upgrade`` after upgrading to this release to migrate it (`#7971 `_) - When the ``activity`` plugin is enabled, every action that creates an activity recorded(i.e. ``package_create``, ``package_update``, ``package_delete``, ``group_*``, ``organization_*``, ``user_*``, ``bulk_update_*``) requires a ``context['user']`` and raises ``ValidationError`` if it's missing or empty. (`#7627 `_) - The configuration option to customize the authorization header name has been renamed to :ref:`apitoken_header_name` from ``apikey_header_name``. - Only sysadmins can now set the ``id`` field of Datasets, Groups, Organizations, Users, Resource Views and Extras (`#8069 `_) - If provided, the value of the ``id`` field needs to be a valid UUID string. Sites using custom ids that are not UUIDs can extend the relevant schema or validate methods to override the validation on the ``id`` field, but are strongly encouraged to use a separate custom field to store the custom id instead. (`#8069 `_) - The ``form_to_db_*`` and ``db_to_form_*`` methods of the ``IGroupForm`` interface are now deprecated, and have been replaced by``create_group_schema()``, ``update_group_schema()`` and ``show_group_schema()``. (`#8069 `_) - Tests performing requests using the test client should authenticate users sending the default ``Authorization`` header with a valid token, as opposed to sending the user name in ``environ_overrides`` (or the older ``extra_environ``) (`#7841 `_) Before:: def test_dataset_new(app): user = factories.User() app.get(url_for("dataset.new"), environ_overrides={"REMOTE_USER": user["name"]}) After:: def test_dataset_new(app): user = factories.UserWithToken() app.get(url_for("dataset.new"), headers={"Authorization": user["token"]}) - Only sysadmins can now set the ``id`` field of Datasets, Groups, Organizations, Users, Resource Views and Extras - If provided, the value of the ``id`` field needs to be a valid UUID v4 string. Sites using custom ids that are not UUIDs can extend the relevant schema or validate methods to override the validation on the ``id`` field, but are strongly encouraged to use a separate custom field to store the custom id instead. - The following interfaces are iterated in reverse order when using :py:class:`~ckan.plugins.core.PluginImplementations(interface)` (`#7609 `_): * ``IConfigDeclaration`` * ``IConfigurer`` * ``ITranslation`` * ``IValidators`` - :py:meth:`~ckanext.datastore.interfaces.IDatastore.datastore_search` of :py:class:`~ckanext.datastore.interfaces.IDatastore` interface is not completely compatible with old version. ``where`` key of the ``query_dict`` returned from this method has a different format. Before it was a collection of tuples with an SQL where-clause with positional/named ``%``-style placeholders on the first position, followed by arbitrary number of parameters:: return { ..., "where": [('"age" BETWEEN %s AND %s', param1, param2, ...), ...] } Now every element of collection must be a tuple that contains SQL where-clause with **named** ``:``-style placeholders and a dict with the values for all the placeholders:: return { ..., "where": [( '"age" BETWEEN :my_ext_min AND :my_ext_max', {"my_ext_min": age_between[0], "my_ext_max": age_between[1]}, )] } In order to avoid name conflicts with placeholders from different plugin, don't use simple names, i.e. ``val``, ``min``, ``name``, and add unique prefix to all the placeholders. (`#7583 `_) - ``snippet/organization.html`` has been moved to ``organization/snippets/info.html`` for consistency with Groups/Packages/Users. (`#7685 `_) - Tracking feature has been moved to its own core extension. Therefore, ``ckan.tracking_enabled`` configuration option should be changed to adding ``tracking`` to CKAN's plugins list. ``g.tracking_enabled`` attribute no longer exist. ``tracking_summary`` info will be returned if the extension is enabled. ``include_tracking`` parameter is no longer required. (`#7772 `_) Removals and deprecations ------------------------- - ``PackageExtra`` and ``GroupExtra`` models will be removed in the next release and replaced by ``Package.extras`` and ``Group.extras`` JSONB fields. Code that accesses these models directly will need to be updated to use the ``Package.extras`` and ``Group.extras`` dicts for updating and JSON queries like ``query(Package, Package.extras['name'] == '"value"')``. (`#8288 `_) - All revision tables will be removed from the database in the next release. If you are upgrading from a ckan older than 2.9 and want to keep the history of changes this release is the last chance to run the ``migrate_package_activity.py`` script as described in the 2.9.0 :ref:`migration-notes-2.9`. (`#8320 `_) - The ``form_to_db_*`` and ``db_to_form_*`` methods of the ``IGroupForm`` interface are now deprecated, and have been replaced by``create_group_schema()``, ``update_group_schema()`` and ``show_group_schema()``. (`#8069 `_) - Removes ``dataset-form`` and ``dataset-resource-form`` classes from our HTML templates since they do not exist in our CSS files. (`#7164 `_) - The ``resource`` blueprint will be removed in the future. The blueprint ``_resource`` is preferred. E.g. use ``dataset_resource.read`` instead of ``resource.read`` (`#7373 `_) - Removes all calls and references to the deprecated ``check_data_dict`` method. (`#7420 `_) - The ``site_read`` authz function has been removed since it always returned True. (`#7544 `_) - SQLAlchemy's ``Metadata`` object (:py:attr:`ckan.model.meta.metadata`) is no longer bound the the DB engine. `A number of operations `_ such as ``table.exists()``, ``table.create()``, ``metadata.create_all()``, ``metadata.reflect()``, now produce an :py:class:`sqlalchemy.exc.UnboundExecutionError` error (`#7583 `_) . Depending on the situation, the following changes may be required: * Instead of creating tables via custom CLI command or during application startup, use `Alembic migrations `_ * If there is no other way, change ``table.create()``/``table.exists()`` to ``table.create(engine)``/``table.exists()``. Get ``engine`` by calling :py:func:`~ckan.model.ensure_engine`. - The Bootstrap 3 based templates have been removed. (`#7637 `_) - ``template_head_end`` and ``template_footer_end`` config options have been removed. You can achieve the same effect by extending the ``base.html`` template. (`#7672 `_) - ``ckan.dumps_url`` and ``ckan.dumps_format`` config options have been removed. You can achieve the same effect by extending ``package/search.html``. (`#7673 `_) - The ``build_extra_admin_nav`` helper and ``ckan.admin_tabs`` config have been removed. To achieve the same result it is possible to add a nav icon by extending the ``content_primary_nav`` block in ``ckan/templates/admin/base.html`` (`#7674 `_) :: {% ckan_extends %} {% block content_primary_nav %} {{ super() }} {{ h.build_nav_icon('example_extension.endpoint', _('My Cool Feature'), icon='trophy') }} {% endblock %} - The ``ckan.homepage_style`` configuration options and the ``homepage_style`` variable have been removed. ``layout1.html`` code has been moved into ``home/index.html``, as it will be the only layout available. (`#7677 `_) - The Recline-based view plugins (``recline_view``, ``recline_grid_view``, ``recline_map_view``, etc) have been removed and are no longer available. Users are encouraged to use the DataTables-based view (``datatables_view``) or some of the `community maintained alternatives `_ `#7918 `_) - Move datastore-specific download logic from ``ckan/templates/package/resource_read.html`` to ``ckanext/datastore/templates/package/resource_read.html`` (`#7927 `_) - The deprecated methods with the form ``after_`` and ``before_`` of the :py:class:`~ckan.plugins.interfaces.IPackageController` and :py:class:`~ckan.plugins.interfaces.IResourceController` interfaces have been removed. The form ``after__`` must be used from now on. E.g. ``after_create()`` -> ``after_dataset_create()`` or ``after_resource_create()``. (`#7976 `_) - All plugins need to be instances of p.SingletonPlugin, they can't inherit from a base class that is an instance itself. For example, you need to move from this (`#7976 `_) :: class FirstPlugin(p.SingletonPlugin): p.implements(ISomething) def some_method(self): pass class SecondPlugin(FirstPlugin): p.implements(IAnything) To this:: class BasePlugin(): def some_method(self): pass class FirstPlugin(p.SingletonPlugin, BasePlugin): p.implements(ISomething) class SecondPlugin(p.SingletonPlugin, BasePlugin): p.implements(IAnything) v.2.10.9 2025-10-29 =================== Migration notes --------------- - This version requires a requirements upgrade on source installations - Restore handling of plugin order for ``ITemplateHelpers`` to align with ``add_template_directory`` precedence again (first plugin wins). You might have to change the order of a plugin in ``ckan.plugins`` if you relied on overriding a template helper from another plugin. (`#9069 `_) - A new config option :ref:`ckan.uploads_enabled` was added to prevent a critical error being shown in the logs about a missing :ref:`ckan.storage_path` setting. This is not required and existing sites should work as before. When the ``ckan.uploads_enabled`` is not set, uploads will be shown in the UI if ``ckan.storage_path`` is defined or there is an ``IUploader`` plugin configured. (`#8977 `_) Minor changes ------------- - Replaced lint tool flake8 with ruff. (`#8964 `_) - Remove unused `ckan.static_max_age` config option (`#9005 `_) - Include CHANGELOG.rst in MANIFEST.in. (`#9083 `_) Bugfixes -------- - `CVE-2025-XXXX `_: Rotate session identifiers to prevent Session Cookie Fixation . - `CVE-2025-54384 `_: Stored XSS vector in Markdown description fields. - Fix auth check for ``resource_view_reorder`` (`#9131 `_) - Restore handling of plugin order for ``ITemplateHelpers`` to align with ``add_template_directory`` precedence again (first plugin wins). (`#9091 `_) - Hide users data from `stats` if the `ckan.auth.public_user_details` setting is set to `False`. This ensures that user details are not exposed in the statistics when public user details are disabled. (`#9030 `_) - Fixes the font of the sort indicator in datatablesview by properly closing the opening `_) - Historical versions of a custom dataset type ignores base template specified by the ``IDatasetForm`` interface. (`#8875 `_) - Fix ``EXPLAIN JSON`` output sometimes being auto-decoded in multithreaded environments by setting json_deserializer at engine level. (`#8929 `_) - Catch ``NotAuthorized`` exception raised by ``datastore_search`` (`#8989 `_) - Remove override of group_dict['package_count'] to display the correct dataset count (`#8252 `_ v.2.10.8 2025-05-07 =================== Minor changes ------------- - Register pytest plugins as entrypoints to make them available to all extensions (`#8507 `_) - Update release process docs (`#8586 `_) - Migrate CI checks to GitHub Actions (`#8909 `_) - Upgrade Jinja2 requirement to address CVE-2025-27516 Bugfixes -------- - Default to a long CSRF token timeout to fix "The CSRF token is invalid." errors (`#8803 `_) - Fix search button styling by removing deprecated wrapper (`#8737 `_) - Catch an error in datastore to avoid 500 error in POSTs to `datatables/ajax/` (`#8149 `_) - Invalidate cached pages and load fresh ones if cookies change (`#6955 `_) - Remove unsupported legacy API keys from documentation (`#8195 `_) - Ensure ``session["last_active"]`` is stored as an iso string instead of a `datetime` so that it can be serialized to JSON (e.g. in cookies). (`#8379 `_) - Fix ``check_access`` order for resource create view (`#8588 `_) - Fix auth check for datastore data dictionary view (`#8639 `_) - Add missing boolean_validator to sysadmin field in user schema (`#8674 `_) - ``authz.has_user_permission_for_some_org`` returns True for sysadmins. (`#8680 `_) - Apply ``humanize_entity_type`` helper consistently across the group listing page (`#8682 `_) - ``datastore_search``: fix for sort on array column types (`#8709 `_) - Fix some ``.btn-default`` classes that were mistakenly changed to ``.btn-light``. (`#8828 `_) v.2.10.7 2025-02-05 =================== Migration notes --------------- * Going forward, if both ``ckan.upload.[type].mimetypes`` and ``ckan.upload.[type].types`` are empty, no uploads will be allowed for this object type (e.g. ``user`` or ``group``). It previously meant that all file types were allowed. To keep the old behaviour use the string ``*`` as value in both options (this is dangerous and **not** recommended). Minor changes ------------- - Adapt login failure message if reCAPTCHA is enabled (`#8627 `_) - Update release process docs (`#8586 `_) - Support 2.11 version of the Solr schema in CKAN 2.10 (``5acfeda6e``) Bugfixes -------- - `CVE-2025-24372 `_: Fix potential XSS vector through user and group/organization images. - Invalidate cached pages and load fresh ones if cookies change (`#6955 `_) - Fix `check_access` order for resource create view (`#8588 `_) - Fix auth check for datastore data dictionary view (`#8639 `_) v.2.10.6 2024-12-11 =================== Minor changes ------------- - `datastore_info` action method now has `side_effect_free`, allowing it to be available via GET requests in the API. (`#8457 `_) - Add id attribute to AnonymousUser (`#8571 `_) - Automate publishing CKAN package to PyPI (`#8520 `_) - Automate creation of GitHub release (`#8570 `_) Bugfixes -------- - Fixed context in `set_datastore_active_flag` to solve possible solr errors during `index_package` (`#7571 `_) - POST request to GET-only endpoint causes 500 error (`#7616 `_) - Set license model `od_conformance` and `osd_conformance` attributes' default values to `False` to prevent errors. (`#8268 `_) - Load the right i18n files for Chinese locales in DataTables View. (`#8432 `_) - Fixed server error on robots.txt when bootstrap 3 templates were used. (`#8536 `_) - Include ``public`` folder in MANIFEST.in (`#8565 `_) v.2.10.5 2024-08-21 =================== Migration notes --------------- - This version requires a requirements upgrade on source installations - The minimum Python version for this version is Python 3.8. It has been tested up to Python 3.11 Minor changes ------------- - Support for Python 3.11 (`#8171 `_) - Upgrade requirements to address security vulnerabilities (`#8349 `_) - Added :ref:`ckan.datatables.null_label` config option. Datatables views will now show blank cells for NoneType field values by default. (`#7574 `_) Bugfixes -------- - `CVE-2024-43371 `_: SSRF prevention mechanisms. Added support for the :ref:`ckan.download_proxy` setting in the `Resource Proxy `_ plugin. - `CVE-2024-41674 `_: fixed Solr credentials leak via error message in ``package_search`` action. - `CVE-2024-41675 `_: fixed XSS vector in DataTables view. - Allow using ``.`` in Solr local parser parameters (`#8138 `_) - Fix misplaced CSRF token in the BS3 collaborator_new.html. (`#8204 `_) - Prevent exception in Datatables view when the size field is missing (`#8284 `_) v.2.10.4 2024-03-13 =================== Migration notes --------------- - The default format for accepted uploads for user, groups and organization images is now limited to PNG, GIF anf JPG. If you need to add additional formats you can use the :ref:`ckan.upload.user.mimetypes` and :ref:`ckan.upload.group.mimetypes`) (`#7028 `_) - Public user registration is disabled by default, ie users can not create new accounts from the UI. With this default value, new users can be created by being invited by an organization admin, being created directly by a sysadmin in the ``/user/register`` endpoint or being created in the CLI using ``ckan user add``. To allow public registration see :ref:`ckan.auth.create_user_via_web`, but it's strongly encouraged to put some measures in place to avoid spam. (`#7028 `_) (`#7208 `_) Minor changes ------------- - Define allowed alternative Solr query parsers via the :ref:`ckan.search.solr_allowed_query_parsers` config option (`#8053 `_) Bugfixes -------- - `CVE-2024-27097 `_: fixed potential log injection in reset user endpoint. - use custom group type from the activity object if it's not supplied, eg on user activity streams (`#7980 `_) - Removes extra <<`_) - CKAN does not start without ``beaker.session.validate_key`` option introduced in v2.10.3 (`#8023 `_) - Editing of resources unavailable from package view page. (`#8025 `_) - Pass custom package types through to the 'new resource' activity item (`#8034 `_) - Fix Last Modified sort parameter for bulk-process page (`#8048 `_) - Detect XLSX mimetypes correctly in uploader (`#8088 `_) - Remove nginx cache as configuration from documentation (`#8031 `_) - Fix `clean_db` fixtures breaking when tables are missing (`#8054 `_) - Fix JS error in flash message when adding a Member (`#8104 `_) v.2.10.3 2023-12-13 =================== Minor changes ------------- - New sites now default to cookie-based sessions (the default value for ``beaker.session.type`` is now ``cookie``. The ``beaker.session.samesite`` configuration option has been introduced, allowing you to specify the ``SameSite`` attribute for session cookies. This attribute determines how cookies are sent in cross-origin requests, enhancing security and privacy. .. note:: When using cookie-based sessions, it is now required to set ``beaker.session.validate_key`` appropriately. - Skip interactive mode of ``ckan user setpass`` using ``-p``/``--password`` option. (`#7530 `_) - Added support for Solr 9. Users of the `official Docker images `_ can use the ``ckan/ckan-solr:2.10-solr9`` tag. (`#7693 `_) - Update requirements to support more Python versions (`#7935 `_) - Add tooltips when links are truncated, to show the full text. (`#7743 `_) - Added pages to confirm User delete and Dataset Collaborator delete. Fixed cancellation of Group Member delete. (`#7813 `_) - The ``validators`` attribute of a declared config option makes tries to parse arguments to validators as python literals. If **all** arguments can be parsed, they are passed to a validator factory with original types. If at least one argument is not a valid Python literal, all values are passed as a string (this was the previous behavior). Space characters are still not allowed inside arguments, use the ``\\x20`` symbol if you need a space in a literal (`#7615 `_):: # Not changed `validators: v(xxx)` # v("xxx") `validators: v("xxx",yyy)` # v("xxx", "yyy") `validators: v(1,2,none)` # v("1", "2", "none") `validators: v("hello\\x20world")` # v("hello world") # Changed `validators: v("xxx")` # v("xxx") `validators: v("xxx",1)` # v("xxx", 1) `validators: v(1,2,None)` # v(1, 2, None) - Automatically add the ``not_empty`` validator to any config option declared with ``required: true`` (`#7658 `_) Bugfixes -------- - `CVE-2023-50248 `_: fix potential out of memory error when submitting the dataset form with a specially-crafted field. - Fix ``deprecated`` decorator (`#7939 `_) - Fix for missing Tag facets on Home page (`#7520 `_) - Fix errors when running the `ckan db upgrade` command (`#7681 `_) - Fix datastore_search + downloading datastore resources as json with null values (`#6713 `_) - ``CONFIG_FROM_ENV_VARS`` takes precedence over config file and extensions but those settings are not normalized. (`#7502 `_) - Fixed server not recognizing SSL settings in configuration .ini file (`#7758 `_) - Fix error when indexing a full ISO date with timezone info (`#7775 `_) - Aligned `member_create` with `group_member_save` to prevent possible member duplication. (`#7804 `_) - datastore-only resources now have a visible download button on the resource page (`#7806 `_) - update resource ``datastore_active`` with a single statement on ``datastore_create/delete`` (`#7832 `_) - Fixed Octet Streaming for Datastore Dump requests. (`#7839 `_) - Fixed restricting anonymous users in actions to check user in context. (`#7871 `_) - Empty string in ``beaker.session.timeout`` produces an error instead of never-expiring session (`#7881 `_) - Updated Bootstrap alert-error class to alert-danger (`#7901 `_) - Changed dataset query to check for ``+state:`` in the ``fq_list`` as well as the `fq` parameter before forcing ``state:active`` (`#7905 `_) - View modules use pluggable ``ckan.plugins.toolkit.h`` instead of `ckan.lib.helpers` (`#7923 `_) - Fix HTML5 validation failing on resource uploads (`#7925 `_) - Fixed issues with the ``ckan views create`` CLI sub-command. (`#7944 `_) - Improve handling of date fields in Solr (`#7775 `_) - Fix URL validator does not support ":" for specifying ports (`#7891 `_) - Fix user_show for ``ckan.auth.public_user_details`` (`#7866 `_) - Add missing translations to aria-label attributes (`#7947 `_) - Catch AttributeErrors in license retrieval (`#7931 `_) - Fix downloading datastore resources as json with null values in json columns (`#7545 `_) v.2.10.2 ======== Unreleased v.2.10.1 2023-05-24 =================== Bug fixes --------- - `CVE-2023-32321 `_: fix potential path traversal, remote code execution, information disclosure and DOS vulnerabilities via crafted resource ids. - Redirect on password reset form error now maintains root_path and locale (`#7006 `_) - Fix display of Popular snippet (`#7205 `_) - Fixes missing CSRF token when trying to remove a group from a package. (`#7417 `_) - ``IMiddleware`` implementations produce an error mentioning missing ``app.after_request`` attribute. (`#7426 `_) - Application hangs during startup when using config chains. (`#7427 `_) - Fix exception in ``license_list`` action (`#7454 `_) - In tests, templates from ``ckan.plugins`` set by the config file are used even if these plugins are disabled for the test via ``pytest.mark.ckan_config("ckan.plugins", "")`` (`#7483 `_) - Fix usage of ``defer_commit`` in context in create actions for users, datasets, organizations and groups. - ``model.Dashboard.get()`` no longer creates a dashboard object under the hood if it does not exist in the database (`#7487 `_) - "Groups" link in the header is not translated. (`#7500 `_) - Names are now quoted in From and To addresses in emails, meaning that site titles with commas no longer break email clients. (`#7508 `_) - Pagination widget is not styled in Bootstrap 5 templates. (`#7528 `_) - Fix missing resource URL on update resource with uploaded file (`#7449 `_) - Fix custom macro styles (`#7461 `_) - Fix mobile layout styles (`#7467 `_) - Fix fontawesome icons, replace unavailable FA v3 icons (`#7474 `_) - Fix promote sysadmin layout (`#7476 `_) - Fix markdown macros regression (`#7485 `_) - Set session scope for migrate_db_for fixture (`#7563 `_) Migration notes --------------- - The default storage backend for the session data used by the Beaker library uses the Python ``pickle`` module, which is considered unsafe. While there is no direct known vulnerability using this vector, a safer alternative is to store the session data in the `client-side cookie `_. This will probably be the default behaviour in future CKAN versions:: # ckan.ini beaker.session.type = cookie beaker.session.data_serializer = json # Use a long, random string for this setting beaker.session.validate_key = CHANGE_ME beaker.session.httponly = True beaker.session.secure = True beaker.session.samesite = Lax # or Strict, depending on your setup .. note:: You might need to install an additional library that can provide AES encryption, e.g. ``pip install cryptography`` v.2.10.0 2023-02-15 =================== Overview -------- - CKAN 2.10 supports Python 3.7 to 3.10 - This version requires a requirements upgrade on source installations - This version requires a database upgrade - This version does not require a Solr schema upgrade if you are already using the 2.9 schema, but it is recommended to upgrade to the 2.10 Solr schema. - Make sure to check the :ref:`migration-notes-2.10` Major features -------------- - Added **CSRF protection** to the frontend forms to protect against Cross-Site Request Forgery attacks. This feature is enabled by default in CKAN core, extensions are excluded from the CSRF protection to give time to update them, but CSRF protection will be enforced in the future. To enforce the CSRF protection in extensions you can use the ``ckan.csrf_protection.ignore_extensions`` setting. See the :ref:`CSRF section ` in the extension best practices for more information on how to enable it. (`#6920 `_) - Refactored the **Authentication logic** to use `Flask-login `_ instead of repoze.who. This has implications on how login sessions are managed (e.g. when and why users might be logged out) and will affect all plugins that modify the standard authentication process. Please check the *Migration notes* section below to learn more (`#6560 `_). - **Configuration declaration**: declare configuration options to ensure validation and default values. All declared CKAN configuration options are validated and converted to the expected type during the application startup. See the *Migration notes* section below to understand the changes involved and check the :ref:`documentation `. (`#6467 `_) - Add **Signals** support to allow subscriptor-based features in extensions. See :doc:`extensions/signals` (`#5359 `_) - Add **Blanket implementations**: decorators providing common implementations of simple interfaces to reduce boilerplate in plugins. See the ``blanket()`` method in the :doc:`/extensions/plugins-toolkit` (`#5169 `_) - Add CLI commands for API Token management (`#5868 `_) - The CKAN source code is fully typed now (`#5924 `_) - Add extensible snippet for resource uploads (`#6226 `_) - Migrated to **Bootstrap 5** from v3 for the default CKAN theme. Bootstrap v3 templates are still available for use by specifying the base template folder in the configuration (`#6307 `_):: ckan.base_public_folder=public-bs3 ckan.base_templates_folder=templates-bs3 - Removed the **Docker** related files from the main CKAN repository. A brand new official Docker setup can be found at the `ckan/ckan-docker `_ repository. (`#7370 `_) - Added new command ``ckan shell`` that opens an interactive python shell with the Flask's application context preloaded (among other useful objects). (`#6919 `_) - Added new sub-commands to the ``search-index`` command (`#7044 `_ and `#7175 `_): - ``list-orphans`` lists all public package IDs which exist in the solr index, but do not exist in the database. - ``clear-orphans`` clears the search index for all the public orphaned packages. - ``list-unindexed`` lists all ununindexed packages - Add new group command: ``clean``. Add ``clean users`` command to delete users containing images with formats not supported in ``ckan.upload.user.mimetypes`` config option. (`#7241 `_) - Activities now receive the full dict of the object they refer to in their ``data`` section. This allows greater flexibility when creating custom activities from plugins. (`#6557 `_) - Site maintainers can choose to completely ignore cookie based by using ``ckan.auth.enable_cookie_auth_in_api``. When set to False, all API requests must use :ref:`API Tokens `. Note that this is likely to break some existing JS modules from the frontend that perform API calls, so it should be used with caution. (`#7088 `_) - CKAN now records the last time a user was active on the site. The minimum interval between records can be controlled with the :ref:`ckan.user.last_active_interval` config option. (`#6466 `_) - :py:class:`~ckan.plugins.toolkit.BaseModel` class for declarative SQLAlchemy models added to :py:mod:`ckan.plugins.toolkit`. Models extending ``BaseModel`` class are attached to the SQLAlchemy's metadata object automatically:: from ckan.plugins import toolkit class ExtModel(toolkit.BaseModel): __tablename__ = "ext_model" id = Column(String(50), primary_key=True) ... (`#7351 `_) - Add dev containers / GitHub Codespaces config (See the `documentation `_ Minor changes ------------- - Test factories extends SQLAlchemy factory, are available via fixtures and produce more random entities using faker library. (`#6335 `_) - Migrated preprocessor from LESS to SCSS for preliminary work for Bootstrap upgrade. (`#6175 `_) - Add ``ckan.plugins.core.plugin_loaded`` to the core helpers as ``plugin_loaded`` (`#7011 `_) - Make HTTP response returned on a private dataset if not authorized configurable (`#6641 `_) - Allow ``_id`` for ``datastore_upsert`` unique key (`#6793 `_) - Add functionality to ``user_show`` to fetch own details when logged in without passing id (`#5490 `_) - ``datastore_info`` now returns more detailed info. It returns database-level metadata in addition to rowcount (aliases, id, size, index_size, db_size and table_type), and the data dictionary with database-level schemata (native_type, index_name, is_index, notnull & uniquekey). See the documentation at :py:func:`~ckanext.datastore.logic.action.datastore_info` (`#5831 `_) - ``datastore_info`` now works with aliases, and can be used to dereference aliases. (`#5832 `_) - Document new ``ckan.download_proxy`` config value for extensions that download external URLs (`#xloader-127 `_) - Add `organization_followee_count` to the get api (`#2628 `_) - Environment variables prefixed with `CKAN_` can be used as variables inside config file via ``option = %(CKAN_***)s`` (`#6192 `_) - CLI command ``less`` is now renamed to ``sass`` as the preprocessor was changed in #6175. (`#6287 `_) - Support including file attachments when sending emails (`#6535 `_) - Reworked the JavaScript for the view filters to allow for special characters as well as colons and pipes, which previously caused errors. Added a new helper (``decode_view_request_filters()``) to easily decode the new flattened filter string. (`#6747 `_) - Add an index on column resource_id in table resource_view. (`#7134 `_) - Non-sysadmin users are no longer able to change their own state (`#6956 `_) - The "rank" field is no longer returned in datastore_search results unless explicitly defined in the fields parameter (`#6961 `_) - Upgrade requirements to the latest version whenever possible (`#7064 `_) - Create a ``fresh_context()`` function to allow cleaning the ``context`` dict preserving some common values (``user``, ``model``, etc) (`#7112 `_) - Add ``--quiet`` option to ``ckan user token add`` command to mak easier to integrate with automated scripts (`#7217 `_) - Updated and documented input param for ``api_token_list`` from ``user`` to ``user_id``. ``user`` is still supported for backwards compatibility but it might be removed in the future. (`#7344 `_) Bugfixes -------- - Stable default ordering when consuming resource content from datastore (`#2317 `_) - Fix missing activities from UI when internal processes are run by ignored users (`#5699 `_) - Fix the datapusher trigger in case of resource_update via API (`#5727 `_) - package_revise now returns some errors in normal keys instead of under 'message' (`#5888 `_) - Allow multi-level config inheritance (`#6000 `_) - Fix Chinese locales. Note that the URLs for the `zh_CN` and `zh_TW` locales have changed but there are redirects in place, eg http://localhost:5000/zh_CN/dataset -> http://localhost:5000/zh_Hans_CN/dataset (`#6008 `_) - Fix performance bottleneck in activity queries (`#6028 `_) - Keep repeatable facets inside pagination links (`#6084 `_) - Consistent CLI behavior when when no command provided and when using `--help` options (`#6120 `_) - Variables from extended config files (``use = config:...``) have lower precedence. In the following example:: ;; a.ini output = %(var)s ;; b.ini use = config:a.ini var = B ;; c.ini use = config:b.ini var = C final value of the ``output`` config option will be ``C``. (`#6192 `_) - Restore error traceback for `search-index rebuild -i` CLI command (`#6329 `_) - Prevent Traceback to logged for HTTP Exception until debug is true Add the HTTP status Code in logging for HTTP requests (`#6340 `_) - Improve rendering data types in resource view (`#6356 `_) - Snippet names rendered into HTML as comments in non-debug mode. (`#6406 `_) - h.remove_url_param fail with minimal set of params (`#6414 `_) - Type of uploads for group and user image can be restricted via the `ckan.upload.{object_type}.types` and `ckan.upload.{object_type}.mimetypes` config options (eg `ckan.upload.group.types`, `ckan.upload.user.mimetypes`) (`#6477 `_) - ``*_patch`` actions call their ``*_update`` equivalents via ``get_action`` allowing plugins to override them consistently (`#6519 `_) - Fixed and simplified organization and group forms breadcrumb inheritance (`#6637 `_) - Ensure that locale exists on i18n JS API (`#6698 `_) - Configuration options that were used to specify a CSS file with a base theme have been removed. Use the alternatives below in order to specify an _asset_ (see :doc:`theming/webassets`) with a base theme for application (`#6817 `_): * ``ckan.main_css`` replaced by :ref:`ckan.theme` * ``ckan.i18n.rtl_css`` replaced by :ref:`ckan.i18n.rtl_theme` - prepare_dataset_blueprint: support dataset type (`#7031 `_) - Changed default sort key for group and user lists from ASCII Alphebitized to new `strxfrm` helper, resulting in human-readable alphebitization. (`#7039 `_) - Fix resource file size not updating with resource_patch (`#7075 `_) - Revert Flask requirement from 2.2.2 to 2.0.3. (`#7082 `_) - restore original plugin template directory order after update_config order change (`#7085 `_) - Fix urls containing unicode encoded in hex (`#7107 `_) - Fix a bug that causes CKAN to only register the first blueprint of plugins. (`#7108 `_) - remove old deleted resources on package_update so that performance is consistent over time (no longer degrading) (`#7119 `_) - Beaker session config variables need to be initialised in a newly generated ckan config file (`#7133 `_) - Fixed broken organization delete form (`#7150 `_) - Fix the current year reference for CKAN documentation (`#7153 `_) - Fix bootstrap 3 webassets files to point to valid assets. (`#7161 `_) - Fix the display of the License select element in the Dataset form. (`#7162 `_) - Build CSS files with latest updates. (`#7163 `_) - Fix activity stream icon on Bootstrap 5. Migrate activity CSS classes to the extension folder. (`#7169 `_) - Fix 404 error when selecting the same date in the changes view (`#7191 `_) - Fix display of Popular snippet. Removes old `ckan-icon` scss class. (`#7205 `_) - Fix icons and alignment in resource datastore tab. (`#7247 `_) - Make heading semantic in bug report template (`#7186 `_) - Add title attribute to iframe (`#7187 `_) - Fix color contrast in dashboard buttons for web accessibility (`#7193 `_) - Make skip to content visible for keyboard-only user (`#7194 `_) - Fix color contrast issue in add dataset page (`#7195 `_) - Fix color contrast of delete button in user edit page for web accessibility (`#7199 `_) .. _migration-notes-2.10: Migration notes --------------- - Changes in the authenticated users management (logged in users): The old ``auth_tkt`` cookie created by repoze.who does not exist anymore. Flask-login stores the logged-in user identifier in the Flask session. CKAN uses `Beaker `_ to manage the session, and the default session backend stores this session information as files on the server (on ``/tmp``). This means that **if the session data is deleted in the server, all users will be logged out of the site**. This can happen for instance: * if the CKAN container is redeployed in a Docker / cloud setup and the session directory is not persisted * if the sessions are periodically cleaned by an external script Here's a summary of the behaviour changes between CKAN versions: .. list-table:: :widths: 40 30 30 :header-rows: 1 * - Action - CKAN < 2.10 - CKAN >= 2.10 * - Clear cookies - User logged out - User logged out (If ``remember_me`` cookie is deleted) * - Clear server sessions - User still logged in - User logged out The way to keep the old behaviour with the Beaker backend is to store the session data in the `cookie itself `_ (note that this stores *all* session data, not just the user identifier). This will probably be the default behaviour in future CKAN versions:: # ckan.ini beaker.session.type = cookie beaker.session.validate_key = CHANGE_ME beaker.session.httponly = True beaker.session.secure = True beaker.session.samesite = Lax # or Strict Alternatively you can configure another persistent backend for the sessions in the server, like an SQL Database or Redis (see the `Beaker configuration `_ for details). - It is recommended that you review the :ref:`session-settings` and :ref:`flask-login-remember-me-cookie-settings` to make sure they cover your security requirements. - Due to the newly introduced :ref:`declare-config-options`, all declared CKAN configuration options are validated and converted to the expected type during the application startup:: debug = config.get("debug") # CKAN <= v2.9 assert type(debug) is str assert debug == "false" # or any value that is specified in the config file # CKAN >= v2.10 assert type(debug) is bool assert debug is False # or ``True`` The ``aslist``, ``asbool``, ``asint`` converters from ``ckan.plugins.toolkit`` will keep the current behaviour:: # produces the same result in v2.9 and v2.10 assert tk.asbool(config.get("debug")) is False assert tk.asint(config.get("ckan.devserver.port")) == 5000 assert tk.aslist(config.get("ckan.plugins")) == ["stats"] If you are using custom logic, the code requires a review. For example, the following code will produce an ``AttributeError`` exception, because ``ckan.plugins`` is converted into a list during the application's startup:: # AttributeError plugins = config.get("ckan.plugins").split() Depending on the desired backward compatibility, one of the following expressions can be used instead:: # if both v2.9 and v2.10 are supported plugins = tk.aslist(config.get("ckan.plugins")) # if only v2.10 is supported plugins = config.get("ckan.plugins") The second major change affects default values for configuration options. Starting from CKAN 2.10, the majority of the config options have a declared default value. It means that whenever you invoke ``config.get`` method, the *declared default* value is returned instead of ``None``. Example:: # CKAN v2.9 assert config.get("search.facets.limit") is None # CKAN v2.10 assert config.get("search.facets.limit") == 10 The second argument to ``config.get`` should be only used to get the value of a missing *undeclared* option:: assert config.get("not.declared.and.missing.from.config", 1) == 1 The above is the same for any extension that *declares* its config options using ``IConfigDeclaration`` interface or ``config_declarations`` blanket. (`#6467 `_) - Public registration of users has been disabled by default (`#7210 `_) - User and group/org image upload formats have been restricted by default (`#7210 `_) - The activities feature has been extracted into a separate ``activity`` plugin. To keep showing the activities in the UI and enable the activity related API actions you need to add the ``activity`` plugin to the :ref:`ckan.plugins` config option. This change doesn't affect activities already stored in the DB. They are still available once the plugin is enabled. Note that some imports have changed (`#6790 `_):: `ckan.model.Activity` -> `ckanext.activity.model.Activity` - Users of the Xloader or DataPusher need to provide a valid API Token in their configurations using the ``ckanext.xloader.api_token`` or ``ckan.datapusher.api_token`` keys respectively. (`#7139 `_) - Only user-defined functions can be used as validators. An attempt to use a mock-object, built-in function or class will cause a ``TypeError``. (`#6048 `_) - The language code for the Norwegian language has been updated from ``no`` to ``nb_NO``. There are redirects in place from the old code to the new one for localized URLs, but please update your links. If you were using the old ``no`` code in a config option like ``ckan.default_locale`` or ``ckan.locales_offered`` you will need to update the value to ``nb_NO``. (`#6746 `_) - `toolkit.aslist` now converts any iterable other than ``list`` and `tuple` into a ``list``: ``list(value)``. Before, such values were just wrapped into a list, i.e: ``[value]`` (`#7257 `_). .. list-table:: Short overview of changes :widths: 40 30 30 :header-rows: 1 * - Expression - Before - After * - ``aslist([1,2])`` - ``[1, 2]`` - ``[1, 2]`` * - ``aslist({1,2})`` - ``[{1, 2}]`` - ``[1, 2]`` * - ``aslist({1: "one", 2: "two"})`` - ``[{1: "one", 2: "two"}]`` - ``[1, 2]`` * - ``aslist(range(1,3))`` - ``[range(1, 3)]`` - ``[1, 2]`` Removals and deprecations ------------------------- - Legacy API keys are no longer supported for Authentication and have been removed from the UI. API Tokens should be used instead. See :ref:`api authentication` for more details (`#6247 `_) - ``build_nav_main()``, ``build_nav_icon()`` and ``build_nav()`` helpers no longer support Pylons route syntax. eg use ``dataset.search`` instead of ``controller=dataset, action=search``. (`#6263 `_) - The following old helper functions have been removed and are no longer available: ``submit()``, ``radio()``, ``icon_url()``, ``icon_html()``, ``icon()``, ``resource_icon()``, ``format_icon()``, ``button_attr()``, ``activity_div()`` (`#6272 `_) - The following methods are deprecated and should be replaced with their respective new versions in the plugin interfaces: - `ckan.plugins.interfaces.IResourceController`: - change ``before_create`` to ``before_resource_create`` - change ``after_create`` to ``after_resource_create`` - change ``before_update`` to ``before_resource_update`` - change ``after_update`` to ``after_resource_update`` - change ``before_delete`` to ``before_resource_delete`` - change ``after_delete`` to ``after_resource_delete`` - change ``before_show`` to ``before_resource_show`` - `ckan.plugins.interfaces.IPackageController`: - change ``after_create`` to ``after_dataset_create`` - change ``after_update`` to ``after_dataset_update`` - change ``after_delete`` to ``after_dataset_delete`` - change ``after_show`` to ``after_dataset_show`` - change ``before_search`` to ``before_dataset_search`` - change ``after_search`` to ``after_dataset_search`` - change ``before_index`` to ``before_dataset_index`` | (`#6501 `_) - The ``ckan seed`` command has been removed in favour of ``ckan generate fake-data`` for generating test entities in the database. Refer to ``ckan generate fake-data --help`` for some usage examples. (`#6504 `_) - The ``IRoutes`` interface has been removed since it was part of the old Pylons architecture. (`#6594 `_) - Remove ``ckan.cache_validated_datasets`` config (`#6628 `_) - Remove ``ckan.search.automatic_indexing`` config (`#6639 `_) - The ``PluginMapperExtension`` has been removed since it was no longer used in core and it had a deprecated dependency. (`#6648 `_) - Remove deprecated ``fields`` parameter in ``resource_search`` method. (`#6687 `_) - The ``ISession`` interface has been removed from CKAN. To extend SQLAlchemy use event listeners instead. (`#6699 `_) - ``unselected_facet_items`` helper has been removed. You can use ``get_facet_items_dict`` with ``exclude_active=True`` instead. (`#6765 `_) - The Recline-based view plugins (``recline_view``, ``recline_grid_view``, ``recline_graph_view`` and ``recline_map_view``) are deprecated and will be removed in future versions. Check :doc:`maintaining/data-viewer` for alternatives. (`#7078 `_) - The requirement-setuptools.txt file has been removed (`#7271 `_) - ``ckan.route_after_login`` renamed to ``ckan.auth.route_after_login`` (`#7350 `_) v.2.9.11 2024-03-13 =================== Minor changes ------------- - Define allowed alternative Solr query parsers via the :ref:`ckan.search.solr_allowed_query_parsers` config option (`#8053 `_). Note that the 2.9 version of this patch does not use pyparsing to parse the local parameters string, so some limitations are in place, mainly that no quotes are allowed in the local parameters definition. - Get default formats for DataStore views from config (`#8095 `_) Bugfixes -------- - `CVE-2024-27097 `_: fixed potential log injection in reset user endpoint. - Fixed Octet Streaming for Datastore Dump requests. (`#7899 `_) - Fix Password Reset Keys with multiple accounts (`#8079 `_) - Detect XLSX mimetypes correctly in uploader (`#8088 `_) v.2.9.10 2023-12-13 =================== Bugfixes -------- - `CVE-2023-50248 `_: fix potential out of memory error when submitting the dataset form with a specially-crafted field. - Update resource datastore_active with a single statement (`#7833 `_) - Fix downloading datastore resources as json with null values in json columns (`#7545 `_) - Fix errors when running the `ckan db upgrade` command (`#7681 `_) - Fix ``deprecated`` decorator (`#7939 `_) - Changed dataset query to check for ``+state:`` in the ``fq_list`` as well as the `fq` parameter before forcing ``state:active`` (`#7905 `_) v.2.9.9 2023-05-24 ================== Bugfixes -------- - `CVE-2023-32321 `_: fix potential path traversal, remote code execution, information disclosure and DOS vulnerabilities via crafted resource ids. - Names are now quoted in From and To addresses in emails, meaning that site titles with commas no longer break email clients. (`#7508 `_) Migration notes --------------- - The default storage backend for the session data used by the Beaker library uses the Python ``pickle`` module, which is considered unsafe. While there is no direct known vulnerability using this vector, a safer alternative is to store the session data in the `client-side cookie `_. This will probably be the default behaviour in future CKAN versions:: # ckan.ini beaker.session.type = cookie beaker.session.data_serializer = json beaker.session.validate_key = CHANGE_ME beaker.session.httponly = True beaker.session.secure = True beaker.session.samesite = Lax # or Strict, depending on your setup v.2.9.8 2023-02-15 ================== Major changes ------------- - Disable public registration of users by default (`#7210 `_) - Restrict user and group/org image upload formats by default (`#7210 `_) Minor changes ------------- - Add dev containers / GitHub Codespaces config for CKAN 2.9 (See the `documentation `_ - Add new group command: ``clean``. Add ``clean users`` command to delete users containing images with formats not supported in ``ckan.upload.user.mimetypes`` config option (`#7241 `_) - Set the ``resource`` blueprint to not auto register. (`#7374 `_) - ``prepare_dataset_blueprint``: support dataset type (`#7031 `_) - Add ``--quiet`` option to ``ckan user token add`` command to mak easier to integrate with automated scripts (`#7217 `_) Bugfixes -------- - Fix ``package_update`` performance (`#7219 `_) - Fix ``_()`` function override (`#7232 `_) - Fix 404 when selecting the same date in the changes view (`#7192 `_) - Enable DateTime to be returned through Actions, allowing ``datapusher_status`` to be accessed through the API. (`#7110 `_) - Fixed broken organization delete form (`#7150 `_) v.2.9.7 2022-10-26 ================== Bugfixes -------- * CVE-2022-43685: fix potential user account takeover via user create * Fix Datatables view download format selector (`#7147 `_) * Revert deletions included in 2.9.6 as part of #6187 (`#7118 `_) v.2.9.6 2022-09-28 ================== Note: This release includes requirements upgrades to address security issues Bugfixes -------- - Fixes incorrectly encoded url current_url (`#6685 `_) - Check if locale exists on i18n JS API (`#6698 `_) - Add ``csrf_input()`` helper for cross-CKAN version compatibility (`#7016 `_) - Fix not empty validator (`#6658 `_) - Use ``get_action()`` in patch actions to allow custom logic (`#6519 `_) - Allow to extend organization_facets (`#6682 `_) - Expose check_ckan_version to templates (`#6741 `_) - Allow get_translated helper to fall back to base version of a language (`#6815 `_) - Fix server error in tag autocomplete when vocabulary does not exist (`#6820 `_) - Check if locale exists on i18n JS API (`#6698 `_) - Fix updating a non-existing resource causes an internal sever error (`#6928 `_) - Remove extra comma (`#6774 `_) - Fix test data creation issues (`#6805 `_) - Fix for updating non-existing resource - Avoid storing the session on each request (`#6954 `_) - Return zero results instead of raising NotFound when vocabulary does not exist - Fix the datapusher trigger in case of resource_update via API (`#5727 `_) - Consistent CLI behavior when when no command provided and when using `--help` options (`#6120 `_) - Fix regression when validating resource subfields (`#6546 `_) - Fix resource file size not updating with resource_patch (`#7075 `_) - Prevent non-sysadmin users to change their own state (`#6956 `_) - Use user id in auth cookie rather than name - Reorder resource view button: allow translation (`#6089 `_) - Optimize temp dir creation on uploads (`#6578 `_) - Exclude site_user from user_listi (`#6618 `_) - Fix race condition in creating the default site user (`#6638 `_) - gettext not for metadata fields (`#6660 `_) - Include root_path in activity email notifications (`#6743 `_) - Extract translations from emails (`#5857 `_) - Use the headers Reply-to value if its set in the extensions (`#6838 `_) - Improve error when downloading resource (`#6832 `_) - ``ckan_config`` test mark works with request context (`#6868 `_) - Fix caching logic on logged in users (`#6864 `_) - Fix member delete (`#6892 `_) - Concurrent-safe resource updates (`#6439 `_) - Fix error when listing tokens in the CLI in py2 (`#6789 `_) Minor changes ------------- - The ``ckan.main_css`` and ``ckan.i18.rtl_css`` settings, which were not working, have been replaced by :ref:`ckan.theme` and :ref:`ckan.i18n.rtl_theme` respectively. Both expect the name of an *asset* with a base theme for the application (`#6817 `_) - The type of uploads for group and user image can be restricted via the `ckan.upload.{object_type}.types` and `ckan.upload.{object_type}.mimetypes` config options (eg :ref:`ckan.upload.group.types`, :ref:`ckan.upload.user.mimetypes`) (`#6477 `_) - Allow to use PDB and IDE debuggers (`#6798 `_) - Unpin pytz, upgrade zope.interface (`#6665 `_) - Update sqlparse version - Bump markdown requirement to support Python 3.9 - Update psycopg2 to support PostgreSQL 12 - Add auth functions for 17 actions that didn't have them before (`#7045 `_) - Add no-op ``csrf_input()`` helper to help extensions with cross-CKAN version support (`#7030 `_) v.2.9.5 2022-01-19 ================== Major features -------------- - Solr 8 support. Starting from version 2.9.5, CKAN supports Solr versions 6 and 8. Support for Solr 6 will be dropped in the next CKAN minor version (2.10). Note that if you want to use Solr 8 you need to use the ``ckan/config/solr/schema.solr8.xml`` file, or alternatively you can use the ``ckan/ckan-solr:2.9-solr8`` Docker image which comes pre-configured. (`#6530 `_) Bugfixes -------- - Consistent CLI behavior when no command is provided and when using `--help` (`#6120 `_) - Fix regression when validating resource subfields (`#6546 `_) - Fix user create/edit email validators (`#6399 `_) - Error opening JS translations on Python 2 (`#6531 `_) - Set logging level to error in error mail handler (`#6577 `_) - Add RootPathMiddleware to flask stack to support non-root installs running on python 3 (`#6556 `_) - Use correct auth function when editing organizations (`#6622 `_) - Fix invite user with existing email error (`#5880 `_) - Accept empty string in one of validator (`#6612 `_) Minor changes ------------- - Add timeouts to requests calls (see `ckan.requests.timeout`) (`#6408 `_) - Types of file uploads for group and user imags can be restricted via the `ckan.upload.{object_type}.types` and `ckan.upload.{object_type}.mimetypes` config options (eg :ref:`ckan.upload.group.types`, :ref:`ckan.upload.user.mimetypes`) (`#6477 `_) - Allow children elements on select2 lists (`#6503 `_) - Enable ``minimumInputLength`` and fix loading message in select2 (`#6554 `_) v.2.9.4 2021-09-22 ================== Note: This release includes requirements upgrades to address security issues Bugfixes -------- - Don't show snippet names in non-debug mode (`#6406 `_) - Show job title on job start/finish log messages (`#6387 `_) - Fix unprivileged users being able to access bulk process (`#6290 `_) - Allow UTF-8 in JS translations (`#6051 `_) - Handle Traceback Exception for HTTP and HTTP status Code in logging (`#6340 `_) - Fix object list validation output (`#6149 `_) - Coerce query string keys/values before passing to quote() (`#6099 `_) - Fix datetime formatting when listing user tokens on py2. (`#6319 `_) - Fix Solr HTTP basic auth cred handling (`#6286 `_) - Remove not accessed user object in resource_update (`#6220 `_) - Fix for g.__timer (`#6207 `_) - Fix guard clause on has_more_facets, #6190 (`#6190 `_) - Fix page render errors when search facets are not defined (`#6181 `_) - Fix exception when using solr_user and solr_password on Py3 (`#6179 `_) - Fix pagination links for custom org types (`#6162 `_) - Fixture for plugin DB migrations (`#6139 `_) - Render activity timestamps with title= attribute (`#6109 `_) - Fix db init error in alembic (`#5998 `_) - Fix user email validator when using name as id parameter (`#6113 `_) - Fix DataPusher error during resource_update (`#5597 `_) - render_datetime helper does not respect ckan.display_timezone configuration (`#6252 `_) - Fix SQLAlchemy configuration for DataStore (`#6087 `_) - Don't cache license translations across requests (`#5586 `_) - Fix tracking.js module preventing links to be opened in new tabs (`#6386 `_) - Fix deleted org/group feeds (`#6368 `_) - Fix runaway preview height (`#6284 `_) - Stable default ordering when consuming resource content from datastore (`#2317 `_) - Several documentation fixes and improvements v.2.9.3 2021-05-19 ================== Bugfixes -------- - Fix Chinese locales. Note that the URLs for the `zh_CN` and `zh_TW` locales have changed but there are redirects in place, eg http://localhost:5000/zh_CN/dataset -> http://localhost:5000/zh_Hans_CN/dataset (`#6008 `_) - Fix performance bottleneck in activity queries (`#6028 `_) - Keep repeatable facets inside pagination links (`#6084 `_) - Ensure order of plugins in PluginImplementations (`#5965 `_) - Fix for Datastore file dump extension (`#5593 `_) - Allow package activity migration on py3 (`#5930 `_) - Fix TemplateSyntaxError in snippets/changes/license.html (`#5972 `_) - Remove hardcoded logging level (`#5941 `_) - Include extra files into ckanext distribution (`#5995 `_) - Fix db init in docker as the directory is not empty (`#6027 `_) - Fix sqlalchemy configuration, add doc (`#5932 `_) - Fix issue with purging custom entity types (`#5859 `_) - Only load view filters on templates that need them - Sanitize user image url - Allow installation of requirements without any additional actions using pip (`#5408 `_) - Include requirements files in Manifest (`#5726 `_) - Dockerfile: pin pip version (`#5929 `_) - Allow uploaders to only override asset / resource uploading (`#6088 `_) - Catch TypeError from invalid thrown by dateutils (`#6085 `_) - Display proper message when sysadmin password is incorrect (`#5911 `_) - Use external library to parse view filter params - Fix auth error when deleting a group/org (`#6006 `_) - Fix datastore_search language parameter (`#5974 `_) - make SQL function whitelist case-insensitive unless quoted (`#5969 `_) - Fix Explore button not working (`#3720 `_) - remove unused var in task_status_update (`#5861 `_) - Prevent guessing format and mimetype from resource urls without path (`#5852 `_) - Multiple documentation improvements Minor changes ------------- - Support for setting host and port on the ini file (`#5939 `_) - Allow to set path to INI file in the WSGI script (`#5987 `_) - Allow multi-level config inheritance (`#6000 `_) v.2.9.2 2021-02-10 ================== General notes: * Note: To use PostgreSQL 12 on CKAN 2.9 you need to upgrade psycopg2 to at least 2.8.4 (more details in `#5796 `_) Major features -------------- - Add CLI commands for API Token management (`#5868 `_) Bugfixes -------- - Persist attributes in chained functions (`#5751 `_) - Fix install documentation (`#5618 `_) - Fix exception when passing limit to organization (`#5789 `_) - Fix for adding directories from plugins if partially string matches existing values (`#5836 `_) - Fix upload log activity sorting (`#5827 `_) - Textview: escape text formats (`#5814 `_) - Add allow_partial_update to fix losing users (`#5734 `_) - Set default group_type to group in group_create (`#5693 `_) - Use user performing the action on activity context on user_update (`#5743 `_) - New block in nav links in user dashboard (`#5804 `_) - Update references to DataPusher documentation - Fix JavaScript error on Edge (`#5782 `_) - Fix error when deleting resource with missing datastore table (`#5757 `_) - ensure HTTP_HOST is bytes under python2 (`#5714 `_) - Don't set old_filename when updating groups (`#5707 `_) - Filter activities from user at the database level (`#5698 `_) - Fix user_list ordering (`#5667 `_) - Allowlist for functions in datastore_search_sql (see :ref:`ckan.datastore.sqlsearch.allowed_functions_file`) - Fix docker install (`#5381 `_) - Fix Click requirement conflict (`#5539 `_) - Return content-type header on downloads if mimetype is (`#5670 `_) - Fix missing activities from UI when internal processes are run by ignored users (`#5699 `_) - Replace 'paster' occurrences with 'ckan' in docs (`#5700 `_) - Include requirements files in Manifest (`#5726 `_) - Fix order which plugins are returned by PluginImplementations changing (`#5731 `_) - Raise NotFound when creating a non-existing collaborator (`#5759 `_) - Restore member edit page (`#5767 `_) - Don't add --ckan-ini pytest option if already added (by pytest-ckan) (`#5774 `_) - Update organization_show package limit docs (`#5784 `_) - Solve encoding errors in changes templates (`#5785 `_) Minor changes ------------- - Add aria attribute and accessible screen reader text to the mobile nav button. (`#5555 `_) - Remove jinja2 blocks from robots.txt (`#5648 `_) - Allow to run the development server using SSL (`#5825 `_) - Update extension template, migrate tests to GitHub Actions (`#5797 `_) v.2.9.1 2020-10-21 ================== General notes: * Note: This version requires a database upgrade with ``ckan db upgrade`` (You should always backup your database first) Bugfixes -------- - Restore `stats` extension with reduced functionality (`#5215 `_) - Allow IAuthenticator methods to return responses (`#5259 `_) - Emit activities when updating datasets in bulk (`#5479 `_) - Catch IndexError from date parsing during dataset indexation (`#5535 `_) - Remove foreign keys relationships in revision tables to avoid purge errors (`#5542 `_) - Fix fullscreen for resource webpageview (`#5552 `_) - Fix skip to content link hiding on screen readers (`#5556 `_) - Fix KeyErrors in change list detection (`#5562 `_) - Fix instantiation of smtp on python 3.8 (`#5595 `_) - Fix `unflatten` function and DataDictionary/package extras update bug (`#5611 `_) - Fix managing resources by collaborators (`#5620 `_) - package_revise: allow use by normal users (`#5637 `_) - Fix reloader option on ckan run command (`#5639 `_) - Allow config-tool to be used with an incomplete config file (`#5647 `_) Minor changes ------------- - Add aria attribute and accessible screen reader text to the mobile nav button. (`#5555 `_) - Remove jinja2 blocks from robots.txt (`#5648 `_) v.2.9.0 2020-08-05 ================== .. _migration-notes-2.9: Migration notes --------------- - This version does require a requirements upgrade on source installations - This version does require a database upgrade - This version does not require a Solr schema upgrade if you are already using the 2.8 schema, but it is recommended to upgrade to the 2.9 Solr schema. - This version requires changes to the ``who.ini`` configuration file. If your setup doesn't use the one bundled with this repo, you will have to manually change the following lines:: use = ckan.lib.auth_tkt:make_plugin to:: use = ckan.lib.repoze_plugins.auth_tkt:make_plugin And also:: use = repoze.who.plugins.friendlyform:FriendlyFormPlugin to:: use = ckan.lib.repoze_plugins.friendly_form:FriendlyFormPlugin Otherwise, if you are using symbolinc link to ``who.ini`` under vcs, no changes required. (`#4796 `_) - All the static CSS/JS files must be bundled via a `webassets.yml` file, as opposed to the previously used, optional `resource.config` file. Check the `Assets documentation `_ for more details. (`#4614 `_) - When ``ckan.cache_enabled`` is set to ``False`` (default) all requests include the ``Cache-control: private`` header. If ``ckan.cache_enabled`` is set to ``True``, when the user is not logged in and there is no session data, a ``Cache-Control: public`` header will be added. For all other requests the ``Cache-control: private`` header will be added. Note that you will also need to set the ``ckan.cache_expires`` config option to allow caching of requests. (`#4781 `_) - A full history of dataset changes is now displayed in the Activity Stream to admins, and optionally to the public. By default this is enabled for new installs, but disabled for sites which upgrade (just in case the history is sensitive). When upgrading, open data CKANs are encouraged to make this history open to the public, by setting this in production.ini: ``ckan.auth.public_activity_stream_detail = true`` (`#3972 `_) - When upgrading from previous CKAN versions, the Activity Stream needs a migrate_package_activity.py running for displaying the history of dataset changes. This can be performed while CKAN is running or stopped (whereas the standard `paster db upgrade` migrations need CKAN to be stopped). Ideally it is run before CKAN is upgraded, but it can be run afterwards. If running previous versions or this version of CKAN, download and run migrate_package_activity.py like this:: cd /usr/lib/ckan/default/src/ckan/ wget https://raw.githubusercontent.com/ckan/ckan/2.9/ckan/migration/migrate_package_activity.py wget https://raw.githubusercontent.com/ckan/ckan/2.9/ckan/migration/revision_legacy_code.py python migrate_package_activity.py -c /etc/ckan/production.ini Future versions of CKAN are likely to need a slightly different procedure. Full info about this migration is found here: https://github.com/ckan/ckan/wiki/Migrate-package-activity (`#4784 `_) - The :ref:`config_file` default name has been changed to ``ckan.ini`` across the documentation regardless of the environment. You can use any name including the legacy ``development.ini`` and ``production.ini`` but to keep in sync with the documentation is recommended to update the name. - The old `paster` CLI has been removed in favour of the new `ckan` command. In most cases the commands and subcommands syntax is the same, but the ``-c`` or ``--config`` parameter to point to the ini file needs to provided immediately after the `ckan` command, eg:: ckan -c /etc/ckan/default/ckan.ini sysadmin - The minimum PostgreSQL version required starting from this version is 9.5 (`#5458 `_) Major features -------------- - Python 3 support. CKAN nows supports Python 3.6, 3.7 and 3.8 (`Overview `_). Check `this page `_ for support on how to migrate existing extensions to Python 3. - Dataset collaborators: In addition to traditional organization-based permissions, CKAN instances can also enable the dataset collaborators feature, which allows dataset-level authorization. This provides more granular control over who can access and modify datasets that belong to an organization, or allows authorization setups not based on organizations. It works by allowing users with appropriate permissions to give permissions to other users over individual datasets, regardless of what organization they belong to. To learn more about how to enable it and the different configuration options available, check the documentation on :ref:`dataset_collaborators`. (`#5346 `_) - API Tokens: an alternative to API keys. Tokens can be created and removed on demand (check :ref:`api authentication`) and there is no restriction on the maximum number of tokens per user. Consider using tokens instead of API keys and create a separate token for each use-case instead of sharing the same token between multiple clients. By default API Tokens are JWT, but alternative formats can be implemented using `ckan.plugins.interfaces.IApiToken` interface. (`#5146 `_) - Safe dataset updates with ``package_revise``: This is a new API action for safe concurrent changes to datasets and resources. ``package_revise`` allows assertions about current package metadata, selective update and removal of fields at any level, and multiple file uploads in a single call. See the documentation at :py:func:`~ckan.logic.action.update.package_revise` (`#4618 `_) - Refactor frontend assets management to use `webassets `_, including support for :ref:`x-sendfile` (`#4614 `_) - Users can now upload or link to custom profile pictures. By default, if a user picture is not provided it will fall back to gravatar. Alternatively, gravatar can be completely disabled by setting ``ckan.gravatar_default = disabled``. In that case a placeholder image is shown instead, which can be customized by overriding the ``templates/user/snippets/placeholder.html`` template. (`#5272 `_) - Add `plugin_extras` field allowing extending User object for internal use (`#5382 `_) Minor changes ------------- - New command for running database migrations from extensions. See :ref:`extensions db migrations` for details, (`#5150 `_) - For navl schemas, the 'default' validator no longer applies the default when the value is False, 0, [] or {} (`#4448 `_) - Use alembic instead of sqlalchemy-migrate for managing database migrations (`#4450 `_) - If you've customized the schema for package_search, you'll need to add to it the limiting of ``row``, as per default_package_search_schema now does. (`#4484 `_) - Several logic functions now have new upper limits to how many items can be returned, notably ``group_list``, ``organization_list`` when ``all_fields=true``, ``datastore_search`` and ``datastore_search_sql``. These are all configurable. (`#4562 `_) - Give users the option to define which page they want to be redirected to after logging in via `ckan.route_after_login` config variable. (`#4770 `_) - Add cache control headers to flask (`#4781 `_) - Create recline_view on ods files by default (`#4936 `_) - Replace nosetests with pytest (`#4996 `_) - Make creating new tags in autocomplete module optional (`#5012 `_) - Allow reply to emails (`#5024 `_) - Improve and reorder resource_formats.json (`#5034 `_) - Email unique validator (`#5100 `_) - Preview for multimedia files (`#5103 `_) - Allow extensions to define Click commands (`#5112 `_) - Add organization and group purge (`#5127 `_) - HTML emails (`#5132 `_) - Unified workflow for creating/applying DB migrations from extensions (`#5150 `_) - Use current package_type for urls (`#5189 `_) - Werkzeug dev server improvements (`#5195 `_) - Allow passing arguments to the RQ enqueue_call function (`#5208 `_) - Add option to configure labels of next/prev page button and pager format. (`#5223 `_) - DevServer: threaded mode and extra files (`#5303 `_) - Make default sorting configurable (`#5314 `_) - Allow initial values in group form (`#5345 `_) - Make ckan more accessible (`#5360 `_) - Update date formatters (`#5376 `_) - Allow multiple `ext_*` params in search views (`#5398 `_) - Always 404 on non-existing user lookup (`#5464 `_) Bugfixes -------- - 500 error when calling `resource_search` by `last_modified` (`#4130 `_) - Action function "datastore_search" would calculate the total, even if you set ``include_total=False``. (`#4448 `_) - Emails not sent from flask routes (`#4711 `_) - Admin of organization can add himself as a member/editor to the organization and lose admin rights (`#4821 `_) - Error when posting empty array with type json using datastore_create (`#4826 `_) - ValueError when you configure exception emails (`#4831 `_) - Dataset counts incorrect on Groups listing (`#4987 `_) - Fix broken layout in organization bulk_process (`#5147 `_) - Index template with template path instead of numeric index (`#5172 `_) - Add metadata_modified field to resource (`#5236 `_) - Send the right URL of CKAN to datapusher (`#5281 `_) - Multiline translation strings not translated (`#5339 `_) - Allow repeaded params in h.add_url_param (`#5373 `_) - Accept timestamps with seconds having less than 6 decimals (`#5417 `_) - RTL css fixes (`#5420 `_) - Prevent account presence exposure when `ckan.auth.public_user_details = false` (`#5432 `_) - `ckan.i18n_directory` config option ignored in Flask app. (`#5436 `_) - Allow lists in resource extras (`#5453 `_) Removals and deprecations ------------------------- - Revision and History UI is removed: `/revision/*` & `/dataset/{id}/history` in favour of `/dataset/changes/` visible in the Activity Stream. ``model.ActivityDetail`` is no longer used and will be removed in the next CKAN release. (`#3972 `_) - ``c.action`` and ``c.controller`` variables should be avoided. ``ckan.plugins.toolkit.get_endpoint`` can be used instead. This function returns tuple of two items(depending on request handler): 1. Flask blueprint name / Pylons controller name 2. Flask view name / Pylons action name In some cases, Flask blueprints have names that are differs from their Pylons equivalents. For example, 'package' controller is divided between 'dataset' and 'resource' blueprints. For such cases you may need to perform additional check of returned value: >>> if toolkit.get_endpoint()[0] in ['dataset', 'package']: >>> do_something() In this code snippet, will be called if current request is handled via Flask's dataset blueprint in CKAN>=2.9, and, in the same time, it's still working for Pylons package controller in CKAN<2.9 (`#4319 `_) - The following logic functions have been removed (`#4627 `_): * ``dashboard_activity_list_html`` * ``organization_activity_list_html`` * ``user_activity_list_html`` * ``package_activity_list_html`` * ``group_activity_list_html`` * ``organization_activity_list_html`` * ``recently_changed_packages_activity_list_html`` * ``dashboard_activity_list_html`` * ``activity_detail_list`` - Remove Bootstrap 2 templates (`#4779 `_) - Extensions that add CLI commands should note the deprecation of ``ckan.lib.cli.CkanCommand`` and all other helpers in ckan.lib.cli. Extensions should instead implement CLIs using the new IClick interface. (`#5112 `_) - Remove paster CLI (`#5264 `_) v.2.8.12 2022-10-26 =================== Bugfixes -------- * CVE-2022-43685: fix potential user account takeover via user create v.2.8.11 2022-09-28 =================== Fixes: * Fixes incorrectly encoded url current_url (`#6685 `_) * Check if locale exists on i18n JS API (`#6698 `_) * Add ``csrf_input()`` helper for cross-CKAN version compatibility (`#7016 `_) * Fix not empty validator (`#6658 `_) * Use ``get_action()`` in patch actions to allow custom logic (`#6519 `_) * Allow to extend organization_facets (`#6682 `_) * Expose check_ckan_version to templates (`#6741 `_) * Allow get_translated helper to fall back to base version of a language (`#6815 `_) * Fix server error in tag autocomplete when vocabulary does not exist (`#6820 `_) * Check if locale exists on i18n JS API (`#6698 `_) * Fix updating a non-existing resource causes an internal sever error (`#6928 `_) v.2.8.10 2022-01-19 =================== Fixes: * Add timeouts to requests calls (see `ckan.requests.timeout`) (`#6408 `_) * Fix user create/edit email validators (`#6399 `_) * Allow children elements on select2 lists (`#6503 `_) v.2.8.9 2021-09-22 ================== Fixes: * render_datetime helper does not respect ckan.display_timezone configuration (`#6252 `_) * Fix SQLAlchemy configuration for DataStore (`#6087 `_) * Don't cache license translations across requests (`#5586 `_) * Fix tracking.js module preventing links to be opened in new tabs (`#6386 `_) * Fix deleted org/group feeds (`#6368 `_) * Fix runaway preview height (`#6284 `_) * Fix unreliable ordering of DataStore results (`#2317 `_) v.2.8.8 2021-05-19 ================== * Fix Chinese locales (`#4413 `_) * Allow installation of requirements without any additional actions using pip (`#5408 `_) * Include requirements files in Manifest (`#5726 `_) * Dockerfile: pin pip version (`#5929 `_) * Allow uploaders to only override asset / resource uploading (`#6088 `_) * Catch TypeError from invalid thrown by dateutils (`#6085 `_) * Display proper message when sysadmin password is incorrect (`#5911 `_) * Use external library to parse view filter params * Fix auth error when deleting a group/org (`#6006 `_) * Fix datastore_search language parameter (`#5974 `_) * make SQL function whitelist case-insensitive unless quoted (`#5969 `_) * Fix Explore button not working (`#3720 `_) * remove unused var in task_status_update (`#5861 `_) * Prevent guessing format and mimetype from resource urls without path (`#5852 `_) v.2.8.7 2021-02-10 ================== General notes: * Note: To use PostgreSQL 12 on CKAN 2.8 you need to upgrade SQLAlchemy to 1.2.17 and vdm to 0.15 (more details in `#5796 `_) Fixes: * Persist attributes in chained functions (`#5751 `_) * Fix install documentation (`#5618 `_) * Fix exception when passing limit to organization (`#5789 `_) * Fix for adding directories from plugins if partially string matches existing values (`#5836 `_) * Fix upload log activity sorting (`#5827 `_) * Textview: escape text formats (`#5814 `_) * Add allow_partial_update to fix losing users (`#5734 `_) * Set default group_type to group in group_create (`#5693 `_) * Use user performing the action on activity context on user_update (`#5743 `_) * New block in nav links in user dashboard (`#5804 `_) * Update references to DataPusher documentation * Fix JavaScript error on Edge (`#5782 `_) * Fix error when deleting resource with missing datastore table (`#5757 `_) * ensure HTTP_HOST is bytes under python2 (`#5714 `_) * Don't set old_filename when updating groups (`#5707 `_) * Filter activities from user at the database level (`#5698 `_) * Fix user_list ordering (`#5667 `_) * Allowlist for functions in datastore_search_sql (see :ref:`ckan.datastore.sqlsearch.allowed_functions_file`) v.2.8.6 2020-10-21 ================== Fixes: * Allow IAuthenticator methods to return responses (`#5259 `_) * Fix skip to content link hiding on screen readers (`#5556 `_) * Fix unflattening of dataset extras (`#5602 `_) * Fix minified JS files in 2.7 (`#5557 `_) * Send the right URL of CKAN to datapusher (`#5281 `_) * Fix fullscreen for resource webpageview (`#5552 `_) * PackageSearchIndex.index_package(): catch IndexError from date parsing (`#5535 `_) * Fix collapsible menu in mobile view (`#5448 `_) * Refactor query string parsing module v.2.8.5 2020-08-05 ================== Fixes: * Add RTL support (`#5413 `_) * Fix UnicodeDecodeError on abort function (`#4829 `_) * Improve and reorder resource_formats.json (`#5034 `_) * Allow passing arguments to the RQ enqueue_call function (`#5208 `_) * Fix dashboard follower filter (`#5412 `_) * Update dictionary.html for bs2 version (`#5365 `_) * Prevent password reset exposing account presence (`#5431 `_) * Add class dropdown to 'New view' menu (`#5470 `_) * Update jQuery to 3.5.0 (`#5364 `_) * Fix dashboard activity filter (`#5424 `_) * Prevent account presence exposure when ckan.auth.public_user_details = false (`#5432 `_) * Fix resource upload filename fetching in IE (`#5438 `_) * Unflatten: allow nesting >1 level (`#5444 `_) * Allow lists in resource extras (`#5453 `_) * Only add error to tag_errors if not empty (`#5454 `_) * Fix order_by param in user_list action (`#5342 `_) * Fix for Resources validation errors display (`#5335 `_) v.2.8.4 2020-04-15 ================== General notes: * Note: This version does not requires a requirements upgrade on source installations * Note: This version does not requires a database upgrade * Note: This version does not require a Solr schema upgrade * Note: This version includes changes in the way the ``SameSite`` flag is set on the ``auth_tkt`` authorization cookie. The new default setting for it is ``SameSite=Lax``, which aligns with the behaviour of all major browsers. If for some reason you need a different value, you can set it via the `who.samesite` configuration option. You can find more information on the ``SameSite`` attribute `here `_. Fixes: * Fix for number of datasets displayed on the My organizations tab (`#3580 `_) * Allow chaining of core actions (`#4509 `_) * Password reset request - generally tighten it up (`#4636 `_) * Fix start option in data_dict (`#4920 `_) * Add missing get_action calls in activity actions (`#4967 `_) * Fix datetime comparison in resource_dict_save (`#5033 `_) * Fix wrong _ function reference in user blueprint (`#5046 `_) * Allow vocabulary_id in /api/2/util/tag/autocomplete (`#5071 `_) * Fetch less data for `get_all_entity_ids` (`#5201 `_) * Show error in text view if xhr failed (`#5271 `_) * Fix code injection in autocomplete module (`#5064 `_) * Check for the existence of tracking summary data before attempting to load it (`#5030 `_) * Disable streaming for pylons requests (`#4431 `_) * Filter revisions shown according to dataset permissions * Fix wrong resource URL after ValidationErrors (`#5152 `_) * Update JS vendor libraries * Samesite support in auth cookie (`#5255 `_) * Handle missing resources in case we have a race condition with the DataPusher (`#3980 `_) * Add the g object to toolkit * Use returned facets in group controller (`#2713 `_) * Updated translations * Fix broken translation in image view placeholder (`#5099 `_) v.2.8.3 2019-07-03 ================== General notes: * Note: This version does not requires a requirements upgrade on source installations * Note: This version does not requires a database upgrade * Note: This version does not require a Solr schema upgrade Fixes: * Fix `include_total` in `datastore_search` (`#4446 `_) * Fix problem with reindex-fast (`#4352 `_) * Fix `ValueError` in `url_validator` (`#4629 `_) * Strip local path when uploading file in IE (`#4608 `_) * Increase size of h1 headings to 1.8em (`#4665 `_) * Fix broken div nesting in the `user/read_base.html` (`#4672 `_) * `package_search` parameter `fl` accepts list-like values (`#4464 `_) * Use `chained_auth_function` with core auth functions (`#4491 `_) * Allow translation of custom licenses (`#4594 `_) * Fix delete button links (`#4598 `_) * Fix hardcoded root paths (`#4662 `_) * Fix reCaptcha (`#4732 `_) * Fix incremented follower-counter (`#4767 `_) * Fix breadcrumb on /datasets (`#4405 `_) * Fix `root_path` when using mod_wsgi (`#4452 `_) * Correctly insert root_path for urls generated with _external flag (`#4722 `_) * Make reorder resources button translatable (`#4838 `_) * Fix `feeds` urls generation (`#4854 `_) * More robust auth functions for `resource_view_show` (`#4827 `_) * Allow to customize the DataProxy URL (`#4874 `_) * Allow custom CKAN callback URL for the DataPusher (`#4878 `_) * Add `psycopg>=2.8` support (`#4841 `_) v.2.8.2 2018-12-12 ================== General notes: * This version requires a requirements upgrade on source installations * Note: This version does not requires a database upgrade * Note: This version does not require a Solr schema upgrade Fixes: * Strip full URL on uploaded resources before saving to DB (`#4382 `_) * Fix user not being defined in check_access function (`#4574 `_) * Remove html5 shim from stats extension (`#4236 `_) * Fix for datastore_search distinct=true option (`#4236 `_) * Fix edit slug button (`#4379 `_) * Don't re-register plugin helpers on flask_app (`#4414 `_) * Fix for Resource View Re-order (`#4416 `_) * autocomplete.js: fix handling of comma key codes (`#4421 `_) * Flask patch update (`#4426 `_) * Allow plugins to define multiple blueprints (`#4495 `_) * Fix i18n API encoding (`#4505 `_) * Allow to defined legacy route mappings as a dict in config (`#4521 `_) * group_patch does not reset packages (`#4557 `_) v.2.8.1 2018-07-25 ================== General notes: * Note: This version does not requires a requirements upgrade on source installations * Note: This version does not requires a database upgrade * Note: This version does not require a Solr schema upgrade Fixes: * "Add Filter" Performance Issue (`#4162 `_) * Error handler update (`#4257 `_) * "New view" button does not work (`#4260 `_) * Upload logo is not working (`#4262 `_) * Unable to pip install ckan (`#4271 `_) * The "License" Icon in 2.8 is wrong (`#4272 `_) * Search - input- border color is overly specific in CSS (`#4273 `_) * Site logo image does not scale down when very large (`#4283 `_) * Validation Error on datastore_search when sorting timestamp fields (`#4288 `_) * Undocumented changes breaking error_document_template (`#4303 `_) * Internal server error when viewing /dashboard when logged out (`#4305 `_) * Missing c.action attribute in 2.8.0 templates (`#4310 `_) * [multilingual] AttributeError: '_Globals' object has no attribute 'fields' (`#4338 `_) * `search` legacy route missing (`#4346 `_) v.2.8.0 2018-05-09 ================== General notes: * This version requires a requirements upgrade on source installations * This version requires a database upgrade * This version requires a Solr schema upgrade * This version requires re-running the ``datastore set-permissions`` command (assuming you are using the DataStore). See: :ref:`datastore-set-permissions` Otherwise new and updated datasets will not be searchable in DataStore and the logs will contain this error:: ProgrammingError: (psycopg2.ProgrammingError) function populate_full_text_trigger() does not exist CKAN developers should also re-run set-permissions on the test database: :ref:`datastore-test-set-permissions` * There are several old features being officially deprecated starting from this version. Check the *Deprecations* section to be prepared. Major changes: * New revamped frontend templates based on Bootstrap 3, see "Changes and deprecations" (#3547) * Allow datastore_search_sql on private datasets (#2562) * New Flask blueprints migrated from old Pylons controllers: user, dashboard, feeds, admin and home (#3927, #3870, #3775, #3762) * Improved support for custom groups and organization types (#4032) * Hide user details to anonymous users (#3915) Minor changes: * Allow chaining of authentication functions (#3679) * Show custom dataset types in search pages (#3807) * Overriding datastore authorization system (#3679) * Standardize on url_for (#3831) * Deprecate notify_after_commit (#3633) * _mail_recipient header override (#3781) * Restrict access to member forms (#3684) * Clean up template rendering code (#3923) * Permission labels are indexed by type text in SOLR (#3863) * CLI commands require a Flask test request context (#3760) * Allow IValidator to override existing validators (#3865) * Shrink datastore_create response size (#3810) * Stable version URLs CKAN for documentation (#4209) * API Documentation update (#4136) * Documentation of Data Dictionary (#3989) * Remove datastore legacy mode (#4041) * Map old Pylons routes to Flask ones (#4066) Bug fixes: * File uploads don't work on new Flask based API (#3869) * {% ckan_extends %} not working on templates served by Flask (#4044) * Problems in background workers with non-core database relations (#3606) * Render_datetime can't handle dates before year 1900 (#2228) * DatapusherPlugin implementation of notify() can call 'datapusher_submit' multiple times (#2334) * Dataset creation page generates incorrect URLs with Chrome autocomplete (#2501) * Search buttons need accessible labels (#2550) * Column name length limit for datastore upload (#2804) * #2373: Do not validate packages or resources from database to views (#3016) * Creation of dataset - different behaviour between Web API & CKAN Interface functionality (#3528) * Redirecting to same page in non-root hosted ckan adds extra root_path to url (#3499) * Beaker 1.8.0 exception when the code is served from OSX via Vagrant (#3512) * Add "Add Dataset" button to user's and group's page (#2794) * Some links in CKAN is not reachable (#2898) * Exception when specifying a directory in the ckan.i18n_directory option (#3539) * Resource view filter user filters JS error (#3590) * Recaptcha v1 will stop working 2018-3-31 (#4061) * "Testing coding standards" page in docs is missing code snippets (#3635) * Followers count not updated immediately on UI (#3639) * Increase jQuery version (#3665) * Search icon on many pages is not properly vertically aligned (#3654) * Datatables view can't be used as a default view (#3669) * Resource URL is not validated on create/update (#3660) * Upload to Datastore tab shows incorrect time at Upload Log (#3588) * Filter results button is not working (#3593) * Broken link in "Upgrading CKAN’s dependencies" doc page (#3637) * Default logo image not properly saved (#3656) * Activity test relies on datetime.now() (#3644) * Info block text for Format field not properly aligned in resource form page (#3663) * Issue upon creating new organization/group through UI form (#3661) * In API docs "package_create" lists "owner_org" as optional (#3647) * Embed modal window not working (#3731) * Frontend build command does not work on master (#3688) * Loading image duplicated (#3716) * Datastore set-up error - logging getting in the way (#3694) * Registering a new account redirects to an unprefixed url (#3834) * Exception in search page when not authorized (#4081) * Datastore full-text-search column is populated by postgres trigger rather than python (#3785) * Datastore dump results are not the same as data in database (#4150) * Adding filter at resource preview doesn't work while site is setup with ckan.root_path param (#4140) * No such file or directory: '/usr/lib/ckan/default/src/ckan/requirement-setuptools.txt' during installation from source (#3641) * Register user form missing required field indicators (#3658) * Datastore full-text-search column is populated by postgres trigger rather than python (#3786) * Add missing major changes to change log (#3799) * Paster/CLI config-tool requires _get_test_app which in turn requires a dev-only dependency (#3806) * Change log doesn't mention necessary Solr scheme upgrade (#3851) * TypeError: expected byte string object, value of type unicode found (#3921) * CKAN's state table clashes with PostGIS generated TIGER state table (#3929) * [Docker] entrypoint initdb.d sql files copied to root (#3939) * DataStore status page throws TypeError - Bleach upgrade regression (#3968) * Source install error with who.ini (#4020) * making a JSONP call to the CKAN API returns the wrong mime type (#4022) * Deleting a resource sets datastore_active=False to all resources and overrides their extras (#4042) * Deleting first Group and Organization custom field is not possible (#4094) Changes and deprecations: * The default templates included in CKAN core have been updated to use Bootstrap 3. Extensions implementing custom themes are encouraged to update their templates, but they can still make CKAN load the old Bootstrap 2 templates during the transition using the following configuration options:: ckan.base_public_folder = public-bs2 ckan.base_templates_folder = templates-bs2 * The API versions 1 and 2 (also known as the REST API), ie ``/api/rest/*`` have been completely removed in favour of the version 3 (action API, ``/api/action/*``). * The old Celery based background jobs have been removed in CKAN 2.8 in favour of the new RQ based jobs (https://docs.ckan.org/en/latest/maintaining/background-tasks.html). Extensions can still of course use Celery but they will need to handle the management themselves. * After introducing dataset blueprint, `h.get_facet_items_dict` takes search_facets as second argument. This change is aimed to reduce usage of global variables in context. For a while, it has default value of None, in which case, `c.search_facets` will be used. But all template designers are strongly advised to specify this argument explicitly, as in future it'll become required. * The ``ckan.recaptcha.version`` config option is now removed, since v2 is the only valid version now (#4061) v.2.7.12 2021-09-22 =================== Fixes: * Fix tracking.js module preventing links to be opened in new tabs (`#6384 `_) * Fix deleted org/group feeds (`#6367 `_) * Fix runaway preview height (`#6283 `_) * Fix unreliable ordering of DataStore results (`#2317 `_) v.2.7.11 2021-05-19 =================== Fixes: * Allow uploaders to only override asset / resource uploading (`#6088 `_) * Catch TypeError from invalid thrown by dateutils (`#6085 `_) * Use external library to parse view filter params * Fix auth error when deleting a group/org (`#6006 `_) * Fix datastore_search language parameter (`#5974 `_) * make SQL function whitelist case-insensitive unless quoted (`#5969 `_) * Fix Explore button not working (`#3720 `_) * "New view" button fix (`#4260 `_) * remove unused var in task_status_update (`#5861 `_) * Prevent guessing format and mimetype from resource urls without path (`#5852 `_) v.2.7.10 2021-02-10 =================== Fixes: * Fix install documentation (`#5618 `_) * Fix exception when passing limit to organization (`#5789 `_) * Fix for adding directories from plugins if partially string matches existing values (`#5836 `_) * Fix upload log activity sorting (`#5827 `_) * Textview: escape text formats (`#5814 `_) * Add allow_partial_update to fix losing users (`#5734 `_) * Set default group_type to group in group_create (`#5693 `_) * Use user performing the action on activity context on user_update (`#5743 `_) * New block in nav links in user dashboard (`#5804 `_) * Update references to DataPusher documentation * Fix JavaScript error on Edge (`#5782 `_) * Fix error when deleting resource with missing datastore table (`#5757 `_) * ensure HTTP_HOST is bytes under python2 (`#5714 `_) * Don't set old_filename when updating groups (`#5707 `_) * Filter activities from user at the database level (`#5698 `_) * Fix user_list ordering (`#5667 `_) * Allow list for functions in datastore_search_sql (see :ref:`ckan.datastore.sqlsearch.allowed_functions_file`) v.2.7.9 2020-10-21 ================== Fixes: * Fix unflattening of dataset extras (`#5602 `_) * Fix minified JS files in 2.7 (`#5557 `_) * Send the right URL of CKAN to datapusher (`#5281 `_) * Fix fullscreen for resource webpageview (`#5552 `_) * PackageSearchIndex.index_package(): catch IndexError from date parsing (`#5535 `_) * Fix collapsible menu in mobile view (`#5448 `_) * Refactor query string parsing module v.2.7.8 2020-08-05 ================== Fixes: * Fix UnicodeDecodeError on abort function (`#4829 `_) * Improve and reorder resource_formats.json (`#5034 `_) * Allow passing arguments to the RQ enqueue_call function (`#5208 `_) * Fix dashboard follower filter (`#5412 `_) * Update dictionary.html for bs2 version (`#5365 `_) * Prevent password reset exposing account presence (`#5431 `_) * Add class dropdown to 'New view' menu (`#5470 `_) * Update jQuery to 3.5.0 (`#5364 `_) * Fix dashboard activity filter (`#5424 `_) * Prevent account presence exposure when ckan.auth.public_user_details = false (`#5432 `_) * Fix resource upload filename fetching in IE (`#5438 `_) * Unflatten: allow nesting >1 level (`#5444 `_) * Allow lists in resource extras (`#5453 `_) * Only add error to tag_errors if not empty (`#5454 `_) * Fix order_by param in user_list action (`#5342 `_) * Fix for Resources validation errors display (`#5335 `_) v.2.7.7 2020-04-15 ================== General notes: * Note: This version does not requires a requirements upgrade on source installations * Note: This version does not requires a database upgrade * Note: This version does not require a Solr schema upgrade * Note: This version includes changes in the way the ``SameSite`` flag is set on the ``auth_tkt`` authorization cookie. The new default setting for it is ``SameSite=Lax``, which aligns with the behaviour of all major browsers. If for some reason you need a different value, you can set it via the `who.samesite` configuration option. You can find more information on the ``SameSite`` attribute `here `_. Fixes: * Fix for number of datasets displayed on the My organizations tab (`#3580 `_) * Password reset request - generally tighten it up (`#4636 `_) * Add missing get_action calls in activity actions (`#4967 `_) * Fix datetime comparison in resource_dict_save (`#5033 `_) * Allow vocabulary_id in /api/2/util/tag/autocomplete (`#5071 `_) * Fetch less data for `get_all_entity_ids` (`#5201 `_) * Show error in text view if xhr failed (`#5271 `_) * Fix code injection in autocomplete module (`#5064 `_) * Check for the existence of tracking summary data before attempting to load it (`#5030 `_) * Fix broken translation in image view placeholder (`#5099 `_) * Filter revisions shown according to dataset permissions * Update JS vendor libraries * Use returned facets in group controller (`#2713 `_) * Samesite support in auth cookie (`#5255 `_) * Handle missing resources in case we have a race condition with the DataPusher (`#3980 `_) * Add the g object to toolkit v.2.7.6 2019-07-03 ================== General notes: * Note: This version does not requires a requirements upgrade on source installations * Note: This version does not requires a database upgrade * Note: This version does not require a Solr schema upgrade Fixes: * Fix problem with reindex-fast (`#4352 `_) * Fix `include_total` in `datastore_search` (`#4446 `_) * Fix `ValueError` in `url_validator` (`#4629 `_) * Strip local path when uploading file in IE (`#4608 `_) * Increase size of h1 headings to 1.8em (`#4665 `_) * Fix broken div nesting in the `user/read_base.html` (`#4672 `_) * Use `get_action` to call activity actions (`#4684 `_) * Make reorder resources button translatable (`#4838 `_) * More robust auth functions for `resource_view_show` (`#4827 `_) * Allow to customize the DataProxy URL (`#4874 `_) * Allow custom CKAN callback URL for the DataPusher (`#4878 `_) v2.7.5 2018-12-12 ================= * Strip full URL on uploaded resources before saving to DB (`#4382 `_) * Fix for datastore_search distinct=true option (`#4236 `_) * Fix edit slug button (`#4379 `_) * Don't re-register plugin helpers on flask_app (`#4414 `_) * Fix for Resource View Re-order (`#4416 `_) * autocomplete.js: fix handling of comma key codes (`#4421 `_) * Flask patch update (`#4426 `_) * Allow plugins to define multiple blueprints (`#4495 `_) * Fix i18n API encoding (`#4505 `_) * Allow to defined legacy route mappings as a dict in config (`#4521 `_) * group_patch does not reset packages (`#4557 `_) v2.7.4 2018-05-09 ================= * Adding filter at resource preview doesn't work while site is setup with ckan.root_path param (#4140) * Datastore dump results are not the same as data in database (#4150) v2.7.3 2018-03-15 ================= General notes: * As with all patch releases this one does not include requirement changes. However in some scenarios you might encounter the following error while installing or upgrading this version of CKAN:: Error: could not determine PostgreSQL version from '10.2' This is due to a bug in the psycopg2 version pinned to the release. To solve it, upgrade psycopg2 with the following command:: pip install --upgrade psycopg2==2.8.2 * This release does not require a Solr schema upgrade, but if you are having the issues described in #3863 (datasets wrongly indexed in multilingual setups), you can upgrade the Solr schema and reindex to solve them. * #3422 (implemented in #3425) introduced a major bug where if a resource was deleted and the DataStore was active extras from all resources on the site where changed. This is now fixed as part of this release but if your database is already affected you will need to run a script to restore the extras to their previous state. Remember, you only need to run the script if all the following are true: 1. You are currently running CKAN 2.7.0 or 2.7.2, and 2. You have enabled the DataStore, and 3. One or more resources with data on the DataStore have been deleted (or you suspect they might have been) If all these are true you can run the following script to restore the extras to their previous state: https://github.com/ckan/ckan/blob/dev-v2.7/scripts/4042_fix_resource_extras.py This issue is described in #4042 Fixes: * Fix toggle bars header icon (#3880) * Change CORS header keys and values to string instead of unicode (#3855) * Fix cors header when all origins are allowed (#3898) * Update SOLR schema.xml reference in Dockerfile * Build local SOLR container by default * Create datastore indexes only if they are not exist * Properly close file responses * Use javascript content-type for jsonp responses (#4022) * Add Data Dictionary documentation (#3989) * Fix SOLR index delete_package implementation * Add second half of DataStore set-permissions command(Docs) * Fix extras overriding for removed resources (#4042) * Return a 403 if not authorized on the search page (#4081) * Add support for user/pass for Solr as ENV var * Change permission_labels type to string in schema.xml (#3863) * Disallow solr local parameters * Improve text view rendering * Update Orgs/Groups logic for custom fields delete and update (#4094) * Upgrade Solr Docker image v2.7.2 2017-09-28 ================= * Include missing minified JavaScript files v2.7.1 2017-09-27 ================= * add field_name to image_upload macro when uploading resources (#3766) * Add some missing major changes to change log. (#3799) * _mail_recipient header override (#3781) * skip url parsing in redirect (#3499) * Fix multiple errors in i18n of JS modules (#3590) * Standardize on url_for on popup (#3831) v2.7.0 2017-08-02 ================= General notes: * Starting from this version, CKAN requires at least Postgres 9.3 * Starting from this version, CKAN requires a Redis database. Please refer to the new `ckan.redis.url `_ configuration option. * This version requires a requirements upgrade on source installations * This version requires a database upgrade * This version requires a Solr schema upgrade * There are several old features being officially deprecated starting from this version. Check the *Deprecations* section to be prepared. Major changes: * New datatables_view resource view plugin for tabular data (#3444) * IDataStoreBackend plugins for replacing the default DataStore Postgres backend (#3437) * datastore_search new result formats and performance improvements (#3523) * PL/PGSQL triggers for DataStore tables (#3428) * DataStore dump CLI commands (#3384) * Wrap/override actions defined in other plugins (#3494) * DataStore table data dictionary stored as postgres comments (#3414) * Common session object for Flask and Pylons (#3208) * Rename deleted datasets when they conflict with new ones (#3370) * DataStore dump more formats: CSV, TSV, XML, JSON; BOM option (#3390) * Common requests code for Flask and Pylons so you can use Flask views via the new IBlueprint interface (#3212) * Generate complete datastore dump files (#3344) * A new system for asynchronous background jobs (#3165) * Chaining of action functions (#3494) Minor changes: * Renamed example theme plugin (#3576) * Localization support for groups (#3559) * Create new resource views when format changes (#3515) * Email field validation (#3568) * datastore_run_triggers sysadmin-only action to apply triggers to existing data (#3565) * Docs updated for Ubuntu 16.04 (#3544) * Upgrade leaflet to 0.7.7 (#3534) * Datapusher CLI always-answer-yes option (#3524) * Added docs for all plugin interfaces (#3519) * DataStore dumps nested columns as JSON (#3487) * Faster/optional datastore_search total calculation (#3467) * Faster group_activity_query (#3466) * Faster query performance (#3430) * Marked remaining JS strings translatable (#3423) * Upgrade font-awesome to 4.0.3 (#3400) * group/organization_show include_dataset_count option (#3385) * image_formats config option for image viewer (#3380) * click may now be used for CLI interfaces: use load_config instead of CkanCommand (#3384) * package_search option to return only names/ids (#3427) * user_list all_fields option (#3353) * Error controller may now be overridden (#3340) * Plural translations in JS (#3211) * Support JS translations in extensions (#3272) * Requirements upgraded (#3305) * Dockerfile updates (#3295) * Fix activity test to use utcnow (#3644) * Changed required permission from 'update' to 'manage_group' (#3631) * Catch invalid sort param exception (#3630) * Choose direction of recreated package relationship depending on its type (#3626) * Fix render_datetime for dates before year 1900 (#3611) * Fix KeyError in 'package_create' (#3027) * Allow slug preview to work with autocomplete fields (#2501) * Fix filter results button not working for organization/group (#3620) * Allow underscores in URL slug preview on create dataset (#3612) * Fallback to po file translations on ``h.get_translated()`` (#3577) * Fix Fanstatic URL on non-root installs (#3618) * Fixed escaping issues with ``helpers.mail_to`` and datapusher logs * Autocomplete fields are more responsive - 300ms timeout instead of 1s (#3693) * Fixed dataset count display for groups (#3711) * Restrict access to form pages (#3684) * Render_datetime can handle dates before year 1900 (#2228) API changes: * ``organization_list_for_user`` (and the ``h.organizations_available()`` helper) now return all organizations a user belongs to regardless of capacity (Admin, Editor or Member), not just the ones where she is an administrator (#2457) * ``organization_list_for_user`` (and the ``h.organizations_available()`` helper) now default to not include package_count. Pass include_dataset_count=True if you need the package_count values. * ``resource['size']`` will change from string to long integer (#3205) * Font Awesome has been upgraded from version 3.2.1 to 4.0.3 .Please refer to https://github.com/FortAwesome/Font-Awesome/wiki/Upgrading-from-3.2.1-to-4 to upgrade your code accordingly if you are using custom themes. Deprecations: * The API versions 1 and 2 (also known as the REST API, ie ``/api/rest/*`` will removed in favour of the version 3 (action API, ``/api/action/*``), which was introduced in CKAN 2.0. The REST API will be removed on CKAN 2.8. * The default theme included in CKAN core will switch to use Bootstrap 3 instead of Bootstrap 2 in CKAN 2.8. The current Bootstrap 2 based templates will still be included in the next CKAN versions, so existing themes will still work. Bootstrap 2 templates will be eventually removed though, so instances are encouraged to update their themes using the available documentation (https://getbootstrap.com/migration/) * The activity stream related actions ending with ``*_list`` (eg ``package_activity_list``) and ``*_html`` (eg ``package_activity_list_html``) will be removed in CKAN 2.8 in favour of more efficient alternatives and are now deprecated. * The legacy revisions controller (ie ``/revisions/*``) will be completely removed in CKAN 2.8. * The old Celery based background jobs will be removed in CKAN 2.8 in favour of the new RQ based jobs (https://docs.ckan.org/en/latest/maintaining/background-tasks.html). Extensions can still of course use Celery but they will need to handle the management themselves. v.2.6.9 2020-04-15 ================== General notes: * Note: This version does not requires a requirements upgrade on source installations * Note: This version does not requires a database upgrade * Note: This version does not require a Solr schema upgrade Fixes: * Fix for number of datasets displayed on the My organizations tab (`#3580 `_) * Fix datetime comparison in resource_dict_save (`#5033 `_) * Fetch less data for `get_all_entity_ids` (`#5201 `_) * Show error in text view if xhr failed (`#5271 `_) * Allow vocabulary_id in /api/2/util/tag/autocomplete (`#5071 `_) * Fix code injection in autocomplete module (`#5064 `_) * Fix broken translation in image view placeholder (`#5099 `_) * Filter revisions shown according to dataset permissions * Update JS vendor libraries * Use returned facets in group controller (`#2713 `_) * Samesite support in auth cookie (`#5255 `_) * Handle missing resources in case we have a race condition with the DataPusher (`#3980 `_) * Add the g object to toolkit v.2.6.8 2019-07-03 ================== General notes: * Note: This version does not requires a requirements upgrade on source installations * Note: This version does not requires a database upgrade * Note: This version does not require a Solr schema upgrade Fixes: * Fix broken div nesting in the `user/read_base.html` (`#4672 `_) * Strip local path when uploading file in IE (`#4608 `_) * Increase size of h1 headings to 1.8em (`#4665 `_) * Fix `ValueError` in `url_validator` (`#4629 `_) * More robust auth functions for `resource_view_show` (`#4827 `_) * Allow to customize the DataProxy URL (`#4874 `_) * Allow custom CKAN callback URL for the DataPusher (`#4878 `_) v2.6.7 2018-12-12 ================= * Fix for Resource View Re-order (`#4416 `_) * autocomplete.js: fix handling of comma key codes (`#4421 `_) * group_patch does not reset packages (`#4557 `_) v2.6.6 2018-05-09 ================= * Adding filter at resource preview doesn't work while site is setup with ckan.root_path param (#4140) * Stable version URLs CKAN for documentation (#4209) * Add Warning in docs sidebar (#4209) v2.6.5 2018-03-15 ================= Note: This version requires a database upgrade * Activity Time stored in UTC (#2882) * Migration script to adjust current activity timestamps to UTC * Change CORS header keys and values to string instead of unicode (#3855) * Fix cors header when all origins are allowed (#3898) * Update SOLR schema.xml reference in Dockerfile * Build local SOLR container by default * Create datastore indexes only if they don't exist * Properly close file responses * Use javascript content-type for jsonp responses (#4022) * Fix SOLR index delete_package implementation * Add second half of DataStore set-permissions command (Docs) * Return a 403 if not authorized on the search page (#4081) * Add support for user/pass for Solr as ENV var * Disallow solr local parameters * Improve text view rendering * Update Orgs/Groups logic for custom fields delete and update (#4094) v2.6.4 2017-09-27 ================= * Mail recipient header override (#3781) * Skip url parsing in redirect (#3499) * Support non root for fanstatic (#3618) v2.6.3 2017-08-02 ================= * Fix in organization / group form image URL field (#3661) * Fix activity test to use utcnow (#3644) * Changed required permission from 'update' to 'manage_group' (#3631) * Catch invalid sort param exception (#3630) * Choose direction of recreated package relationship depending on its type (#3626) * Fix render_datetime for dates before year 1900 (#3611) * Fix KeyError in 'package_create' (#3027) * Allow slug preview to work with autocomplete fields (#2501) * Fix filter results button not working for organization/group (#3620) * Allow underscores in URL slug preview on create dataset (#3612) * Create new resource view if resource format changed (#3515) * Fixed escaping issues with `helpers.mail_to` and datapusher logs * Autocomplete fields are more responsive - 300ms timeout instead of 1s (#3693) * Fixed dataset count display for groups (#3711) * Restrict access to form pages (#3684) v2.6.2 2017-03-22 ================= * Use fully qualified urls for reset emails (#3486) * Fix edit_resource for resource with draft state (#3480) * Tag fix for group/organization pages (#3460) * Setting of datastore_active flag moved to separate function (#3481) v2.6.1 2017-02-22 ================= * Fix DataPusher being fired multiple times (`#3245 `_) * Use the url_for() helper for datapusher URLs (`#2866 `_) * Resource creation date use datetime.utcnow() (`#3447 `_) * Fix locale error when using fix ckan.root_path * `render_markdown` breaks links with ampersands * Check group name and id during package creation * Use utcnow() on dashboard_mark_activities_old (`#3373 `_) * Fix encoding error on DataStore exception * Datastore doesn't add site_url to resource created via API (`#3189 `_) * Fix memberships after user deletion (`#3265 `_) * Remove idle database connection (`#3260 `_) * Fix package_owner_org_update action when called via the API (`#2661 `_) * Fix French locale (`#3327 `_) * Updated translations v2.6.0 2016-11-02 ================= Note: Starting from this version, CKAN requires at least Python 2.7 and Postgres 9.2 Note: This version requires a requirements upgrade on source installations Note: This version requires a database upgrade Note: This version does not require a Solr schema upgrade (You may want to upgrade the schema if you want to target Solr>=5, see `#2914 `_) Major: * Private datasets are now included in the default dataset search results (`#3191 `_) * package_search API action now has an include_private parameter (`#3191 `_) Minor: * Make resource name default to file name (`#1372 `_) * Customizable email templates (`#1527 `_) * Change solrpy library to pysolr (`#2352 `_) * Cache SQL query results (`#2353 `_) * File Upload UX improvements (`#2604 `_) * Helpers for multilingual fields (`#2678 `_) * Improve Extension translation docs (`#2783 `_) * Decouple configuration from Pylons (`#3163 `_) * toolkit: add h, StopOnError, DefaultOrganizationForm (`#2835 `_) * Remove Genshi support (`#2833 `_) * Make resource URLs optional (`#2844 `_) * Use 403 when actions are forbidden, not 401 (`#2846 `_) * Upgrade requirements version (`#3004 `_, `#3005 `_) * Add icons sources (`#3048 `_) * Remove lib/dumper (`#2879 `_) * ckan.__version__ available as template helper (`#3103 `_) * Remove `site_url_nice` from app_globals (`#3117 `_) * Remove `e.message` deprecation warning when running tests (`#3121 `_) * Drop Python 2.6 support (`#3126 `_) * Update Recline version (`#3184 `_) * Refactor config/middleware.py to more closely match poc-flask-views (`#3116 `_) * Creation of datasets sources with no organization specified (`#3046 `_) Bug fixes: * DataPusher called multiple times when creating a dataset (`#2856 `_) * Default view is re-added when removed before DataStore upload is complete (`#3011 `_) * "Data API" button disappears on resource page after empty update (`#3012 `_) * Uncaught email exceptions on user invite (`#3077 `_) * Resource view description is not rendered as Markdown (`#3128 `_) * Fix broken html5lib dependency (`#3180 `_) * ZH_cn translation formatter fix (`#3238 `_) * Incorrect i18n-paths in extension's setup.cfg (`#3275 `_) * Changing your user name produces an error and logs you out (`#2394 `_) * Fix "Load more" functionality in the dashboard (`#2346 `_) * Fix filters not working when embedding a resource view (`#2657 `_) * Proper sanitation of header name on SlickGrid view (`#2923 `_) * Fix unicode error when indexing field of type JSON (`#2969 `_) * Fix group feeds returning no datasets (`#2955 `_) * Replace MapQuest tiles in Recline with Stamen Terrain (`#3162 `_) * Fix bulk operations not taking effect (`#3199 `_) * Raise validation errors on group/org_member_create (`#3108 `_) * Incorrect warnings when ckan.views.default_views is empty (`#3093 `_) * Don't show deleted users/datasets on member_list (`#3078 `_) * Fix Tag pagination widget styling (`#2399 `_) * Fix package_owner_org_update standalone (`#2661 `_) * Don't template fanstatic error pages (`#2770 `_) * group_controller() on IGroupForm not in interface (`#2771 `_) * Fix assert_true to test for message in response (`#2802 `_) * Add user parameter to paster profile command (`#2815 `_) * make context['user'] always username or None (`#2817 `_) * remove some deprecated compatibility hacks (`#2818 `_) * Param use_default_schema does not work on package_search (`#2848 `_) * Sanitize offset when listing group activity (`#2859 `_) * Incorrect 'download resource' hyperlink when a resource is unable to upload to datastore (`#2873 `_) * Resolve datastore_delete erasing the database when filters was blank. (`#2885 `_) * DomainObject.count() doesn't return count (`#2919 `_) * Fix response code test failures (`#2931 `_) * Fixed the url_for_* helpers when both SCRIPT_NAME and ckan.root_path are defined (`#2936 `_) * Escape special characters in password while db loading (`#2952 `_) * Fix redirect not working with non-root (`#2968 `_) * Group pagination does not preserve sort order (`#2981 `_) * Remove LazyJSONObject (`#2983 `_) * Deleted users appear in sysadmin user lists (`#2988 `_) * Server error at /organization if not authorized to list organizations (`#2990 `_) * Slow page rendering when using lots of snippets (`#3000 `_) * Only allow JSONP callbacks on GET requests (`#3002 `_) * Attempting to access non-existing helpers should raise HelperException (`#3041 `_) * Deprecate h.url, make it use h.url_for internally (`#3055 `_) * Tests fail when LANG environment variable is set to German (`#3060 `_) * Fix pagination style (CSS) (`#3067 `_) * Login fails with 404 when using root_path (`#3089 `_) * Resource view description is not rendered as Markdown (`#3128 `_) * Clarify package_relationship_update documentation (`#3132 `_) * `q` parameter in followee_list action has no effect (`#3167 `_) * Zh cn translation formatter fix (`#3238 `_) * Users are not removed in related tables if the main user entry is deleted (`#3265 `_) API changes and deprecations: * Replace `c.__version__` with new helper `h.ckan_version()` (`#3103 `_) v2.5.9 2018-05-09 ================= * Adding filter at resource preview doesn't work while site is setup with ckan.root_path param (#4140) * Add Warning in docs sidebar (#4209) * Point API docs to stable URL (#4209) v2.5.8 2018-03-15 ================= Note: This version requires a database upgrade * Fix language switcher * Activity Time stored in UTC (#2882) * Migration script to adjust current activity timestamps to UTC * Change CORS header keys and values to string instead of unicode (#3855) * Fix cors header when all origins are allowed (#3898) * Create datastore indexes only if they are not exist * Use javascript content-type for jsonp responses (#4022) * Fix SOLR index delete_package implementation * Add second half of DataStore set-permissions command(Docs) * Update SOLR client (pysolr -> solrpy) * Return a 403 if not authorized on the search page (#4081) * Add support for user/pass for Solr as ENV var * Disallow solr local parameters * Improve text view rendering * Update Orgs/Groups logic for custom fields delete and update (#4094) v2.5.7 2017-09-27 ================= * Allow overriding email headers (#3781) * Support non-root instances on fanstatic (#3618) * Add missing close button on organization page (#3814) v2.5.6 2017-08-02 ================= * Fix in organization / group form image URL field (#3661) * Fix activity test to use utcnow (#3644) * Changed required permission from 'update' to 'manage_group' (#3631) * Catch invalid sort param exception (#3630) * Choose direction of recreated package relationship depending on its type (#3626) * Fix render_datetime for dates before year 1900 (#3611) * Fix KeyError in 'package_create' (#3027) * Allow slug preview to work with autocomplete fields (#2501) * Fix filter results button not working for organization/group (#3620) * Allow underscores in URL slug preview on create dataset (#3612) * Create new resource view if resource format changed (#3515) * Fixed incorrect escaping in `mail_to` and datapusher's log * Autocomplete fields are more responsive - 300ms timeout instead of 1s (#3693) * Fixed dataset count display for groups (#3711) * Restrict access to form pages (#3684) v2.5.5 2017-03-22 ================= * Use fully qualified urls for reset emails (#3486) * Fix edit_resource for resource with draft state (#3480) * Tag fix for group/organization pages (#3460) * Setting of datastore_active flag moved to separate function (#3481) v2.5.4 2017-02-22 ================= * Fix DataPusher being fired multiple times (#3245) * Use the url_for() helper for datapusher URLs (#2866) * Resource creation date use datetime.utcnow() (#3447) * Fix locale error when using fix ckan.root_path * `render_markdown` breaks links with ampersands * Check group name and id during package creation * Use utcnow() on dashboard_mark_activities_old (#3373) * Fix encoding error on DataStore exception * Datastore doesn't add site_url to resource created via API (#3189) * Fix memberships after user deletion (#3265) * Remove idle database connection (#3260) * Fix package_owner_org_update action when called via the API (#2661) v2.5.3 2016-11-02 ================= * DataPusher called multiple times when creating a dataset (#2856) * Default view is re-added when removed before DataStore upload is complete (#3011) * "Data API" button disappears on resource page after empty update (#3012) * Uncaught email exceptions on user invite (#3077) * Resource view description is not rendered as Markdown (#3128) * Fix broken html5lib dependency (#3180) * ZH_cn translation formatter fix (#3238) * Incorrect i18n-paths in extension's setup.cfg (#3275) * Changing your user name produces an error and logs you out (#2394) * Fix "Load more" functionality in the dashboard (#2346) * Fix filters not working when embedding a resource view (#2657) * Proper sanitation of header name on SlickGrid view (#2923) * Fix unicode error when indexing field of type JSON (#2969) * Fix group feeds returning no datasets (#2955) * Replace MapQuest tiles in Recline with Stamen Terrain (#3162) * Fix bulk operations not taking effect (#3199) * Raise validation errors on group/org_member_create (#3108) * Incorrect warnings when ckan.views.default_views is empty (#3093) * Don't show deleted users/datasets on member_list (#3078) v2.5.2 2016-03-31 ================= Bug fixes: * Avoid submitting resources to the DataPusher multiple times (#2856) * Use `resource.url` as raw_resource_url (#2873) * Fix DomainObject.count() to return count (#2919) * Prevent unicode/ascii conversion errors in DataStore * Fix datastore_delete erasing the db when filters is blank (#2885) * Avoid package_search exception when using use_default_schema (#2848) * Encode EXPLAIN SQL before sending to datastore * Use `ckan.site_url` to generate urls of resources (#2592) * Fixed the url for the organization_item template v2.5.1 2015-12-17 ================= Note: This version requires a requirements upgrade on source installations Note: This version requires a database upgrade Note: This version does not require a Solr schema upgrade Major: * CKAN extension language translations integrated using ITranslations interface (#2461, #2643) * Speed improvements for displaying a dataset (#2234), home page (#2554), searching (#2382, #2724) and API actions: package_show (#1078) and user_list (#2752). * An interface to replace the file uploader, allowing integration with other cloud storage providers (IUploader interface) (#2510) Minor: * package_purge API action added (#1572) * revision_list API action now has paging (#1431) * Official Ubuntu 14.04 LTS support (#1651) * Require/validate current password before allowing a password change (#1940) * recline_map_view now recognizes GeoJSON fields (#2387) * Timezone setting (#2494) * Updating a resource via upload now saves the last_modified value in the resource (#2519) * DataPusher can be customized using the new IDataPusher interface (#2571) * Exporting and importing users, with their passwords (if sysadmin) (#2647) Bug fixes: * Fix to allow uppercase letters in local part of email when sending user invitations (#2415) * License pick-list changes would cause old values in datasets to be overwritten when edited (#2472) * Schema was being passed to package_create_default_resource_views (#2484) * Arabic translation format string issue (#2493) * Error when deleting organizations (#2512) * When DataPusher had an error storing a resource in Data Store, the resource data page gave an error (#2518) * Data preview failed when it comes from a server that gives 403 error from a HEAD request (#2530) * 'paster views create' failed for non-default dataset types (#2532) * DataPusher didn't work for TSV files (#2553) * DataPusher failed sometimes due to 'type mismatch' (#2581) * IGroupForm wasn't allowing new groups (of type 'group') to use group_form (#2617, #2640) * group_purge left behind a Member if it has a parent group/org (#2631) * organization_purge left orphaned datasets still with owner_id (#2632) * Fix Markdown rendering issue * Return default error page on fanstatic errors * Prevent authentication when using API callbacks Changes and deprecations ------------------------ * The old RDF templates to output a dataset in RDF/XML or N3 format have been removed. These can be now enabled using the ``dcat`` plugin on *ckanext-dcat*: https://github.com/ckan/ckanext-dcat#rdf-dcat-endpoints * The library used to render markdown has been changed to python-markdown. This introduces both ``python-markdown`` and ``bleach`` as dependencies, as ``bleach`` is used to clean any HTML provided to the markdown processor. * This is the last version of CKAN to support Postgresql 8.x, 9.0 and 9.1. The next minor version of CKAN will require Postgresql 9.2 or later. v2.5.0 2015-12-17 ================= Cancelled release v2.4.9 2017-09-27 ================= * Allow overriding email headers (#3781) * Support non-root instances on fanstatic (#3618) * Add missing close button on organization page (#3814) v2.4.8 2017-08-02 ================= * Fix in organization / group form image URL field (#3661) * Fix activity test to use utcnow (#3644) * Changed required permission from 'update' to 'manage_group' (#3631) * Catch invalid sort param exception (#3630) * Choose direction of recreated package relationship depending on its type (#3626) * Fix render_datetime for dates before year 1900 (#3611) * Fix KeyError in 'package_create' (#3027) * Allow slug preview to work with autocomplete fields (#2501) * Fix filter results button not working for organization/group (#3620) * Allow underscores in URL slug preview on create dataset (#3612) * Create new resource view if resource format changed (#3515) * Fixed incorrect escaping in `mail_to` * Autocomplete fields are more responsive - 300ms timeout instead of 1s (#3693) * Fixed dataset count display for groups (#3711) * Restrict access to form pages (#3684) v2.4.7 2017-03-22 ================= * Use fully qualified urls for reset emails (#3486) * Fix edit_resource for resource with draft state (#3480) * Tag fix for group/organization pages (#3460) * Fix for package_search context (#3489) v2.4.6 2017-02-22 ================= * Use the url_for() helper for datapusher URLs (#2866) * Resource creation date use datetime.utcnow() (#3447) * Fix locale error when using fix ckan.root_path * `render_markdown` breaks links with ampersands * Check group name and id during package creation * Use utcnow() on dashboard_mark_activities_old (#3373) * Fix encoding error on DataStore exception * Datastore doesn't add site_url to resource created via API (#3189) * Fix memberships after user deletion (#3265) * Remove idle database connection (#3260) * Fix package_owner_org_update action when called via the API (#2661) v2.4.5 2017-02-22 ================= Cancelled release v2.4.4 2016-11-02 ================= * Changing your user name produces an error and logs you out (#2394) * Fix "Load more" functionality in the dashboard (#2346) * Fix filters not working when embedding a resource view (#2657) * Proper sanitation of header name on SlickGrid view (#2923) * Fix unicode error when indexing field of type JSON (#2969) * Fix group feeds returning no datasets (#2955) * Replace MapQuest tiles in Recline with Stamen Terrain (#3162) * Fix bulk operations not taking effect (#3199) * Raise validation errors on group/org_member_create (#3108) * Incorrect warnings when ckan.views.default_views is empty (#3093) * Don't show deleted users/datasets on member_list (#3078) v2.4.3 2016-03-31 ================= Bug fixes: * Use `resource.url` as raw_resource_url (#2873) * Fix DomainObject.count() to return count (#2919) * Add offset param to organization_activity (#2640) * Prevent unicode/ascii conversion errors in DataStore * Fix datastore_delete erasing the db when filters is blank (#2885) * Avoid package_search exception when using use_default_schema (#2848) * resource_edit incorrectly setting action to new instead of edit * Encode EXPLAIN SQL before sending to datastore * Use `ckan.site_url` to generate urls of resources (#2592) * Don't hide actual exception on paster commands v2.4.2 2015-12-17 ================= Note: This version requires a requirements upgrade on source installations Bug fixes: * Fix Markdown rendering issue * Return default error page on fanstatic errors * Prevent authentication when using API callbacks v2.4.1 2015-09-02 ================= Note: #2554 fixes a regression where ``group_list`` and ``organization_list`` where returning extra additional fields by default, causing performance issues. This is now fixed, so the output for these actions no longer returns ``users``, ``extras``, etc. Also, on the homepage template the ``c.groups`` and ``c.group_package_stuff`` context variables are no longer available. Bug fixes: * Fix dataset count in templates and show datasets on featured org/group (#2557) * Fix autodetect for TSV resources (#2553) * Improve character escaping in DataStore parameters * Fix "paster db init" when celery is configured with a non-database backend * Fix severe performance issues with groups and orgs listings (#2554) v2.4.0 2015-07-22 ================= Note: This version requires a database upgrade Note: This version requires a Solr schema upgrade Major: * CKAN config can now be set from environment variables and via the API (#2429) Minor: * API calls now faster: ``group_show``, ``organization_show``, ``user_show``, ``package_show``, ``vocabulary_show`` & ``tag_show`` (#1886, #2206, #2207, #2376) * Require/validate current password before allowing a password change (#1940) * Added ``organization_autocomplete`` action (#2125) * Default authorization no longer allows anyone to create datasets etc (#2164) * ``organization_list_for_user`` now returns organizations in hierarchy if they exist for roles set in ``ckan.auth.roles_that_cascade_to_sub_groups`` (#2199) * Improved accessibility (text based browsers) focused on the page header (#2258) * Improved IGroupForm for better customizing groups and organization behaviour (#2354) * Admin page can now be extended to have new tabs (#2351) Bug fixes: * Command line ``paster user`` failed for non-ascii characters (#1244) * Memory leak fixed in datastore API (#1847) * Modifying resource didn't update it's last updated timestamp (#1874) * Datastore didn't update if you uploaded a new file of the same name as the existing file (#2147) * Files with really long file were skipped by datapusher (#2057) * Multi-lingual Solr schema is now updated so it works again (#2161) * Resource views didn't display when embedded in another site (#2238) * ``resource_update`` failed if you supplied a revision_id (#2340) * Recline could not plot GeoJSON on a map (#2387) * Dataset create form 404 error if you added a resource but left it blank (#2392) * Editing a resource view for a file that was UTF-8 and had a BOM gave an error (#2401) * Email invites had the email address changed to lower-case (#2415) * Default resource views not created when using a custom dataset schema (#2421, #2482) * If the licenses pick-list was customized to remove some, datasets with old values had them overwritten when edited (#2472) * Recline views failed on some non-ascii characters (#2490) * Resource proxy failed if HEAD responds with 403 (#2530) * Resource views for non-default dataset types couldn't be created (#2532) Changes and deprecations ------------------------ * The default of allowing anyone to create datasets, groups and organizations has been changed to False. It is advised to ensure you set all of the :ref:`authorization-settings` options explicitly in your CKAN config. (#2164) * The ``package_show`` API call does not return the ``tracking_summary``, keys in the dataset or resources by default any more. Any custom templates or users of this API call that use these values will need to pass: ``include_tracking=True``. * The legacy `tests` directory has moved to `tests/legacy`, the `new_tests` directory has moved to `tests` and the `new_authz.py` module has been renamed `authz.py`. Code that imports names from the old locations will continue to work in this release but will issue a deprecation warning. (#1753) * ``group_show`` and ``organization_show`` API calls no longer return the datasets by default (#2206) Custom templates or users of this API call will need to pass ``include_datasets=True`` to include datasets in the response. * The ``vocabulary_show`` and ``tag_show`` API calls no longer returns the ``packages`` key - i.e. datasets that use the vocabulary or tag. However ``tag_show`` now has an ``include_datasets`` option. (#1886) * Config option ``site_url`` is now required - CKAN will not abort during start-up if it is not set. (#1976) v2.3.5 2016-11-02 ================= * Fix "Load more" functionality in the dashboard (#2346) * Fix filters not working when embedding a resource view (#2657) * Proper sanitation of header name on SlickGrid view (#2923) * Fix unicode error when indexing field of type JSON (#2969) * Fix group feeds returning no datasets (#2955) * Replace MapQuest tiles in Recline with Stamen Terrain (#3162) * Fix bulk operations not taking effect (#3199) * Raise validation errors on group/org_member_create (#3108) * Incorrect warnings when ckan.views.default_views is empty (#3093) * Don't show deleted users/datasets on member_list (#3078) v2.3.4 2016-03-31 ================= Bug fixes: * Use `resource.url` as raw_resource_url (#2873) * Fix DomainObject.count() to return count (#2919) * Prevent unicode/ascii conversion errors in DataStore * Fix datastore_delete erasing the db when filters is blank (#2885) * Avoid package_search exception when using use_default_schema (#2848) * resource_edit incorrectly setting action to new instead of edit * Use `ckan.site_url` to generate urls of resources (#2592) * Don't hide actual exception on paster commands v2.3.3 2015-12-17 ================= Note: This version requires a requirements upgrade on source installations Bug fixes: * Fix Markdown rendering issue * Return default error page on fanstatic errors * Prevent authentication when using API callbacks v2.3.2 2015-09-02 ================= Bug fixes: * Fix autodetect for TSV resources (#2553) * Improve character escaping in DataStore parameters * Fix "paster db init" when celery is configured with a non-database backend v2.3.1 2015-07-22 ================= Bug fixes: * Resource views won't display when embedded in another site (#2238) * ``resource_update`` failed if you supplied a revision_id (#2340) * Recline could not plot GeoJSON on a map (#2387) * Dataset create form 404 error if you added a resource but left it blank (#2392) * Editing a resource view for a file that was UTF-8 and had a BOM gave an error (#2401) * Email invites had the email address changed to lower-case (#2415) * Default resource views not created when using a custom dataset schema (#2421, #2482) * If the licenses pick-list was customized to remove some, datasets with old values had them overwritten when edited (#2472) * Recline views failed on some non-ascii characters (#2490) * Resource views for non-default dataset types couldn't be created (#2532) v2.3 2015-03-04 =============== Note: This version requires a requirements upgrade on source installations Note: This version requires a database upgrade Note: This version requires a Solr schema upgrade Note: This version requires a DataPusher upgrade on source installations. You should target DataPusher=>0.0.6 and upgrade its dependencies. Major: * Completely refactored resource data visualizations, allowing multiple persistent views of the same data an interface to manage and configure them. (#1251, #1851, #1852, #2204, #2205) Check the updated documentation to know more, and the "Changes and deprecations" section for migration details: https://docs.ckan.org/en/latest/maintaining/data-viewer.html * Responsive design for the default theme, that allows nicer rendering across different devices (#1935) * Improved DataStore filtering and full text search capabilities (#1792, #1830, #1838, #1815) * Added new extension points to modify the DataStore behaviour (#1725) * Simplified two-step dataset creation process (#1659) * Ability for users to regenerate their own API keys (#1412) * New ``package_patch`` action to allow individual fields dataset updates (#1416, #1679) * Changes on the authentication mechanism to allow more secure setups (``httponly`` and ``secure`` cookies, disable CORS, etc). (#2004. #2050, #2052 ...) See "Changes and deprecations" section for more details and "Troubleshooting" for migration instructions. * Better support for custom dataset types (#1795, #2083) * Extensions can combine free-form extras and ``convert_to_extras`` fields (#1894) * Updated documentation theme, now clearer and responsive (#1845) Minor: * Adding custom fields tutorial (#790) * Add metadata created and modified fields to the dataset page (#655) * Improve IFacets plugin interface docstrings (#781) * Remove help string from API calls (#1318) * Add "datapusher submit" command to upload existing resources data (#1792) * More template blocks to allow for easier extension maintenance (#1301) * CKAN API - remove help string from standard calls (#1318) * Hide activity by selected users on activity stream (#1330) * Documentation and clarification about "CKAN Flavored Markdown" (#1332) * Resource formats are now guessed automatically (#1350) * New JavaScript modules tutorial (#1377) * Allow overriding dataset, group, org validation (#1400) * Remove ResourceGroups, show package_id on resources (#1407) * Better errors for NAVL junk (#1418) * DataPusher integration improvements (#1446) * Allow people to create unowned datasets when they belong to an org (#1473) * Add res_type to Solr schema (#1495) * Separate data and metadata licenses on create dataset page (#1503) * Allow CKAN (and paster) to find config from envvar (#1597) * Added xlsx and tsv to the defaults for ckan.datapusher.formats. (#1644) * Add resource extras to Solr search index (#1709) * Prevent packages update in organization_update (#1711) * Programmatically log user in after registration (#1721) * New plugin interfaces: IValidators.get_validators and IConverters.get_converters (#1841) * Index resource name in Solr (#1905) * Update search index after membership changes (#1917) * resource_show: use package_show to get validated data (#1921) * Serve placeholder images locally (#1951) * Don't get all datasets when loading the org in the dataset page (#1978) * Text file preview - lack of vertical scroll bar for long files (#1982) * Changes to allow better use of custom group types in IGroupForm extensions (#1987) * Remove moderated edits (#2006) * package_create: allow sysadmins to set package ids (#2102) * Enable a logged in user to move dataset to another organization (#2218) * Move PDF views into a separate extension (#2270) * Do not provide email configuration in default config file (#2273) * Add custom DataStore SQLAlchemy properties (#2279) Bug fixes: * Set up stats extension as namespace plugin (#291) * Fix visibility validator for datasets (#1188) * Select boxes with autocomplete are clearing their placeholders (#1278) * Default search ordering on organization home page is broken (#1368) * related_list logic function throws a 503 without any parameters (#1384) * Exception on group dictize due to 'with_capacity' on context (#1390) * Wrong template on Add member page (#1392) * Overflowing email address on user page (#1398) * The reset password e-mail is using an incorrect translation string (#1409) * You can't view a group when there is an IGroupForm (#1420) * Disabling activity_streams borks editing groups and user (#1421) * Use a more secure default for the repoze secret key (#1422) * Duplicated Required Fields notice on Group form (#1426) * UI language reset after account creation (#1429) * num_followers and package_count not in default_group_schema (#1434) * Fix extras deletion (#1449) * Fix resource reordering (#1450) * Datastore callback fails when browser url is different from site_url (#1451) * sysadmins should not create datasets without org when config is set (#1453) * Member Editing Fixes (#1454) * Bulk editing broken on IE7 (#1455) * Fix group deletion on IE7 (#1460) * Organization ATOM feed is broken (#1463) * Users can not delete a dataset that not belongs to an organization (#1471) * Error during authorization in datapusher_hook (#1487) * Wrong datapusher hook callback URL on non-root deployments (#1490) * Wrong breadcrumbs on new dataset form and resource pages (#1491) * Atom feed Content-Type returned as 'text/html' (#1504) * Invite to organization causes Internal Server error (#1505) * Dataset tags autocomplete doesn't work (#1512) * Activity Stream from: Organization Error group not found (#1519) * Improve password hashing algorithm (#1530) * Can't download resources with geojson extension (#1534) * All datasets for featured group/organization shown on home page (#1569) * Able to list private datasets via the API (#1580) * Don't lowercase the names of uploaded files (#1584) * Show more facets only if there are more facts to show (#1612) * resource_create should break when called without URL (#1641) * Creating a DataStore resource with the package_id fails for a normal user (#1652) * Fix package permission checks for create+update (#1664) * bulk_process page for non-existent organization throws Exception (#1682) * Catch NotFound error in resource_proxy (#1684) * Fix int_validator (#1692) * Current date indexed on empty "_date" fields (#1701) * Possible to show a resource inside an arbitrary dataset (#1707) * Edit member page shows wrong fields (#1723) * Insecure content warning when running Recline under SSL (#1729) * Flash messages not displayed as part of page.html (#1743) * package_show response includes solr rubbish when using ckan.cache_validated_datasets (#1764) * "Add some resources" link shown to unauthorized users (#1766) * email notifications via paster plugin post erroneously demands authentication (#1767) * Inserting empty arrays in JSON type fields in datastore fails (#1776) * Ordering a dataset listing loses the existing filters (#1791) * Don't delete all cookies whose names start with "ckan" (#1793) * Upgrade some major requirements (eg SQLAlchemy, Requests) (#1817, #1819) * list of member roles disappears on add member page (#1873) * Stats plugin should only show active datasets (#1936) * Featured group on homepage not linking to group (#1996) * --reload doesn't work on the 'paster serve' command (#2013) * Can not override auth config options from tests (#2035) * Fix ``resource_create`` authorization (#2037) * package_search gives internal server error if page < 1 (#2042) * Fix organization pagination (#2141) * Resource extras can not be updated (#2158) * package_show doesn't validate when a custom schema is used (#2175) * Update jQuery minified version to match the unminified one (#1750) * Fix exception during database upgrade (#2029) * Fix resources disappearing on dataset update (#1779) * Fix activity stream queries performance on large instances (#2008) * Only link to http, https and ftp resource urls (#2085) * Avoid private and deleted datasets on stats plugin (#1936) * Fix tags count and group links in stats extension (#1649) * Make resource_create auth work against package_update (#2037) * Fix DataStore permissions check on startup (#1374) * Fix datastore docs link (#2044) * Clean up field names before rendering the Recline table (#2319) * Don't "normalize" resource URL in recline view (#2324) * Don't assume resource format is there on text preview (#2320) * And many, many more! Changes and deprecations ------------------------ * By convention, view plugin names now end with ``_view`` rather than ``_preview`` (eg ``recline_view`` rather than ``recline_preview``). You will need to update them on the :ref:`ckan.plugins` setting. * The way resource visualizations are created by default has changed. You might need to set the :ref:`ckan.views.default_views` configuration option and run a migration command on existing instances. Please refer to the migration guide for more details: https://docs.ckan.org/en/latest/maintaining/data-viewer.html#migrating-from-previous-ckan-versions * The PDF Viewer extension has been moved to a separate extension: https://github.com/ckan/ckanext-pdfview. Please install it separately if you are using the ``pdf_view`` plugin (or the old ``pdf_preview`` one). * The action API (v3) no longer returns the full help for the action on each request. It rather includes a link to a separate call to get the action help string. * The ``user_show`` API call does not return the ``datasets``, ``num_followers`` or ``activity`` keys by default any more. Any custom templates or users of this API call that use these values will need to specify parameters: ``include_datasets`` or ``include_num_followers``. ``activity`` has been removed completely as it was actually a list of revisions, rather than the activity stream. If you want the actual activity stream for a user, call ``user_activity_list`` instead. * The output of ``resource_show`` now contains a ``package_id`` key that links to the parent dataset. * ``helpers.get_action()`` (or ``h.get_action()`` in templates) is deprecated. Since action functions raise exceptions and templates cannot catch exceptions, it's not a good idea to call action functions from templates. Instead, have your controller method call the action function and pass the result to your template using the ``extra_vars`` param of ``render()``. Alternatively you can wrap individual action functions in custom template helper functions that handle any exceptions appropriately, but this is likely to make your the logic in your templates more complex and templates are difficult to test and debug. Note that logic.get_action() and toolkit.get_action() are *not* deprecated, core code and plugin code should still use ``get_action()``. * Cross-Origin Resource Sharing (CORS) support is no longer enabled by default. Previously, Access-Control-Allow-* response headers were added for all requests, with Access-Control-Allow-Origin set to the wildcard value ``*``. To re-enable CORS, use the new ``ckan.cors`` configuration settings (:ref:`ckan.cors.origin_allow_all` and :ref:`ckan.cors.origin_whitelist`). * The HttpOnly flag will be set on the authorization cookie by default. For enhanced security, we recommend using the HttpOnly flag, but this behaviour can be changed in the ``Repoze.who`` settings detailed in the Config File Options documentation (`who.httponly`). * The OpenID login option has been removed and is no longer supported. See "Troubleshooting" if you are upgrading an existing CKAN instance as you may need to update your ``who.ini`` file. Template changes ---------------- * Note to people with custom themes: If you've changed the ``{% block secondary_content %}`` in templates/package/search.html pay close attention as this pull request changes the structure of that template block a little. Also: There's a few more bootstrap classes (especially for grid layout) that are now going to be in the templates. Take a look if any of the following changes might effect your content blocks: https://github.com/ckan/ckan/pull/1935 Troubleshooting: ---------------- * Login does not work, for existing and new users. You need to update your existing ``who.ini`` file. - In the ``[plugin:auth_tkt]`` section, replace:: use = ckan.config.middleware:ckan_auth_tkt_make_app with:: use = ckan.lib.auth_tkt:make_plugin - In ``[authenticators]``, add the ``auth_tkt`` plugin Also see the next point for OpenID related changes. * Exception on first load after upgrading from a previous CKAN version:: ImportError: has no 'OpenIDAuthenticator' attribute or:: ImportError: No module named openid There are OpenID related configuration options in your ``who.ini`` file which are no longer supported. This file is generally located in ``/etc/ckan/default/who.ini`` but its location may vary if you used a custom deployment. The options that you need to remove are: - The whole ``[plugin:openid]`` section - In ``[general]``, replace:: challenge_decider = repoze.who.plugins.openid.classifiers:openid_challenge_decider with:: challenge_decider = repoze.who.classifiers:default_challenge_decider - In ``[identifiers]``, remove ``openid`` - In ``[authenticators]``, remove ``ckan.lib.authenticator:OpenIDAuthenticator`` - In ``[challengers]``, remove ``openid`` This is a diff with the whole changes: https://github.com/ckan/ckan/pull/2058/files#diff-2 Also see the previous point for other ``who.ini`` changes. v2.2.4 2015-12-17 ================= Note: This version requires a requirements upgrade on source installations Bug fixes: * Fix Markdown rendering issue * Return default error page on fanstatic errors * Prevent authentication when using API callbacks v2.2.3 2015-07-22 ================= Bug fixes: * Allow uppercase emails on user invites (#2415) * Fix broken boolean validator (#2443) * Fix auth check in resources_list.html (#2037) * Key error on resource proxy (#2425) * Ignore revision_id passed to resources (#2340) * Add reset for reset_key on successful password change (#2379) v2.2.2 2015-03-04 ================= Bug fixes: * Update jQuery minified version to match the unminified one (#1750) * Fix exception during database upgrade (#2029) * Fix resources disappearing on dataset update (#1779) * Fix activity stream queries performance on large instances (#2008) * Only link to http, https and ftp resource urls (#2085) * Avoid private and deleted datasets on stats plugin (#1936) * Fix tags count and group links in stats extension (#1649) * Make resource_create auth work against package_update (#2037) * Fix DataStore permissions check on startup (#1374) * Fix datastore docs link (#2044) * Fix resource extras getting lost on resource update (#2158) * Clean up field names before rendering the Recline table (#2319) * Don't "normalize" resource URL in recline view (#2324) * Don't assume resource format is there on text preview (#2320) v2.2.1 2014-10-15 ================= Bug fixes: * Organization image_url is not displayed in the dataset view. (#1934) * list of member roles disappears on add member page if you enter a user that doesn't exist (#1873) * group/organization_member_create do not return a value. (#1878) * i18n: Close a tag in French translation in Markdown syntax link (#1919) * organization_list_for_user() fixes (#1918) * Don't show private datasets to group members (#1902) * Incorrect link in Organization snippet on dataset page (#1882) * Prevent reading system tables on DataStore SQL search (#1871) * Ensure that the DataStore is running on legacy mode when using PostgreSQL < 9.x (#1879) * Select2 in the Tags field is broken(#1864) * Edit user encoding error (#1436) * Able to list private datasets via the API (#1580) * Insecure content warning when running Recline under SSL (#1729) * Add quotes to package ID in Solr query in _bulk_update_dataset to prevent Solr errors with custom dataset IDs. (#1853) * Ordering a dataset listing loses the existing filters (#1791) * Inserting empty arrays in JSON type fields in datastore fails (#1776) * email notifications via paster plugin post erroneously demands authentication (#1767) * "Add some resources" link shown to unauthorized users (#1766) * Current date indexed on empty "\*_date" fields (#1701) * Edit member page shows wrong fields (#1723) * programmatically log user in after registration (#1721) * Dataset tags autocomplete doesn't work (#1512) * Deleted Users bug (#1668) * UX problem with previous and next during dataset creation (#1598) * Catch NotFound error in resources page (#1685) * _tracking page should only respond to POST (#1683) * bulk_process page for non-existent organization throws Exception (#1682) * Fix package permission checks for create+update (#1664) * Creating a DataStore resource with the package_id fails for a normal user (#1652) * Trailing whitespace in resource URLs not stripped (#1634) * Move the closing div inside the block (#1620) * Fix open redirect (#1419) * Show more facets only if there are more facts to show (#1612) * Fix breakage in package groups page (#1594) * Fix broken links in RSS feed (#1589) * Activity Stream from: Organization Error group not found (#1519) * DataPusher and harvester collision (#1500) * Can't download resources with geojson extension (#1534) * Oversized Forgot Password button and field (#1508) * Invite to organization causes Internal Server error (#1505) v2.2 2014-02-04 =============== Note: This version does not require a requirements upgrade on source installations Note: This version requires a database upgrade Note: This version requires a Solr schema upgrade (The Solr schema file has been renamed, the schema file from the previous release is compatible with this version, but users are encouraged to point to the new one, see "API changes and deprecations") Major: * Brand new automatic importer of tabular data to the DataStore, the DataPusher. This is much more robust and simple to deploy and maintain than its predecessor (ckanext-datastorer). Whole new UI for re-importing data to the DataStore and view the import logs (#932, #938, #940, #981, #1196, #1200 ...) * Completely revamped file uploads that allow closer integration with resources and the DataStore, as well as making easir to integrate file uploads in other features. For example users can now upload images for organizations and groups. See "API changes and deprecations" if you are using the current FileStore. (#1273, #1173 ... ) * UI and API endpoints for resource reordering (#1277) * Backend support for organization hierarchy, allowing parent and children organizations. Frontend needs to be implemented in extensions (#1038) * User invitations: it is now possible to create new users with just their email address. An invite email is sent to them, allowing to change their user name and password (#1178) * Disable user registration with a configuration option (#1226) * Great effort in improving documentation, specially for customizing CKAN, with a complete tutorial for writing extensions and customizing the theme. User and sysadmin guides have also been moved to the main documentation (#943, #847, #1253) Minor: * Homepage modules to allow predefined layouts (#1126) * Ability to delete users (#1163) * Dedicated dataset groups page for displaying and managing them (#1102) * Implement organization_purge and group_purge action functions (#707) * Improve package_show performance (#1078) * Support internationalization of rendered dates and times (#1041) * Improve plugin load handling (#549) * Authorization function auditing for action functions (#1060) * Improve datetime rendering (#518) * New SQL indexes to improve performance (#1164) * Changes in requirements management (#1149) * Add offset/limit to package_list action (#1179) * Document all available configuration options (#848) * Make CKAN sqlalchemy 0.8.4 compatible (#1427) * UI labelling and cleanup (#1030) * Better UX for empty groups/orgs (#1094) * Improve performance of group_dictize when the group has a lot of packages (#1208) * Hide __extras from extras on package_show (#1218) * "Clear all" link within each facet block is unnecessary (#1263) * Term translations of organizations (#1274) * '--reset-db' option for when running tests (#1304) Bug fixes: * Fix plugins load/unload issues (#547) * Improve performance when new_activities not needed (#1013) * Resource preview breaks when CSV headers include percent sign (#1067) * Package index not rebuilt when resources deleted (#1081) * Don't accept invalid URLs in resource proxy (#1106) * UI language reset after account creation (#1429) * Catch non-integer facet limits (#1118) * Error when deleting custom tags (#1114) * Organization images do not display on Organization user dashboard page (#1127) * Can not reactivate a deleted dataset from the UI (#607) * Non-existent user profile should give error (#1068) * Recaptcha not working in CKAN 2.0 (jinja templates) (#1070) * Groups and organizations can be visited with interchangeable URLs (#1180) * Dataset Source (url) and Version fields missing (#1187) * Fix problems with private / public datasets and organizations (#1188) * group_show should never return private data (#1191) * When editing a dataset, the organization field is not set (#1199) * Fix resource_delete action (#1216) * Fix trash purge action redirect broken for CKAN instances not at / (#1217) * Title edit for existing dataset changes the URL (#1232) * 'facet.limit' in package_search wrongly handled (#1237) * h.SI_number_span doesn't close correctly (#1238) * CkanVersionException wrongly raised (#1241) * (group|organization)_member_create only accepts username (and not id) (#1243) * package_create uses the wrong parameter for organization (#1257) * ValueError for non-int limit and offset query params (#1258) * Visibility field value not kept if there are errors on the form (#1265) * package_list should not return private datasets (#1295) * Fix 404 on organization activity stream and about page (#1298) * Fix placeholder images broken on non-root locations (#1309) * "Add Dataset" button shown on org pages when not authorized (#1348) * Fix exception when visiting organization history page (#1359) * Fix search ordering on organization home page (#1368) * datastore_search_sql failing for some anonymous users (#1373) * related_list logic function throws a 503 without any parameters (#1384) * Disabling activity_streams borks editing groups and user (#1421) * Member Editing Fixes (#1454) * Bulk editing broken in IE7 (#1455) * Fix group deletion in IE7 (#1460) * And many, many more! API changes and deprecations: * The Solr schema file is now always named ``schema.xml`` regardless of the CKAN version. Old schema files have been kept for backwards compatibility but users are encouraged to point to the new unified one (#1314) * The FileStore and file uploads have been completely refactored and simplified to only support local storage backend. The links from previous versions of the FileStore to hosted files will still work, but there is a command available to migrate the files to new Filestore. See this page for more details: https://docs.ckan.org/en/latest/filestore.html#filestore-21-to-22-migration * By default, the authorization for any action defined from an extension will require a logged in user, otherwise a :py:class:`ckan.logic.NotAuthorized` exception will be raised. If an action function allows anonymous access (eg search, show status, etc) the ``auth_allow_anonymous_access`` decorator (available on the plugins toolkit) must be used (#1210) * ``package_search`` now returns results with custom schemas applied like ``package_show``, a ``use_default_schema`` parameter was added to request the old behaviour, this change may affect customized search result templates (#1255) * The ``ckan.api_url`` configuration option has been completely removed and it can no longer be used (#960) * The ``edit`` and ``after_update`` methods of IPackageController plugins are now called when updating a resource using the web frontend or the resource_update API action (#1052) * Dataset moderation has been deprecated, and the code will probably be removed in later CKAN versions (#1139) * Some front end libraries have been updated, this may affect existing custom themes: Bootstrap 2.0.3 > 2.3.2, Font Awesome 3.0.2 > 3.2.1, jQuery 1.7.2 > 1.10.2 (#1082) * SQLite is officially no longer supported as the tests backend Troubleshooting: * Exception on startup after upgrading from a previous CKAN version:: AttributeError: 'instancemethod' object has no attribute 'auth_audit_exempt' Make sure that you are not loading a 2.1-only plugin (eg ``datapusher-ext``) and update all the plugin in your configuration file to the latest stable version. * Exception on startup after upgrading from a previous CKAN version:: File "/usr/lib/ckan/default/src/ckan/ckan/lib/dictization/model_dictize.py", line 330, in package_dictize result_dict['metadata_modified'] = pkg.metadata_modified.isoformat() AttributeError: 'NoneType' object has no attribute 'isoformat' One of the database changes on this version is the addition of a ``metadata_modified`` field in the package table, that was filled during the DB migration process. If you have previously migrated the database and revert to an older CKAN version the migration process may have failed at this step, leaving the fields empty. Also make sure to restart running processes like harvesters after the update to make sure they use the new code base. v2.1.6 2015-12-17 ================= Note: This version requires a requirements upgrade on source installations Bug fixes: * Fix Markdown rendering issue * Return default error page on fanstatic errors * Prevent authentication when using API callbacks v2.1.5 2015-07-22 ================= Bug fixes: * Fix broken boolean validator (#2443) * Key error on resource proxy (#2425) * Ignore revision_id passed to resources (#2340) * Add reset for reset_key on successful password change (#2379) v2.1.4 2015-03-04 ================= Bug fixes: * Only link to http, https and ftp resource urls (#2085) * Avoid private and deleted datasets on stats plugin (#1936) * Fix tags count and group links in stats extension (#1649) * Make resource_create auth work against package_update (#2037) * Fix DataStore permissions check on startup (#1374) * Fix datastore docs link (#2044) * Fix resource extras getting lost on resource update (#2158) * Clean up field names before rendering the Recline table (#2319) * Don't "normalize" resource URL in recline view (#2324) * Don't assume resource format is there on text preview (#2320) v2.1.3 2014-10-15 ================= Bug fixes: * Organization image_url is not displayed in the dataset view. (#1934) * i18n: Close a tag in French translation in Markdown syntax link (#1919) * organization_list_for_user() fixes (#1918) * Incorrect link in Organization snippet on dataset page (#1882) * Prevent reading system tables on DataStore SQL search (#1871) * Ensure that the DataStore is running on legacy mode when using PostgreSQL < 9.x (#1879) * Edit user encoding error (#1436) * Able to list private datasets via the API (#1580) * Insecure content warning when running Recline under SSL (#1729) * Add quotes to package ID in Solr query in _bulk_update_dataset to prevent Solr errors with custom dataset IDs. (#1853) * Ordering a dataset listing loses the existing filters (#1791) * Inserting empty arrays in JSON type fields in datastore fails (#1776) * programmatically log user in after registration (#1721) * Deleted Users bug (#1668) * Catch NotFound error in resources page (#1685) * bulk_process page for non-existent organization throws Exception (#1682) * Default search ordering on organization home page is broken (#1368) * Term translations of organizations (#1274) * Preview fails on private datastore resources (#1221) * Strip whitespace from title in model dictize (#1228) v2.1.2 2014-02-04 ================= Bug fixes: * Fix context for group/about setup_template_variables (#1433) * Call setup_template_variables in group/org read, about and bulk_process (#1281) * Remove repeated sort code in package_search (#1461) * Ensure that check_access is called on activity_create (#1421) * Fix visibility validator (#1188) * Remove p.toolkit.auth_allow_anonymous_access as it is not available on 2.1.x (#1373) * Add organization_revision_list to avoid exception on org history page (#1359) * Fix activity and about organization pages (#1298) * Show 404 instead of login page on user not found (#1068) * Don't show Add Dataset button on org pages unless authorized (#1348) * Fix datastore_search_sql authorization function (#1373) * Fix extras deletion (#1449) * Better word breaking on long words (#1398) * Fix activity and about organization pages (#1298) * Remove limit of number of arguments passed to ``user add`` command. * Fix related_list logic function (#1384) * Avoid UnicodeEncodeError on feeds when params contains non ascii characters v2.1.1 2013-11-8 ================ Bug fixes: * Fix errors on preview on non-root locations (#960) * Fix place-holder images on non-root locations (#1309) * Don't accept invalid URLs in resource proxy (#1106) * Make sure came_from url is local (#1039) * Fix logout redirect in non-root locations (#1025) * Wrong auth checks for sysadmins on package_create (#1184) * Don't return private datasets on package_list (#1295) * Stop tracking failing when no lang/encoding headers (#1192) * Fix for paster db clean command getting frozen * Fix organization not set when editing a dataset (#1199) * Fix PDF previews (#1194) * Fix preview failing on private datastore resources (#1221) v2.1 2013-08-13 =============== Note: This version requires a requirements upgrade on source installations Note: This version requires a database upgrade Note: This version does not require a Solr schema upgrade .. note:: The ``json_preview`` plugin has been renamed to ``text_preview`` (see #266). If you are upgrading CKAN from a previous version you need to change the plugin name on your CKAN config file after upgrading to avoid a PluginNotFound exception. Major: * Bulk updates of datasets within organizations (delete, make public/private) (#278) * Organizations and Groups search (#303) * Generic text preview extension for JSON, XML and plain text files (#226) * Improve consistency of the Action API (#473) * IAuthenticator interface for plugging into authorization platforms (Work in progress) (#1007) * New clearer dashboard with more information easier to access (#626) * New ``rebuild_fast`` command to speed up reindex using multiple cores (#700) * Complete restructure of the documentation, with updated sections on installation, upgrading, release process, etc and guidelines on how to write new documentation (#769 and multiple others) Minor: * Add group members page to templates (#844) * Show search facets on organization page (#776) * Changed default sort ordering (#869) * More consistent display of buttons across pages (#890) * History page ported to new templates (#368) * More blocks to templates to allow further customization (#688) * Improve imports from lib.helpers (#262) * Add support for callback parameter on Action API (#414) * Create site_user at startup (#952) * Add warning before deleting an organization (#803) * Remove flags from language selector (#822) * Hide the Data API button when datastore is disabled (#752) * Pin all requirements and separate minimal requirements in a separate file (#491, #1149) * Better preview plugin selection (#1002) * Add new functions to the plugins toolkit (#1015) * Improve ExampleIDatasetFormPlugin (#2750) * Extend h.sorted_extras() to do substitutions and auto clean keys (#440) * Separate default database for development and testing (#517) * More descriptive Solr exceptions when indexing (#674) * Validate datastore input through schemas (#905) Bug fixes: * Fix 500 on password reset (#264) * Fix exception when indexing a wrong date on a _date field (#267) * Fix datastore permissions issues (#652) * Placeholder images are not linked with h.url_for_static (#948) * Explore dropdown menu is hidden behind other resources in IE (#915) * Buttons interrupt file uploading (#902) * Fix resource proxy encoding errors (#896) * Enable streaming in resource proxy (#989) * Fix cache_dir and beaker paths on deployment.ini_tmpl (#888) * Fix multiple issues on create dataset form on IE (#881) * Fix internal server error when adding member (#869) * Fix license faceting (#853) * Fix exception in dashboard (#830) * Fix Google Analytics integration (#827) * Fix ValueError when resource size is not an integer (#1009) * Catch NotFound on new resource when package does not exist (#1010) * Fix Celery configuration to allow overriding from config (#1027) * came_from after login is validated to not redidirect to another site (#1039) * And many, many more! Deprecated and removed: * The ``json_preview`` plugin has been replaced by a new ``text_preview`` one. Please update your config files if using it. (#226) Known issues: * Under certain authorization setups the frontend for the groups functionality may not work as expected (See #1176 #1175). v2.0.8 2015-12-17 ================= Note: This version requires a requirements upgrade on source installations Bug fixes: * Fix Markdown rendering issue * Return default error page on fanstatic errors * Prevent authentication when using API callbacks v2.0.7 2015-07-22 ================= Bug fixes: * Fix broken boolean validator (#2443) * Key error on resource proxy (#2425) * Ignore revision_id passed to resources (#2340) * Add reset for reset_key on successful password change (#2379) v2.0.6 2015-03-04 ================= Bug fixes: * Only link to http, https and ftp resource urls (#2085) * Avoid private and deleted datasets on stats plugin (#1936) * Fix tags count and group links in stats extension (#1649) * Make resource_create auth work against package_update (#2037) * Fix datastore docs link (#2044) * Fix resource extras getting lost on resource update (#2158) * Clean up field names before rendering the Recline table (#2319) * Don't "normalize" resource URL in recline view (#2324) * Don't assume resource format is there on text preview (#2320) v2.0.5 2014-10-15 ================= Bug fixes: * organization_list_for_user() fixes (#1918) * Incorrect link in Organization snippet on dataset page (#1882) * Prevent reading system tables on DataStore SQL search (#1871) * Ensure that the DataStore is running on legacy mode when using PostgreSQL < 9.x (#1879) * Current date indexed on empty "\*_date" fields (#1701) * Able to list private datasets via the API (#1580) * Insecure content warning when running Recline under SSL (#1729) * Inserting empty arrays in JSON type fields in datastore fails (#1776) * Deleted Users bug (#1668) v2.0.4 2014-02-04 ================= Bug fixes: * Fix extras deletion (#1449) * Better word breaking on long words (#1398) * Fix activity and about organization pages (#1298) * Show 404 instead of login page on user not found (#1068) * Remove limit of number of arguments passed to ``user add`` command. * Fix related_list logic function (#1384) v2.0.3 2013-11-8 ================ Bug fixes: * Fix errors on preview on non-root locations (#960) * Don't accept invalid URLs in resource proxy (#1106) * Make sure came_from url is local (#1039) * Fix logout redirect in non-root locations (#1025) * Don't return private datasets on package_list (#1295) * Stop tracking failing when no lang/encoding headers (#1192) * Fix for paster db clean command getting frozen v2.0.2 2013-08-13 ================= Bug fixes: * Fix markdown in group descriptions (#303) * Fix resource proxy encoding errors (#896) * Fix datastore exception on first run (#907) * Enable streaming in resource proxy (#989) * Fix in user search (#1024) * Fix Celery configuration to allow overriding from config (#1027) * Undefined function on organizations controller (#1036) * Fix license not translated in orgs/groups (#1040) * Fix link to documentation from the footer (#1062) * Fix missing close breadcrumb tag in org templates (#1071) * Fix recently_changed_packages_activity_stream function (#1159) * Fix Recline map sidebar not showing in IE 7-8 (#1133) v2.0.1 2013-06-11 ================= Bug fixes: * Use IDatasetForm schema for resource_update (#897) * Fixes for CKAN being run on a non-root URL (#948, #913) * Fix resource edit errors losing info (#580) * Fix Czech translation (#900) * Allow JSON filters for datastore_search on GET requests (#917) * Install vdm from the Python Package Index (#764) * Allow extra parameters on Solr queries (#739) * Create site user at startup if it does not exist (#952) * Fix modal popups positioning (#828) * Fix wrong redirect on dataset form on IE (#963) v2.0 2013-05-10 =============== .. note:: Starting on v2.0, issue numbers with four digits refer to the old ticketing system at https://trac.ckan.org and the ones with three digits refer to GitHub issues. For example: * #3020 is https://trac.ckan.org/ticket/3020 * #271 is https://github.com/ckan/ckan/issues/271 Some GitHub issues URLs will redirect to GitHub pull request pages. .. note:: v2.0 is a huge release so the changes listed here are just the highlights. Bug fixes are not listed. Note: This version requires a requirements upgrade on source installations Note: This version requires a database upgrade Note: This version requires a Solr schema upgrade Organizations based authorization (see :doc:`/maintaining/authorization`): CKAN's new "organizations" feature replaces the old authorization system with a new one based on publisher organizations. It replaces the "Publisher Profile and Workflow" feature from CKAN 1.X, any instances relying on it will need to be updated. * New organization-based authorization and organization of datasets * Supports private datasets * Publisher workflow * New authorization ini file options New frontend (see :doc:`/theming/index`): CKAN's frontend has been completely redesigned, inside and out. There is a new default theme and the template engine has moved from Genshi to Jinja2. Any custom templates using Genshi will need to be updated, although there is a ``ckan.legacy_templates`` setting to aid in the migration. * Block-based template inheritance * Custom jinja tags: {% ckan_extends %}, {% snippet %} and {% url_for %} (#2502, #2503) * CSS "primer" page for theme developers * We're now using LESS for CSS * Scalable font icons (#2563) * Social sharing buttons (google plus, facebook, twitter) (this replaces the ckanext-social extension) * Three-stage dataset creation form (#2501) * New `paster front-end-build` command does everything needed to build the frontend for a production CKAN site (runs `paster less` to compile the css files, `paster minify` to minify the css and js files, etc.) Plugins & Extensions: * New plugins toolkit provides a stable set of utility and helper functions for CKAN plugins to depend on. * The IDatasetForm plugin interface has been redesigned (note: this breaks backwards-compatibility with existing IDatasetForm plugins) (#649) * Many IDatasetForm bugs were fixed * New example extensions in core, and better documentation for the relevant plugin interfaces: example_itemplatehelpers (#447), example_idatasetform (#2750), hopefully more to come in 2.1! * New IFacets interface that allows to modify the facets shown on various pages. (#400) * The get_action() function now automatically adds 'model' and 'session' to the context dict (this saves on boiler-plate code, and means plugins don't have to import ckan.model in order to call get_action()) (#172) Activity Streams, Following & User Dashboard: * New visual design for activity streams (#2941) * Group activity streams now include activities for changes to any of the group's datasets (#1664) * Group activity streams now appear on group pages (previously they could only be retrieved via the api) * Dataset activity streams now appear on dataset pages (previously they could only be retrieved via the api) (#3024) * Users can now follow groups (previously you could only follow users or datasets) (#3005) * Activity streams and following are also supported for organizations (#505) * When you're logged into CKAN, you now get a notifications count in the top-right corner of the site, telling you how many new notifications you have on your dashboard. Clicking on the count takes you to your dashboard page to view your notifications. (#3009) * Optionally, you can also receive notifications by email when you have new activities on your dashboard (#1635) * Infinite scrolling of activity streams (if you scroll to the bottom of a an activity stream, CKAN will automatically load more activities) (#3018) * Redesigned user dashboard (#3028): - New dropdown-menu enables you to filter you dashboard activity stream to show only activities from a particular user, dataset, group or organization that you're following - New sidebar shows previews and unfollow buttons (when the activity stream is filtered) * New :ref:`ckan.activity_streams_enabled` config file setting allows you to disable the generation of activity streams (#654) Data Preview: * PDF files preview (#2203) * JSON files preview * HTML pages preview (in an iframe) (#2888) * New plugin extension point that allows plugins to add custom data previews for different data types (#2961) * Improved Recline Data Explorer previews (CSV files, Excel files..) * Plain text files preview API: * The Action API is now CKAN's default API, and the API documentation has been rewritten (#357) Other highlights: * CKAN now has continuous integration testing at https://travis-ci.org/ckan/ckan/ * Dataset pages now have . Main highlights: * Package Resource object (multiple download urls per package): each package can have multiple 'resources' (urls) with each resource having additional metadata such as format, description and hash (#88, #89, #229) * "Full-text" searching of packages (#187) * Semantic web integration: RDFization of all data plus integration with an online RDF store (e.g. for http://www.ckan.net/ at http://semantic.ckan.net/ or Talis store) (#90 #163) * Package ratings (#77 #194) * i18n: we now have translations into German and French with deployments at http://de.ckan.net/ and http://fr.ckan.net/ (#202) * Package diffs available in package history (#173) * Minor: * Package undelete (#21, #126) * Automated CKAN deployment via Fabric (#213) * Listings are sorted alphabetically (#195) * Add extras to rest api and to ckanclient (#158 #166) * Infrastructural: * Change to UUIDs for revisions and all domain objects * Improved search performance and better pagination * Significantly improved performance in API and WUI via judicious caching v0.10 2009-09-30 ================ * Switch to repoze.who for authentication (#64) * Explicit User object and improved user account UI with recent edits etc (#111, #66, #67) * Generic Attributes for Packages (#43) * Use sqlalchemy-migrate to handle db/model upgrades (#94) * "Groups" of packages (#105, #110, #130, #121, #123, #131) * Package search in the REST API (#108) * Full role-based access control for Packages and Groups (#93, #116, #114, #115, #117, #122, #120) * New CKAN logo (#72) * Infrastructural: * Upgrade to Pylons 0.9.7 (#71) * Convert to use formalchemy for all forms (#76) * Use paginate in webhelpers (#118) * Minor: * Add author and maintainer attributes to package (#91) * Change package state in the WUI (delete and undelete) (#126) * Ensure non-active packages don't show up (#119) * Change tags to contain any character (other than space) (#62) * Add Is It Open links to package pages (#74) v0.9 2009-07-31 =============== * (DM!) Add version attribute for package * Fix purge to use new version of vdm (0.4) * Link to changed packages when listing revision * Show most recently registered or updated packages on front page * Bookmarklet to enable easy package registration on CKAN * Usability improvements (package search and creation on front page) * Use external list of licenses from license repository * Convert from py.test to nosetests v0.8 2009-04-10 =============== * View information about package history (ticket:53) * Basic datapkg integration (ticket:57) * Show information about package openness using icons (ticket:56) * One-stage package create/registration (r437) * Reinstate package attribute validation (r437) * Upgrade to vdm 0.4 v0.7 2008-10-31 =============== * Convert to use SQLAlchemy and vdm v0.3 (v. major) * Atom/RSS feed for Recent Changes * Package search via name and title * Tag lists show number of associated packages v0.6 2008-07-08 =============== * Autocompletion (+ suggestion) of tags when adding tags to a package. * Paginated lists for packages, tags, and revisions. * RESTful machine API for package access, update, listing and creation. * API Keys for users who wish to modify information via the REST API. * Update to vdm v0.2 (SQLObject) which fixes ordering of lists. * Better immunity to SQL injection attacks. v0.5 2008-01-22 =============== * Purging of a Revision and associated changes from cli and wui (ticket:37) * Make data available in machine-usable form via sql dump (ticket:38) * Upgrade to Pylons 0.9.6.* and deploy (ticket:41) * List and search tags (ticket:33) * (bugfix) Manage reserved html characters in urls (ticket:40) * New spam management utilities including (partial) blacklist support v0.4 2007-07-04 =============== * Preview support when editing a package (ticket:36). * Correctly list IP address of of not logged in users (ticket:35). * Improve read action for revision to list details of changed items (r179). * Sort out deployment using modpython. v0.3 2007-04-12 =============== * System now in a suitable state for production deployment as a beta * Domain model versioning via the vdm package (currently released separately) * Basic Recent Changes listing log messages * User authentication (login/logout) via open ID * License page * Myriad of small fixes and improvements v0.2 2007-02 ============ * Complete rewrite of ckan to use pylons web framework * Support for full CRUD on packages and tags * No support for users (authentication) * No versioning of domain model objects v0.1 2006-05 ============ NB: not an official release * Almost functional system with support for persons, packages * Tag support only half-functional (tags are per package not global) * Limited release and file support