Validator functions reference

Validators in CKAN are user-defined functions that serves two purposes:

• Ensuring that the input satisfies certain requirements

• Converting the input to an expected form

Validators can be defined as a function that accepts one, two or four arguments. But this is an implementation detail and validators should not be called directly. Instead, the ckan.plugins.toolkit.navl_validate() function must be used whenever input requires validation.

import ckan.plugins.toolkit as tk
from ckanext.my_ext.validators import is_valid

data, errors = tk.navl_validate(
    {"input": "value"},
    {"input": [is_valid]},
)

And in order to be more flexible and allow overrides, don’t import validator functions directly. Instead, register them via the IValidators interface and use the ckan.plugins.tookit.get_validator() function:

import ckan.plugins as p
import ckan.plugins.toolkit as tk

def is_valid(value):
    return value

class MyPlugin(p.SingletonPlugin)
    p.implements(p.IValidators)

    def get_validators(self):
        return {"is_valid": is_valid}

...
# somewhere in code
data, errors = tk.navl_validate(
    {"input": "value"},
    {"input": [tk.get_validator("is_valid")]},
)

As you should have already noticed, navl_validate requires two parameters and additionally accepts an optional one. That’s their purpose:

1. Data that requires validation. Must be a dict object, with keys being the names of the fields.

2. The validation schema. It’s a mapping of field names to the lists of validators for that particular field.

3. Optional context. Contains any extra details that can change validation workflow in special cases. For the simplicity sake, we are not going to use context in this section, and in general is best not to rely on context variables inside validators.

Let’s imagine an input that contains two fields first and second. The first field must be an integer and must be provided, while the second field is an optional string. If we have following four validators:

• is_integer

• is_string

• is_required

• is_optional

we can validate data in the following way:

input = {"first": "123"}
schema = {
    "first": [is_required, is_integer],
    "second": [is_optional, is_string],
}

data, errors = tk.navl_validate(input, schema)

If the input is valid, data contains validated input and errors is an empty dictionary. Otherwise, errors contains all the validation errors for the provided input.

Built-in validators

ckan.lib.navl.validators.keep_extras(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → None

Convert dictionary into simple fields.

data, errors = tk.navl_validate(
    {"input": {"hello": 1, "world": 2}},
    {"input": [keep_extras]}
)
assert data == {"hello": 1, "world": 2}

ckan.lib.navl.validators.not_missing(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → None

Ensure value is not missing from the input, but may be empty.

data, errors = tk.navl_validate(
    {},
    {"hello": [not_missing]}
)
assert errors == {"hello": [error_message]}

ckan.lib.navl.validators.not_empty(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → None

Ensure value is available in the input and is not empty.

data, errors = tk.navl_validate(
    {"hello": None},
    {"hello": [not_empty]}
)
assert errors == {"hello": [error_message]}

ckan.lib.navl.validators.if_empty_same_as(other_key: str) → Callable[[...], Any]

Copy value from other field when current field is missing or empty.

data, errors = tk.navl_validate(
    {"hello": 1},
    {"hello": [], "world": [if_empty_same_as("hello")]}
)
assert data == {"hello": 1, "world": 1}

ckan.lib.navl.validators.both_not_empty(other_key: str) → Callable[[Any], Any] | Callable[[Any, Context], Any] | Callable[[tuple[Any, ...], dict[tuple[Any, ...], Any], dict[tuple[Any, ...], list[str]], Context], None]

Ensure that both, current value and other field has value.

data, errors = tk.navl_validate(
    {"hello": 1},
    {"hello": [], "world": [both_not_empty("hello")]}
)
assert errors == {"world": [error_message]}

data, errors = tk.navl_validate(
    {"world": 1},
    {"hello": [], "world": [both_not_empty("hello")]}
)
assert errors == {"world": [error_message]}

data, errors = tk.navl_validate(
    {"hello": 1, "world": 2},
    {"hello": [], "world": [both_not_empty("hello")]}
)
assert not errors

ckan.lib.navl.validators.empty(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → None

Ensure that value is not present in the input.

data, errors = tk.navl_validate(
    {"hello": 1},
    {"hello": [empty]}
)
assert errors == {"hello": [error_message]}

ckan.lib.navl.validators.ignore(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → NoReturn

Remove the value from the input and skip the rest of validators.

data, errors = tk.navl_validate(
    {"hello": 1},
    {"hello": [ignore]}
)
assert data == {}

ckan.lib.navl.validators.default(default_value: Any) → Callable[[Any], Any] | Callable[[Any, Context], Any] | Callable[[tuple[Any, ...], dict[tuple[Any, ...], Any], dict[tuple[Any, ...], list[str]], Context], None]

Convert missing or empty value to the default one.

data, errors = tk.navl_validate(
    {},
    {"hello": [default("not empty")]}
)
assert data == {"hello": "not empty"}

ckan.lib.navl.validators.configured_default(config_name: str, default_value_if_not_configured: Any) → Callable[[Any], Any] | Callable[[Any, Context], Any] | Callable[[tuple[Any, ...], dict[tuple[Any, ...], Any], dict[tuple[Any, ...], list[str]], Context], None]

When key is missing or value is an empty string or None, replace it with a default value from config, or if that isn’t set from the default_value_if_not_configured.

ckan.lib.navl.validators.ignore_missing(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → None

If the key is missing from the data, ignore the rest of the key’s schema.

By putting ignore_missing at the start of the schema list for a key, you can allow users to post a dict without the key and the dict will pass validation. But if they post a dict that does contain the key, then any validators after ignore_missing in the key’s schema list will be applied.

Raises:

ckan.lib.navl.dictization_functions.StopOnError – if data[key] is ckan.lib.navl.dictization_functions.missing or None

Returns:

None

ckan.lib.navl.validators.ignore_empty(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → None

Skip the rest of validators if the value is empty or missing.

data, errors = tk.navl_validate(
    {"hello": ""},
    {"hello": [ignore_empty, isodate]}
)
assert data == {}
assert not errors

ckan.lib.navl.validators.convert_int(value: Any) → int

Ensure that the value is a valid integer.

data, errors = tk.navl_validate(
    {"hello": "world"},
    {"hello": [convert_int]}
)
assert errors == {"hello": [error_message]}

ckan.lib.navl.validators.unicode_only(value: Any) → str

Accept only unicode values

data, errors = tk.navl_validate(
    {"hello": 1},
    {"hello": [unicode_only]}
)
assert errors == {"hello": [error_message]}

ckan.lib.navl.validators.unicode_safe(value: Any) → str

Make sure value passed is treated as unicode, but don’t raise an error if it’s not, just make a reasonable attempt to convert other types passed.

This validator is a safer alternative to the old ckan idiom of using the unicode() function as a validator. It tries not to pollute values with Python repr garbage e.g. when passed a list of strings (uses json format instead). It also converts binary strings assuming either UTF-8 or CP1252 encodings (not ASCII, with occasional decoding errors)

ckan.lib.navl.validators.limit_to_configured_maximum(config_option: str, default_limit: int) → Callable[[Any], Any] | Callable[[Any, Context], Any] | Callable[[tuple[Any, ...], dict[tuple[Any, ...], Any], dict[tuple[Any, ...], list[str]], Context], None]

If the value is over a limit, it changes it to the limit. The limit is defined by a configuration option, or if that is not set, a given int default_limit.

ckan.logic.validators.owner_org_validator(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → Any

Validate organization for the dataset.

Depending on the settings and user’s permissions, this validator checks whether organization is optional and ensures that specified organization can be set as an owner of dataset.

ckan.logic.validators.package_id_not_changed(value: Any, context: Context) → Any

Ensures that package’s ID is not changed during the update.

ckan.logic.validators.int_validator(value: Any, context: Context) → Any

Return an integer for value, which may be a string in base 10 or a numeric type (e.g. int, long, float, Decimal, Fraction). Return None for None or empty/all-whitespace string values.

Raises:

ckan.lib.navl.dictization_functions.Invalid for other inputs or non-whole values

ckan.logic.validators.natural_number_validator(value: Any, context: Context) → Any

Ensures that the value is non-negative integer.

ckan.logic.validators.is_positive_integer(value: Any, context: Context) → Any

Ensures that the value is an integer that is greater than zero.

ckan.logic.validators.datetime_from_timestamp_validator(value: Any, context: Context) → Any

ckan.logic.validators.boolean_validator(value: Any, context: Context) → Any

Return a boolean for value. Return value when value is a python bool type. Return True for strings ‘true’, ‘yes’, ‘t’, ‘y’, and ‘1’. Return False in all other cases, including when value is an empty string or None

ckan.logic.validators.isodate(value: Any, context: Context) → Any

Convert the value into datetime.datetime object.

ckan.logic.validators.package_id_exists(value: str, context: Context) → Any

Ensures that the value is an existing package’s ID or name.

ckan.logic.validators.package_id_does_not_exist(value: str, context: Context) → Any

Ensures that the value is not used as a package’s ID or name.

ckan.logic.validators.resource_id_does_not_exist(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → Any

ckan.logic.validators.group_id_does_not_exist(value: str, context: Context) → Any

Ensures that the value is not used as a ID or name.

ckan.logic.validators.user_id_does_not_exist(value: str, context: Context) → Any

Ensures that the value is not used as a ID or name.

ckan.logic.validators.resource_view_id_does_not_exist(value: str, context: Context) → Any

Ensures that the value is not used as a ID or name.

ckan.logic.validators.package_name_exists(value: str, context: Context) → Any

Ensures that the value is an existing package’s name.

ckan.logic.validators.package_id_or_name_exists(package_id_or_name: str, context: Context) → Any

Return the given package_id_or_name if such a package exists.

Raises:

ckan.lib.navl.dictization_functions.Invalid if there is no package with the given id or name

ckan.logic.validators.resource_id_exists(value: Any, context: Context) → Any

Ensures that the value is not used as a resource’s ID or name.

ckan.logic.validators.uuid_validator(value: Any) → Any

ckan.logic.validators.user_id_exists(user_id: str, context: Context) → Any

Raises Invalid if the given user_id does not exist in the model given in the context, otherwise returns the given user_id.

ckan.logic.validators.user_id_or_name_exists(user_id_or_name: str, context: Context) → Any

Return the given user_id_or_name if such a user exists.

Raises:

ckan.lib.navl.dictization_functions.Invalid if no user can be found with the given id or user name

ckan.logic.validators.group_id_exists(group_id: str, context: Context) → Any

Raises Invalid if the given group_id does not exist in the model given in the context, otherwise returns the given group_id.

ckan.logic.validators.group_id_or_name_exists(reference: str, context: Context) → Any

Raises Invalid if a group identified by the name or id cannot be found.

ckan.logic.validators.name_validator(value: Any, context: Context) → Any

Return the given value if it’s a valid name, otherwise raise Invalid.

If it’s a valid name, the given value will be returned unmodified.

This function applies general validation rules for names of packages, groups, users, etc.

Most schemas also have their own custom name validator function to apply custom validation rules after this function, for example a package_name_validator() to check that no package with the given name already exists.

Raises:

ckan.lib.navl.dictization_functions.Invalid – if value is not a valid name

ckan.logic.validators.package_name_validator(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → Any

Ensures that value can be used as a package’s name

ckan.logic.validators.package_version_validator(value: Any, context: Context) → Any

Ensures that value can be used as a package’s version

ckan.logic.validators.duplicate_extras_key(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → Any

Ensures that there are no duplicated extras.

ckan.logic.validators.group_name_validator(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → Any

Ensures that value can be used as a group’s name

ckan.logic.validators.tag_length_validator(value: Any, context: Context) → Any

Ensures that tag length is in the acceptable range.

ckan.logic.validators.tag_name_validator(value: Any, context: Context) → Any

Ensures that tag does not contain wrong characters

ckan.logic.validators.tag_not_uppercase(value: Any, context: Context) → Any

Ensures that tag is lower-cased.

ckan.logic.validators.tag_string_convert(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → Any

Takes a list of tags that is a comma-separated string (in data[key]) and parses tag names. These are added to the data dict, enumerated. They are also validated.

ckan.logic.validators.ignore_not_package_admin(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → Any

Ignore if the user is not allowed to administer the package specified.

ckan.logic.validators.ignore_not_sysadmin(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → Any

Ignore the field if user not sysadmin or ignore_auth in context.

ckan.logic.validators.limit_sysadmin_update(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → Any

Should not be able to modify your own sysadmin privs, or the system user’s

ckan.logic.validators.ignore_not_group_admin(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → Any

Ignore if the user is not allowed to administer for the group specified.

ckan.logic.validators.user_name_validator(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → Any

Validate a new user name.

Append an error message to errors[key] if a user named data[key] already exists. Otherwise, do nothing.

Raises:

ckan.lib.navl.dictization_functions.Invalid – if data[key] is not a string

Return type:

None

ckan.logic.validators.user_both_passwords_entered(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → Any

Ensures that both password and password confirmation is not empty

ckan.logic.validators.user_password_validator(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → Any

Ensures that password is safe enough.

ckan.logic.validators.user_passwords_match(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → Any

Ensures that password and password confirmation match.

ckan.logic.validators.user_password_not_empty(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → Any

Only check if password is present if the user is created via action API. If not, user_both_passwords_entered will handle the validation

ckan.logic.validators.user_about_validator(value: Any, context: Context) → Any

Ensures that user’s about field does not contains links.

ckan.logic.validators.vocabulary_name_validator(name: str, context: Context) → Any

Ensures that the value can be used as a tag vocabulary name.

ckan.logic.validators.vocabulary_id_not_changed(value: Any, context: Context) → Any

Ensures that vocabulary ID is not changed during the update.

ckan.logic.validators.vocabulary_id_exists(value: Any, context: Context) → Any

Ensures that value contains existing vocabulary’s ID or name.

ckan.logic.validators.tag_in_vocabulary_validator(value: Any, context: Context) → Any

Ensures that the tag belongs to the vocabulary.

ckan.logic.validators.tag_not_in_vocabulary(key: tuple[Any, ...], tag_dict: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → Any

Ensures that the tag does not belong to the vocabulary.

ckan.logic.validators.url_validator(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → Any

Checks that the provided value (if it is present) is a valid URL

ckan.logic.validators.user_name_exists(user_name: str, context: Context) → Any

Ensures that user’s name exists.

ckan.logic.validators.role_exists(role: str, context: Context) → Any

Ensures that value is an existing CKAN Role name.

ckan.logic.validators.datasets_with_no_organization_cannot_be_private(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → Any

ckan.logic.validators.list_of_strings(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → Any

Ensures that value is a list of strings.

ckan.logic.validators.if_empty_guess_format(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → Any

Make an attempt to guess resource’s format using URL.

ckan.logic.validators.clean_format(format: str)

Normalize resource’s format.

ckan.logic.validators.no_loops_in_hierarchy(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → Any

Checks that the parent groups specified in the data would not cause a loop in the group hierarchy, and therefore cause the recursion up/down the hierarchy to get into an infinite loop.

ckan.logic.validators.filter_fields_and_values_should_have_same_length(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → Any

ckan.logic.validators.filter_fields_and_values_exist_and_are_valid(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → Any

ckan.logic.validators.extra_key_not_in_root_schema(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → Any

Ensures that extras are not duplicating base fields

ckan.logic.validators.empty_if_not_sysadmin(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → Any

Only sysadmins may pass this value

ckan.logic.validators.strip_value(value: str)

Trims the Whitespace

ckan.logic.validators.email_validator(value: Any, context: Context) → Any

Validate email input

ckan.logic.validators.collect_prefix_validate(prefix: str, *validator_names: str) → Callable[[Any], Any] | Callable[[Any, Context], Any] | Callable[[tuple[Any, ...], dict[tuple[Any, ...], Any], dict[tuple[Any, ...], list[str]], Context], None]

Return a validator that will collect top-level keys starting with prefix then apply validator_names to each one. Results are moved to a dict under the prefix name, with prefix removed from keys

ckan.logic.validators.dict_only(value: Any) → dict[Any, Any]

Ensures that the value is a dictionary

ckan.logic.validators.email_is_unique(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → Any

Validate email is unique

ckan.logic.validators.one_of(list_of_value: Container[Any]) → Callable[[Any], Any] | Callable[[Any, Context], Any] | Callable[[tuple[Any, ...], dict[tuple[Any, ...], Any], dict[tuple[Any, ...], list[str]], Context], None]

Checks if the provided value is present in a list or is an empty string

ckan.logic.validators.json_object(value: Any) → Any

Make sure value can be serialized as a JSON object

ckan.logic.validators.extras_valid_json(extras: Any, context: Context) → Any

Ensures that every item in the value dictionary is JSON-serializable.

ckan.logic.converters.convert_to_extras(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → Any

Convert given field into an extra field.

ckan.logic.converters.convert_from_extras(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → Any

Restore field using object’s extras.

ckan.logic.converters.extras_unicode_convert(extras: dict[tuple[Any, ...], Any], context: Context)

Convert every value of the dictionary into string.

ckan.logic.converters.free_tags_only(key: tuple[Any, ...], data: dict[tuple[Any, ...], Any], errors: dict[tuple[Any, ...], list[str]], context: Context) → Any

Ensure that none of the tags belong to a vocabulary.

ckan.logic.converters.convert_to_tags(vocab: Any) → Callable[[tuple[Any, ...], dict[tuple[Any, ...], Any], dict[tuple[Any, ...], list[str]], Context], None]

Convert list of tag names into a list of tag dictionaries

ckan.logic.converters.convert_from_tags(vocab: Any) → Callable[[tuple[Any, ...], dict[tuple[Any, ...], Any], dict[tuple[Any, ...], list[str]], Context], None]

ckan.logic.converters.convert_user_name_or_id_to_id(user_name_or_id: Any, context: Context) → Any

Return the user id for the given user name or id.

The point of this function is to convert user names to ids. If you have something that may be a user name or a user id you can pass it into this function and get the user id out either way.

Also validates that a user with the given name or id exists.

Returns:

the id of the user with the given user name or id

Return type:

string

Raises:

ckan.lib.navl.dictization_functions.Invalid if no user can be found with the given id or user name

ckan.logic.converters.convert_package_name_or_id_to_id(package_name_or_id: Any, context: Context) → Any

Return the package id for the given package name or id.

The point of this function is to convert package names to ids. If you have something that may be a package name or id you can pass it into this function and get the id out either way.

Also validates that a package with the given name or id exists.

Returns:

the id of the package with the given name or id

Return type:

string

Raises:

ckan.lib.navl.dictization_functions.Invalid if there is no package with the given name or id

ckan.logic.converters.convert_group_name_or_id_to_id(group_name_or_id: Any, context: Context) → Any

Return the group id for the given group name or id.

The point of this function is to convert group names to ids. If you have something that may be a group name or id you can pass it into this function and get the id out either way.

Also validates that a group with the given name or id exists.

Returns:

the id of the group with the given name or id

Return type:

string

Raises:

ckan.lib.navl.dictization_functions.Invalid if there is no group with the given name or id

ckan.logic.converters.convert_to_json_if_string(value: Any, context: Context) → Any

Parse string value as a JSON object.

ckan.logic.converters.as_list(value: Any)

Convert whitespace separated string into a list of strings.

ckan.logic.converters.convert_to_list_if_string(value: Any) → Any

Transform string into one-item list

ckan.logic.converters.json_or_string(value: Any) → Any

parse string values as json, return string if that fails

ckan.logic.converters.json_list_or_string(value: Any) → Any

parse string values as json or comma-separated lists, return string as a one-element list if that fails

ckan.logic.converters.remove_whitespace(value: Any, context: Context) → Any

Trim whitespaces from the value.