I followed the CKAN official installation documentation here which installs ckan 2.9.10 with docker compose. But it fails to run the ckan container which is one of the containers built.
The building returned error after error for various packages from the file requirements-pyv2.txt (e.g., packages webob,zope.interface, repoze.who-friendlyform, lxml ...). For now I managed to fix them by removing the versions specified.
The final blocker I have, is that the ckan container is not running and exited.
Below is the command line and final error (to keep it short) and right at the end of my post you have the full output.
~/github/ckan/contrib/docker$ docker compose up
ckan | from collections import Mapping
ckan | ImportError: cannot import name 'Mapping' from 'collections' (/usr/lib/python3.11/collections/__init__.py)
ckan exited with code 1
How could I solve this please? Note that I'm completely new with CKAN and my experience in web development is also very limited and I'm learning all that with this CKAN project and any help/explanation is GREATLY appreciated. Thanks!
[+] Running 11/11_solr_data" Created 0.0s
✔ Network docker_default Created 0.0s
✔ Volume "docker_ckan_storage" Created 0.0s
✔ Volume "docker_pg_data" Created 0.0s
✔ Volume "docker_solr_data" Created 0.0s
✔ Volume "docker_ckan_config" Created 0.0s
✔ Volume "docker_ckan_home" Created 0.0s
✔ Container solr Created 1.0s
✔ Container db Created 1.0s
✔ Container datapusher Created 1.0s
✔ Container redis Created 1.0s
✔ Container ckan Created 2.0s
Attaching to ckan, datapusher, db, redis, solr
solr | Starting Solr 6.6.5
redis | 1:C 06 Feb 2024 14:09:12.054 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis | 1:C 06 Feb 2024 14:09:12.055 * Redis version=7.2.4, bits=64, commit=00000000, modified=0, pid=1, just started
redis | 1:C 06 Feb 2024 14:09:12.055 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis | 1:M 06 Feb 2024 14:09:12.055 * monotonic clock: POSIX clock_gettime
redis | 1:M 06 Feb 2024 14:09:12.055 * Running mode=standalone, port=6379.
redis | 1:M 06 Feb 2024 14:09:12.056 * Server initialized
redis | 1:M 06 Feb 2024 14:09:12.056 * Ready to accept connections tcp
db | The files belonging to this database system will be owned by user "postgres".
db | This user must also own the server process.
db |
db | The database cluster will be initialized with locale "en_US.utf8".
db | The default database encoding has accordingly been set to "UTF8".
db | The default text search configuration will be set to "english".
db |
db | Data page checksums are disabled.
db |
db | fixing permissions on existing directory /var/lib/postgresql/data/db ... ok
db | creating subdirectories ... ok
db | selecting default max_connections ... 100
db | selecting default shared_buffers ... 128MB
db | selecting dynamic shared memory implementation ... posix
ckan | db:5432 - no response
db | creating configuration files ... ok
db | running bootstrap script ... ok
solr | 2024-02-06 14:09:12.991 INFO (main) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
db | performing post-bootstrap initialization ... ok
solr | 2024-02-06 14:09:13.211 INFO (main) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.6.5
solr | 2024-02-06 14:09:13.212 INFO (main) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port 8983
solr | 2024-02-06 14:09:13.212 INFO (main) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: /opt/solr
db | syncing data to disk ... ok
db |
db | Success. You can now start the database server using:
db |
db | pg_ctl -D /var/lib/postgresql/data/db -l logfile start
db |
db |
db | WARNING: enabling "trust" authentication for local connections
db | You can change this by editing pg_hba.conf or using the option -A, or
db | --auth-local and --auth-host, the next time you run initdb.
solr | 2024-02-06 14:09:13.220 INFO (main) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2024-02-06T14:09:13.212Z
solr | 2024-02-06 14:09:13.229 INFO (main) [ ] o.a.s.c.SolrResourceLoader Using system property solr.solr.home: /opt/solr/server/solr
solr | 2024-02-06 14:09:13.232 INFO (main) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /opt/solr/server/solr/solr.xml
db | waiting for server to start....2024-02-06 14:09:13.234 UTC [44] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db | 2024-02-06 14:09:13.247 UTC [45] LOG: database system was shut down at 2024-02-06 14:09:12 UTC
db | 2024-02-06 14:09:13.251 UTC [44] LOG: database system is ready to accept connections
solr | 2024-02-06 14:09:13.297 INFO (main) [ ] o.a.s.c.SolrResourceLoader [null] Added 0 libs to classloader, from paths: []
db | done
db | server started
solr | 2024-02-06 14:09:13.512 INFO (main) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=600000&connTimeout=60000&retry=true
ckan | db:5432 - no response
solr | 2024-02-06 14:09:13.592 INFO (main) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /opt/solr/server/solr
solr | 2024-02-06 14:09:13.593 INFO (main) [ ] o.a.s.c.CorePropertiesLocator Cores are: [ckan]
solr | 2024-02-06 14:09:13.597 INFO (coreLoadExecutor-6-thread-1) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
db | CREATE DATABASE
db |
db |
db | /usr/local/bin/docker-entrypoint.sh: sourcing /docker-entrypoint-initdb.d/00_create_datastore.sh
solr | 2024-02-06 14:09:13.637 INFO (coreLoadExecutor-6-thread-1) [ x:ckan] o.a.s.c.SolrResourceLoader [ckan] Added 53 libs to classloader, from paths: [/opt/solr/contrib/clustering/lib, /opt/solr/contrib/extraction/lib, /opt/solr/contrib/langid/lib, /opt/solr/contrib/velocity/lib, /opt/solr/dist]
solr | 2024-02-06 14:09:13.638 INFO (main) [ ] o.e.j.s.Server Started @981ms
db | CREATE ROLE
solr | 2024-02-06 14:09:13.673 INFO (coreLoadExecutor-6-thread-1) [ x:ckan] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.0.0
solr | 2024-02-06 14:09:13.738 INFO (coreLoadExecutor-6-thread-1) [ x:ckan] o.a.s.s.ManagedIndexSchemaFactory The schema is configured as managed, but managed schema resource managed-schema not found - loading non-managed schema schema.xml instead
solr | 2024-02-06 14:09:13.761 INFO (coreLoadExecutor-6-thread-1) [ x:ckan] o.a.s.s.IndexSchema [ckan] Schema name=ckan
solr | 2024-02-06 14:09:13.822 WARN (coreLoadExecutor-6-thread-1) [ x:ckan] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.SynonymFilterFactory]. Please consult documentation how to replace it accordingly.
solr | 2024-02-06 14:09:13.824 WARN (coreLoadExecutor-6-thread-1) [ x:ckan] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.WordDelimiterFilterFactory]. Please consult documentation how to replace it accordingly.
solr | 2024-02-06 14:09:13.843 WARN (coreLoadExecutor-6-thread-1) [ x:ckan] o.a.s.s.IndexSchema 'defaultSearchField' is deprecated, please use 'df' on request instead.
solr | 2024-02-06 14:09:13.843 INFO (coreLoadExecutor-6-thread-1) [ x:ckan] o.a.s.s.IndexSchema [ckan] default search field in schema is text.
solr | 2024-02-06 14:09:13.844 WARN (coreLoadExecutor-6-thread-1) [ x:ckan] o.a.s.s.IndexSchema [ckan] query parser default operator is AND. WARNING: Deprecated, please use 'q.op' on request instead. Will not work from Solr 7
solr | 2024-02-06 14:09:13.895 INFO (coreLoadExecutor-6-thread-1) [ x:ckan] o.a.s.s.IndexSchema Loaded schema ckan/2.9 with uniqueid field index_id
solr | 2024-02-06 14:09:13.914 INFO (coreLoadExecutor-6-thread-1) [ x:ckan] o.a.s.s.ManagedIndexSchema Upgraded to managed schema at /opt/solr/server/solr/ckan/conf/managed-schema
solr | 2024-02-06 14:09:13.921 INFO (coreLoadExecutor-6-thread-1) [ x:ckan] o.a.s.s.ManagedIndexSchemaFactory After upgrading to managed schema, renamed the non-managed schema /opt/solr/server/solr/ckan/conf/schema.xml to /opt/solr/server/solr/ckan/conf/schema.xml.bak
solr | 2024-02-06 14:09:13.922 INFO (coreLoadExecutor-6-thread-1) [ x:ckan] o.a.s.c.CoreContainer Creating SolrCore 'ckan' using configuration from instancedir /opt/solr/server/solr/ckan, trusted=true
solr | 2024-02-06 14:09:13.946 INFO (coreLoadExecutor-6-thread-1) [ x:ckan] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
solr | 2024-02-06 14:09:13.950 INFO (coreLoadExecutor-6-thread-1) [ x:ckan] o.a.s.c.SolrCore [[ckan] ] Opening new SolrCore at [/opt/solr/server/solr/ckan], dataDir=[/opt/solr/server/solr/ckan/data/]
db | CREATE DATABASE
db | GRANT
db |
db | /usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/20_postgis_permissions.sql
solr | 2024-02-06 14:09:14.040 INFO (coreLoadExecutor-6-thread-1) [ x:ckan] o.a.s.r.XSLTResponseWriter xsltCacheLifetimeSeconds=5
solr | 2024-02-06 14:09:14.202 INFO (coreLoadExecutor-6-thread-1) [ x:ckan] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
solr | 2024-02-06 14:09:14.202 INFO (coreLoadExecutor-6-thread-1) [ x:ckan] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
solr | 2024-02-06 14:09:14.210 INFO (coreLoadExecutor-6-thread-1) [ x:ckan] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms;
solr | 2024-02-06 14:09:14.210 INFO (coreLoadExecutor-6-thread-1) [ x:ckan] o.a.s.u.CommitTracker Soft AutoCommit: disabled
solr | 2024-02-06 14:09:14.225 INFO (coreLoadExecutor-6-thread-1) [ x:ckan] o.a.s.s.SolrIndexSearcher Opening [Searcher@836ebb1[ckan] main]
solr | 2024-02-06 14:09:14.236 INFO (coreLoadExecutor-6-thread-1) [ x:ckan] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /opt/solr/server/solr/ckan/conf
solr | 2024-02-06 14:09:14.252 INFO (coreLoadExecutor-6-thread-1) [ x:ckan] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
solr | 2024-02-06 14:09:14.256 INFO (coreLoadExecutor-6-thread-1) [ x:ckan] o.a.s.s.DirectSolrSpellChecker init: {name=default,field=_text_,classname=solr.DirectSolrSpellChecker,distanceMeasure=internal,accuracy=0.5,maxEdits=2,minPrefix=1,maxInspections=5,minQueryLength=4,maxQueryFrequency=0.01}
solr | 2024-02-06 14:09:14.258 INFO (coreLoadExecutor-6-thread-1) [ x:ckan] o.a.s.h.c.QueryElevationComponent Loading QueryElevation from: /opt/solr/server/solr/ckan/conf/elevate.xml
solr | 2024-02-06 14:09:14.269 INFO (coreLoadExecutor-6-thread-1) [ x:ckan] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
solr | 2024-02-06 14:09:14.288 INFO (searcherExecutor-7-thread-1-processing-x:ckan) [ x:ckan] o.a.s.c.QuerySenderListener QuerySenderListener sending requests to Searcher@836ebb1[ckan] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
solr | 2024-02-06 14:09:14.288 INFO (searcherExecutor-7-thread-1-processing-x:ckan) [ x:ckan] o.a.s.c.QuerySenderListener QuerySenderListener done.
solr | 2024-02-06 14:09:14.289 INFO (searcherExecutor-7-thread-1-processing-x:ckan) [ x:ckan] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
solr | 2024-02-06 14:09:14.288 INFO (coreLoadExecutor-6-thread-1) [ x:ckan] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1790158888541093888
solr | 2024-02-06 14:09:14.290 INFO (searcherExecutor-7-thread-1-processing-x:ckan) [ x:ckan] o.a.s.c.SolrCore [ckan] Registered new searcher Searcher@836ebb1[ckan] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
ckan | db:5432 - no response
db | CREATE EXTENSION
db | ALTER VIEW
db | ALTER TABLE
db |
db |
db | /usr/local/bin/docker-entrypoint.sh: sourcing /docker-entrypoint-initdb.d/postgis.sh
db | CREATE DATABASE
db | UPDATE 1
db | Loading PostGIS extensions into template_postgis
db | CREATE EXTENSION
db | CREATE EXTENSION
db | CREATE EXTENSION
ckan | db:5432 - no response
db | CREATE EXTENSION
db | Loading PostGIS extensions into ckan
db | CREATE EXTENSION
db | NOTICE: extension "postgis" already exists, skipping
db | CREATE EXTENSION
db | CREATE EXTENSION
db | CREATE EXTENSION
db |
db | waiting for server to shut down....2024-02-06 14:09:16.073 UTC [44] LOG: received fast shutdown request
db | 2024-02-06 14:09:16.075 UTC [44] LOG: aborting any active transactions
db | 2024-02-06 14:09:16.076 UTC [44] LOG: background worker "logical replication launcher" (PID 51) exited with exit code 1
db | 2024-02-06 14:09:16.080 UTC [46] LOG: shutting down
db | 2024-02-06 14:09:16.176 UTC [44] LOG: database system is shut down
db | done
db | server stopped
db |
db | PostgreSQL init process complete; ready for start up.
db |
db | 2024-02-06 14:09:16.288 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
db | 2024-02-06 14:09:16.288 UTC [1] LOG: listening on IPv6 address "::", port 5432
db | 2024-02-06 14:09:16.292 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db | 2024-02-06 14:09:16.305 UTC [108] LOG: database system was shut down at 2024-02-06 14:09:16 UTC
db | 2024-02-06 14:09:16.309 UTC [1] LOG: database system is ready to accept connections
ckan | db:5432 - accepting connections
ckan | Generating config at /etc/ckan/production.ini...
ckan | Traceback (most recent call last):
ckan | File "/usr/local/bin/ckan", line 5, in <module>
ckan | from ckan.cli.cli import ckan
ckan | File "/usr/lib/ckan/venv/src/ckan/ckan/cli/cli.py", line 11, in <module>
ckan | import ckan.plugins as p
ckan | File "/usr/lib/ckan/venv/src/ckan/ckan/plugins/__init__.py", line 3, in <module>
ckan | from ckan.plugins.core import *
ckan | File "/usr/lib/ckan/venv/src/ckan/ckan/plugins/core.py", line 14, in <module>
ckan | from ckan.common import asbool
ckan | File "/usr/lib/ckan/venv/src/ckan/ckan/common.py", line 13, in <module>
ckan | import flask
ckan | File "/usr/lib/ckan/venv/lib/python3.11/site-packages/flask/__init__.py", line 14, in <module>
ckan | from jinja2 import escape
ckan | File "/usr/lib/ckan/venv/lib/python3.11/site-packages/jinja2/__init__.py", line 33, in <module>
ckan | from jinja2.environment import Environment, Template
ckan | File "/usr/lib/ckan/venv/lib/python3.11/site-packages/jinja2/environment.py", line 16, in <module>
ckan | from jinja2.defaults import BLOCK_START_STRING, \
ckan | File "/usr/lib/ckan/venv/lib/python3.11/site-packages/jinja2/defaults.py", line 32, in <module>
ckan | from jinja2.tests import TESTS as DEFAULT_TESTS
ckan | File "/usr/lib/ckan/venv/lib/python3.11/site-packages/jinja2/tests.py", line 13, in <module>
ckan | from collections import Mapping
ckan | ImportError: cannot import name 'Mapping' from 'collections' (/usr/lib/python3.11/collections/__init__.py)
ckan exited with code 1
Have you tried taking a look at the video walkthroughs Brett has posted on Link Digital’s YouTube Channel?
2.9 and 2.10 are there for the Docker method: https://m.youtube.com/@link-digital