I am trying to build docker with airflow and Postgres nevertheless getting many errors during build as shown below. I've tried to add into requirements.txt firstly some found solution on internet advised to downgrading version of markup nevertheless it didn't help at all. What can I do to solve that issue?
markupsafe==2.1.1 and then also markupsafe==2.0.1 Error:
ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/usr/local/lib/python3.7/site-packages/markupsafe/__init__.py) Docker file:
version: '3' services: postgres: image: postgres:11.2 environment: - POSTGRES_USER=airflow - POSTGRES_PASSWORD=airflow - POSTGRES_DB=airflow ports: - "5432:5432" webserver: image: puckel/docker-airflow:1.10.4 build: context: dockerfile: Dockerfile args: AIRFLOW_DEPS: gcp_api,s3 restart: always depends_on: - postgres environment: - LOAD_EX=n - EXECUTOR=Local - FERNET_KEY=jsDPRErfv8Z_eVTnGfF8ywd19j4pyqE3NpdUBA_oRTo= volumes: - ./sparkify/dags:/usr/local/airflow/dags # Uncomment to include custom plugins - ./sparkify/plugins:/usr/local/airflow/plugins ports: - "8081:8081" command: webserver healthcheck: test: ["CMD-SHELL", "[ -f /usr/local/airflow/airflow-webserver.pid ]"] interval: 30s timeout: 30s retries: 3 requirements: image: python:3.8-alpine volumes: - .:/sparkify working_dir: /sparkify command: pip3 install -r requirements.txt Requirments.txt:
markupsafe==2.0.1 Full error:
Creating network "airflowwithpostgres_default" with the default driver Creating airflowwithpostgres_postgres_1 ... done Creating airflowwithpostgres_requirements_1 ... done Creating airflowwithpostgres_webserver_1 ... done Attaching to airflowwithpostgres_requirements_1, airflowwithpostgres_postgres_1, airflowwithpostgres_webserver_1 postgres_1 | The files belonging to this database system will be owned by user "postgres". postgres_1 | This user must also own the server process. postgres_1 | webserver_1 | Tue May 10 18:38:28 UTC 2022 - waiting for Postgres... 1/20 postgres_1 | The database cluster will be initialized with locale "en_US.utf8". postgres_1 | The default database encoding has accordingly been set to "UTF8". postgres_1 | The default text search configuration will be set to "english". postgres_1 | postgres_1 | Data page checksums are disabled. postgres_1 | postgres_1 | fixing permissions on existing directory /var/lib/postgresql/data ... ok postgres_1 | creating subdirectories ... ok postgres_1 | selecting default max_connections ... 100 postgres_1 | selecting default shared_buffers ... 128MB postgres_1 | selecting dynamic shared memory implementation ... posix postgres_1 | creating configuration files ... ok postgres_1 | running bootstrap script ... ok postgres_1 | performing post-bootstrap initialization ... ok postgres_1 | syncing data to disk ... postgres_1 | WARNING: enabling "trust" authentication for local connections postgres_1 | You can change this by editing pg_hba.conf or using the option -A, or postgres_1 | --auth-local and --auth-host, the next time you run initdb. postgres_1 | ok postgres_1 | postgres_1 | Success. You can now start the database server using: postgres_1 | postgres_1 | pg_ctl -D /var/lib/postgresql/data -l logfile start postgres_1 | postgres_1 | waiting for server to start....2022-05-10 18:38:29.428 UTC [44] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432" postgres_1 | 2022-05-10 18:38:29.441 UTC [45] LOG: database system was shut down at 2022-05-10 18:38:29 UTC postgres_1 | 2022-05-10 18:38:29.446 UTC [44] LOG: database system is ready to accept connections postgres_1 | done postgres_1 | server started postgres_1 | CREATE DATABASE postgres_1 | postgres_1 | postgres_1 | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/* postgres_1 | postgres_1 | 2022-05-10 18:38:29.859 UTC [44] LOG: received fast shutdown request postgres_1 | waiting for server to shut down....2022-05-10 18:38:29.861 UTC [44] LOG: aborting any active transactions postgres_1 | 2022-05-10 18:38:29.865 UTC [44] LOG: background worker "logical replication launcher" (PID 51) exited with exit code 1 postgres_1 | 2022-05-10 18:38:29.865 UTC [46] LOG: shutting down postgres_1 | 2022-05-10 18:38:29.882 UTC [44] LOG: database system is shut down postgres_1 | done postgres_1 | server stopped postgres_1 | postgres_1 | PostgreSQL init process complete; ready for start up. postgres_1 | postgres_1 | 2022-05-10 18:38:29.974 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432 postgres_1 | 2022-05-10 18:38:29.974 UTC [1] LOG: listening on IPv6 address "::", port 5432 postgres_1 | 2022-05-10 18:38:29.976 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432" postgres_1 | 2022-05-10 18:38:29.989 UTC [62] LOG: database system was shut down at 2022-05-10 18:38:29 UTC postgres_1 | 2022-05-10 18:38:29.994 UTC [1] LOG: database system is ready to accept connections requirements_1 | Collecting markupsafe==2.1.1 requirements_1 | Downloading MarkupSafe-2.1.1-cp38-cp38-musllinux_1_1_x86_64.whl (29 kB) requirements_1 | Installing collected packages: markupsafe requirements_1 | Successfully installed markupsafe-2.1.1 requirements_1 | WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: airflowwithpostgres_requirements_1 exited with code 0 postgres_1 | 2022-05-10 18:38:33.377 UTC [69] LOG: incomplete startup packet webserver_1 | Traceback (most recent call last): webserver_1 | File "/usr/local/bin/airflow", line 21, in <module> webserver_1 | from airflow import configuration webserver_1 | File "/usr/local/lib/python3.7/site-packages/airflow/__init__.py", line 39, in <module> webserver_1 | from airflow.models import DAG webserver_1 | File "/usr/local/lib/python3.7/site-packages/airflow/models/__init__.py", line 21, in <module> webserver_1 | from airflow.models.baseoperator import BaseOperator # noqa: F401 webserver_1 | File "/usr/local/lib/python3.7/site-packages/airflow/models/baseoperator.py", line 30, in <module> webserver_1 | import jinja2 webserver_1 | File "/usr/local/lib/python3.7/site-packages/jinja2/__init__.py", line 33, in <module> webserver_1 | from jinja2.environment import Environment, Template webserver_1 | File "/usr/local/lib/python3.7/site-packages/jinja2/environment.py", line 15, in <module> webserver_1 | from jinja2 import nodes webserver_1 | File "/usr/local/lib/python3.7/site-packages/jinja2/nodes.py", line 19, in <module> webserver_1 | from jinja2.utils import Markup webserver_1 | File "/usr/local/lib/python3.7/site-packages/jinja2/utils.py", line 642, in <module> webserver_1 | from markupsafe import Markup, escape, soft_unicode webserver_1 | ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/usr/local/lib/python3.7/site-packages/markupsafe/__init__.py) webserver_1 | Traceback (most recent call last): webserver_1 | File "/usr/local/bin/airflow", line 21, in <module> webserver_1 | from airflow import configuration webserver_1 | File "/usr/local/lib/python3.7/site-packages/airflow/__init__.py", line 39, in <module> webserver_1 | from airflow.models import DAG webserver_1 | File "/usr/local/lib/python3.7/site-packages/airflow/models/__init__.py", line 21, in <module> webserver_1 | from airflow.models.baseoperator import BaseOperator # noqa: F401 webserver_1 | File "/usr/local/lib/python3.7/site-packages/airflow/models/baseoperator.py", line 30, in <module> webserver_1 | import jinja2 webserver_1 | File "/usr/local/lib/python3.7/site-packages/jinja2/__init__.py", line 33, in <module> webserver_1 | from jinja2.environment import Environment, Template webserver_1 | File "/usr/local/lib/python3.7/site-packages/jinja2/environment.py", line 15, in <module> webserver_1 | from jinja2 import nodes webserver_1 | File "/usr/local/lib/python3.7/site-packages/jinja2/nodes.py", line 19, in <module> webserver_1 | from jinja2.utils import Markup webserver_1 | File "/usr/local/lib/python3.7/site-packages/jinja2/utils.py", line 642, in <module> webserver_1 | from markupsafe import Markup, escape, soft_unicode webserver_1 | ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/usr/local/lib/python3.7/site-packages/markupsafe/__init__.py) webserver_1 | Traceback (most recent call last): webserver_1 | File "/usr/local/bin/airflow", line 21, in <module> webserver_1 | from airflow import configuration webserver_1 | File "/usr/local/lib/python3.7/site-packages/airflow/__init__.py", line 39, in <module> webserver_1 | from airflow.models import DAG webserver_1 | File "/usr/local/lib/python3.7/site-packages/airflow/models/__init__.py", line 21, in <module> webserver_1 | from airflow.models.baseoperator import BaseOperator # noqa: F401 webserver_1 | File "/usr/local/lib/python3.7/site-packages/airflow/models/baseoperator.py", line 30, in <module> webserver_1 | import jinja2 webserver_1 | File "/usr/local/lib/python3.7/site-packages/jinja2/__init__.py", line 33, in <module> webserver_1 | from jinja2.environment import Environment, Template webserver_1 | File "/usr/local/lib/python3.7/site-packages/jinja2/environment.py", line 15, in <module> webserver_1 | from jinja2 import nodes webserver_1 | File "/usr/local/lib/python3.7/site-packages/jinja2/nodes.py", line 19, in <module> webserver_1 | from jinja2.utils import Markup webserver_1 | File "/usr/local/lib/python3.7/site-packages/jinja2/utils.py", line 642, in <module> webserver_1 | from markupsafe import Markup, escape, soft_unicode webserver_1 | ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/usr/local/lib/python3.7/site-packages/markupsafe/__init__.py) postgres_1 | 2022-05-10 18:38:35.020 UTC [70] LOG: incomplete startup packet webserver_1 | Traceback (most recent call last): webserver_1 | File "/usr/local/bin/airflow", line 21, in <module> webserver_1 | from airflow import configuration webserver_1 | File "/usr/local/lib/python3.7/site-packages/airflow/__init__.py", line 39, in <module> webserver_1 | from airflow.models import DAG webserver_1 | File "/usr/local/lib/python3.7/site-packages/airflow/models/__init__.py", line 21, in <module> webserver_1 | from airflow.models.baseoperator import BaseOperator # noqa: F401 webserver_1 | File "/usr/local/lib/python3.7/site-packages/airflow/models/baseoperator.py", line 30, in <module> webserver_1 | import jinja2 webserver_1 | File "/usr/local/lib/python3.7/site-packages/jinja2/__init__.py", line 33, in <module> webserver_1 | from jinja2.environment import Environment, Template webserver_1 | File "/usr/local/lib/python3.7/site-packages/jinja2/environment.py", line 15, in <module> webserver_1 | from jinja2 import nodes webserver_1 | File "/usr/local/lib/python3.7/site-packages/jinja2/nodes.py", line 19, in <module> webserver_1 | from jinja2.utils import Markup webserver_1 | File "/usr/local/lib/python3.7/site-packages/jinja2/utils.py", line 642, in <module> webserver_1 | from markupsafe import Markup, escape, soft_unicode webserver_1 | ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/usr/local/lib/python3.7/site-packages/markupsafe/__init__.py) webserver_1 | Traceback (most recent call last): webserver_1 | File "/usr/local/bin/airflow", line 21, in <module> webserver_1 | from airflow import configuration webserver_1 | File "/usr/local/lib/python3.7/site-packages/airflow/__init__.py", line 39, in <module> webserver_1 | from airflow.models import DAG webserver_1 | File "/usr/local/lib/python3.7/site-packages/airflow/models/__init__.py", line 21, in <module> webserver_1 | from airflow.models.baseoperator import BaseOperator # noqa: F401 webserver_1 | File "/usr/local/lib/python3.7/site-packages/airflow/models/baseoperator.py", line 30, in <module> webserver_1 | import jinja2 webserver_1 | File "/usr/local/lib/python3.7/site-packages/jinja2/__init__.py", line 33, in <module> webserver_1 | from jinja2.environment import Environment, Template webserver_1 | File "/usr/local/lib/python3.7/site-packages/jinja2/environment.py", line 15, in <module> webserver_1 | from jinja2 import nodes webserver_1 | File "/usr/local/lib/python3.7/site-packages/jinja2/nodes.py", line 19, in <module> webserver_1 | from jinja2.utils import Markup webserver_1 | File "/usr/local/lib/python3.7/site-packages/jinja2/utils.py", line 642, in <module> webserver_1 | from markupsafe import Markup, escape, soft_unicode webserver_1 | ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/usr/local/lib/python3.7/site-packages/markupsafe/__init__.py) webserver_1 | Traceback (most recent call last): webserver_1 | File "/usr/local/bin/airflow", line 21, in <module> webserver_1 | from airflow import configuration webserver_1 | File "/usr/local/lib/python3.7/site-packages/airflow/__init__.py", line 39, in <module> webserver_1 | from airflow.models import DAG webserver_1 | File "/usr/local/lib/python3.7/site-packages/airflow/models/__init__.py", line 21, in <module> webserver_1 | from airflow.models.baseoperator import BaseOperator # noqa: F401 webserver_1 | File "/usr/local/lib/python3.7/site-packages/airflow/models/baseoperator.py", line 30, in <module> webserver_1 | import jinja2 webserver_1 | File "/usr/local/lib/python3.7/site-packages/jinja2/__init__.py", line 33, in <module> webserver_1 | from jinja2.environment import Environment, Template webserver_1 | File "/usr/local/lib/python3.7/site-packages/jinja2/environment.py", line 15, in <module> webserver_1 | from jinja2 import nodes webserver_1 | File "/usr/local/lib/python3.7/site-packages/jinja2/nodes.py", line 19, in <module> webserver_1 | from jinja2.utils import Markup webserver_1 | File "/usr/local/lib/python3.7/site-packages/jinja2/utils.py", line 642, in <module> webserver_1 | from markupsafe import Markup, escape, soft_unicode webserver_1 | ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/usr/local/lib/python3.7/site-packages/markupsafe/__init__.py) airflowwithpostgres_webserver_1 exited with code 1 1 Answer
I'm getting the exact same error!
It turns out markupsafe removed `soft_unicode' which is causing this error! Still working on a solution for this tho :/
EDIT: I got it working!!
Add MarkupSafe==2.0.1 to your PYTHON_DEPS like so:
webserver: image: puckel/docker-airflow:1.10.4 build: context: dockerfile: Dockerfile args: AIRFLOW_DEPS: gcp_api,s3 PYTHON_DEPS: MarkupSafe==2.0.1 Then run docker-compose down then docker-compose up -d --build and all services should be running!
If you get another importerror, you can add the packagae to PYTHON_DEPS like so:
PYTHON_DEPS: MarkupSafe==2.0.1 example==1.1.9 
