error on airflow webserver based on postgres

Question

Basically, I want to build the docker-airflow:

I have the Dockerfile like:

FROM puckel/docker-airflow:1.10.6
COPY ./airflow_home/airflow.cfg /usr/local/airflow/airflow.cfg
...

I have the docker-compose.yml like:

version: "3.4"
services:
    postgres:
        image: "postgres:9.6"
        container_name: "postgres"
        environment:
            - POSTGRES_USER=airflow
            - POSTGRES_PASSWORD=airflow
            - POSTGRES_DB=airflow
        ports:
            - "5432:5432"
        volumes:
            - './data/postgres:/var/lib/postgresql/data'
    # comment initdb after you will have use it at first run
    # set the client and processed log types here
    initdb:
        build: .
        entrypoint: airflow initdb
        entrypoint:
            ['python3', '/usr/local/airflow/__init__.py', '-C', '$Client', '-T', '$Types']
        volumes:
            - './airflow_home/packages:/usr/local/airflow/packages'
        depends_on:
            - postgres

And there is no other airflow library in the requirements.txt.

I always got the error when I did the:

docker-compose up webserver

The error is:

sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedTable) relation "connection" does not exist
webserver_1  | LINE 2: FROM connection GROUP BY connection.conn_id

The "xxx" include log as well.

I originally thought the error would come from the different versions. But I make sure the version has been confirmed right. I used the airflow.cfg comes from the same version and modified the sql_connection with Postgres. The database in Postgres has also been created. Does anyone know how to fix that?

I'm unclear why do you need the initdb section? the puckel docker will run initdb on startup. Also, you didn't include the airflow part in the posted snippet . It seems like initdb didn't run properly, checkout the logs of the started airflow — LiorH
– LiorH, Commented Apr 30, 2020 at 8:54
cuz I need to initialize some parameters for the following processes. I used to test well. The postgres and initdb all run successfully. I will attach the logs soon — Newt
– Newt, Commented Apr 30, 2020 at 8:58
I'd let the airflow image run the initdb part. Most params to airflow can be configured using environment variables. — LiorH
– LiorH, Commented Apr 30, 2020 at 9:01
Thanks for your opinions. Maybe I will give it a go. But now I do not think the problem is on initdb. — Newt
– Newt, Commented Apr 30, 2020 at 9:05

Newt · Accepted Answer · 2020-05-04 03:45:58Z

0

I found the reason. Cuz I used two entrypoints in initdb part. It will overwrite the original init script inside the docker. When I delete the entrypoint:

['python3', '/usr/local/airflow/__init__.py', '-C', '$Client', '-T', '$Types']

It works now.

answered May 4, 2020 at 3:45

Newt

8871 gold badge10 silver badges15 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Newt Over a year ago

I would recommend if you want to execute the command in docker. Better use CMD in Dockerfile. It saves a lot of time to fix the bug brought by the arguments

Collectives™ on Stack Overflow

error on airflow webserver based on postgres

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related