Skip to content

Localstack get connection refused on jenkins #2216

New issue
New issue
Closed
Closed
Localstack get connection refused on jenkins#2216
@mikicho

Description

@mikicho
mikicho
opened on Mar 29, 2020

Type of request: This is a ...

[x] bug report
[ ] feature request

Detailed description

SQS failed to start on Jenkins, works on my local machine.

Expected behavior

The service will start in Jenkins

Actual behavior

Connection refused error (host='127.0.0.1', port=4561):
I wonder why localstack tries to connect to port 4561? this port not listen in the service ports in the docker-compose.

localstack_1  | Waiting for all LocalStack services to be ready
localstack_1  | 2020-03-29 09:44:02,303 CRIT Supervisor is running as root.  Privileges were not dropped because no user is specified in the config file.  If you intend to run as root, you can set user=root in the config file to avoid this message.
localstack_1  | 2020-03-29 09:44:02,307 INFO supervisord started with pid 13
localstack_1  | 2020-03-29 09:44:03,310 INFO spawned: 'dashboard' with pid 19
localstack_1  | 2020-03-29 09:44:03,312 INFO spawned: 'infra' with pid 20
localstack_1  | 2020-03-29 09:44:03,318 INFO success: dashboard entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
localstack_1  | (. .venv/bin/activate; bin/localstack web)
localstack_1  | (. .venv/bin/activate; exec bin/localstack start --host)
localstack_1  | Starting local dev environment. CTRL-C to quit.
localstack_1  | 2020-03-29 09:44:05,027 INFO success: infra entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
localstack_1  | Waiting for all LocalStack services to be ready
localstack_1  | 2020-03-29T09:44:14:INFO:localstack.utils.common: Unable to store certificate file under /tmp/localstack/server.test.pem, using tmp file instead: [Errno 13] Permission denied: '/tmp/localstack/server.test.pem'
localstack_1  | Starting mock SQS (http port 4576)...
localstack_1  | 2020-03-29T09:44:14:DEBUG:localstack.utils.common: Starting download from https://github.com/localstack/localstack-artifacts/raw/master/local-certs/server.key to /tmp/localstack/server.test.pem (2669 bytes)
localstack_1  | 2020-03-29T09:44:14:DEBUG:localstack.utils.common: Writing 3644 bytes (total 3644) to /tmp/localstack/server.test.pem
localstack_1  | 2020-03-29T09:44:14:DEBUG:localstack.utils.common: Done downloading https://github.com/localstack/localstack-artifacts/raw/master/local-certs/server.key, response code 200, total bytes 3644
localstack_1  | 2020-03-29T09:44:14:ERROR:localstack.services.generic_proxy: Error forwarding request: MyHTTPConnectionPool(host='127.0.0.1', port=4561): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f96538ffdc0>: Failed to establish a new connection: [Errno 111] Connection refused')) Traceback (most recent call last):
localstack_1  |   File "/opt/code/localstack/.venv/lib/python3.8/site-packages/urllib3/connection.py", line 156, in _new_conn
localstack_1  |     conn = connection.create_connection(
localstack_1  |   File "/opt/code/localstack/.venv/lib/python3.8/site-packages/urllib3/util/connection.py", line 84, in create_connection
localstack_1  |     raise err
localstack_1  |   File "/opt/code/localstack/.venv/lib/python3.8/site-packages/urllib3/util/connection.py", line 74, in create_connection
localstack_1  |     sock.connect(sa)
localstack_1  | ConnectionRefusedError: [Errno 111] Connection refused
localstack_1  | 
localstack_1  | During handling of the above exception, another exception occurred:
localstack_1  | 
localstack_1  | Traceback (most recent call last):
localstack_1  |   File "/opt/code/localstack/.venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 665, in urlopen
localstack_1  |     httplib_response = self._make_request(
localstack_1  |   File "/opt/code/localstack/.venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 387, in _make_request
localstack_1  |     conn.request(method, url, **httplib_request_kw)
localstack_1  |   File "/usr/lib/python3.8/http/client.py", line 1230, in request
localstack_1  |     self._send_request(method, url, body, headers, encode_chunked)
localstack_1  |   File "/usr/lib/python3.8/http/client.py", line 1276, in _send_request
localstack_1  |     self.endheaders(body, encode_chunked=encode_chunked)
localstack_1  |   File "/usr/lib/python3.8/http/client.py", line 1225, in endheaders
localstack_1  |     self._send_output(message_body, encode_chunked=encode_chunked)
localstack_1  |   File "/usr/lib/python3.8/http/client.py", line 1004, in _send_output
localstack_1  |     self.send(msg)
localstack_1  |   File "/usr/lib/python3.8/http/client.py", line 944, in send
localstack_1  |     self.connect()
localstack_1  |   File "/opt/code/localstack/.venv/lib/python3.8/site-packages/urllib3/connection.py", line 184, in connect
localstack_1  |     conn = self._new_conn()
localstack_1  |   File "/opt/code/localstack/.venv/lib/python3.8/site-packages/urllib3/connection.py", line 168, in _new_conn
localstack_1  |     raise NewConnectionError(
localstack_1  | urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f96538ffdc0>: Failed to establish a new connection: [Errno 111] Connection refused
localstack_1  | 
localstack_1  | During handling of the above exception, another exception occurred:
localstack_1  | 
localstack_1  | Traceback (most recent call last):
localstack_1  |   File "/opt/code/localstack/.venv/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
localstack_1  |     resp = conn.urlopen(
localstack_1  |   File "/opt/code/localstack/.venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 719, in urlopen
localstack_1  |     retries = retries.increment(
localstack_1  |   File "/opt/code/localstack/.venv/lib/python3.8/site-packages/urllib3/util/retry.py", line 436, in increment
localstack_1  |     raise MaxRetryError(_pool, url, error or ResponseError(cause))
localstack_1  | urllib3.exceptions.MaxRetryError: MyHTTPConnectionPool(host='127.0.0.1', port=4561): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f96538ffdc0>: Failed to establish a new connection: [Errno 111] Connection refused'))
localstack_1  | 
localstack_1  | During handling of the above exception, another exception occurred:
localstack_1  | 
localstack_1  | Traceback (most recent call last):
localstack_1  |   File "/opt/code/localstack/localstack/services/generic_proxy.py", line 283, in forward
localstack_1  |     response = self.method(request_url, data=data_to_send,
localstack_1  |   File "/opt/code/localstack/.venv/lib/python3.8/site-packages/requests/api.py", line 119, in post
localstack_1  |     return request('post', url, data=data, json=json, **kwargs)
localstack_1  |   File "/opt/code/localstack/.venv/lib/python3.8/site-packages/requests/api.py", line 61, in request
localstack_1  |     return session.request(method=method, url=url, **kwargs)
localstack_1  |   File "/opt/code/localstack/.venv/lib/python3.8/site-packages/requests/sessions.py", line 530, in request
localstack_1  |     resp = self.send(prep, **send_kwargs)
localstack_1  |   File "/opt/code/localstack/.venv/lib/python3.8/site-packages/requests/sessions.py", line 643, in send
localstack_1  |     r = adapter.send(request, **kwargs)
localstack_1  |   File "/opt/code/localstack/.venv/lib/python3.8/site-packages/requests/adapters.py", line 516, in send
localstack_1  |     raise ConnectionError(e, request=request)
localstack_1  | requests.exceptions.ConnectionError: MyHTTPConnectionPool(host='127.0.0.1', port=4561): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f96538ffdc0>: Failed to establish a new connection: [Errno 111] Connection refused'))
localstack_1  | 
localstack_1  | 2020-03-29T09:44:14:ERROR:localstack.services.generic_proxy: Error forwarding request: MyHTTPConnectionPool(host='127.0.0.1', port=4561): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9653909d00>: Failed to establish a new connection: [Errno 111] Connection refused')) Traceback (most recent call last):
localstack_1  |   File "/opt/code/localstack/.venv/lib/python3.8/site-packages/urllib3/connection.py", line 156, in _new_conn
localstack_1  |     conn = connection.create_connection(
localstack_1  |   File "/opt/code/localstack/.venv/lib/python3.8/site-packages/urllib3/util/connection.py", line 84, in create_connection
localstack_1  |     raise err
localstack_1  |   File "/opt/code/localstack/.venv/lib/python3.8/site-packages/urllib3/util/connection.py", line 74, in create_connection
localstack_1  |     sock.connect(sa)
localstack_1  | ConnectionRefusedError: [Errno 111] Connection refused
localstack_1  | 
localstack_1  | During handling of the above exception, another exception occurred:
localstack_1  | 
localstack_1  | Traceback (most recent call last):
localstack_1  |   File "/opt/code/localstack/.venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 665, in urlopen
localstack_1  |     httplib_response = self._make_request(
localstack_1  |   File "/opt/code/localstack/.venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 387, in _make_request
localstack_1  |     conn.request(method, url, **httplib_request_kw)
localstack_1  |   File "/usr/lib/python3.8/http/client.py", line 1230, in request
localstack_1  |     self._send_request(method, url, body, headers, encode_chunked)
localstack_1  |   File "/usr/lib/python3.8/http/client.py", line 1276, in _send_request
localstack_1  |     self.endheaders(body, encode_chunked=encode_chunked)
localstack_1  |   File "/usr/lib/python3.8/http/client.py", line 1225, in endheaders
localstack_1  |     self._send_output(message_body, encode_chunked=encode_chunked)
localstack_1  |   File "/usr/lib/python3.8/http/client.py", line 1004, in _send_output
localstack_1  |     self.send(msg)
localstack_1  |   File "/usr/lib/python3.8/http/client.py", line 944, in send
localstack_1  |     self.connect()
localstack_1  |   File "/opt/code/localstack/.venv/lib/python3.8/site-packages/urllib3/connection.py", line 184, in connect
localstack_1  |     conn = self._new_conn()
localstack_1  |   File "/opt/code/localstack/.venv/lib/python3.8/site-packages/urllib3/connection.py", line 168, in _new_conn
localstack_1  |     raise NewConnectionError(
localstack_1  | urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f9653909d00>: Failed to establish a new connection: [Errno 111] Connection refused
localstack_1  | 
localstack_1  | During handling of the above exception, another exception occurred:
localstack_1  | 
localstack_1  | Traceback (most recent call last):
localstack_1  |   File "/opt/code/localstack/.venv/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
localstack_1  |     resp = conn.urlopen(
localstack_1  |   File "/opt/code/localstack/.venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 719, in urlopen
localstack_1  |     retries = retries.increment(
localstack_1  |   File "/opt/code/localstack/.venv/lib/python3.8/site-packages/urllib3/util/retry.py", line 436, in increment
localstack_1  |     raise MaxRetryError(_pool, url, error or ResponseError(cause))
localstack_1  | urllib3.exceptions.MaxRetryError: MyHTTPConnectionPool(host='127.0.0.1', port=4561): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9653909d00>: Failed to establish a new connection: [Errno 111] Connection refused'))
...
...
...

Steps to reproduce

Run it on my Jenkins

Command used to start LocalStack

Docker-compose:

  localstack:
    image: localstack/localstack
    ports:
      - "4567-4584:4567-4584"
      - "12345:8080"
    environment:
      - DOCKER_HOST=unix:///var/run/docker.sock
      - SERVICES=sqs
      - DEBUG=1
      - HOSTNAME_EXTERNAL=localstack
      - DEFAULT_REGION=us-west-2
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"
      - "$PWD/localstack:/docker-entrypoint-initaws.d"
    env_file:
      - .env

Client code (AWS SDK code snippet, or sequence of "awslocal" commands)

N/A

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions