Skip to content

Heroku-Redis SSL Issues #493

Open
Open

Description

@akcode47

The following redis.ConnectionPool.from_url(config.REDIS_URL) works with Heroku's REDIS_URL, but due to their TLS changes, I was unable to connect using create_pool(RedisSettings().from_dsn(config.REDIS_URL)) which led to the following SSLCertVerificationError error:

ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:1006)
import redis.asyncio as redis
pool = redis.ConnectionPool.from_url(config.REDIS_URL)

No matter how I pass in ssl=True to the RedisSettings class and set ssl_cert_reqs=none, I get ConnectionErrors, whereas with redis.asyncio, I can establish the pool/client.

While digging around the ARQ code, I saw that the ArqRedis class allows you to pass a pool directly to the pool_or_conn=pool parameter, which works as a workaround for me.

Activity

  1. Graeme22 commented on Dec 23, 2024

    @Graeme22

    This code works for me:

    # create job queue
    redis_settings = RedisSettings.from_dsn(settings.REDIS_URL)
    redis_settings.ssl_cert_reqs = "none"
  2. akcode47 commented on Dec 24, 2024

    @akcode47
    Author

    This code works for me:

    # create job queue
    redis_settings = RedisSettings.from_dsn(settings.REDIS_URL)
    redis_settings.ssl_cert_reqs = "none"

    Thank you - my issue stemmed from the RedisSettings object initialization and modification.

    Previously, I attempted to set ssl_cert_reqs directly in the constructor and immediately chained the from_dsn() class method, which didn't work because from_dsn() is designed to parse and return a new instance of RedisSettings based on the DSN, which overrides any attributes set during object construction.

    Won't work:

    from arq.connections import RedisSettings, create_pool
    async def example():
        redis_settings = RedisSettings(ssl_cert_reqs = "none").from_dsn(config.REDIS_URL)
        return await create_pool(redis_settings)

    In the following working version, the RedisSettings class is initialized with the from_dsn() method, and the instance's ssl_cert_reqs property is set to the string value 'none':

    from arq.connections import RedisSettings, create_pool
    async def example():
        redis_settings = RedisSettings.from_dsn(config.REDIS_URL)
        redis_settings.ssl_cert_reqs = "none"
        return await create_pool(redis_settings)

    Not sure if this is worth mentioning in the docs or if it makes sense to enhance the from_dsn() class method to parse the query parameters and set the ssl_cert_reqs property accordingly. Something like the following:

    conf = urlparse(dsn)
    query_params = parse_qs(conf.query)
    
    # Extract SSL settings from query parameters with a fallback to defaults
    ssl_cert_reqs = query_params.get('ssl_cert_reqs', ['required'])[0]
  3. Graeme22 commented on Dec 24, 2024

    @Graeme22

    No problem! I agree that the constructor should accept and pass on the kwargs. However considering there's an entire rewrite in progress I guess it's not super high priority right now!

  4. samuelcolvin commented on Dec 24, 2024

    @samuelcolvin
    Member

    Thanks for reporting.

    No problem! I agree that the constructor should accept and pass on the kwargs. However considering there's an entire rewrite in progress I guess it's not super high priority right now!

    In some senses, now is a good time to fix this - if we merge a fix and more importantly tests, we'll have to support that behavior after the rewrite.

  5. YuviGold commented on Jan 16, 2025

    @YuviGold

    For anyone who might encounter the same issue, I was able to solve it by implementing the following function

    from urllib.parse import urlparse
    from arq.connections import RedisSettings
    
    def get_redis_settings(redis_url: str) -> RedisSettings:
        url = urlparse(redis_url)
        return RedisSettings(
            host=url.hostname,
            port=url.port,
            password=url.password,
            ssl=(url.scheme == "rediss"),
            ssl_cert_reqs=None,
        )
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions