Skip to content

urllib2 cannot handle https with proxy requiring auth #51540

New issue
New issue
Closed
#161596
Closed
urllib2 cannot handle https with proxy requiring auth#51540
#161596
Assignees
orsenthil
Labels
3.16new features, bugs and security fixesstdlibStandard Library Python modules in the Lib/ directory

Description

@tsujikawa
tsujikawa
mannequin
opened on Nov 9, 2009
BPO 7291
Nosy @ronaldoussoren, @orsenthil, @vadmium, @yan12125, @OneMoreZanuda
Files
  • https_proxy_auth.patch: patch to send Proxy-Authorization header in CONNECT method(https proxy)
  • urllib2_with_proxy_auth_comparison.py
  • 2_7_x.patch: 2.7 maintenance branch patch
  • monkey_2_6_4.py: 2.6.4 monkey patch
  • urllib2_tests.tar.gz: Test code, results and instructions
  • http_proxy_https.patch: Fix handling of 407 and 401 in urllib2 and httplib
  • new_http_proxy_patch.py
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/orsenthil'
closed_at = None
created_at = <Date 2009-11-09.04:38:16.601>
labels = ['3.8', '3.7', 'library', '3.9', '3.10']
title = 'urllib2 cannot handle https with proxy requiring auth'
updated_at = <Date 2020-08-02.19:06:17.855>
user = 'https://bugs.python.org/tsujikawa'

    bugs.python.org fields:

    activity = <Date 2020-08-02.19:06:17.855>
actor = 'alexey.namyotkin'
assignee = 'orsenthil'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2009-11-09.04:38:16.601>
creator = 'tsujikawa'
dependencies = []
files = ['15296', '15669', '20798', '20799', '20822', '20824', '34174']
hgrepos = []
issue_num = 7291
keywords = ['patch']
message_count = 24.0
messages = ['95058', '95060', '95373', '95377', '96659', '96660', '96661', '96844', '96845', '96846', '100840', '103090', '128861', '128862', '128952', '128953', '128955', '211850', '211860', '228406', '244513', '247805', '283306', '374687']
nosy_count = 11.0
nosy_names = ['ronaldoussoren', 'orsenthil', 'mbeachy', 'dieresys', 'tsujikawa', 'b.a.scott', 'martin.panter', 'vzafzal', 'yan12125', 'alexey.namyotkin', 'Jessica Ridgley']
pr_nums = []
priority = 'normal'
resolution = 'accepted'
stage = 'needs patch'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue7291'
versions = ['Python 2.7', 'Python 3.5', 'Python 3.6', 'Python 3.7', 'Python 3.8', 'Python 3.9', 'Python 3.10']

    Linked PRs

    Activity

    1. tsujikawa commented on Nov 9, 2009

      @tsujikawa
      tsujikawamannequin
      on Nov 9, 2009
      MannequinAuthor

      urllib2 cannot handle https with proxy requiring authorization.

      After https_proxy is set correctly,

      Python 2.6.4 (r264:75706, Oct 29 2009, 15:38:25)
[GCC 4.4.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib2
>>> c=urllib2.urlopen("https://sourceforge.net")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.6/urllib2.py", line 124, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/lib/python2.6/urllib2.py", line 389, in open
    response = self._open(req, data)
  File "/usr/lib/python2.6/urllib2.py", line 407, in _open
    '_open', req)
  File "/usr/lib/python2.6/urllib2.py", line 367, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.6/urllib2.py", line 1154, in https_open
    return self.do_open(httplib.HTTPSConnection, req)
  File "/usr/lib/python2.6/urllib2.py", line 1121, in do_open
    raise URLError(err)
urllib2.URLError: <urlopen error Tunnel connection failed: 407 Proxy
Authentication Required>

      This is because HTTPConnection::_tunnel() in httplib.py doesn't send
      Proxy-Authorization header.

    2. tsujikawa
      added
      stdlibStandard Library Python modules in the Lib/ directory
      on Nov 9, 2009

    3. tsujikawa commented on Nov 9, 2009

      @tsujikawa
      tsujikawamannequin
      on Nov 9, 2009
      MannequinAuthor

      I created a patch.
      I added additional argument 'headers' to HTTPConnection::set_tunnel()
      method,
      which is a mapping of HTTP headers to sent with CONNECT method. Since
      authorization
      credential is already set to Request object, in
      AbstractHTTPHandler::do_open(),
      if "Proxy-Authorization" header is found, pass it to set_tunnel().

      It works fine for me.

    4. orsenthil
      self-assigned this
      on Nov 15, 2009

    5. ronaldoussoren commented on Nov 17, 2009

      @ronaldoussoren
      ronaldoussoren
      on Nov 17, 2009
      Contributor

      The patch looks good to me.

      IMHO this should be backported to 2.6 as well.

    6. ronaldoussoren commented on Nov 17, 2009

      @ronaldoussoren
      ronaldoussoren
      on Nov 17, 2009
      Contributor

      I've tested a backport of the patch to 2.6 (just replace set_proxy by
      _set_proxy in the patch) and the resulting version of urllib2 can login to
      the proxy (as expected).

      Thanks for the patch.

    7. orsenthil commented on Dec 20, 2009

      @orsenthil
      orsenthil
      on Dec 20, 2009
      Member

      Fixed and Committed revision 76908 in the trunk.

    8. orsenthil commented on Dec 20, 2009

      @orsenthil
      orsenthil
      on Dec 20, 2009
      Member

      Fixed through reversions r76908, r76909, r76910, r76911

      Thanks for the patch, Tatsuhiro Tsujikawa.

    9. orsenthil
      closed this as completedon Dec 20, 2009

    10. orsenthil commented on Dec 20, 2009

      @orsenthil
      orsenthil
      on Dec 20, 2009
      Member

      meant revisions.

    11. dieresys commented on Dec 23, 2009

      @dieresys
      dieresysmannequin
      on Dec 23, 2009
      Mannequin

      Hi! 2.6 backport is missing an argument in _set_tunnel definition. It
      should be:

          def _set_tunnel(self, host, port=None, headers=None):

    12. dieresys commented on Dec 23, 2009

      @dieresys
      dieresysmannequin
      on Dec 23, 2009
      Mannequin

      The patch fixes only when you pass the authentication info in the proxy
      handler's URL. Like:

          proxy_handler = urllib2.ProxyHandler({'https':
'http://user:pass@proxy-example.com:3128/'})

      But setting the authentication using a ProxyBasicAuthHandler is still
      broken:

          proxy_auth_handler = urllib2.ProxyBasicAuthHandler()
    proxy_auth_handler.add_password('realm', 'proxy-example.com:3128',
'user', 'pass')

      In the attached file (urllib2_with_proxy_auth_comparison.py) we've wrote
      a comparison between what works with HTTP and HTTPS.

      The problem is the 407 error never reaches the ProxyBasicAuthHandler
      because HTTPConnection._tunnel raises an exception when the http
      response status code is not 200.

    13. orsenthil commented on Dec 24, 2009

      @orsenthil
      orsenthil
      on Dec 24, 2009
      Member

      Thanks for the note, Manuel. Fixed it in revision 77013.

    14. orsenthil commented on Mar 11, 2010

      @orsenthil
      orsenthil
      on Mar 11, 2010
      Member

      In this ticket, setting the authentication using a ProxyBasicAuthHandler is not yet addressed yet. (this was informed in the last note). Reopening this one to track it.

    15. orsenthil
      reopened this on Mar 11, 2010

    16. 17 remaining items

    17. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    18. serhiy-storchaka
      added this to Urllib issueson Jul 3, 2026
    19. github-project-automation
      moved this to Todo in Urllib issueson Jul 3, 2026
    20. orsenthil
      added
      3.16new features, bugs and security fixes
      and removed
      3.10 (EOL)end of life
      3.9 (EOL)end of life
      3.8 (EOL)end of life
      3.7 (EOL)end of life
      on Oct 31, 2026

    21. orsenthil commented on Nov 8, 2026

      @orsenthil
      orsenthil
      on Nov 8, 2026
      Member

      A fix for this issue - #161596

    22. orsenthil
      moved this from Todo to In Progress in Urllib issueson Nov 8, 2026
    23. orsenthil
      added 2 commits that reference this issue on Nov 11, 2026

      pythongh-51540: Let proxy auth handlers answer a 407 to CONNECT for h…

      92755b1

      gh-51540: Let proxy auth handlers answer a 407 to CONNECT for https U…

      9f03afd
    24. orsenthil
      closed this as completedin #161596on Nov 11, 2026
    25. github-project-automation
      moved this from In Progress to Done in Urllib issueson Nov 11, 2026
    26. orsenthil
      added a commit that references this issue on Nov 12, 2026

      pythongh-51540: Let proxy auth handlers answer a 407 to CONNECT for h…

      e4a1270
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    Labels

    3.16new features, bugs and security fixesstdlibStandard Library Python modules in the Lib/ directory

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions