urllib2 cannnot handle https and BasicAuth via Proxy. #61720

Activity

1. masatokawamura commented on Mar 22, 2013

   

   masatokawamuramannequin

   on Mar 22, 2013

   MannequinAuthor

   More actions

   When urllib2 module is used, https connections to servers that BasicAuth is required, via proxies, can not be established properly.

   Sample code:
   import urllib2

   def main():
       url = "https://example.com/aplication/rpc"
       
       proxy_support = urllib2.ProxyHandler({'http': 'http://proxy.server.com:8080/','https': 'http://proxy.server.com:8080/'})
   
       pass_mngr = urllib2.HTTPPasswordMgrWithDefaultRealm()
       pass_mngr.add_password(None, url, 'user', 'password')
       auth_handler = urllib2.HTTPBasicAuthHandler(pass_mngr)
       
       opener = urllib2.build_opener(proxy_support,auth_handler)
       req = urllib2.Request(url)
       f = opener.open(req)
       print f.read()

   "opener.open" method throws an exception.

   I found a similar case at http://bugs.python.org/issue7291. However, this issue indicates a case of authentication at a proxy server. In my case, the origin server uses basic authentication.

   The origin server address is not set in Request URI in the https request in case of using proxy servers, because https request gets encripted with SSL when it passes through proxies.

   urllib2 works well for the first on-going request and in-comming return. However, after the first in-comming return (401 code), urllib2 module sends a worng request that includes the origin server URL.

   I suggest modification for set_proxy method in Request class.

   Original code:
   def set_proxy(self, host, type):
   if self.type == 'https' and not self._tunnel_host:
   self._tunnel_host = self.host
   else:
   self.type = type
   self.__r_host = self.__original

           self.host = host

   As a value is already set to tunnel_host when the second out-going request is sent, "self.type == 'https' and not self._tunnel_host" is false.

   Code suggested:
   def set_proxy(self, host, type):
   if self.type == 'https' and not self._tunnel_host:
   self._tunnel_host = self.host
   else:
   self.type = type
   if self.type != 'https':
   self.__r_host = self.__original

           self.host = host

   regards.
2. masatokawamura

   added

   type-crashA hard crash of the interpreter, possibly with a core dumpA hard crash of the interpreter, possibly with a core dump

   stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directory

   on Mar 22, 2013
3. BreamoreBoy commented on Oct 3, 2014

   

   BreamoreBoymannequin

   on Oct 3, 2014

   Mannequin

   More actions

   Slipped under the radar? Note the patch suggested in msg184958.
4. serhiy-storchaka

   added

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   and removed

   type-crashA hard crash of the interpreter, possibly with a core dumpA hard crash of the interpreter, possibly with a core dump

   on Jul 11, 2018
5. ezio-melotti

   transferred this issue fromon Apr 10, 2022
6. crazymerlyn commented on Aug 25, 2024

   

   crazymerlyn

   on Aug 25, 2024

   More actions

   Should this be closed since this only affects Python 2 which is way past it's EOL? The python3 issue #42870 has been fixed.
7. serhiy-storchaka

   added this to Urllib issueson Jul 3, 2026
8. github-project-automation

   moved this to Todo in Urllib issueson Jul 3, 2026
9. github-project-automation

   moved this from Todo to In Progress in Urllib issueson Nov 1, 2026
10. orsenthil commented on Nov 10, 2026

    

    orsenthil

    on Nov 10, 2026

    Member

    More actions

    This will be handled by this - #161596