Issue with httlib and HTTPSConnection Using a Proxy (Proxy)

Question

To reproduce my issue, you need to have fiddler open and watching internet traffic. Fiddler's proxyurl is 127.0.0.1 and port is 8888. But in reality, I get this issue with any proxy over HTTPS. Using the HTTPSConnection class, I set the proxy and port, but I end up getting this error:

'SSLError: [Errno 1] _ssl.c:504: error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol'

This error can be attributed to an invalid proxy, but I get it for any and all proxies.

I am using the python httlib to perform a POST operation to a site:

    content_type, body = self._encode_multipart_formdata(fields, files) # encodes a file to string, standard multipart post stuff

    headers = {
        'content-type': content_type,
        'content-length': str(len(body))
    }

    if proxy_url:
        if ssl:
            h = httplib.HTTPSConnection(proxy_url, proxy_port)

            h.request('POST', 'https://' + host + selector, body, headers)

        else:
            h = httplib.HTTPConnection(proxy_url, proxy_port)
            h.request('POST', 'http://' + host + selector, body, headers)
    else:
        if ssl:
            h = httplib.HTTPSConnection(host,port)
            h.request('POST', selector, body, headers)
        else:
            h = httplib.HTTPConnection(host,port)
            h.request('POST', selector, body, headers)

    resp_data = h.getresponse().read()
    try:

        result = json.loads(resp_data)
    except:
        return None

My python package is written and is thousands of lines of code, so I really can't switch to requests.

The interesting thing, is that I can get the calls to work (non-multipart posts) with urllib2 when doing basic POST and GET operations.

Please advise!

Thank you,

I am using Python 2.7.5

Answer 1

It does not look like that you are actually sending proxy requests. According to your code all you do is to send a plain request to the proxy, but with a https:// URL. This is not how a proxy for HTTPS works. Instead you have to send a CONNECT request to establish a tunnel, then upgrade the socket to SSL and finally send the non-proxy request within this tunnel.