Paul's Programming Notes PostsRSSGithub

UnicodeEncodeError: 'ascii' codec can't encode character u'\ufffd' in position 12: ordinal not in range(128)

The error in the title will occur on pre-0.12.0 versions of python’s Requests library. It seems to be fixed on later versions and maybe even earlier.

Here’s an example of how you save a zip file from behind a corporate proxy with python’s Requests library:

That’s it! Just use the write function with the content of the returned request.

requests.exceptions.SSLError: [Errno 1] _ssl.c:504: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

This requests error means Python couldn’t verify the server’s SSL certificate against its trusted CA bundle. I originally “fixed” it with verify=False. Don’t do that: it turns off certificate verification entirely and leaves the request open to a man-in-the-middle. The error almost always has a real, fixable cause.

If the certificate is signed by an internal or self-signed CA (common for corporate and intranet services), point requests at that CA instead of turning verification off:

r = s.post(url, data=payload, headers=headers, verify="/path/to/internal-ca.pem")

A corporate SSL-inspection proxy re-signs traffic with its own root, and the fix is the same: pass that root CA to verify. If the CA bundle is just out of date, requests verifies against certifi, so pip install --upgrade certifi (or updating your OS CA certs) often clears it. If the server is missing its intermediate certificate, that’s a server-side misconfiguration you can confirm with SSL Labs and fix in the chain.

If you genuinely need to skip verification for a throwaway local test, scope it and silence the warning knowingly, never against anything real:

import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
r = s.post(url, ..., verify=False)  # local debugging only

Requests docs: SSL cert verification.

Writing To File - UnicodeEncodeError: 'ascii' codec can't encode characters

I was getting a lot of errors when I was trying to download a file with the python requests library. The errors looked like the following:

UnicodeEncodeError: ‘ascii’ codec can’t encode characters in position 2-6: ordinal not in range(128)

This code ended up fixing it:

Google Maps Event Firing Order

I wanted to see the order Google Maps fires its events in. Google’s official Map Events example wires up listeners for the map’s events, so it’s a good starting point for seeing how they behave as you pan, zoom, and click.

Tunnel Traffic Through Webserver Using Your Browser

Use the following command in Cygwin (using your IP and Username, of course):

ssh yourUsername@198.168.xxx.xxx -D 2280 -i your/certificate

Set the following settings in Firefox (connect settings, under the advanced tab):

proxy settings