UnicodeEncodeError: 'ascii' codec can't encode character u'\ufffd' in position 12: ordinal not in range(128)

19 Jul 2013

The error in the title will occur on pre-0.12.0 versions of python’s Requests library. It seems to be fixed on later versions and maybe even earlier.

Here’s an example of how you save a zip file from behind a corporate proxy with python’s Requests library:

sess = requests.session() # you may have other get/post requests before you can get to the download page for the file url = 'https://example.com/report.zip' # this may be different for you, i just copied this from an existing request (using fiddler) file_headers = { 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'Accept-Language': 'en-US,en;q=0.8', 'User-Agent': 'Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.0; Trident/5.0)', 'Host': 'example.com', 'Accept-Encoding': 'gzip,deflate,sdch', 'Connection': 'Keep-Alive', 'Content-Type': 'application/octet-stream;charset=text/html;charset=UTF-8', } with open('report.zip', mode='wb') as handle: report_file = sess.get(url, headers=file_headers, prefetch=False, verify=False) handle.write(report_file.content)

That’s it! Just use the write function with the content of the returned request.