Gevent + Requests Performance With verify=True/False 11 Aug 2017 If you use gevent with requests.get on a HTTPS URL with the default verify=True enabled, you’ll see almost 2x longer execution times than with verify=False.
I made a script to test:
import urllib3
import timeit
import gevent
import requests
from gevent import monkey
monkey.patch_all()
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
def test_request(verify):
resp = requests.get(
'https://www.instagram.com/instagram/?__a=1',
verify=verify,
)
resp.raise_for_status()
start_time = timeit.default_timer()
for x in range(100):
test_request(verify=True)
print "verify=True took:", timeit.default_timer() - start_time, "secs"
start_time = timeit.default_timer()
for x in range(100):
test_request(verify=False)
print "verify=False took:", timeit.default_timer() - start_time, "secs"
start_time = timeit.default_timer()
gevent.wait([gevent.spawn(test_request, True) for x in range(100)])
print "gevent verify=True took:", timeit.default_timer() - start_time, "secs"
start_time = timeit.default_timer()
gevent.wait([gevent.spawn(test_request, False) for x in range(100)])
print "gevent verify=False took:", timeit.default_timer() - start_time, "secs"
Here are the results:
verify=True took: 40.3454630375 secs verify=False took: 39.3803040981 secs gevent verify=True took: 2.23735189438 secs gevent verify=False took: 1.58263015747 secs
I suspect that gevent is having trouble using pyopenssl concurrently because it’s a C library.