ImportError: urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the ‘ssl’ module is compiled with LibreSSL 2.8.3

The reason why the error message mentioned OpenSSL 1.1.1+ and LibreSSL 2.8.3 is that urllib3 v2.0 (the version you’ve installed) requires OpenSSL 1.1.1+ to work properly, as it relies on some new features of OpenSSL 1.1.1. The issue is that the version of the ‘ssl’ module that is currently installed in your environment is compiled … Read more

How to retry urllib2.request when fails?

I would use a retry decorator. There are other ones out there, but this one works pretty well. Here’s how you can use it: @retry(urllib2.URLError, tries=4, delay=3, backoff=2) def urlopen_with_retry(): return urllib2.urlopen(“http://example.com”) This will retry the function if URLError is raised. Check the link above for documentation on the parameters, but basically it will retry … Read more

Why do I constantly see “Resetting dropped connection” when uploading data to my database?

Requests uses Keep-Alive by default. Resetting dropped connection, from my understanding, means a connection that should be alive was dropped somehow. Possible reasons are: Server doesn’t support Keep-Alive. There’s no data transfer in established connections for a while, so server drops connections. See https://stackoverflow.com/a/25239947/2142577 for more details.

Python requests is slow and takes very long to complete HTTP or HTTPS request

There can be multiple possible solutions to this problem. There are a multitude of answers on StackOverflow for any of these, so I will try to combine them all to save you the hassle of searching for them. In my search I have uncovered the following layers to this: First, try logging For many problems, … Read more

urllib3 connectionpool – Connection pool is full, discarding connection

No data is being lost! The connection is being discarded after the request is completed (because the pool is full, as mentioned). This means that this particular connection is not going to be re-used in the future. Because a urllib3 PoolManager reuses connections, it will limit how many connections are retained per hos to avoid … Read more

What should I use to open a url instead of urlopen in urllib3

urllib3 is a different library from urllib and urllib2. It has lots of additional features to the urllibs in the standard library, if you need them, things like re-using connections. The documentation is here: https://urllib3.readthedocs.org/ If you’d like to use urllib3, you’ll need to pip install urllib3. A basic example looks like this: from bs4 … Read more