ProxyBroker - HTTP(S), SOCKS4/5.CONNECT80/23

https://raw.githubusercontent.com/co...nd_example.gif
ProxyBroker is an open source tool that asynchronously finds public proxies from multiple sources and concurrently checks them.
Features
Finds more than 7000 working proxies from ~50 sources.
Support protocols: HTTP(S), SOCKS4/5. Also CONNECT method to ports 80 and 23 (SMTP).
Proxies may be filtered by type, anonymity level, response time, country and status in DNSBL.
Work as a proxy server that distributes incoming requests to external proxies. With automatic proxy rotation.
All proxies are checked to support Cookies and Referer (and POST requests if required).
Automatically removes duplicate proxies.
Is asynchronous.
Usage
CLI Examples
Find
Find and show 10 HTTP(S) proxies from United States with the high level of anonymity:
Code:
<pre class="alt2" dir="ltr" style="
margin: 0px;
padding: 6px;
border: 1px solid rgb(0, 0, 0);
width: 640px;
height: 34px;
text-align: left;
overflow: auto;
background: rgb(37, 37, 37) none repeat scroll 0% 0%;
border-radius: 5px;
font-size: 11px;
text-shadow: none;">$ proxybroker find --types HTTP HTTPS --lvl High --countries US --strict -l 10</pre>
https://raw.githubusercontent.com/co...nd_example.gif
Grab
Find and save to a file 10 US proxies (without a check):
Code:
<pre class="alt2" dir="ltr" style="
margin: 0px;
padding: 6px;
border: 1px solid rgb(0, 0, 0);
width: 640px;
height: 34px;
text-align: left;
overflow: auto;
background: rgb(37, 37, 37) none repeat scroll 0% 0%;
border-radius: 5px;
font-size: 11px;
text-shadow: none;">$ proxybroker grab --countries US --limit 10 --outfile ./proxies.txt</pre>
https://raw.githubusercontent.com/co...ab_example.gif
Serve
Run a local proxy server that distributes incoming requests to a pool of found HTTP(S) proxies with the high level of anonymity:
Code:
<pre class="alt2" dir="ltr" style="
margin: 0px;
padding: 6px;
border: 1px solid rgb(0, 0, 0);
width: 640px;
height: 34px;
text-align: left;
overflow: auto;
background: rgb(37, 37, 37) none repeat scroll 0% 0%;
border-radius: 5px;
font-size: 11px;
text-shadow: none;">$ proxybroker serve --host 127.0.0.1 --port 8888 --types HTTP HTTPS --lvl High</pre>
https://raw.githubusercontent.com/co...ve_example.gif
Code:
<pre class="alt2" dir="ltr" style="
margin: 0px;
padding: 6px;
border: 1px solid rgb(0, 0, 0);
width: 640px;
height: 338px;
text-align: left;
overflow: auto;
background: rgb(37, 37, 37) none repeat scroll 0% 0%;
border-radius: 5px;
font-size: 11px;
text-shadow: none;">Requirements
Python 3.5 or higher
aiohttp
https://pypi.python.org/pypi/aiohttp
aiodns
https://pypi.python.org/pypi/aiodns
maxminddb
https://pypi.python.org/pypi/maxminddb
Installation
To install last stable release from pypi:
$ pip install proxybroker
The latest development version can be installed directly from GitHub:
$ pip install -U git+https://github.com/constverum/ProxyBroker.git</pre>
Code:
<pre class="alt2" dir="ltr" style="
margin: 0px;
padding: 6px;
border: 1px solid rgb(0, 0, 0);
width: 640px;
height: 498px;
text-align: left;
overflow: auto;
background: rgb(37, 37, 37) none repeat scroll 0% 0%;
border-radius: 5px;
font-size: 11px;
text-shadow: none;">ProxyBroker$ proxybroker
usage: proxybroker [--max-conn MAX_CONN] [--max-tries MAX_TRIES]
[--timeout SECONDS] [--judge JUDGES] [--provider PROVIDERS]
[--verify-ssl]
[--log [{NOTSET,DEBUG,INFO,WARNING,ERROR,CRITICAL}]]
[--version] [--help]
{find,grab,serve,update-geo} ...
Proxy [Finder | Checker | Server]
Commands:
These are common commands used in various situations
{find,grab,serve,update-geo}
find Find and check proxies
grab Find proxies without a check
serve Run a local proxy server
update-geo Download and use a detailed GeoIP database
Options:
--max-conn MAX_CONN The maximum number of concurrent checks of proxies
--max-tries MAX_TRIES
The maximum number of attempts to check a proxy
--timeout SECONDS, -t SECONDS
Timeout of a request in seconds. The default value is
8 seconds
--judge JUDGES Urls of pages that show HTTP headers and IP address
--provider PROVIDERS Urls of pages where to find proxies
--verify-ssl, -ssl Flag indicating whether to check the SSL certificates
--log [{NOTSET,DEBUG,INFO,WARNING,ERROR,CRITICAL}]
Logging level
--version, -v Show program's version number and exit
--help, -h Show this help message and exit</pre>