|Deletions are marked like this.||Additions are marked like this.|
|Line 18:||Line 18:|
|Sample configuration, make sure you set up the Port and Allow (here, I'm using my localhost)||Sample configuration, make sure you set up the Port and Allow (here, I'm using my localhost). '''N.B.''' Most of these configuration settings are default and can be easily altered to suit. Both the ''tinyproxy.conf'' configuration file and ''LogLevel Info'' ensure that the most verbose help is at hand to understand settings and to debug performance.|
|Line 22:||Line 22:|
|Line 35:||Line 34:|
|Logfile "/var/log/tinyproxy.log"||Logfile "/var/log/tinyproxy/tinyproxy.log"|
|Line 37:||Line 36:|
|PidFile "/var/run/tinyproxy.pid"||PidFile "/var/run/tinyproxy/tinyproxy.pid"|
(Ubuntu 11.04 Natty, Kernel Linux 2.6.38-10-generic, GNOME 2.32.1)
Tinyproxy is a light-weight HTTP/HTTPS proxy daemon for POSIX operating systems. Designed from the ground up to be fast and yet small, it is an ideal solution for use cases such as embedded deployments where a full featured HTTP proxy is required, but the system resources for a larger proxy are unavailable. Fore more information see here.
sudo apt-get install tinyproxy
sudo vi /etc/tinyproxy.conf
Sample configuration, make sure you set up the Port and Allow (here, I'm using my localhost). N.B. Most of these configuration settings are default and can be easily altered to suit. Both the tinyproxy.conf configuration file and LogLevel Info ensure that the most verbose help is at hand to understand settings and to debug performance.
Port 5555 Allow 127.0.0.1 Filter "/etc/filter" FilterURLs On FilterDefaultDeny No #filters will act as a blacklist User nobody Group nogroup ViaProxyName "tinyproxy" ConnectPort 443 ConnectPort 563 Timeout 600 DefaultErrorFile "/usr/share/tinyproxy/default.html" StatFile "/usr/share/tinyproxy/stats.html" Logfile "/var/log/tinyproxy/tinyproxy.log" LogLevel Info PidFile "/var/run/tinyproxy/tinyproxy.pid" MaxClients 100 MinSpareServers 5 MaxSpareServers 20 StartServers 10 MaxRequestsPerChild 0
If necessary (will act as a blacklist, because of FilterDefaultDeny No)
sudo vi /etc/tinyproxy/filter
and add sites urls to be blocked
Commands to Stop,Start, and Restart
sudo /etc/init.d/tinyproxy stop sudo /etc/init.d/tinyproxy start sudo /etc/init.d/tinyproxy restart
Test the proxy with your browser
- For Firefox, menu Preferences, tab General, button Connection settings. Then select Manual Proxy Configuration and enter the host you defined above and the port.
- If you have created the filter above, and browse to google.com, the proxy should block you.
Configure Nutch (Nutch O.8)
Copy the proxy configuration (see below) from conf/nutch-default.xml to conf/nutch-site.xml and fill up with the values of your proxy
<property> <name>http.proxy.host</name> <value>192.168.0.157</value> <description>The proxy hostname. If empty, no proxy is used.</description> </property> <property> <name>http.proxy.port</name> <value>5555</value> <description>The proxy port.</description> </property>
Now if you crawl sites, Nutch will use your proxy. You can monitor it by looking at the logs of Tinyproxy during a crawl:
sudo tail -f /var/log/tinyproxy.log