Transparent Squid - OpenBSD

Squid is a caching proxy that can reduce the bandwidth consumption by caching frequently requested static resources and delivers them to clients offline, saving precious bandwidth and enhances browsing experience.

It is preferably to configure Squid in transperent mode, in which you do configure your firewall "router" as well to intercept all outbound traffic to port 80, for example, and redirect it to Squid on port 3128, the default. At that point, Squid does some magic, pours and mixes chemicals to determine if the requested resource is already in the cache or not, and if it has a valid expiry date. Based on this decission, Squid retreives the requested resource and cache it, or, directly push the rewource from the cache.

The default squid.conf file will suffice for most cases. It is very very well documented and commented. However, there are two directives need to be revised:
http_port 127.0.0.1:3128 transparent
and
acl localnet src 10.10.10.0/24

The first one, http_port, defines the IP address that squid daemon will bind to and the port that is should listen on. The most important of all in the directive is the "transperent" part. If you skipped it, you will have "Invalid Request" errors orinted on the clients' browsers.

The second directive, acl, defines the local network ID. This directive is later interpreted by another directive, acl allow localnet.

Then you have the firewall part. In this document, we will be using PF on an OpenBSD 4.9 machine. The relevant config segements from pf.conf are:

match in on $int_if inet proto tcp from any to any port www rdr-to 127.0.0.1 port 3128
  pass in on $int_if inet proto tcp from any to 127.0.0.1 port 3128 keep state
  pass out on $ext_if inet proto tcp from any to any port www keep state


Reload your PF rules:
pfctl -f /etc/pf.conf

Initialize Squid cache:
squid -z

Confogure Squid to run at system boot:
                vi /etc/rc.local and add:
if [ -x /usr/local/sbin/squid ]; then
           echo -n ' squid'
           /usr/local/sbin/squid
  fi

Now, you can start Squid manually (instead of doing a system reboot) with:
squid

Squid will run in the background. Each subsequent system reboot, Squid will also run with no further interaction.

Use IPtables, RTMPSrv and RTMPDump to Save RTMP Streams

Using iptables and rtmpsrv, you can save the rtmp:// streams, especially the ones  that are well hidden. The rtmp stream is being called from the the .swf file.

You can use packet sniffers to sniff on the macromedia-fs port (1935), bu there is a more convinient and effective way to tackle this.

1. # iptables -t nat -A OUTPUT -p tcp --dport 1935 -j REDIRECT
2. $ rtmpsrv
3. Start playing the online video from a normal web browser.
4. The rtmpsrv command should spit out the rtmp:// URL for the video you are trying to download.
5. # iptables -t nat -D OUTPUT -p tcp --dport 1935 -j REDIRECT
6. $ rtmpdump $url_from_step_4

First, you redirect all the macromedia-fs traffic to your localhost. Then, you start the rtmpsrv to listen on 1935. When you start playing the RTMP video, the request from your browser to the server hosting the video, will get redirected to your localhost where rtmpsrv is listening. What rtmpsrv exactly does, is that it dumps the URL generated from from you client (the swf file) to the media-streaming server. This URL is the most precious, you just take it and as it is and pass it to the rtmpdump tool. The rtmpdump will start dumping the stream to your local disk, amazing!.

In step 5, you just remove the redirection rule.

This howto is meant to be used legally.