httpry is a tool specialized for the analysis of web traffic. The tool itself can be used to capture traffic (httpry -o file) but other other tools are better suited for that such as tcpdump, Snort, Sguil. When it comes to finding out if certain types of files were downloaded via http, this tool does a super job. It can be used in combination with regular expressions (Regex) to find if a file, a script or a malware was downloaded from site or by a host and will ignore everything else. Whether the http traffic is using port 80, 443, 8080, etc, it will parse and display all the web traffic using this simple command:

httpry -i eth0

If you are working with a large pcap file and want to filter on a particular IP or network, httpry support libpcap filters to zoom in on the web traffic you want to analyze. This libpcap filter will show all the web traffic associated with host 192.168.5.25 using this filter:

httpry -r file 'host 192.168.5.25'

07/28/2010 18:00:02 192.168.5.25 216.66.8.10 > GET www.symantec.com /enterprise/security_response/threatexplorer/threats.jsp HTTP/1.0 - -
07/28/2010 18:00:02 216.66.8.10 192.168.5.25 < - - - HTTP/1.0 301 Moved Permanently
07/28/2010 18:00:02 192.168.5.25 216.66.8.16 > GET www.symantec.com /business/security_response/threatexplorer/threats.jsp HTTP/1.0 - -
07/28/2010 18:00:03 216.66.8.16 192.168.5.25 < - - - HTTP/1.0 200 OK
07/28/2010 18:00:03 192.168.5.25 67.97.80.71 > GET vil.nai.com /VIL/newly_discovered_viruses.aspx HTTP/1.0 - -
07/28/2010 18:00:03 192.168.5.25 67.97.80.71 > GET vil.nai.com /VIL/newly_discovered_viruses.aspx HTTP/1.0 - -
07/28/2010 18:00:03 67.97.80.71 192.168.5.25 < - - - HTTP/1.1 200 OK
07/28/2010 18:01:48 74.125.157.101 192.168.5.25 < - - - HTTP/1.1 200 OK
07/28/2010 18:01:48 192.168.5.25 173.194.15.95 > GET safebrowsing-cache.google.com /safebrowsing/rd/ChNnb29nLW1hbHdhcmUtc2hhdmFyEAEYlZQCIJaUAioFFooAAAEyBRWKAAAB HTTP/1.1 - -
07/28/2010 18:01:48 173.194.15.95 192.168.5.25 < - - - HTTP/1.1 200 OK

If you are checking for a particular file extension such as.exe, .js, .msi, .jpg, etc, if you combined your search with grep, httpry can be used to find if any binaries (i.e. malware) were downloaded from a certain site or by a particular client using a pcap captured files. In this example we grep for all the JavaScript transffered by host 192.168.5.25.

httpry -r file 'host 192.168.5.25' | grep "\.js"

07/28/2010 10:57:08 192.168.5.25 69.192.143.238 > GET www.quickquote.lincoln.com /static/com/forddirect/presentation/constants/SkinConstants_lincoln.js HTTP/1.1 - -
07/28/2010 10:57:08 192.168.5.25 69.192.143.238 > GET www.quickquote.lincoln.com /yui/yahoo-dom-event/yahoo-dom-event.js HTTP/1.1 - -
07/28/2010 10:57:08 192.168.5.25 69.192.143.238 > GET www.quickquote.lincoln.com /static/com/forddirect/application/bp20/metrics/s_code.js HTTP/1.1 - -

The httpry website is here. The tarball can be download here and a freeBSD port here.
 

-----------

Guy Bruneau IPSS Inc. gbruneau at isc dot sans dot org