matrix14

Python and Bash Kung Fu

In this blog post I am going to demonstrate some more Kung Fu with Bash/Python to support network forensics.  I hope to demonstrate some basic concepts that can be extracted to leverage in additional use cases, whether they be offensive or defensive.  As a defender you will commonly have to summarize and lookup IP addresses.

I often run into cases when a vendor will release a boat load of Indicators of Compromise (IOCs).  Think Mandiant, FireEye, Dell SecureWorks, etc.  They will give out reports that release a large amount of domains/IPs associated with a particular attack campaign.

Let’s use a recent report released by FireEye on the Sunshop campaign.  We won’t focus on the attack specifics, but rather the boat load of domains provided in this report.

First I will extract the domains from the report and check what they currently resolve to using Google’s DNS.  This task can be quickly accomplished using a combination of bash/python Kung-Fu.  For the examples in this blog post I will focus on a subset of the domains from the report so you can focus more on the CLI Kung Fu.   Below I simply echo the domains to the bash terminal and use awk to pull only the domain and not the bullet point.  Then I pipe this through sed and replace the “

[.]” with just a “.” so I can look the domains up in another script.  Finally I redirect STDOUT to a file with “>>”:

Now I take those domains and throw them at a script that queries Google’s DNS server (8.8.8.8):

Normally this is a very long list so I will echo the output again and then remove any lines with “Failed” by using “grep -v failed”.  Switch v with grep/egrep is very useful because it can remove lines from the output instead of printing the matches.  Then I print just the first element in the line based on a space deliminator using “awk ‘{print $1}’”.  Finally I will pipe this through the sort and uniq utilities to remove duplicate entries and redirect that output to a file to be leveraged by another Python script:

Here I run the ips.txt file I created in the above screen shot which gives me the output you see just before the highlighted text in the screen shot below.  Then I echo that output and using “sed -r ‘s/^/or net /g’” to add “or net” at the beginning of each line.  This makes a BPF filter I can then leverage to hunt for this particular campaign:

This process was broken down into stages so you could see how doing some quick bash Kung Fu can help mold data to fit into a Python script to do more heavy lifting.  Below you can see examples of the code heavily commented to help you understand what the code is doing and hopefully apply it to your own tools:

domain2ip.py:

d2p_script

iptools.py:

cymru_whois_py