Scapy Primer - Part III

The ability to read from and write to files is extremely helpful when it comes to programming and scripting. This ability allows users to either store or read data from a pre-existing source. Where this becomes helpful with a library such as Scapy is that it allows us to read from a packet capture file (pcap). The scenarios in which this type of activity is advantageous would be when a programmer needs process the network traffic that has already occurred. This is a useful ability when we need to analyze, search or replay network traffic.

Here are a few of the methods that are used in order to read from or write to a pcap file.

rdpcap()

Reading packets from a pcap file requires the use of the rdpcap() method. This method requires a file location as a parameter. The best way to ensure that the proper file is given as input is to provide the full path to the file within double quotes.

Let's take a look at an example in which we will attempt to read the file "/tmp/synflood.pcap":

The pcap contains a Syn-Flood attempt. This is a denial of service (DoS) attack that attempts to send a high number of SYN flags to an open port to illicit a SYN/ACK response. This would be an expected response from an open port per the three-way handshake. The source, 192.168.10.1, is spoofed so that any responses are sent in to the void. This attack is antiquated and typically does not work against modern hosts.

packets = rdpcap("/tmp/synflood.pcap")

This line reads each packet in the in the the variable as a list. Checking the variable 'packets' results in a listing that shows 100 packets, which is to be expected as a result of original PCAP.

wrpcap()

Writing packets to a pcap file is very similar to the write process with the exception of the number of parameters that we will provide. The parameters are the file location and the packets that are to be written to file.

The ability to write packets is very much tied to the ability to collect the packets that are to be written to file. Up until this point we have only explored how to collect traffic that has been produced in response to certain stimuli that the programmer has created. This obviously has certain limits when it comes to the types of traffic that we can collect. Now we will explore one additional method that can be used to collect all traffic that passes a network interface card.

sniff()

The sniff() method will sniff traffic from a network interface card of the host machine which the method is being run on. Of course this means that the user running the method should have the ability to read traffic from the network card in promiscuous mode. The sniff() method can take multiple parameters that can specify the type and quantity of traffic it receives.

     iface  - This option specifies the interface that traffic will be collected on.
          
     filter  - This option takes a Berkley Packet Filter in order to limit the type of traffic that is
                  collected.
     count - This option specifies the number of packets that will be collected.

It is not a requirement that the packets sniffed are then stored in a pcap file. They can be read in to a variable as well but for the sake of persistence and relevance, sniffed traffic will be written to file in our examples. Sniffing traffic in this fashion is helpful when we have some idea of the type of traffic we are seeking.

Replay

There may be scenarios where we would like to replay traffic from a pcap file. Research, investigation and malware analysis are a few that come to mind. These situations can benefit from replaying packets in a lab environment.  Scapy makes this a relatively simple task to achieve.

So if we want to replay some traffic on the network. It only takes a few lines of code to do this.

from scapy.all import *
pcap = '/tmp/attack.pcap' 
traffic = rdpcap(pcap)


for packet in traffic: 
     sendp(packet)

We use the sendp() function in this script since the traffic in the pcap contains Layer 2 data. Recall that we previously stated that this function must be used when the packet contains Ethernet level data.

This will allow us to replicate this traffic on a test network, provided that test hosts have network settings that have been modified to match IP addresses in the file. This could save some time when we would like to determine if our systems are vulnerable to certain attacks. Just get a pcap, replay the traffic and observe what happens.

Conclusion

This primer only touches the surface of the possibilities of Scapy. The main purpose was to show just some of the features and abilities. This tool will be used in future posts because it is a great for automating tasks and researching responses from networking devices. It also will provide great benefits for reconnaissance and evasions techniques. A more full featured set of documentation can be found at  http://www.secdev.org/projects/scapy/files/scapydoc.pdf.

Scapy Primer - Part II

Obviously if we create a packet, the next logical step would be to send that packet over the network to a destination. This is a necessary step, especially when attempting to analyze the response to a crafted packet. Since crafting packets allows us to deviate from protocol and RFC standards, an unexpected or unique response may be received. This is a useful technique when it comes to fingerprinting and troubleshooting a host. There are other useful applications when analyzing the response to these crafted packets as well.

Packet Trasmission

A number of functions are available for transmitting packets. The following functions are the most noteworthy and useful.

send()

The send() function is used to send a packet over a network. This function is typically used when we do not want to store the response for further use in the script or program.

     >>> packet = IP(dst="10.10.10.1")/TCP(dport=80)
     >>> ans = send(packet)
     .
     Sent 1 packets.
     >>> ans
     >>>

Notice that no value is returned as a response to the sent packet so the result of printing the variable 'ans' is an empty one.

Let's use tcpdump to get a more detailed picture of what is happening on the network when we use the send() function.

We receive a SYN/ACK as expected since destination port 80 is open on host 10.10.10.1. The host that initiated the conversation, 192.168.93.136, responds to the SYN/ACK with a RST packet. This is due to Scapy making the initial request and not the Linux Kernel. Since this is the case the host does not expect to receive a SYN/ACK, thus it responds with a RST packet.

sr()

The sr() function is very much similar to send(). The difference between the two functions is that sr() will return an answer to the stimulus packet that has been sent. For example, if we were to send an ICMP request to a destination, this function would listen and store the responses within a list that can be enumerated. Unanswered packets will also be returned using sr().

    >>> packet = IP(dst="10.10.10.1")/TCP(dport=80)/"Open Port"
    >>> ans,unsans = sr(packet)
    Begin emission:
    .Finished to send 1 packets.
    *
    Received 2 packets, got 1 answers, remaining 0 packets
    >>> ans
    <Results: TCP:1 UDP:0 ICMP:0 Other:0>
    >>> ans[0]
    (<IP  frag=0 proto=tcp dst=10.10.10.1 |<TCP  dport=http |<Raw  load='Open Port' |>>>,   
    <IP  version=4L ihl=5L tos=0x0 len=40 id=31 flags= frag=0L ttl=64 proto=tcp   
    chksum=0x5a98 src=10.10.10.1 dst=10.0.2.15 options=[] |<TCP  sport=http dport=ftp_data 
    seq=448001 ack=2 dataofs=5L reserved=0L flags=A window=65535 chksum=0xb94d 
    urgptr=0 |<Padding  load='\x00\x00\x00\x00\x00\x00' |>>>)
    >>> packet = IP(dst=["10.10.10.101","10.10.10.251"])/ICMP()/"Ping"
    >>> ans,uans = sr(packet)
    Begin emission:
    .Finished to send 2 packets.
    **
    Received 3 packets, got 2 answers, remaining 0 packets
    >>> ans
    <Results: TCP:0 UDP:0 ICMP:2 Other:0>
    >>> uans
    <Unanswered: TCP:0 UDP:0 ICMP:0 Other:0>
    
Now let's access the elements within this list. We would do this just like any other list in Python. The integer within the brackets represents the index of the stimuli/response pair we would like to access. In this example the first pair is an echo request (Type 8 Code 0) and the echo reply (Type 0) to the original request.

    >>> ans[0]
    (<IP  frag=0 proto=icmp dst=10.10.10.101 |<ICMP  |<Raw  load='Ping' |>>>, <IP  
    version=4L ihl=5L tos=0x0 len=32 id=55 flags= frag=0L ttl=63 proto=icmp chksum=0x5b29 
    src=10.10.10.101 dst=10.0.2.15 options=[] |<ICMP  type=echo-reply code=0 chksum=0x412f 
    id=0x0 seq=0x0 |<Raw  load='Ping' |<Padding  load='\x00\x00\x00\x00\x00\x00\x00\x00\x00
    \x00\x00\x00\x00\x00' |>>>>)

The second pair is the echo request and the ICMP destination unreachable/host unreachable from the gateway (Type 3 Code 1). The host 10.10.10.251 does not exist which is why this response is generated. We are only concerned with these responses for this illustration so the IP and ICMP errors can be ignored at this time.

    >>> ans[1]
    (<IP  frag=0 proto=icmp dst=10.10.10.251 |<ICMP  |<Raw  load='Ping' |>>>, <IP   
    version=4L ihl=5L tos=0xc0 len=60 id=56 flags= frag=0L ttl=63 proto=icmp 
    chksum=0x5a4c src=10.10.10.101 dst=10.0.2.15 options=[] |<ICMP  type=dest-unreach 
    code=host-unreachable chksum=0xdd1f unused=0 |<IPerror  version=4L ihl=5L tos=0x0 
    len=8192 id=256 flags=  frag=0L ttl=63 proto=icmp chksum=0x59c9 src=10.0.2.15
    dst=10.10.10.251 options=[]
    |<ICMPerror  type=echo-request code=0 chksum=0x392f id=0x0 seq=0x0 |<Raw  load='Ping'
    |>>>>>)


The list that is returned as a result of this function is actually multidimensional. We will only explore the first two dimensions but it should be noted that there are many more. Each dimension allows us to dive down deeper in to the packet. We will access the second dimension in the following example. This index relates to the particular transaction that we are interested in accessing within the pair. In this case we will be examining the second ICMP request that was sent to host 10.10.10.251.

    >>> ans[1][0]
    <IP  frag=0 proto=icmp dst=10.10.10.251 |<ICMP  |<Raw  load='Ping' |>>>
    >>> ans[1][0].type
    8
    >>> ans[1][0].code
    0

sr1()

The sr1() function is another variation of the send() function. This function listens for a single response to a packet that has been sent. This is quite useful if we are only concerned with the first response to a packet that has been sent. Unanswered packets are stored within a list as well.


     >>> packet = IP(dst="10.10.10.1")/TCP(dport=22,flags="S")

     >>> ans = sr1(packet)
     Begin emission:
     Finished to send 1 packets.
     *
     Received 1 packets, got 1 answers, remaining 0 packets

We define a packet that is destined for TCP port 22 (Secure Socket Layer) and has a TCP flag that is set to 'S' for SYN. This should initiate the beginning of the TCP handshake and elicit a SYN/ACK flag from the destination host if the port is open.

     >>> ans
     <IP  version=4L ihl=5L tos=0x0 len=44 id=86 flags= frag=0L ttl=64 proto=tcp
     chksum=0x5a5d src=10.10.10.1 dst=10.0.2.15 options=[] |<TCP  sport=ssh dport=ftp_data
     seq=10624001 ack=1 dataofs=6L reserved=0L flags=SA window=65535 chksum=0x5b2f
     urgptr=0 options=[('MSS', 1460)] |<Padding  load='\x00\x00' |>>>
     >>> ans[0][1]
     <TCP  sport=ssh dport=ftp_data seq=10624001 ack=1 dataofs=6L reserved=0L flags=SA
     window=65535 chksum=0x5b2f urgptr=0 options=[('MSS', 1460)] |<Padding  load='\x00\x00'
     |>>
     >>> ans[0][1].flags
     18L



Keep in mind that the variable 'ans' stores the response to the 'packet' that was sent. 'ans[0][1]' specifies the second layer in this response packet. In our case, this would be the TCP layer. When we request the 'flags' field we are presented with the integer '18'. This is the combined value of the TCP flags (ACK = 16, SYN = 2).

srp()

The srp() function is used when sending a layer 2 (data link layer) datagram packet and will listen for responses in much the same way as the previous function does.

     >>> datagram =  
     Ether(src="00:00:00:12:34:ab",dst="14:da:e9:01:3e:23")/IP(dst="10.10.10.101")
     >>> sendp(datagram)
     .
     Sent 1 packets.

This function must be used if the Ether() layer has been defined within a packet. If a function such as send() is used, a warning will be produced indicating that there is an issue.

     >>> send(datagram)
     WARNING: Mac address to reach destination not found. Using broadcast.
     .
     Sent 1 packets.

Now that we have a foundation for crafting and transmitting packets, we can look in to actually reading packets from a packet capture file. This is a tremendously helpful capability when attempting to reproduce activity such as an exploit or attack on a network. We will explore these techniques as well as a few other in next post.

Scapy Primer

Networking is a fundamental knowledge area when it comes to effectively investigating, analyzing and researching the vast majority of security-related topics and events. This makes a lot of sense considering that most computers are on a network, such as the internet, for communication purposes. It is likely that when a system has been attacked and/or compromised, it was done so remotely as opposed to locally. Obviously this is just a result of the way our computers communicate these days. Tools and frameworks that are specifically focused on networking are very useful in this type of environment.

Scapy is a packet manipulation library that has the ability to craft, manipulate and even replay packets on the wire. It can be used either in scripts, programs or interactively at the command line. Obviously this has many practical uses when it comes to both network and security-related research/investigation.  Before we can practically use this tool, a little primer is in order to get a feel for the library and it's capabilities. This is not meant to be a full manual but rather a quick start guide.

Packet Creation

Scapy allows a packet to be created using layers much in the way that they are logically layered in the same way that the OSI and TCP/IP models. You can even go as far as adding additional layers to a packet, although it really wouldn't make a lot of sense and the response may be unpredictable. A remote host wouldn't really understand a packet that has multiple IP layers since it is an abnormal condition. This could be useful if we are looking for a certain response but it is not likely to yield any useful information.

So let's say I want to send a packet to a remote host to confirm if it is running an SMTP (Simple Mail Transfer Protocol) server. SMTP runs on TCP port 25 by default although this isn't a requirement. Let's take a look at how to create a packet for this purpose.

Here are the characteristics of the packet we would like to create:

Source Host:         192.168.93.134
Source Port:          1025
Destination Host: 192.168.93.136
Destination Port:  25

This packet can be created one of two ways. One way is more compact than the other but we will review both.

Method One

The first method is the longest since creation of each layer is performed within a separate declaration.

First, let us define the network layer (Layer 3).

     >>> l3 = IP(dst="192.168.93.134",ttl="128")

Next, we define the transport layer (Layer 4)

      >>> l4 = TCP(sport=1025,dport=25)

Then, we will define the payload that we plan to deliver to the destination.

      >>> data = "Testing"

In this case, we are not using a true SMTP value but a plain-text string.

Now we will combine all three layers in to a single packet. The layers are 'layered' on top of one another using the '/' operator.

      >>> packet = l3/l4/data

Any undefined values such as source host, flags or TCP Options are set to defaults values. The same is true for layers such as the data link layer. These default values can be obtained by using the following notation within interactive mode:
    
     ls(protocol)
     >>> ls(IP)

After a packet has been created, it can be reviewed by simply typing the variable name at the command prompt. This will show the user-defined values of the packet.

     >>> packet
     <IP  frag=0 ttl='128' proto=tcp dst=192.168.93.134 |<TCP  sport=1025 dport=smtp |<Raw  
      load='Testing' |>>>

While the first method provides a more detailed way to craft a packet, the same crafting can also be done in a single line of code. The first method has certain advantages such as reuse of the layer definitions in multiple packet creation. In other words, if we want to change the value at one layer, such as the transport layer, we can change the value of that single layer instead of having to redefine the entire packet.

Method Two

The second method is great for quick packet creation and brevity. All components of the packet can be defined at a single time. Let's take a look at the creation of the same packet that we created in the above section using this shorter method.

Once again, here are the characteristics of the packet we would like to create:

     Source Host:         192.168.93.134
     Source Port:          1025
     Destination Host: 192.168.93.136
     Destination Port:  25

     >>> packet = IP(dst="192.168.93.134",ttl="128")/UDP(sport=1025,dport=25)/"Testing"

This saves us a few lines of code. Let's take a look at the packet.

     >>> packet

     <IP  frag=0 ttl='128' proto=udp dst=192.168.93.134 |<UDP  sport=1025 dport=25 |<Raw  

      load='Testing' |>>>

Packet Details

While packet creation is a great feature, we need a way to extract information from this packet. This includes information such as IP address, port, data, flags and a ton of other header information from each individual layer. 

Scapy provides the following methods for extraction of this information:

getlayer()

Using the getlayer() method will retrieve information pertaining to a single layer of a packet. This information includes non-default fields within the layer.

haslayer()

The haslayer() method allows the programmer to check to see if a layer exists. If the value exists, a '1' will be returned. If it does not exists, a '0' will be returned.

show() 

The show() method can be used to display information about the packet. This is helpful mainly while in interactive mode. It will display a more human friendly view of the packet fields. It will show both user-defined and default values.

The value of a packet header value can be retrieved using the following syntax:

     packet.layer.fieldname

Note:  Requesting flag values will result in a decimal representation of the flags that are set so it

 is up the the programmer to interpret these values to with their associated flag values.

That concludes the basic overview of the packet creation process but there are additional features that we will need to cover before we can begin utilizing Scapy. These features include methods that are used for packet transmission, sniffing and file I/O. While this section is a good launching point, further explanation is needed before any relevant tasks can be performed using Scapy.

CSAW 2014 Reverse Engineering Challenge

Recently I have been competing in a number of InfoSec Capture the Flag competitions with a group of friends and co-workers. The most recent competition was the Polytechnic Institute of New York University's annual CSAW CTF (https://ctf.isis.poly.edu/). This annual, jeopardy-style competition hosts a qualification round that is open to high school students, college students and industry professionals.

During this competition, I focused on the Reverse Engineering category of challenges. The following writeup is for the 200 point category Reverse Engineering challenge. The file can be downloaded here:

https://www.dropbox.com/s/5p2v89z0mlvae2i/csaw2013reversing2.exe?dl=0

I start by running the strings command on the executable file. The motive here is to see if I am able to find any clues or even the flag itself. Using this command can also give a pretty good indication of whether or not the file is actually an executable or has another type of file embedded within it. The results of this command do not yield any valuable clues leading to the discovery of the challenge flag.

The string 'Flag' is present within the output but submitting "RSDS" does not result in success:

One point of interest that I notice is that there are some strings that are related to date information so it's possible this could be a clue:

Maybe the flag can only be obtained when the operating system clock is a certain time, however, more investigation is still needed.

All other human-readable strings appear to be standard Windows API function calls so further analysis is required. I will try to run the file so that I can get a good idea of what it is actually doing and possibly find additional clues. Execution of the file results in a text box pop-up with what appears to be ciphertext.

Pressing any of the three buttons results in the pop-up closing as well as the program.

It would appear that using some basic techniques to analyze this file aren't likely to yield a flag for this challenge. That being the case, it looks like it's time to break out the disassembler and debugger. I prefer to use IDA Pro and OllyDbg but other alternatives are available as well.

Disassembling the program in IDA Pro shows a number of functions, many of which are standard Microsoft DLL functions. The functions names were also present as a result of the strings program output. IDA assigns the function name 'Sub_401000' to the function starting at location 0x401000. This appears to show some promise in regards to finding a solution for this challenge.

Sub_401000 Graphical View

The instructions in the graph indicate that there are two "for loops" in this particular function. There is a notable 'XOR' instruction within the second loop that looks like it could be used for some sort of encryption or decryption. Values are then popped off of the stack and returned.

Looking at these instructions can be a bit of a struggle so utilzing OllyDbg could be helpful in order to see what values are being modified and returned after these loops have concluded.  Let's start by running the program using the debugger.

In order to see what occurs at the end of the second "for loop" a breakpoint should be set within OllyDbg at the 'POP EDI' instruction. So I run the program with the breakpoint set and hit the jackpot:

     Note: It can take some searching to find the location since the program may

     load at a different memory offset in IDA and OllyDbg.

Once the EDI register is popped off the stack, the following ASCII string is found:

"flag{reversing_is_not_that_hard!}"

Submission of this string results in a "Congratulations" message. 

The entire challenge required using both basic and advanced techniques of file analysis. This appears to have been a pretty straightforward reversing challenge but knowing where to look is vital to successful completion.