Streaming Live Captures to CloudShark

We’ve been talking a whole lot about integration lately. From our recent bout at Cisco Toolapalooza, to the great work that’s being done with Meraki, we’re finding that the best way people get comfortable with CloudShark is by incorporating it into their existing tools. There are a great many tools out there that can produce packet captures, and each one can find a different way to get those captures into CloudShark for easier collaboration and management.

One question we’ve got a lot is “can you stream files into CloudShark”? While CloudShark needs completed capture files in order to be able to sort and view them, you can stream packet captures into CloudShark and have them view-able once the capture is completed. Here’s how:

It is possible for external capture tools such as tcpdump and dumpcap to stream live captures directly to CloudShark. The combination of a HTTP/S PUT along with CloudShark’s upload method allows the an incoming capture to be created without knowing the final capture file size. This makes it possible to stream a live capture directly to CloudShark.

Using tcpdump

Here is a simple example with tcpdump. The tcpdump capture output is directed to STDOUT with the -w - argument. The -c 10 argument of tcpdump is used to limit the capture to 10 packets. STDOUT is then piped to curl for the upload. Curl is called with --upload-file - which reads from STDIN as the contents of the file to upload.

$ tcpdump -i en1 -c 10  -w - | curl -X PUT \
    --upload-file - http://cloudshark/api/v1/[token]/upload\?filename=foo.pcap

tcpdump: listening on en1, link-type EN10MB (Ethernet), capture size 65535 bytes
10 packets captured
15 packets received by filter
0 packets dropped by kernel

Using control-C

The simple example above becomes slightly more challenging if you wish to use control-C to stop the capture. In order to stop the tcpdump process, but not stop the curl process, you’ll need to create a helper script that traps the control-C signal.

Here is an example in bash.

!/bin/bash

-- trap SIGINT so we can interupt tcpdump
trap '' 2

-- start a curl upload
curl -X PUT --upload-file - http://cloudshark/api/v1/[token]/upload\?filename=foo.pcap\&additional;_tags=one,two,three

Now this script can be combined with tcpdump or dumpcap. When control-C is entered on the command-line, the dumpcap process will terminate and curl will finish uploading the capture.

$ dumpcap -i en1 -c 1000 -w - | ./upload.sh
Capturing on en1
File: -
Packets captured: 14
Packets received/dropped on interface en1: 14/0 (100.0%)
{"filename":"endless2.pcap","id":"2e6dfaa8339d"}

Using live captures

Streaming live captures directly from a capture tool to CloudShark can be very useful. The filename, additional_tags, and comments options also make it possible to provide additional information at upload time. The capture file is available in CloudShark once the live capture has finished and the upload is completed.

Using ring buffers

Several capture tools have the concept of ring buffers which allow multiple files to be created during a capture. For example, each capture file might be limited to a maximum size. This approach works well with CloudShark since individual capture files can be created. However, the ring buffer approach is not compatible with the streaming approach outlined above. In general, the ring buffer capability of these tools requires an actual file to be created. You can, however, create a different upload script that will watch for new files and upload them as soon as they are available. The inotifywait tool from the inotify-tools package on Linux can help manage this process.

Here is a sample script to upload capture files produced by ring buffers.

!/bin/bash

inotifywait -e close_write -m --format "%f" . | \
while read FILE
do
 curl -X PUT \
    --upload-file $FILE http://cloudshark/api/v1/[token]/upload\?filename=$FILE
 rm $FILE

Want articles like this delivered right to your inbox?

No spam, just good networking

Articles

Streaming Live Captures to CloudShark