NetSecOps problem solvers are never working alone
They say that no person is an island. No one knows this better than NetSecOps people who are in a line of work that requires constant effort and vigilance to keep the network and IT infrastructure running smoothly and securely.
That means that most NetSecOps people work on issues together as a team of analysts and IT professionals, each with different skill sets. Personnel might be dislocated from each other, even across the world. Remote work has become more prevalent and has taken a foothold even among IT professionals as a new norm.
Teams must design their workflow around these realities when troubleshooting issues and resolving network security incidents. This applies perhaps the most to working with packet captures, which, while extremely valuable, hold a lot of data that can be difficult to navigate. Here are some tips to make working with captures as a team easier, more fruitful for the people doing the work, and more efficient for your organization as a whole.
1. Mark your pcap workflow
When you’re collaborating with others on network and security issues, it’s important to include as much information as possible about the work you’ve already done. As long as you’re providing the right context (below), there’s no such thing as too much information. Some markup tips include:
Tag resources
Add as much metadata as you can to your packet captures by tagging and labeling them in the repository you use to organize your packet captures. The most useful tags are those that can tie the capture to a particular incident, ticket, location, or customer.
Tell the story in comments
Take notes as you go by using packet capture comments and tell the story of your investigation. Let them know your conclusions, theories, and unknowns! Comments stay with the capture, so you can write your notes such that they will be accessible to anyone who has access to that resource.
Add annotations
In addition to long-form comments that can apply to a whole capture, pcaps allow you to add annotations directly to individual packets. As you analyze an individual trace file, add annotations that describe your investigative workflow, things you’ve discovered, or questions you might have. In CloudShark, annotations are especially useful, as they support Markdown, letting you add links to other resources and captures.
2. Provide the right context
As the investigation moves from one person to the next, make sure that the next person has the right context when they open a packet capture to continue your work. Context helps cut down on the noise of packet capture data so that an analyst can spend less time getting to the information that is relevant to them.
Start with Zeek or Suricata
Rather than looking purely at list of packets, start with other tools that can process pcaps and provide shortcuts to the data you are looking for. The Zeek network monitoring tool does this by scanning a pcap for high-level information and turning them into a list of “logs”. Zeek is included in CloudShark, with the added ability to pivot to the packets associated with a particular log. You can find more information on how to get the most out of Zeek in our video here.
For security analysis, using an Intrusion Detection System that can scan through pcaps will cut down your investigation time significantly. The Suricata IDS (included with CloudShark) in particular will process a pcap and apply a set of signature-based security rules to it to look for malicious activity and generate a set of alerts. These alerts are the perfect starting point for incident response, and in CloudShark, you can go directly to the packets that caused the alert.
Apply profiles
Use capture profiles to apply the right data columns when looking at the packets. Different data will have different relevant information - for example, 802.11 wireless data includes SSID and signal DBm in the packets, which is helpful for Wi-Fi troubleshooting but probably not for HTTP analysis.
Share your work
Provide context with links directly to the view that an analyst needs to see. If you’ve built a capture view with Wireshark filters, use a system that lets those filters be applied as soon as an analyst looks at it. In CloudShark, filters are applied through the capture URL, so your teammates will see what you see when you send it or include it in a ticket or report. This applies to the Zeek and Suricata views mentioned above as well.
3. Level up your teammates
The key to NetSecOps collaboration is making sure that everyone is on the same page. This applies particularly when entry level personnel are starting an analysis or escalating/receiving next steps to/from their higher-level experts. It also means helping those people learn faster using the tools at their disposal.
Share profiles
You’ve built profiles like we mentioned above to look at specific problems in a specific way. The next step is to save them and share them with the rest of your team to preserve and leverage the expert knowledge your team has gathered over the years. Not only will this help entry-level analysts do their jobs faster, but will help educate them on solving particular problems, making them future experts.
Save examples
Save the best of your completed investigations as examples for other team members. Include all of your packet captures, notes, annotations, and profiles together as a collection. Not only does this help with training new analysts, but also acts as an incredible resource for retrospective investigation of issues. When you have a full report with all of the evidence and context you need in a form that is easy to work with, you’ll avoid repeating the same analysis over and over again when a new but similar problem arises.
Track resolution
It should be obvious, but make sure to track the step-by-step resolution of an issue so that the steps can be repeated. Include links to the packets involved in an investigation at each stage, so that anyone reading a report can easily follow what happened, whether they are entry-level or C-level.
