Investigation Scenario 🔎

You received an alert referencing an exploitation attempt against Apache (see image).

What do you look for to investigate whether an incident occurred and what was affected?

Assume you have access to whatever digital evidence source you need.

#InvestigationPath #DFIR #SOCAnalyst

There are many paths you could take with this scenario. At a high level, the big question you want to be answered is whether the user or an attacker set up the forwarding rule. But, you've got to ask other, more specific questions to figure that out.

A lot of great responses this week so I won't rehash every path, but there's an opportunity to explore the disposition and prevalence of the client IP, the timing of the rule creation versus AD auth, potential outgoing spam activity,

A few folks pointed out the timing of the rule creation, which is undoubtedly significant. Was the rule created well before djenkins went on their trip? Right before? During it? Those timings all have different implications.

As a general focus, you could spend time finding anomalies from the user's normal baseline or identifying signs of compromise on their system. Either way, you'll build a timeline of relevant events and draw inferences from that.

Many folks jumped straight to containment, and I can understand why because that's an easy call in this scenario -- high risk and low-impact decision in some cases. However, you still have to investigate the scenario even after you've contained it.

My response of the week is from @ckure, who was very detailed and covered a lot of meaningful angles. https://infosec.exchange/@ckure/109983206481584311

By the way, how would you go about auditing the inbox/forwarding rules setup across your user base? Something to think about... 🚀

#InvestigationPath #DFIR

Investigation Scenario 🔎

While hunting through O365 logs, you discovered the depicted entry related to one of your user’s mailboxes.

The user is on vacation for a week and unreachable. What do you look for to investigate whether an incident occurred?

Assume you have access to whatever digital evidence source you need.

#InvestigationPath #DFIR #SOCAnalyst

This week's scenario presents a good opportunity for forecasting... thinking through what other events might have happened if the scenario event was malicious OR benign.

The User Agent string is trivial to set or change for anything sending data over HTTP and related protocols. There are many reasons why you might encounter a blank one. For example...

- A malicious script

- A misconfigured script

- A misconfigured proxy or logging error (more common than you think)

- Malware downloaders

- Malware C2

- Default behaviors (this is the case for PowerShell's Net.WebClient.DownloadString).

- and more...

From an investigative perspective, you might want to start by understanding what was transmitted (but not easy since I said no PCAP) or locating and assessing the disposition of the process responsible for the network traffic. There are lots of good ideas for doing that here: https://twitter.com/chrissanders88/status/1630953588582129664

Once you've identified the responsible process, you can begin understanding its function. We've talked about that here a lot... searching threat intel for the hash, sandbox output, org prevalence, user role context, file source, and so on.

Lots of folks mentioned profiling the remote host based on reputational threat intel. Plenty useful! But, often not something that'll help you establish disposition on its own for a case like this.

I chose this response from @JoeSchottman (on Twitter) as my response of the week: https://twitter.com/JoeSchottman/status/1630779645518831616. While it's only one thing to look for, it's a valuable insight into the types of hosts where you might see this behavior and a quick path to potentially proving a benign disposition by making a phone call.

By the way, I mentioned logging errors... what are some ways that logging errors might obscure actual event information in a manner detrimental to an investigation? Something to think about...

New Investigation Scenario next Tuesday! 🚀

#InvestigationPath #DFIR #SOCAnalyst

Investigation Scenario 🔎

Proxy logs indicate that a host on your network made a few HTTP requests with no User Agent string (the field is empty).

What do you look for to investigate why this is happening and whether an incident has occurred?

You don’t have access to PCAP, but you can use whatever additional data sources you like.

#InvestigationPath #DFIR #SOCAnalyst

A few thoughts on this week's #InvestigationPath Scenario...

You're focused here on whether an incident occurred, and you start with a process name that you know executed. That opens up lots of doors! You could quickly pivot to lots of other artifacts (file hash, parent process, execution location, executing user, and so on).

Probably the most notable thing here is that sqlserver.exe is not the name of the actual SQL Server process from Microsoft... so we've perhaps got something mimicking a legit process name. That tips the disposition scale in favor of malicious activity, but more work to be done. It's always worth your time to Google file names even if you think you know what they are... shout out to @Dfendir_ on twitter, my response of the week who did that thing and found that the activity could be associated with a coin miner (https://twitter.com/Dfendir_/status/1628125705803337745).

When you find malware that might be relevant, you can look for other artifacts of that malware to see if it's present, focusing on things that are strong indicators and easy to find. Of course, the file name is abstracted from the file itself. If you can get the hash and search on that, it will be less abstracted and perhaps get you closer to the capabilities of what you're dealing with.

Beyond that, you've got lots of analysis techniques available:
- Have you seen it anywhere else? (Prevalence)
- Was it launched from an unexpected parent process? (Hierarchy)
- Is it making or receiving suspicious network connections?
... and lots more in the replies.

In this scenario, I also provided some superfluous information about the server sitting in the Amazon cloud. That lets you know about other available evidence (Couldtrail logs), but you don't need those to start this investigation. They might come in handy later.

Speaking of Cloudtrail logs... do you have assets in the cloud? Do you know what unique logging capabilities you can access from your cloud provider and what they can prove? Something to think about... 🚀

#DFIR #SOCAnalyst #Infosec

Investigation Scenario 🔎

One of your web servers hosted in the Amazon cloud launched a new process named sqlserver.exe.

What do you look for to investigate whether an incident occurred?

Assume you have access to whatever digital evidence source you need. Be specific.

#InvestigationPath #DFIR #SOCAnalyst

A lot of responses this week! In an ideal situation, you would know WHY the Jr analyst thinks Cobalt Strike is present on this system and work from that, but you don't know that here. So...

A few folks provided some great critiques on the approach. One recurring theme was that focusing on these specific file names was unlikely to yield results because most attackers will change those defaults. That makes sense... file names are abstractions from the file itself and are easily changed. However... they're also pretty low-effort questions to answer. I think looking for them is fine, but there are lots more things to be done.

Most folks focused on identifying more specific things they would look for from their research and experience. PowerShell artifacts, named pipes, and so on. As I've talked about often, whenever you have named malware, it's always a great idea to find specific known artifacts of the malware you can quickly look for. Pointing this analyst in that direction makes good sense.

So much of learning to be a good analyst is figuring out which investigation path to take... they are many in a scenario like this one. It's learning to spend your time wisely and when to move on to other things. Even some techniques that probably won't yield results, like looking for default named files, are so trivial to execute they're worth a thirty-second SIEM search. There are a lot of ways to spend too much time on this investigation. Focusing too broadly on information-dense evidence sources like PCAP/Memory analysis without specific goals in mind, looking for broader behavioral artifacts instead of CS-specific artifacts, and so on.

One technique I like here that doesn't get mentioned often enough is the strategic use of signature-based detection tools. We think of those tools for real-time detection, but they're super useful in a response scenario too. I'm mostly thinking about techniques like running Suricata against network traffic, YARA against the file system, and Chainsaw + Sigma Rules against the system logs. There's also THOR, which can match YARA + Sigma rules, among others. Those tools cast a broad net, but they're going to automate some of the analysis work and give you a place to dig in, absent some other lead. I'm choosing @danaepp on Twitter as my response of the week for mentioning how the Jr analyst can leverage existing detection rules (here with Velociraptor + YARA) to identify more specific courses of action: https://twitter.com/DanaEpp/status/1622978719118934016. Whenever you're trying to investigate something, looking for existing detection logic across a variety of sources, even without leveraging tools that scan for these things, is an incredibly valuable technique.

All in all, I'd say the Jr analyst from the initial prompt was on the right track with a lot of their ideas. There's more research needed here to find some more specific paths to pursue and a few misconceptions that need clearing up, but those are workable things.

Speaking of information-dense artifact sources. Do you ever over-rely on that sort of data? How would you know if you were? That's something to think about... 🚀 #InvestigationPath #DFIR #ThreatIntel #Malware #SOCAnalyst

Investigation Scenario 🔎

A junior analyst is concerned that Cobalt Strike may be running on a Windows workstation. They proposed the steps in the image below to investigate the event.

What feedback would you give them on their proposed investigation path? What would you prioritize, deprioritize, add, or change?

#InvestigationPath #DFIR #SOCAnalyst #ThreatIntel

Investigation Scenario 🔎

You received a Suricata NIDS alert indicating that a host attempted a DNS lookup for a domain associated with BAZAR malware command and control.

What do you look for to investigate whether an incident occurred and its extent?

#InvestigationPath #DFIR #SOCAnalyst #malware

For this week's scenario, you know that exfil almost certainly occurred, but not when or how... so that's where you're focused. A lot of folks recognized that immediately -- great! Let's talk about it...

Investigations start with the evidence you already have. In this case, you start with the sample data from the forum post. It can't tell you everything, but it can give you a hint about where to start looking. The first step is determining where that data is normally housed.

The location of that data (whether at rest or in transit) becomes the focus of the investigation. You can consider how users normally access that data (like through a web app) or how someone with root on the box might access it (like direct to files or a DB).

You're taking an inside-out approach here. You know where the finish line was but not which road the attacker took to get there. Each of those pathways involves a different set of artifacts to consider. We're talking about attack surface here.

For example, if the data is normally accessed through a web app. You should consider the use of stolen credentials to access that web app and retrieve the data.

As another example, the data might be accessible (although not in normal practice) through files on disk. Here, you can consider abnormal authentications to that system or executions related to file archiving and transmission, among other things.

I'd refer to this scenario as a data-centric investigation because it's the knowledge of what the attacker accessed that starts the investigation -- we don't encounter these as frequently as some other starting points.

From my research. I find plenty of evidence that good analysts constantly consider the attack surface relevant to data/hosts throughout an investigation. In this scenario, you've got to do that right off the bat to efficiently drive the investigation forward.

Shout out to Sajjad who is my response of the week. He worked under the assumption the data come from a DB server and mentioned some normal and abnormal methods that data might be accessed. (https://www.linkedin.com/feed/update/urn:li:activity:7023664326146220032?commentUrn=urn%3Ali%3Acomment%3A(activity%3A7023664326146220032%2C7023722653874040832)

An investigation like this one could be time-consuming, particularly if you don't have a great collection and monitoring infrastructure in place. It's also a good reminder to document where data lives + the people who manage it and can help identify access pathways.

Speaking of attack surface, how do you normally characterize the attack surface of a given host? There are a few ways to go about that, but, it's something to think about... 🚀

#InvestigationPath #DFIR #SOCAnalyst

Investigation Scenario 🔎

You have discovered an ad on a dark web forum claiming to have a list of your customers identifying information for sale. The sample data provided in the ad appears real.

Where and what do you look for to investigate this potential breach?

Assume you have access to whatever digital evidence source you need, but try to be precise with your strategy. Don't just list the data sources you want... describe what you would look for within them.

#InvestigationPath #DFIR #SOC

For this scenario, I want to focus on the need to understand how the event occurred. At a high level, you can consider that services stop in only a few ways…

1. The system hosting the process stops running
2. The process is manually shut down
3. The process crashes

It may sound silly, but you have to consider that the system was simply shut down. Common things are common. There are many ways to identify system shutdowns, but one of the easiest is System EID 1074.

Shout out to @ameerahmed on Twitter, who is my response of the week for mentioning this as a starting point: https://twitter.com/ameerahmed/status/1615378187063812100?s=20&t=6vqdEJ5cwi5PeYtBoK1eog

Shutting down the event log process manually is not entirely trivial. If you try to do it yourself, you’ll notice that get errors about dependencies, and if you try to shut those down you’ll get errors too. But, this is still doable.

One technique is to set the dependencies to disabled and reboot the system. Then, you can shut the event log service down.

That action generates Security EID 1100, but you’ll also see this during a normal system shutdown too, so looking at the timing (post reboot rather than pre-reboot) is important. You can also look for System EID 7035/7040 to find where those services were disabled.

It’s also reasonable that an attacker might clear the event log before disabled it. That in itself will generate Security EID 1102. Those are always pretty noteworthy in general.

Some attackers will accomplish the steps I mentioned with a script, so you can look for evidence of those executions in places like Security EID 4688, Sysmon EID 1, or in PowerShell logs.

There are lots of ways processes crash, and what artifacts get logged there depends a lot on how the process is crashed. You can look for things like Security EID 7023/7024 or Application EID 1000/1002, among others.

In this scenario, I did mention specifically that the process was stopped. Processes can also be tampered with — still running but not functioning properly. There are multiple techniques to do this with the Event Log service. A few are documented here: https://svch0st.medium.com/event-log-tampering-part-1-disrupting-the-eventlog-service-8d4b7d67335c

That post also goes on to describe ways that event logs can be manipulated. That isn’t common bc of the effort/reward, but it’s still possible. That means you can’t fully rely on the event log entries when investigating event log tampering. Gotta diversify those sources.

The catch here is that the event log was likely disabled for a reason if done maliciously. What is that person trying to hide? That’s where a lot of folks went immediately with their investigation. I’ll not rehash all those paths, but look at the responses to get some good ideas!

I hope this scenario demonstrates the value of having a lab to play around with. Here, you'll want to play around with starting/stopping the event log service if you haven’t before. It’s all about identifying novelty, seeking to understand it, and carrying that knowledge forward.

The Event Log service stopping can sound like a weird and malicious thing if you’ve never considered it before. There are lots of behaviors in an operating system that could be described that way. Can you think of a few others? Something to think about… 🚀

I post an investigation scenario every week!

❓Tuesday: I post the scenario to comment on
✅ Friday: I share some of my thoughts and pick a favorite response

If you like these scenarios, you’ll probably also like some of my writing and courses here: https://chrissanders.org/links/

#InvestigationPath #DFIR

Investigation Scenario 🔎

The Windows Event Log service was stopped on a workstation for 16 minutes, then started back up.

What do you look for to investigate whether an incident occurred?

Assume you have access to whatever digital evidence source you need.

#InvestigationPath #DFIR #SOCAnalyst #threathunting #infosec

RT @chrissanders88@twitter.com

Investigation Scenario 🔎

You’ve been alerted to outgoing network communication from a Linux host with the HTTP user agent E9BC3BD76216AFA560BFB5ACAF5731A3.

What do you look for to investigate whether an incident occurred?

#InvestigationPath #DFIR #SOCAnalyst

🐦🔗: https://twitter.com/chrissanders88/status/1612826638634336257

Investigation Scenario 🔎

You’ve been alerted to outgoing network communication from a Linux host with the HTTP user agent E9BC3BD76216AFA560BFB5ACAF5731A3.

What do you look for to investigate whether an incident occurred?

Assume you have access to whatever digital evidence source you need.

#InvestigationPath #DFIR #SOCAnalyst #Linux

RT @chrissanders88@twitter.com

Investigation Scenario 🔎

During IR, you discover a recently created file named 1.wad on a system the attacker accessed.

What do you look for to investigate whether the file is involved in the compromise and how the attacker might have used it?

#InvestigationPath #DFIR

🐦🔗: https://twitter.com/chrissanders88/status/1610293907878813699

Investigation Scenario 🔎

During incident response, you discover a recently created file named 1.wad on a system the attacker accessed.

What do you look for to investigate whether the file is involved in the compromise and how the attacker might have used it?

Assume you have access to whatever digital evidence source you need.

#InvestigationPath #DFIR #SOCAnalyst

Investigation Scenario 🔎

A user reported their mouse moving around on the screen by itself for a few minutes. The cursor appeared to open and close a few documents on the desktop.

What do you look for to investigate whether an incident has occurred?

You can use any evidence source you like.

#InvestigationPath #DFIR #SOcAnalyst #ThreatHunting

There's a lot we could talk about with this scenario, but I think one of the more important takeaways is how the mention of IIS represents a novelty cue for many analysts.

A novelty cue is something that indicates the presence of an unknown threat, capability, or technology that the analyst doesn't understand well. Most analysts have heard of IIS, but not as many understand how it's configured, or its logging capabilities. You don't have to use IIS logs to investigate this scenario if you have PCAP, proxy, or Zeek data, but you can. They'll certainly be easier to parse through than PCAP. Whenever you encounter a novelty cue related to some software or service on your network, it's an opportunity to learn more about it, and one that you should usually take. It's these little knowledge gains that accumulate over time that allow analysts to get 1% better every day.

On to the investigation... a key step here is identifying whether the spike in traffic is within the normal scope of the application or something else. That'll also help you characterize the traffic a bit and point towards some possible explanations. One approach is aggregating/stacking the POST requests by URI (whole or partial by parameters) and comparing them to another time period or existing knowledge of the app. That could reveal many things.

Shout out to Daniel on Twitter, who mentioned that right off the bat, and is the response of the week for this scenario: https://twitter.com/DanielOfService/status/1602711368884518915.

With this system being exposed to the internet, it could be all sorts of benign scanning, which is the most likely thing. But, you also have to consider web shells and authenticated enumeration, among other possibilities.

Early on, it's worth checking the patch level of the system, IIS, and the components running on it. A missing patching exposing a critical vulnerability could mean likely exploitation with the exposed attack surface.

Thinking about web shells, a helpful strategy would be to check the system for new files in the web directories, check for a new application pool, or requests to pages never observed before. Focus on common web extensions like ASPX, JS, PHP, and so on.

As always, lots of other good ideas were presented in the thread. We could explore more about the traffic itself, the source, or the state of the web server. But, the first steps matter so you want those to be efficient and relevant.

#InvestigationPath #DFIR