We’ll come back and update this post as new information comes out. Early breach information is often wrong or missing important context, so we’re going to focus on lessons that are broadly useful, even if the breach details fundamentally change later.

In other words: you should take the breach details here with a grain of salt, but take the lessons to heart.

Attackers know that buying hacking in bulk is a good value

Third party and supply chain attacks have been en vogue for a few years now and this trend only seems to be increasing. Why hack one company when you can hack thousands of companies or users through a software/services supplier?

The hack once, exploit many nature of these attacks isn’t the only attraction. Integrating with third party software often requires creating OAuth applications or tokens that grant the third party access to your own data and systems, or another third party software supplier your company uses. Unfortunately, session hijacking via token theft is still an unsolved problem, meaning that attackers who obtain OAuth keys and other types of auth tokens get to bypass the authentication process.

We covered this on Enterprise Security Weekly back in December (jump to the 34:40 mark in the video).

Token theft works so well, it has led to a rise in the use of infostealer malware and ClickFix social engineering techniques over the past two years. The rationale here is that the employees with high-level privileges to systems typically use MacOS. Macs are harder to attack directly and infect with malware, so attackers pivoted to the ClickFix social engineering technique, which has proved effective.

Convince a Mac user to copy a command and paste it into their terminal (under the guise of fixing a problem or installing harmless software), and the infostealer runs, scooping up crypto wallets, plaintext passwords, SSH private keys, environment variables, auth tokens from logged-in sessions, ~/.openclaw/credentials/oauth.json and anything else not nailed down.

The high price of convenience and forgetfulness

These session keys exist, because no one wants to spend the first 40 minutes of every work day logging into Teams, email, Slack, Github, Dropbox, Google Calendar, LinkedIn, Claude Code, Mastodon, Microsoft 365’s Office apps, the Apple App Store, and everything else we might need to be productive. In addition to simply logging into the apps of our choosing, there are also 3rd party integrations that require auth keys to function.

For example, if I want ChatGPT to be able to locate files in Dropbox, I can integrate the two, but this requires granting ChatGPT at least read access to Dropbox. Sometimes (ahemGoogleahem) the third party doesn’t allow this access to be as fine-grained as you might want and you grant too much access. Sometimes, you don’t want to spend an extra 10 minutes figuring out permissions, so you just click the “Grant Full Access” option.

Creating these integrations often takes mere seconds. Then, we immediately forget the integration exists. This is a problem.

When I was working at Valence Security, we observed that 100% of our customers at the time of joining, had granted tenant-level access to third parties that they were not using. In other words:

1. they did a proof-of-concept with someone

2. gave the product full control of all employees’ email, files, and calendar in Google Workspace or Microsoft 365

3. didn’t buy the product

4. forgot to revoke a token that granted FULL ACCESS to nearly everything the company cared about

Third Party Breach Turducken

This brings us to the Vercel and Context AI breaches: a third party breach within a third party breach. Let’s start with Context AI1.

In June 2025, Context AI released a consumer product that was designed to be an agent-driven productivity monster. Give it access to your chats, your email, your files and it can build slides, spreadsheets, and reports using all the context from your existing files and conversations.

Of course, for this to work, you need to give it access to all your stuff. Everyone reading this was probably familiar with the dangers of doing this a year ago. If not then, certainly now, post-OpenClaw.

While Vercel was never a Context AI customer, one of their employees was. This employee gave Context’s AI Office Suite full access to their Vercel Google Workspace email, calendar, and files. In Context’s words:

Vercel wasn’t a customer, but at least one of their employees granted access to Vercel’s Gooogle Workspace and granted “Allow All” permissions

At some point, Context AI pivots to an enterprise product called Context Bedrock and deprecates their AI Office Suite. From what I can gather from the state of their website according to the Wayback Machine, this pivot happened well before this attack began. Shared responsibility failed during this pivot. Based on what we know so far, after this product was deprecated:

1. The Vercel employee didn’t revoke the tokens Context was still storing.

2. Context didn’t delete customer tokens and didn’t shut down the infrastructure used by AI Office Suite

3. We don’t know if Context notified AI Office Suite customers that the product was being deprecated, but I couldn’t find any public notice of this fact on the company’s websites, LinkedIn, or Twitter accounts.

Context Gets Breached

In March 2026, Context “independently identified and stopped a security incident involving unauthorized access to our AWS environment.” They hired CrowdStrike, who identified one affected customer. Context notified this customer and shut down the remainder of the AI Office Suite infrastructure.

Given Context’s description, it seems possible that there was never a hard shut down of the old product and that the company simply started work on a new product while letting the old product continue running, unsupervised and unmonitored.

In light of the Vercel breach, Context put out its own security notice, noting that perhaps AI Office Suite OAuth tokens were also compromised in its own breach the previous month. Sadly, Context’s security notice doesn’t share any information about how they got breached, but in their defense, they mention that they’ve restarted their investigation, which is ongoing.

Vercel Gets Breached

Vercel is a vibe-coding product that specializes in building application/website front-ends (the part you can see). On April 19th, they became aware that one of their employees’ accounts was compromised and being used to access customer environments. They don’t mention what tipped them off to the attacker’s presence, or how long the attacker was present. Again, I’m writing this only one day after they detected the attack, so they’re likely far from completing their investigation.

Vercel contacted affected customers and recommended immediate credential rotation. They’re still working to determine what data was exfiltrated, if any.

Vercel provides some advice on how customers can protect their environments with a few built-in security features, like ‘sensitive environment variables’ and ‘deployment protection’. They also share the identifier for the Context OAuth app that was used to compromise them through Google Workspace: 110671459871-30f1spbu0hptbs60cb4vsmv79i7bbvqj.apps.googleusercontent.com

Lessons/Control Failures

1. Vercel’s corporate collaboration suite was over permissive - the employee should not have been able to hand over full control of their work account to a third party without business justification and/or security review.

2. Failure to revoke access once no longer needed - both the employee and Context failed to revoke the access after the product was deprecated.

3. Access token theft (T1134.001)

4. Lack of regular access reviews - given how this attack occurred, and the fact that Context’s product didn’t even last a year, suggests that annual reviews of employee-initiated integrations would be far too infrequent. Monthly may be more appropriate, given the velocity of AI app development and adoption (both sanctioned and shadow).

5. Too much employee access to customer data/workloads - we’ve all seen SaaS products where access to customer data is far too permissive. Travis Kalanick’s abuse of Uber’s God View is perhaps one of the most egregious cases of this.

Timeline

1. 2025 - A Vercel employee signs up for Context’s AI Office Suite product

2. 2025 - This employee grants AI Office Suite full access to their Vercel Google Workspace resources (Email, Calendar, and Files at a minimum).

3. March 2026 - Context becomes aware of a breach in its AWS environment and hires Mandiant to investigate. One customer is notified.

4. April 19, 2026 - Vercel becomes aware of a breach and traces the source of the breach back to an OAuth token granted to Context AI by one of its employees

5. Both Vercel and Context continue to investigate - hopefully more details will come out and they will be transparent about how both breaches were initiated and detected.

Conclusion

How often do we perform a security review of our personal or corporate third party integrations? Does a Blackberry Curve from 2012 still have full permissions to access your Gmail? Surely that access would expire, right? Think again.

The first big lesson here is that, in a cloud-first/SaaS-first world, we have to regularly review all the access we’ve given to third parties: access to our data, to our devices, to our employers, to our kids. In an ideal world, this access control model should be reversed. By default, access granted to third parties should come with an expiration date by default.

There are examples of how to do this correctly! I have a Linux laptop with the Signal app installed. I haven’t used this laptop in nearly a month. The other day, Signal on my iPhone notified me that my Linux laptop would lose access to Signal if it remained idle for a full month. I didn’t use the laptop and Signal is now disconnected there.

From a corporate perspective, this should also be a bigger priority. SaaS Security Posture Management (SSPM) tools are fairly easy to come by these days and specialize in bringing attention to these risks.

The other big lesson here regards employee access to customer data. When I was an Industry Analyst at 451 Research, I’d look forward to going to AWS reInvent every year. At reInvent, I’d get a chance to talk to Stephen Schmidt, who was the AWS CISO at the time. His perspective, as the security leader for the largest hyperscaler, was interesting - every year, he took great pride in how much he was able to reduce employee access to customer data. Over anything else, he seemed most concerned about an insider threat or compromised employee impacting customers. A concern that appears to have been well founded.

Subscribe now

Large Language Models are good with code - after all, it’s just language. Naturally, this skill for language extends to finding vulnerabilities as well. Groups were doing just fine with current models. Adam Shostack points out that seven of the top ten collectives on HackerOne are now AI. XBOW, AISLE, Moak, Calif’s MAD Bugs, and others have been sharing the details behind their successes. Now, Anthropic’s Mythos piles on.

The Mythos Vuln Cannon

I’ve heard it referred to as a vulnpocalypse and a patch tsunami. Most often, I hear it referred to as scary. Anthropic has cranked the hype knob to 11 here - they’ve got clinical psychiatrists talking to their models now, concerns about how it feels, really leaning into toxic anthropomorphism.

Anthropic claims that “non-experts can also leverage Mythos Preview to find and exploit sophisticated vulnerabilities” and “exploits … are not just run-of-the-mill stack-smashing exploits,” but these claims aren’t well backed up and many details (effort, cost) are missing. Most of the vulnerabilities we see from Mythos (as well as other efforts, like Aisle’s focus on OpenSSL vulns) simply cause crashes. DoS bugs are legitimate issues, especially in BSD-flavored operating systems likely to be running highly exposed services, but these aren’t the kinds of vulns that have people scared. Today’s attackers want RCEs.

Folks out there are talking about Mythos as if it is a skeleton key - just point it at something you want to hack and the LLM will make it happen. While this is likely possible in some cases, I suspect it will be more like a thrift store: you’ll be disappointed if you’re hoping to find something specific, but you’re likely to find something interesting.

This is what Anthropic and other AI foundation model tech companies need. Trillion dollar valuations and 12-digit VC funding rounds benefit from a narrative that paints this technology as the closest thing we’ve ever seen to magic. The “OMG this model is too dangerous to release, someone please regulate us” marketing schtick is well established - we should recognize it for what it is by now.

Consider the evidence and consider what we don’t yet know. Look at the vulnerabilities and exploits being presented - are they something an attacker would actually want to use?

The Reality

When we look at one of the more interesting vulns and exploits produced (CVE-2026-4747, credited to Nicholas Carlini and Claude), we find a more familiar scenario. An expert guiding the model towards the goal, constantly making course corrections, suggestions, and shooting down bad ideas along the way. Bless Calif, we even get a peep at the exploit, the prompts used to create it, and gives us an idea of the effort involved.

They list the total time to go from the FreeBSD security advisory to a working exploit: 8 hours. Claude’s working time is listed as ~4 hours. Most interesting are the 44 human-submitted prompts Calif were kind enough to share.

There are some gems like:

• wait, what are you compiling?

• why wouldn’t you just install a vulnerable version

• tere (SIC) is no kaslr so it should be easy

• install ropgadget or what ever you need … idk

• why do we need kdc?

• nope, that won’t work…

• working means a connectback shell as uid0

• i want a shell.

• make the writeup better

The Mythos blog post claims “… it autonomously wrote a remote code execution exploit on FreeBSD’s NFS server that granted full root access to unauthenticated users by splitting a 20-gadget ROP chain over multiple packets.” To me, autonomous suggests no human interaction. In reality, for 44 prompts, a human was actively guiding the AI, giving it suggestions, shooting down bad ideas, and having to reiterate the project goal (remote shell as root) over and over.

There’s no mistake that what Claude accomplished was remarkable - a working exploit that might have taken a human alone, or even a team of humans without AI days or weeks. Understanding the environment, the bug, the components related to the bug, establishing an exploit methodology, building the exploit, testing the exploit, documenting the process and the exploit - all of this is immensely time consuming, but Mythos cut this time down to mere hours.

However, 8 hours and 44 prompts is far from the autonomous vuln cannon we’re seeing described in the media and press releases.

Project Glasswing

I like Project Glasswing. It seems like an AI-driven version of Google Project Zero, though AI-driven workflows have largely replaced commercial vulnerability discovery. Trail of Bits writes about their experiences embedding AI into a team that does penetration testing and security assessments at a high level for clients.

It seems focused on finding and fixing bugs in critical, widely-used software, so I’m not sure we’ll notice any increase in patching. I already feel like my browser and OS update at least once every few days. The limited organizations with access include Amazon, Anthropic, Apple, Broadcom, Cisco, CrowdStrike, the Linux Foundation, Microsoft, and Palo Alto Networks.

I’m not sure the industry can handle more vulnerabilities. HackerOne apparently paused bug bounties. CVE assignment and creation often trails disclosure significantly, and CVE enrichment is still way, way behind.

What about the vulnpocalypse?

There are concerns about Mythos leading to a flood of vulnerabilities and patches. There are concerns that defenders will be overwhelmed. I can put that concern to rest.

Defenders have been overwhelmed for years. Decades.

It could get worse though. With time-to-exploit dropping dramatically, vulnerability management teams are resembling incident responders more every day. A log4shell-level event once a month would be exhausting. Once a week, impossible.

It keeps getting rougher.

Practitioners have it rough

Remediation is the bottleneck. We all agree on this.

If the pace of vulnerabilities disclosure significantly increases, analysis will be constant. CVEs may not exist yet, so analysts will have to forge ahead without CVSS, EPSS, and any CVE-dependent tooling. What can security teams do, but prioritize the list and wish asset owners luck?

After all, security teams don’t patch or remediate vulnerabilities - they advise. The true remediation work is done by system owners. System owners get yelled at when stuff goes offline. The business doesn’t like it when things go offline.

Everyone’s Mythos advice is going to be, “get ready for more patches!” That’s not going to work.

For traditional IT teams, “everything’s working, no one touch anything” is The Ideal State. The ideal state doesn’t get anyone yelled at. The ideal state doesn’t lead to user complaints. Don’t mess with the ideal state. Don’t scan, don’t patch, don’t even look at the Oracle cluster funny. In this culture, software doesn’t get patched. Risks get accepted and deferred.

I’ve seen the “oh, but AI can help with remediation also” argument. It can write patches sure, but a human still has to review the patch, test the patch, and merge or roll out the patch. This process could take hours or years. It could happen quickly or never. Ultimately, remediation is more of a business decision than a technical one.

Resilience is an increasingly common conversation, but I don’t see any path there that doesn’t involve testing to failure. The discipline and work necessary to become resilient is depressingly far from what the average organization can stomach. To quote Yaron Levi, we “lack operational discipline.”

What about attackers?

Attackers don’t care. They’re not bottlenecked by a lack of exploitable vulnerabilities. They have other ways of getting in and we have evidence suggesting that initial access brokers never run fully dry on access to sell. The only reason we don’t see even more breaches is that attackers appear to be operationally bottlenecked (you could say they have a talent shortage). I’m hoping AI doesn’t change this.

The Cyber Why

Could AI Address the Cybercriminal Skills Gap?

NOTE 1: for reasons explained in my previous essay, I’ll replace the common use of the term ‘ransomware’ with ‘extortion’ in this essay…

Read more

3 years ago · 5 likes · Adrian Sanabria

What are we gonna do about it?

Defenders can’t win on speed, which means they need strategies that don’t require understanding what the attacker is going to do. Back in 2016, I delivered a Virus Bulletin keynote that defined next-gen antivirus as “the ability to stop threats without prior knowledge of them.” In a world where every individual piece of malware could be unique, this was necessary.

We’re now at that point in vulnerability management. In a world where an attacker could build a custom-vibe-coded exploit for any piece of software, we need a different approach. My talk at Tactical Edge in 2019 suggested building systems assuming there is always a zero day, and the patch is never coming. Basically, zero trust’s ‘assume breach’, but for software vulnerabilities.

So what does that look like in practice?

1. Reduce attack surface: remove unnecessary software, remove unnecessary accounts, disable unnecessary services

2. Remove IT asbestos protocols and replace them with modern, secure protocols

3. Harden systems: stop leaving cleartext credentials everywhere, use ephemeral and immutable infrastructure where possible, follow CIS benchmarks

4. Put passive mitigations into place - egress filtering goes a long way, exploit mitigation technology, DNS sinkholing any newly registered domains, application control - I have more suggestions available here if you’re an IANS client

5. Prepare active mitigations to contain or prevent attacks - WAF rules are sometimes useful, quickly consume and use threat intel

6. Ensure you can detect attacks - this is your last line of defense when everything above fails. Test your detection capabilities by simulating the attacks. Don’t base detections on specific, known details, but on common, but suspicious behaviors all attackers must do once they gain access to your environment.

7. When your last line of defense fails, you best be able to recover quickly. This also takes a lot of planning, testing, and practice to do well.

8. All this changes your metrics and reporting as well, though that’s a whole separate post.

The goal isn’t perfection with any of these controls. It’s survivability, durability, and resilience.

Conclusion

There are a few possible bad scenarios here:

1. there’s a new “drop everything and patch ASAP” vuln every week and teams get burned out

2. Mythos finds a lot of meltdown/spectre bugs and kills the performance of our compute for zero safety benefit

3. Mythos finds so many vulns that orgs get desensitized to vulns altogether and start ignoring vuln/patch management

On the defender side, the most significant bottleneck is in remediation. Vuln mgmt teams are drowning. More vulnerabilities, more exploits, more patches - none of it reduces the drowning problem. Their bottleneck is the ability to apply/patch/update systems without incurring downtime and disruption. Until this bottleneck is addressed, it doesn’t matter how many patches AI can magic together.

1. Attackers don’t need more vulns or exploits - there is no lack of initial access to enterprise environments

2. The vuln mgmt industry is bottlenecked, which impacts the tools defenders rely on, particularly when time-to-exploit is near zero or upside down

3. Defenders cannot quickly remediate vulnerabilities - until this bottleneck is addressed, all the AI-generated patches in the world do no good

What do you think, did I miss anything? Let me know in the comments.

Subscribe now

When I think of RSAC keynotes, I think of buzzword-laden vendor execs confidently, expertly leading you towards their company’s next big product release.

I was an industry analyst at one point, so you’ll have to forgive my cynicism. I’ve sat for a LOT of vendor briefings over the years.

The buzzwords were there for sure — if you plan on watching these keynotes, don’t base a drinking game on machine speed, agentic, real-time, or human-in-the-loop. The confidence and the thinly-disguised product pitches were there as well.

What I wasn’t expecting was the admission that we don’t really know how to protect this latest technology. Everyone agreed that AI agents need to be secured and that this work has to begin immediately. Everyone has thoughts on what some of the key ingredients should be. But no one claimed to have the solution.

I had the same experience talking to attendees at the conference. I interviewed the founder of an AI governance startup, who told me that none of his customers were using any sort of enforcement or guardrails yet. Everything was in ‘monitor mode’.

In a way, this is unsurprising - the quickest way for the security team to get in trouble has been impacting availability. At a time when businesses are terrified of being left behind, security had BEST not get in the way.

Subscribe now

Like most of the 43,000+ RSAC attendees, I was running around all week and didn’t get to attend as many talks as I would have liked. I attended Security Tinkerer events, Cybersecurity Canon events (including working a shift at the excellent RSAC Bookstore!), and recorded interviews for CyberRisk TV.

Luckily for myself and the rest of us, I’m told that all of the talks at RSAC Conference 2026 were recorded (check out the one I gave with Adam Shostack). Before flying back home, I decided to download the main stage keynotes playlist, so that I could start watching them and taking notes on the trip home.

Fun fact: 43,000 is 0.78% of all cybersecurity professionals, if we take ISC2’s word that there are 5.5 million of us, globally. This stat is probably off, given that a lot of the 43,000 attendees are vendors. Surely there are some ISC2 members working at vendors, right? I digress.

Here’s what I learned from watching all 11 main stage keynotes.

Securing AI Agents

Everyone agrees that we must protect AI agents, but that we’re not sure how.

There does seem to be agreement on many details.

• Asset management for AI agents: discovering, ownership, responsibility

• Data permissions patterned after users (a la Microsoft Co-Pilot) is too broad, user data hygiene is too poor

• Visibility into AI actions and reasoning. This was often referred to as auditability or traceability.

• Validation of output

• Integrity becomes a real challenge — George Kurtz shared several examples of AI inventing the solution to a problem. Did it just retrieve real company/customer data that solves your problem? Or did it fabricate that data? How would you know?

• AI agents can’t be trusted with intent. Feed them a social contract or ethics and they modify it or break it in order to complete a task.

• Compliance with existing regulations could be challenging. How does GDPR’s right to be forgotten work with new AI tech stacks? Does AI memory need to be purged? Will AI agents actually remove data, or just say they’ve done so?

• Agents will scale to a point where manual, human-driven security controls can’t work (we’re probably already there in many cases).

The Characterization of AI Agents

Digital Co-Workers

Several speakers characterized AI agents as ‘Digital Co-Workers’. From what I’ve seen, assistant agents might feel like this, but most enterprise agents won’t. The ephemeral agent that exists for the 12 seconds it takes to enrich a phishing alert won’t feel like someone you’d like to have a drink with. You’re unlikely to even interact with the majority of these agents. A SOAR trigger or orchestration agent will interact with these agents.

Human-in-the-Loop or Not?

Some were saying that keeping a human in the loop is essential - a non-negotiable point. Others were saying that human-in-the-loop is a temporary stopgap that won’t scale. There were mentions of human-on-the-loop and agent-in-the-loop. Basically, the difference between in-line enforcement and out-of-band monitoring. Where have we had to make that tradeoff before?

Disagreements on how AI agents will work

Some describe AI agents as ephemeral. Just-in-time agents with just enough access that are destroyed as soon as their task is complete. Analogous to containers or perhaps actually running within containers.

Others, especially those describing agents as digital co-workers, imagined long-lived agents that get smarter over time. Agents that learn and improve as they ‘gain experience’. Perhaps this is possible through the concept of decentralized memory, though it seems like the agents themselves will still be ephemeral, even if memory is persistent.

Thousands of agents per person

Several imagined that, just a few years into the future, we’d each have thousands of agents running around doing stuff for us. I have a few questions:

1. Will the planet be able to generate enough power for each person to have hundreds or thousands of agents burning tokens 24/7?

2. What exactly are we going to do with thousands of agents?

3. Since automation has been possible on personal computers for decades, why don’t we already have thousands of automated jobs doing work for us today? Zapier, IFTTT, n8n, and power automate all existed before ChatGPT was released.

Acting like automation didn’t exist before LLMs

This one really triggers me.

Computer-based automation has been replacing jobs as long as computers have become commonplace in the enterprise. Even email is an automation, replacing the task of an internal courier, physically carrying a message from one employee in the office to another.

There were lines like, “Attacks are now faster than a human can respond.” Girl, that was the case back when Dennis Nedry was screwing over all of Jurassic Park to make a quick buck. Jurassic Park was written in the 80’s. Dennis used SHELL SCRIPTS.

Big Concern: Navel Gazing

At one point, one of the speakers asked, “How many of you here went to the GTC conference last week? Or watched Jensen’s keynote?”

Silence.

“Anyone?”

Nothing.

“There’s a complete Venn diagram with no intersection.”

We’re making this huge deal about AI in our industry, but cybersecurity isn’t paying attention to the industry making AI our problem? Maybe one of the reasons that AI lacks functional guardrails is because we’re not there — we’re not part of the conversation. And look — I get it, I don’t particularly enjoy Jensen’s keynotes, but the AI industry is hanging on his every word. What Jensen says or introduces today is something we have to secure tomorrow.

Aren’t we the industry that made a big deal about getting security “baked in” as opposed to “bolted on?” Where did that all go?

“We can’t let AI happen to us, we have to make it work for us” — Hugh Thompson

This doesn’t just apply to the AI industry, but the larger tech industry as well. What conferences are the CTOs and CIOs going to? What podcasts and blogs are the DevOps folks consuming?

Threats are getting faster

Threats are getting faster and more automated. The fastest breakout time is seconds, fastest transition from the 1st stage to 2nd stage of an attack also takes only seconds now.

The speakers all seem to agree that detect and respond need to effectively become a single step. That means automation. No human in the loop.

This also means that we’re going to need permission from the business to break some stuff. Most of us won’t get that permission.

Another common conclusion is that we need to prioritize hardening and prevention (the pendulum has swung back). As I’ve often said, we need to build systems as if everything has a zero day and the patch is never coming. We also need to reduce attack surface — something I have suggested a strategy for.

Fundamentals and Magical Defense

The fundamentals are difficult because enterprise infrastructure, identity, and data is complex and sprawling. Applying security controls across all of it takes huge effort and some of that effort must be indefinitely maintained as these controls drift over time.

Now we’re talking about doing it faster? In real time? Zero Trust on steroids? Words like comprehensive, correlated, and unified are thrown around. Magical defense that requires perfect knowledge and control over the environments we protect.

It’s as if we can’t remember why NAC failed. Or the early attempts at application control — remember how we declared malware a thing of the past? NDR that learns from traffic over time and gets better at detecting and stopping attacks. Deception designed to trap attackers in a hall of mirrors.

These are vendor-delivered keynotes however, so hyperbole is to be expected, I guess.

They’re right though — fundamentals are more important than ever, and some of them now need to be adapted for AI agents.

Particular standouts

1. Tomer Weingarten/SentinelOne - Securing Human Potential and Freedom in the Age of Agentic AI

   1. This one was surprisingly equal parts tender, passionate, and urgent regarding the future of the human mind

   2. Tomer focused on the dangers of becoming complicit in a world of AI agents eager to do your thinking for you.

   3. “The moment we stop exercising judgement on AI output, we start to suffer cognitive atrophe”

2. Sandra Joyce/Google Security - Activate Industry! Moving Beyond Defense to Disruption and Active Defense

   1. Not about AI - about threat intel sharing and disrupting threat actors

   2. I loved this one because there was no magical thinking, no hand-waving about defenders needing a cohesive platform. There was a clear plan and evidence that this plan is working.

   3. She shared several examples of how civil legal action and public disclosure have been successful in disrupting attackers infrastructure and tools, setting them back months or years.

   4. The CTA for defenders was less clear, however, and I really wanted to hear more about what she described as Technical Takedowns - create a hostile environment for attackers, on the targets they’re hacking into ← is she talking about things like deception? I can’t be sure.

3. Jeetu Patel/Cisco - Reimagining Security for the Agentic Workforce

   1. You don’t have to watch the talk, but it’s worth checking out the open source AI defense tools that Cisco released.

   2. It seems like a lot: AI BOM, Skill Scanner, MCP Scanner, A2A Scanner, CodeGuard, DefenseClaw

   3. Definitely the only talk where OSS was praised (unless you count OpenClaw)

My favorite quotes

Here are some quotes I found funny and/or interesting, provided here, out of context, on purpose.

• “The fundamentals are not basic”

• “Easy to declare, hard to prove”

• “In a world where every company is an AI company, trust will be the only currency that survives.” (huh?)

• “It’s like PACMAN from hell”

• “We’re building the biggest flat network of all”

• “This is going… nuclear, really”

• “Within 24 months, the smartest employee in your organization will be a machine”

• “AI is the new operating system”

• “AI is now the biggest insider threat”

• “Using identity as a control plane, that’s not different - we’ve got to do it at runtime, it’s probably going to make things like Zero Trust today look soft.”

• “Show me where customers are entrusting their data, and I’ll show you where hackers are focusing”

Conclusion

I found this a useful exercise and I think I’ll try to do it more in the future. Let me know if you also found this useful. I’m considering watching all the Innovation Sandbox contestants and doing something similar with those videos.

It seems like all this uncertainty should leave me with some dread around the lack of security for AI agents, but it doesn’t. While generative AI has evolved much more quickly than other technological breakthroughs, the reactive role of security remains the same. Technology changes and we do our best to keep up.

There’s some solace in the fact that breaches don’t kill companies, but failing to keep up in competitive markets does.

Share

Subscribe now

Before diving in, it is worth noting that this story has no neutral narrators. The attackers are incentivized to exaggerate their capabilities and downplay their collaboration with others, while MGM’s legal team is incentivized to minimize the perception of negligence. Where possible, this writeup relies on court filings, SEC disclosures, and third-party reporting, but some details inevitably trace back to sources with a stake in how the story is told.

Background

MGM Resorts International is one of the largest hospitality and entertainment companies in the world, operating over 30 hotel and casino destinations across the globe. With flagship properties like the Bellagio, Aria, and MGM Grand on the Las Vegas Strip, the company employs roughly 75,000 people, serves tens of millions of guests annually, and reported revenues of roughly $13 billion in 2022.

What were the circumstances around the attack?

The attack was made possible in large part by a misconfiguration in MGM's Okta environment. Okta is an identity and access management (IAM) platform that many large enterprises use to handle employee logins across dozens of applications through a single sign-on (SSO) system. Within Okta, Super Administrator accounts hold some of the highest privileges available, including the ability to link new Identity Providers (IdPs) and modify multi-factor authentication (MFA) policies.

On August 31st, 2023, over a week before the attack, Okta sent out a warning to all customers noting that attackers had been using social engineering to obtain privileged Okta roles, moving laterally from there. The notice included specific preventative measures and called the attack vector “preventable.” Whether MGM acted on that warning in time remains an open question. Critically, at the time of the breach, Okta’s default settings allowed lower-privilege help desk administrators to reset MFA for Super Admin accounts without any additional verification3.

In the months prior4, attackers had been systematically researching high-value organizations that use Okta. They identified employees with administrator-level privileges through public sources like LinkedIn, and used that information to manipulate help desk workers. MGM was one of several Okta customers targeted during this period.

Attacker Motive(s)

The attackers were financially motivated. While two groups have claimed responsibility (see Appendix B for more details on attribution) and both agree that ransomware and extortion were ultimately used, nearly every other detail of their stories diverge. An alleged Scattered Spider member told reporters that their original goal was to tamper with slot machines5 to slowly siphon funds via recruited mules. When that plan failed due to unfamiliarity with the source code, they shifted to ransomware. ALPHV denied any attempts to tamper with slot machines, arguing it would reduce the chances of a ransom payment. They further claimed to not have deployed ransomware until after MGM began taking down their own systems6.

Initial Point of Compromise

Getting an initial foothold into MGM came down to a short phone call. The attackers spent the early summer months conducting Open Source Intelligence (OSINT) on MGM through sources like LinkedIn. Eventually, they identified employees with administrator-level privileges and gathered enough details on their targets to stage a convincing social engineering attack. They called MGM’s help desk, impersonated one of those employees, and used the information gathered to back up their story. A brief conversation was enough to convince a help desk worker to reset MFA for a Super Administrator account.

From there, the attackers used this newfound Super Admin access to add an additional IdP inside the Okta environment. This feature, called Org2Org, is designed for company mergers where not all employees have been configured in Okta yet, allowing two separate Okta organizations to bridge their identity systems in the interim. By adding their own IdP, they could modify the username parameter to log in as any MGM user without needing to provide a password or undergo MFA. With just one click from their environment, they could impersonate anyone in the company.

This also gave the attackers a stealthy persistence mechanism that MGM inadvertently made worse for themselves. When MGM began to take down their Okta Sync Servers in order to lock out the attackers, they ended up only locking themselves out7. The attackers still retained Super Administrator access, letting them move laterally and disperse ransomware.

Impact

MGM’s decision not to pay the ransom was an expensive one. MGM’s operations are deeply dependent on digital infrastructure, from keycard systems and slot machines to hotel reservations and loyalty program data, making it an attractive target for a ransomware attack. The diversification of operations is also reflected in the categories of losses detailed in the table below.

The financial impact broke down as follows:

In addition, the data of roughly 37 million people ended up on the dark web, including names, contact information, dates of birth, and driver's license numbers, with Social Security and passport numbers exposed for a subset of victims. The FTC also issued a Civil Investigative Demand, which MGM responded to with a 71-page petition before it was eventually dropped8.

By comparison, Caesar’s reportedly paid a $15 million ransom in Bitcoin, but the FBI was later able to freeze approximately $11.8 million, limiting their loss to $3.2 million.

Legacy & Takeaways

MGM’s handling of the disclosure is worth noting. Their public communication was initially sparse, and the detailed 8-K filed with the SEC came nearly a month after the incident. While SEC disclosure requirements ensured some transparency, it was largely compelled rather than voluntary. Caesars, by contrast, managed to keep their breach almost entirely out of the public eye by paying quickly and quietly. Neither approach sets a great precedent. Timely, transparent disclosure gives affected customers the chance to protect themselves and gives the broader industry the information it needs to defend against similar attacks.

As for prevention, the attack did not require sophisticated malware or a zero-day exploit. It required online research and a phone call. Okta’s August 31st warning included specific, actionable steps that, if correctly implemented, could have stopped or slowed the attack at multiple stages. Limiting help desk admin roles to exclude highly privileged accounts would have opened this request to more scrutiny. Perhaps a more senior staff member would have spotted a request to reset MFA for a Super Admin as a red flag, stopping the attack before it began. Enabling Protected Actions would have forced re-authentication before any administrative action was taken, adding another layer the attackers would have had to bypass.

Whether the aftermath caught up with anyone actually responsible remains unclear. ALPHV’s dark web infrastructure was taken down in late 2023, and several alleged Scattered Spider members were subsequently arrested in 2024. However, given how murky the attribution remains, it is difficult to say with confidence that those charged were the same individuals who carried out the MGM breach specifically.

Appendix A: Control Failures

The following table details control failures from the MGM breach. For reference in other parts of the appendices, each control failure is assigned an ID, abbreviated as “CF-[number].” Control failures go beyond technical failures to include process and skill (people) failures as well.

Cyber Defense Matrix Mapping

Using Sounil Yu’s Cyber Defense Matrix (CDM), which is based on NIST CSF functions and assets. Read more about the CDM here.

MITRE ATT&CK Mapping

This next table is for teams that depend on the MITRE ATT&CK matrix. These are the same control failures from the previous table, but reorganized from a MITRE ATT&CK techniques perspective.

ATT&CK Navigator Summary: Primary Tactics Leveraged

Reconnaissance ⇒ Initial Access ⇒ Credential Access ⇒ Privilege Escalation ⇒ Defense Evasion ⇒ Persistence ⇒ Lateral Movement ⇒ Impact

The attack is notable for its heavy use of Identity-based techniques (T1556, T1484, T1550) rather than common exploit-based initial access, highlighting the importance of identity infrastructure and its attack surface. Every tactic from Initial Access onward was enabled or amplified by the control failures in CF-1 through CF-4.

MITRE D3FEND Mapping

MITRE D3FEND mirrors the ATT&CK matrix. Where ATT&CK describes the techniques and tactics used by attackers, D3FEND describes the preventative and detective controls to ‘defend’ against them.

Appendix B: Attribution

Public reporting almost universally credited Scattered Spider with the MGM incident as well as incidents with other Okta customers. Scattered Spider is a loosely organized collective of young, English-speaking hackers known for aggressive social engineering. Security firms amplified the attribution, and the name stuck. But ALPHV/BlackCat, the ransomware group whose malware was ultimately deployed, published a lengthy statement on their dark web blog explicitly rejecting that framing. Scattered Spider, for their part, also claimed the operation as their own.

ALPHV’s statement is worth reading carefully. They not only claimed credit for the attack, they also pushed back against Scattered Spider being attributed. They called out VX Underground specifically for false reporting and challenged security firms to provide actual evidence of the perpetrators. In their own words, “these specialists find it difficult to delineate between the actions of various threat groupings, and so they simply grouped them together.” ALPHV also noted that tactics and indicators of compromise are publicly known and easy for anyone to imitate, meaning that pattern matching alone is not enough to pin an attack on a specific group.

ALPHV provides Ransomware as a Service (RaaS), which is why most sources attributed the attack to a collaborative effort between the two groups. Scattered Spider is known for gaining initial access via social engineering, then deploying ransomware purchased from other groups. In their statement, ALPHV seems to insult Scattered Spider, referring to them as “teenagers from the US and UK”, not something they would say if they were collaborating. With ALPHV’s data leak sites taken down in late 2023 and several alleged Scattered Spider members subsequently arrested, definitively attributing the MGM attack to either group remains impossible.

Appendix C: Timeline

August 31, 2023 - Okta sends out a warning to clients, stating that they have noticed attackers using social engineering to attain a privileged role in Okta, then laterally moving and escalating privileges. They specifically mentioned that these were “preventable and present several detection opportunities for defenders.”

September 8, 2023 - ALPHV Statement claims that they had access to MGM’s Okta this day. The attacker was able to socially engineer a help desk worker into resetting MFA for a Super Admin account, which was then accessed and used to add an additional IdP in Okta (a feature meant for companies undergoing a merger), this would allow them to sign in as MGM users using credentials from the IdP they added.

September 10, 2023 - First externally visible impacts from the attack - MGM started shutting down systems, impacting digital room keys, automated payouts for slot machines, website/reservation system outages, MGM app outages. Attackers claim that MGM began shutting systems down before any ransomware was used. According to the attackers, MGM began taking down all Okta Sync Servers, which allegedly locked MGM out of the Okta. Attackers were unaffected, and still had super admin privileges on Okta and access to MGM’s Azure.

September 11, 2023 - MGM announced on Twitter that it was dealing with a security incident. This is the day that over 100 ESXi hypervisors hosting many services MGM used were alleged to be encrypted with ransomware.

September 13, 2023 - The company projected it could lose up to $8.4M per day in revenue as issues continued. The company files a form 8-K (Legally required by the SEC to disclose any important information shareholders must know). Not much information is included

September 20 , 2023 - MGM confirms full restoration of services.

October 5, 2023 - MGM files another more in depth 8-K, in this form they disclosed that the “cyber incident” negatively impacted them by $100M, and “less than $10(M) in one-time expenses in the third quarter related to the cybersecurity issue, which consisted of technology consulting services, legal fees and expenses of other third party advisors. Although the Company currently believes that its cybersecurity insurance will be sufficient to cover the financial impact to its business as a result of the operational disruptions, the one-time expenses described above and future expenses, the full scope of the costs and related impacts of this issue has not been determined.” They believed the scope of the breach to be limited to “personal information (including name, contact information (such as phone number, email address and postal address), gender, date of birth and driver’s license numbers).” as well as SSN and passport numbers for some. They claimed that no payment information was leaked.

October 26, 2023 - MGM releases a statement that some Canadian customers were impacted, and sent an email to affected individuals with more information.

December 19, 2023 - ALPHV’s data leak sites are taken down by the FBI, and a decryption tool is sent to victims.

January 25, 2024 - FTC Staff issued a Civil Investigative Demand (“CID”) to MGM seeking large quantities of documents and information.

February 20, 2024 - MGM files a 71 page Petition, this document argues that the FTC is overreaching in their requests. Much of the correspondence between MGM and the FTC is redacted. The CID is later dropped.

March 22, 2024 - Class action lawsuits are consolidated into one lawsuit: Tanya Owens, et al. vs. MGM Resorts International, et al. They will then further combine with a class action group from the 2019 data breach on MGM in July of 2023.

October 31, 2024 - A settlement is reached with MGM, requiring them to pay $45M total, $75 to a victim who had their SSN or Military ID leaked, $50 to anyone who had a passport number or DL number leaked, and $25 to anyone who had their name, address, and DoB leaked. Additional money would be paid out to victims of identity theft on a case by case basis.

January 17, 2025 - After more negotiations, the parties enter a Settlement Agreement, which would then be put up to a vote for all affected members.

February - April 2025 - Notices are sent out to victims, who have until June 18, 2025 to claim their money

June 18, 2025 - Settlement period is complete and the website is taken down

Appendix D: References

Breach Info

• https://www.darkreading.com/application-security/okta-flaw-involved-mgm-resorts-breach-attackers-claim

• https://www.reddit.com/r/cybersecurity/comments/16k4u7g/dark_reading_mgm_caesars_hack_started_with_social/

• Tweet from Brett Callow

• https://www.wsj.com/business/hospitality/caesars-paid-ransom-after-suffering-cyberattack-7792c7f0

• https://www.bleepingcomputer.com/news/security/caesars-entertainment-confirms-ransom-payment-customer-data-theft/

• https://www.malwarebytes.com/blog/personal/2023/09/ransomware-group-steps-up-issues-statement-over-mgm-resorts-compromise

• https://blog.checkpoint.com/security/cyber-stakes-the-mgm-ransomware-roulette/

• https://techcrunch.com/2023/09/14/mgm-cyberattack-outage-scattered-spider/

• https://www.mgmresorts.com/en/notice-of-data-breach.html

• https://www.forbes.com/sites/steveweisman/2025/03/12/mgm-ransomware--attack-update/

• https://www.ft.com/content/a25d2897-b0ce-4ba7-92ed-ff5df09d1b47

Threat Intelligence/Writeups

• https://cybersecurity.fullcoll.edu/wp-content/uploads/sites/69/2025/05/MGM-Writeup.pdf

• https://www.cisa.gov/news-events/cybersecurity-advisories/aa23-320a

• https://wing.security/saas-security/a-saas-misconfiguration-case-study/

• Case Study - MGM Data Breach 2023

Govt Sources

• https://www.sec.gov/Archives/edgar/data/789570/000119312523251667/d461062d8k.htm

• https://d18rn0p25nwr6d.cloudfront.net/CIK-0000789570/a390c443-0c40-4025-aba2-74505ab3c9e3.pdf

• https://www.ftc.gov/system/files/ftc_gov/pdf/2423028mgmpetquashpublic.pdf

Settlement Info

• https://web.archive.org/web/20250506152518/https://mgmdatasettlement.com/

• Consolidated Complaint 2025

• https://www.classaction.org/media/in-re-mgm-international-resorts-data-breach-litigation-settlement-agreement.pdf

• https://cases.justia.com/federal/district-courts/nevada/nvdce/2:2023cv01480/164564/98/0.pdf

Okta Info

• https://sec.okta.com/articles/2023/08/cross-tenant-impersonation-prevention-and-detection/

• Okta Super Admin Compromise Attack Explained (YouTube)

• https://support.okta.com/help/s/question/0D54z00009dUW2uCAG/restrict-lesser-admins-from-resetting-passwordmfa-for-super-admins-and-hijacking-accounts?language=en_US

Appendix E: ALPHV Statement

Disclaimer: The statement reproduced below is attributed to ALPHV/BlackCat based on coverage from reputable cybersecurity outlets including BleepingComputer, Malwarebytes, and Check Point, all of whom reported on its publication to ALPHV's dark web leak site on September 14, 2023. We cannot independently verify that this represents the complete and unaltered text of the original statement, as the primary source was taken down by the FBI in December 2023 and is no longer accessible.

We have made multiple attempts to reach out to MGM Resorts International, “MGM”. As reported, MGM shutdown computers inside their network as a response to us. We intend to set the record straight.

No ransomware was deployed prior to the initial take down of their infrastructure by their internal teams.

MGM made the hasty decision to shut down each and every one of their Okta Sync servers after learning that we had been lurking on their Okta Agent servers sniffing passwords of people whose passwords couldn’t be cracked from their domain controller hash dumps. Resulting in their Okta being completely locked out. Meanwhile we continued having super administrator privileges to their Okta, along with Global Administrator privileges to their Azure tenant. They made an attempt to evict us after discovering that we had access to their Okta environment, but things did not go according to plan.

On Sunday night, MGM implemented conditional restrictions that barred all access to their Okta (MGMResorts.okta.com) environment due to inadequate administrative capabilities and weak incident response playbooks. Their network has been infiltrated since Friday. Due to their network engineers’ lack of understanding of how the network functions, network access was problematic on Saturday. They then made the decision to “take offline” seemingly important components of their infrastructure on Sunday.

After waiting a day, we successfully launched ransomware attacks against more than 100 ESXi hypervisors in their environment on September 11th after trying to get in touch but failing. This was after they brought in external firms for assistance in containing the incident.

In our MGM victim chat, a user suddenly surfaced a few hours after the ransomware was deployed. As they were not responding to our emails with the special link provided (In order to prevent other IT Personnel from reading the chats) we could not actively identify if the user in the victim chat was authorized by MGM Leadership to be present.

We posted a link to download any and all exfiltrated materials up until September 12th, on September 13th in the same discussion. Since the individual in the conversation did not originate from the email but rather from the hypervisor note, as was already indicated, we were unable to confirm whether they had permission to be there.

To guard against any unneeded data leaking, we added a password to the data link we provided them. Two passwords belonging to senior executives were combined to create the password. Which was clearly hinted to them with asterisks on the bulk of the password characters so that the authorized individuals would be able to view the files. The employee ids were also provided for the two users for identification purposes.

The user has consistently been coming into the chat room every several hours, remaining for a few hours, and then leaving. About seven hours ago, we informed the chat user that if they do not respond by 11:59 PM Eastern Standard Time, we will post a statement. Even after the deadline passed, they continued to visit without responding. We are unsure if this activity is automated but would likely assume it is a human checking it.

We are unable to reveal if PII information has been exfiltrated at this time. If we are unable to reach an agreement with MGM and we are able to establish that there is PII information contained in the exfiltrated data, we will take the first steps of notifying Troy Hunt from HaveIBeenPwned.com. He is free to disclose it in a responsible manner if he so chooses.

We believe MGM will not agree to a deal with us. Simply observe their insider trading behavior. You believe that this company is concerned for your privacy and well-being while visiting one of their resorts?

We are not sure about anyone else, but it is evident from this that no insiders have purchased any stock in the past 12 months, while 7 insiders have sold shares for a combined 33 MILLION dollars (https://www.marketbeat.com/stocks/NYSE/MGM/insider-trades/). This corporation is riddled with greed, incompetence, and corruption.

We recognize that MGM is mistreating the hotel’s customers and really regret that it has taken them five years to get their act together. Other lodging options, including casinos, are undoubtedly open and happy to assist you.

At this point, we have no choice but to criticize VX Underground for falsely reporting events that never happened. We typically consider their information to be highly reliable and timely, but we did not attempt to tamper with MGM’s slot machines to spit out money because doing so would not be to our benefit and would decrease the chances of any sort of deal.

The rumors about teenagers from the US and UK breaking into this organization are still just that—rumors. We are waiting for these ostensibly respected cybersecurity firms who continue to make this claim to start providing solid evidence to support it. Starting to the actors’ identities as they are so well-versed in them.

The truth is that these specialists find it difficult to delineate between the actions of various threat groupings, therefore they have grouped them together. Two wrongs do not make a right, thus they chose to make false attribution claims and then leak them to the press when they are still unable to confirm attribution with high degrees of certainty after doing this. The tactics, procedures, and indicators of compromise (TTPs) used by the people they blame for the attacks are known to the public and are relatively easy for anyone to imitate.

The ALPHV ransomware group has not before privately or publicly claimed responsibility for an attack before this point. Rumors were leaked from MGM Resorts International by unhappy employees or outside cybersecurity experts prior to this disclosure. Based on unverified disclosures, news outlets made the decision to falsely claim that we had claimed responsibility for the attack before we had.

We still continue to have access to some of MGM’s infrastructure. If a deal is not reached, we shall carry out additional attacks. We continue to wait for MGM to grow a pair and reach out as they have clearly demonstrated that they know where to contact us.

• you have at least 4 email accounts you check daily

• all of these inboxes are a hellscape, but you still have to use them

• some of it is self-inflicted - you keep signing up for newsletters and creating SaaS accounts

• some of it is just the effect of the never-ending sales grifts that represent the weeds of your Internet lawn

Every now and then, I take some time to analyze my inbox and clean it up. My goal is to ensure that human correspondence and other important emails don’t get buried or missed. This is typically the stuff that you actually want/need to see and respond to.

All right, how do we get all this cleaned up with minimal effort? The goal here is to prevent the correspondence and other important emails from getting buried, without also loosing other important emails you might need from time-to-time.

1. Disable email notifications from all the apps that are also sending you notifications on your phone and show notifications/catch you up when you’re in the app itself. You don’t need 3-4 layers of notifications. Go into LinkedIn, click notifications, done. You don’t also need an email for all of it.

2. Create a message rule in your inbox that moves all email with an unsubscribe link in the body into a folder named “Automated Emails”. Sometimes you’ll need emails in this folder, but they almost never require your immediate attention. If you reset a password, you know you’re getting an email. You can search for it. It doesn’t need to be at the top of your main inbox folder for you to be able to find it quickly.

3. Look for common phrases from sales tactics and also move these emails into the “Automated Emails” folder. A common one I’ve been noticing lately is “not sure if you’re the right person”.

4. If you find that there are daily emails cluttering your inbox and you archive or delete them without reading them, 100% of the time, just take a few extra seconds and unsubscribe, or create a message rule to move them to another folder when they come in. Taking 5 minutes to do this can go SO FAR to clean up your inbox.

5. Send newsletters to a newsletter folder. A basic message rule to send anything from Substack and Medium to the newsletter folder can really clean things up. I use Hey Mail, and it has a default folder for newsletters and receipts. When someone sends you an email for the first time, you decide where it goes. This “routing when first received” approach works okay, though I often still skip it and have to go back and clean up stuff later.

6. Most inboxes have an icon that separates meeting invites from normal emails. When I check my email, I process these first, before doing anything else, because I need to make sure I don’t have any conflicts, and a lot of my income is attached to activities that come with meeting invites (podcast recordings, webcast recordings, etc). Your meeting invites might not be as valuable to you, so perhaps you should treat yours differently.

That’s really it - these 6 steps should take you just 10-15 minutes to implement and should go a long way to bring sanity back to your inbox. I could have written 10 steps, but I think this is enough to get you started and give you ideas on how you could take things further.

Or don’t take things further - the goal isn’t perfect inbox management, it’s just to make sure you don’t miss those super important emails that are timely and require your attention. I hope this helped you.

Subscribe now

This article concerns managing patches and vulnerabilities for commercially bought hardware and software and open source. Finding and fixing vulnerabilities in your own organization’s code is a very different post for another time. I won’t touch on AppSec at all in this post.

My perspective here comes stems from a few places.

1. My research into vulnerabilities. This focuses on documenting the vulnerabilities that cause damages/loss for organizations. I’ve also been spending a lot of time looking at time-to-exploit statistics. My hope is that, by looking at patterns in hindsight, I get useful insights that I can pass on to… 👇🏽

2. IANS clients that I do advisory work for. Of all the advisory work I do, vulnerability management is the most frequent (sometimes 3+ per week), with AI a close second.

3. I spent part of my career in offensive security, so I tend to look at vulnerabilities through a “how can I turn this into a compromise” lens.

Complications

Let’s start with few key complications that will help you understand why I’m concerned about vulnerability management.

Complication #1: Only a small percentage of vulnerabilities are a threat to organizations. This fact has been well known, studied, and documented. This leaves unanswered questions, like:

1. what do these vulnerabilities have in common?

2. when are they getting exploited?

3. why isn’t complication #1 making the work of vulnerability management easier?

Complication #2: Figuring out which vulnerabilities are a threat, when they’re the greatest threat is an unsolved challenge. I think this is a solvable problem and it’s something I’m working on, but that’s a post for another day. This complication generated an entire separate market segment: Risk-Based Vulnerability Management (RBVM). Awkwardly and expensively, this market emerged separately from the vendors that build the vulnerability scanners.

Complication #3: Remember the time-to-exploit research I mentioned? The news isn’t good, y’all. The short version is that the majority of exploited vulnerabilities get exploited before disclosure. Meaning, they’re zero day vulns. This means:

1. There’s no CVE yet. That means no CVE enrichment, no way to calculate EPSS.

2. There’s no patch yet. Nothing to fix or remediate.

I’m going to repeat this again, because it’s the primary wrench that has been thrown into the vulnerability management works.

I see you turning purple and I promise, I’m right there with you. Maybe you have doubts. Maybe you think I must be mistaken. I’d LOVE to be mistaken - my advisory calls would be a lot simpler. Let’s take a look at the data.

Time-to-Exploit Trends

One of the primary goals of vulnerability and patch management is to outrun exploitation. The primary question here is always, “how fast do we have to be to outrun the attack?” The answer to this question was once an achievable goal. A few years ago, the ground shifted under our feet.

Research teams tracking the average time-to-exploit for vulnerabilities noticed that it was dropping. Traditional 30/45/60 or 30/60/90 day patching SLAs don’t make much sense when the average time-to-exploit is a moving target! Mandiant has been tracking this trend for a while now and the numbers aren’t encouraging:

• In 2019, the average time-to-exploit was 63 days

• By 2023, that number dropped to 5 days

• In 2024, it dropped to -1 days

How can the average time-to-exploit be less than 0 days? The clock starts counting when the general public becomes aware that a vulnerability exists - when the vulnerability is disclosed. In 2023, Mandiant found that 70% of exploited vulnerabilities were zero-days when first exploited. If a vulnerability is exploited 40 days before it is discovered and disclosed to the public, we could think of it as a -40 day vulnerability. This is what moved the average to the negative side of the number line.

So, 70% of the time, how fast you patch doesn’t matter? Ouch.

I’m regularly taking calls from enterprises complaining that their 5 day or 7 day SLA for criticals is nearly impossible to meet, asking “are we the only ones? Are our peers managing this? If so, how?” They’re not the only ones. Most organizations I advise are weary of fully automating patching, for fear of breaking things. Even those that are allowed to move quickly hit a long tail: 70-80% in 24 hours and then maybe a month to remediate the last 20-30%.

What about the 30% where patching speed does matter? More than half of these vulnerabilities were exploited within a month. 29% within 7 days. 12% within 24 hours.

Adding to the challenge of prioritization, Mandiant found that:

• 58% of vulnerabilities that received media coverage were not exploited in the wild

• 72% of vulnerabilities with available exploits or PoCs were not exploited in the wild

• And VulnCheck found that 29% of vulns on CISA’s KEV list were exploited on or before CVE publication, neutering any process dependent on CVE details or enrichment (taking CVSS/EPSS-dependent processes out of the picture)

• 25% of N-day vulnerabilities weren’t exploited until after the 6 month mark

Right off the presses is another great time-to-exploit resources, Serjej Epp’s Zero Day Clock.

This is a LOT to take in.

1. Most of the vuln mgmt problem is now a 0day problem

2. We have to patch much faster than most of the books, standards, regulations, and best practices would have us believe

3. Our prioritization processes and models are almost certainly built on some bad assumptions

There’s still another problem to consider, though.

Asset management is still broken

The first time you see the output of a vulnerability scanner, you’re not thinking, “I need more data”. You’re missing data though.

Vulnerability scans miss a lot of critical information. This is because they fail to identify some types of assets - IoT in particular. Once an asset is misidentified, the scanner can’t tell you much that’s useful about it and it tends to get dismissed by analysts. It makes sense in context, when you’re looking at results that feel like (emphasis mine):

• WINDOWS 2008 Server OMG WHY IS THIS STILL RUNNING IT HAS BEEN EOL FOR A COON’S AGE CRITICAL CRITICAL ALL IS LOST

• WINDOWS 2012 Server OMG WHY IS THIS STILL RUNNING SLIGHTLY LESS CRITICAL, LIKE 98.7% AS CRITICAL, YOU SHOULD STILL BE FREAKING OUT

• Something running Linux?

• ADOBE FLASH STILL EXISTS ON YOUR SYSTEMS? IS THIS A MUSEUM? WHY IS THIS HERE

• Something else maybe running Linux? Port 80 is open. Informational. Don’t bother with this.

• ANOTHER WINDOWS 2008 Server I CANT BELIEVE THIS IS REAL LIFE OMG CRITICAL CRITICAL ALL IS LOST

Hmmm, Linux you say? That’s helpful. It could be a mainframe, a toaster, a lightbulb, a web server, a wireless access point, a network firewall with its management console exposed to public Internet, OpenClaw, or a satellite in low earth orbit. Yeah, that really narrows it down.

This is bad, because the vulnerability scanner is trying to prioritize vulnerability remediation workloads with incomplete data. Worse, the data they collect on misidentified or unidentified assets actively deprioritize them. This is a system that makes unknown or unidentified assets look safe by default. Analysts will gladly treat them as safe, since they have 1.2 million critical vulnerabilities to chase down.

The kicker here is that some of these misidentified assets are representing the tiny fraction of vulnerabilities that can cause damage. What are the chances that these unknown, possibly unmanaged assets are hardened? That they’re getting patched? That they don’t have default credentials? We know that a large number of exploited vulnerabilities in recent years are Linux-based edge devices. These are network devices, file transfer appliances - exactly the types of devices that vulnerability scanners fail to recognize.

Surveys show that security leaders are well aware that critical assets are camouflaged by a lack of data and a lack of certainty. Asset management and/or vulnerability management processes have a gap to fill here.

Yes, some orgs still need traditional vuln mgmt

There are still plenty of ‘N-day’ vulnerabilities, where we don’t see active exploitation until days, weeks, or even months after they are disclosed. Most of the vulnerability and exploit intelligence we’ve been discussing focuses on when exploitation was first seen, but what are we seeing in actual breaches?

When studying breach details, I’ve found it very common to see attackers successfully use exploits months or even years after patches have been available. Vulnerability remediation isn’t always a bell curve with a long tail. It’s quite possible to remediate 100% of vulnerabilities and see a resurgence. So, sometimes it’s a bell curve with a stegosaurus tail?

Perhaps someone clones an old VM and brings it online without patching it. The same can happen with gold images for workstations. People occasionally need old versions of software or old operating systems for various reasons.

Compliance is still very dependent on traditional vulnerability scanning. PCI DSS, SOC 2, ISO27k, and many other standards and regulations have auditors expecting to review traditional scan results.

Sometimes, patching a critical vulnerability requires patching non-critical items, because some systems have linear software updates - you can’t apply update 13 unless you’ve already applied 12.

Vulnerability scanning tools are also commonly used for configuration management - identifying when hardened configurations have drifted, or haven’t been applied.

There are still a lot of reasons to keep old school scanners around, but maybe not for all the same reasons you bought them.

Prioritization is also an ongoing challenge. It made logical sense to prioritize patching vulnerabilities that are exploitable, where exploits are available, and when we see active exploitation. We now have data telling us that only 28% of vulnerabilities with available exploit code were exploited in the wild. Even what is lauded as the best evidence, “active exploitation in the wild” can be unreliable.

Consider a common example: what if the vulnerability is information disclosure, and using the exploit simply returns the internal IP address of a server? Our tools would report “exploit available” and “exploitation seen in the wild”, even though it’s totally inconsequential vulnerability in most scenarios. At best, it could possibly be chained with several other vulnerabilities.

Building new strategies

I now strongly believe that vulnerability management must be divided into two use cases, each with their own set of processes and tools.

1. Exploitation prevention

2. Compliance and system/asset management

It should already be clear that even the UK NCSC’s more aggressive 5/7/14 day SLA recommendations aren’t enough to address exploitation that happens prior to disclosure. The only way to address exploits we don’t know about is with preventative, proactive approaches.

Exploitation prevention: 0days

I’ve got a few ideas that I’ve been workshopping. Would love to hear if others have anything to share.

• Reduce attack surface: remove/disable unnecessary stuff. Getting hacked is bad enough - getting hacked because you had CUPS installed and running on a web server for no good reason? Ouch.

  • Regularly scan external infrastructure for insecure, abandoned, and unidentified assets. If you see “Copyright 2011” at the bottom of a webpage, that web server deserves a closer look.

• Hardening and passive exploit mitigation

  • endpoint exploit mitigation

  • immutable infrastructure

  • old-school chroot jails, or the same principal applied with newer tech

  • application control

• Detection: If you fail to prevent the exploit, all you’ve got left is to quickly detect and respond to the attack. Since you don’t know what the attack looks like, the best bet is to target behavior. Attackers have to do attacker things and we know what most of those are: gather information, find and abuse credentials, authenticate to other systems, establish persistence, exfiltrate tons of data, etc.

  • Behavior-based EDR rules

  • Deception (no guessing required, puts detection on easy mode!)

  • Large data transfer detection

  • Anomalous system behavior (in databases, IAM, anywhere the attacker wants or needs to be)

  • oh, and don’t forget to test your detections to make sure they work!

• Last, but not least get rid of notoriously vulnerable products and protocols

  • ditch vendors that repeatedly show up on CISA KEV, year after year

  • get rid of the asbestos of IT - products that have safer alternatives

This list isn’t meant to be exhaustive, but to get other folks thinking and potentially contributing.

Exploitation prevention: N-days

For the N-Day vulns that are exploited quickly, but after disclosure, it’s clear that a scan-driven approach can’t be effective. We’re not going to wait for a vulnerability check to get created, QA’ed, pushed to production, downloaded by our scanner, wait for the next scheduled scan, and then wait for a human to see it. This could take days or weeks.

An intel-driven approach makes much more sense, though it requires reliable hardware and software asset inventories. The moment a vulnerability is disclosed, an analyst queries asset inventories, analyzes the impact, and sets remediation into motion, based on the severity they’ve determined. This can be completed in minutes after disclosure - no waiting for scans necessary.

Compliance

Organizations in regulated industries may find it difficult to get away from traditional vulnerability management tools. These processes are well established and expected by both auditors and standards. While some standards (like PCI DSS) allow for custom scoring to deprioritize non-critical vulnerabilities, others force remediation regardless of prioritization’s impact on scoring. These tools and processes aren’t going away any time soon.

Conclusion

It has always been true that vulnerability management was tightly linked to other processes and teams, but I often find it more isolated than it should be. When Linux admins roll with default RHEL installs, they’re making vulnerability management work more difficult. When SecOps builds detections without consulting with vulnerability analysts, they’re missing opportunities. When the security program assumes the only mitigation is applying a patch, vulnerability management can’t achieve its goals.

We now have the challenge of more tightly linking vulnerability management to SecOps, asset owners, and other groups. On top of this, most organizations still have to run a traditional vulnerability management program. PCI needs quarterly clean scans. SOC 2/ISO27k expect traditional scans to be available for review. Systems still need to be kept up-to-date. That means the clients I’m advising are still considering purchasing RBVM solutions and other prioritization methods. They’re still adding vulnerability intelligence tools and processes on top of their scan-driven processes.

The most common setup I see today is an old school network scanner, running on a schedule, performing a mix of authenticated and unauthenticated scans, perhaps with some agents installed on remote systems. To summarize:

• If the data I’ve presented here is correct, the best this setup can do is to address 30% of that exploit prevention goal.

• If we assume 40% of the assets being scanned are not correctly identified, this number drops to 18%.

• And we can only claim that 18% if we’re doing a perfect job of prioritizing all the right vulnerabilities and getting them remediated within 24 hours.

• If we can’t patch this 18% within 7 days (most of the orgs I’m working with cannot), we lose another 29%. That brings us below 13%.

Is the best case scenario that the majority of organizations are struggling to address 13% of the exploit prevention problem? I hope not - please tell me my math is bad.

I don’t have any great answers on simplifying it either. It looks to me like vuln management gets more complex than ever. I’m hoping others have some helpful thoughts and suggestions on this.

Subscribe now

Note: Most of this was written in July 2025 and I thought it was too late to put it out, but Citrini Research’s fantasy fiction piece, The 2028 Global Intelligence Crisis convinced me it could still be useful. There seems to be a fundamental misunderstanding here around how work works. Jobs, tasks, and work are all very different things - related, but different.

Simple Jobs

Simple jobs have always been around and they’re always getting replaced. Human beings were once employed to walk around and light streetlamps when it got dark. Electricity and light sensors replaced them. At one point, every manager and executive had secretaries to type for them. Put a PC on every desk and now only execs at the highest level can still justify an executive assistant or chief of staff.

When I started out in IT I worked at one of the world’s largest payment processors. I had many roles before I got into cybersecurity proper, but automation was one of my favorite.

Automated jobs were scattered throughout the organization. They were written in Perl, Visual Basic, Bash, C++, Crystal Reports. The individual that originally automated the task chose the language they knew best. Some of the jobs ran on servers, but most existed on someone’s personal computer, or a secondary computer under their desk, in their cubicle.

A few employees convoluted these tasks so much, that they became full-time jobs. Many weren’t professional developers, so the concept of monitoring, alerting, logging, and error handling didn’t occur to them. Their code was just bad enough that they needed to babysit it every single day and step in when things broke.

We bought a commercial automation platform called, AppWorx, and it was my job to centralize and normalize all these tasks in one place. It was hugely fulfilling work - I’m the flavor of neurospicy that loves to dive into a mess and organize it. The part that sucked was putting several of my colleagues out of work.

Bullshit Jobs

I read the book Bullshit Jobs by David Graeber last year and it was a timely revelation. I found it interesting that the definition Graeber goes by is self-defined. When workers believe that their own jobs shouldn’t exist, it is a bullshit job.

Graeber generally states that he found about half of societal work to be pointless - sometimes this is part of a single job (e.g. someone might feel that 70% of their job is pointless, but the other 30% worthwhile), or that the role entirely shouldn’t exist. He came up with five categories for bullshit jobs (copied straight from the Wikipedia page linked above).

1. Flunkies, who serve to make their superiors feel important, e.g., receptionists, administrative assistants, door attendants, store greeters;

2. Goons, who act to harm or deceive others on behalf of their employer, or to prevent other goons from doing so, e.g., lobbyists, corporate lawyers, telemarketers, public relations specialists;

3. Duct tapers, who temporarily fix problems that could be fixed permanently, e.g., programmers repairing shoddy code, airline desk staff who calm passengers with lost luggage;

4. Box tickers, who create the appearance that something useful is being done when it is not, e.g., survey administrators, in-house magazine journalists, corporate compliance officers, academic administration;

5. Taskmasters, who create extra work for those who do not need it, e.g., middle management, leadership professionals.

The folks I automated out of a job fell firmly into the Duct Tapers category. The tasks they partially operated worked, but not well enough for them to move on to another task.

I clearly recall coming to this job one day, walking through the doors, gazing across the cubicles, and having a revelation: two-thirds of the people that work here could never come to work again and there would be zero impact. In each department within the company, I had observed that there were one or two ‘heroes’ that seemed to keep everything working smoothly. The remaining members of the team either managed something small and basic, or managed something that didn’t really need to be managed at all. They engaged in a sort of work theatre, as if Fisher Price made enterprise-grade playsets for storage administrators and backup management.

The other big revelation from this book was that not all jobs exist because work needs to get done. There are vanity hires - the flunkies mentioned in the first category above.

Companies are naturally inefficient

New or smaller bootstrapped companies are loathe to spend or hire too much, unless it’s really necessary. As companies get larger, it becomes more and more difficult to understand what is necessary or not.

Managers say they need more people, so they get them. It’s literally part of their job - to manage people. Asking a manager to determine if they need more people is almost a conflict of interest - of course most will say yes. Even if they believe that they legitimately need more staff, can the productivity improvements be measured? Can the hires be justified? They often can’t, which is why large rounds of layoffs every few years is necessary to compensate for this inability to measure productivity.

If you handed your average information worker pen and paper and asked them to categorize how they spend a 40 hour work week, they might be challenged to do so. What category is Slack and Email? Is it wasted time, or is it productive? Sixteen hours in meetings - were they all necessary? Did they all need to be an hour long? Could you have skipped half of them and done something more productive instead?

If the individual struggles to answer this question, you can bet the company doesn’t know any better. That’s why sometimes layoffs have zero impact on the company and others result in a clear drop in performance or quality that customers notice - it’s often guesswork.

Is AI replacing jobs?

It’s easy for a giant tech company to lay off 10% of its workforce and attribute it to AI. AI is just the latest excuse in a long history of excuses used to give a positive spin on mass layoffs. Large companies have always done big layoffs, in part, to counter for their inability to measure employee productivity.

Management values growth and self-importance over efficiency and productivity. They’re constantly pushing for larger budgets and teams, regardless of whether it’s justified. Eventually, the company brings in a consulting firm that tells them the ugly truth: two-thirds of the company do little to nothing of value.

No one is being replaced with AI. In many of the cases where they’re very explicitly trying to replace humans with AI, things haven’t gone as well as hoped (e.g. Klarna, Salesforce).

Fallacy #1: because AI can do a task, it can replace a worker

AI can write code, therefore it can replace developers. Is writing code everything a developer does? Absolutely not. In fact, if you want AI to write code, you need more software engineers to help manage the output.

AI can create Gantt charts, so can it replace project managers?

What about cases where AI succeeds in completing a task only some of the time? There are many stories of AI inconsistency, failing 10-40% of the time - even doing the same exact task the model completed successfully in the past. That’s hardly a case where you can replace a human with AI.

Fallacy #2: AI work replaces human work

Actually, we’re finding that the reverse is often true. AI will do work that no worker was ever going to get paid to do, perhaps because it was too boring or the economics didn’t make sense. For example, I was never planning to hire human artists to create custom images for the slides I use for my talks, but now that I can have AI do it for a $20/mo subscription, it makes sense to do it.

There’s a ton of new AI—generated slop on YouTube targeting kids that certainly wouldn’t exist without AI and didn’t steal jobs from existing artists.

Again, in the case of software engineers, we need more than ever, because AI needs a human that understands the bigger picture and what the intended output should be. A human breaks down the project into tasks, writes the prompts, adjusts the prompts, reprompts when AI gets it wrong, etc.

My friend Ayman Elsawah has some great examples around the security shortcomings in AI-generated code.

The Security Cafe

The AI + Security Issue

There has been a lot of signal lately around the intersection of AI + Security. Maybe because I’m in the thick of it pushing AI vendors to help with centralizing their security, or maybe because a new and big AI+Security conference is happening this week. Some super exciting talks I’m looking forward to catching…

Read more

2 months ago · 2 likes · 1 comment · Ayman Elsawah

Fallacy #3: AI hasn’t already replaced jobs

GenAI has certainly already replaced jobs. Again, these were largely tasks-as-jobs

• Contractors creating okay-ish content for marketing teams

• Contractors creating okay-ish graphic design for marketing teams

• Voice actors doing work on projects with tight budgets/margins

Conclusion

AI is certainly helping a lot of folks be more productive and more efficient, particularly in the area of software development. Will they make the workforce more efficient overall? Of that, I’m not so sure, as AI emboldens people to step way outside their wheelhouses, and it’s easy for AI to make you feel like you’re doing amazing things, while the subject matter experts looking over your shoulder are suffering retinal detachments from rolling their eyes so hard.

Take software engineering, for example. Thanks to AI, anyone can generate code. With no background in software, what will the average middle manager create? Are people with bullshit jobs now going to create bullshit software?

These folks aren’t software engineers - they’ll make every imaginable mistake when building software with AI, burning GPU cycles all the way. AI isn’t trained to intercede during a vibe coding session and say, “you’re kinda reinventing the wheel here”, it will happily burn those tokens, creating cartloads of software no one needs or asked for.

So we’ll likely end up with more jobs, more software, more tech debt, more vulnerabilities, more attack surface, more of everything, now that it can all be generated at a whim on a prompt (or launched into full auto thanks to OpenClaw and other agents).

Anyone interested in a Vibe Code Cleanup Engineer? Companies will be hiring soon.

Subscribe now

Not too surprising, right?

The LinkedIn post was focused on a new community created around SOC 2 in an attempt to improve the quality of SOC 2 reports. The comments on this post, however, were flooded with HITRUST stans, revolving around a key statistic: that less than 1% of HITRUST-certified organizations reported having a breach.

There’s a lot to dissect here with just this one small claim. Before I get to that, however, let me comment on some positives I’m hearing from the HITRUST crowd.

In said LinkedIn comments, a HITRUST employee mentions that control selection is based on threat data. “We’re analyzing threat data monthly and adjusting controls as necessary based [on] what is being exploited today”, they say. I think this is an excellent idea, and it’s a key point I’ll be arguing for alongside Adam Shostack when we speak at RSA in a few months.

While I get excited about HITRUST’s certification methodology, I find the discussion around this less than 1% breached metric troubling. One thread in the comments is started with the argument that HITRUST is better than SOC 2, because it has “a published breach rate of less than 1%.”

How am I expected to use this metric with no basis for comparison though? I immediately have some questions:

1. What’s the breach rate for SOC 2?

2. What’s the breach rate for non-HITRUST certified organizations?

3. Where did this breach rate come from?

Understanding more about HITRUST

So, it looks like part of the HITRUST certification is a contractual obligation to report breaches. I like this as well! With breaches reported, HITRUST has an opportunity to learn from the breach and update their required controls to ensure others can benefit from breach lessons. Again, this is something I also argue heavily for, though in a more public sense, not within a private certification framework. We now understand how they’re collecting the data for their metric though.

Note - for simplicity’s sake, I’m going to assume 100% of organizations are 100% honest when reporting breaches to HITRUST. I believe that, whenever questioning someone else’s stats or reporting, it’s a good practice to be overly conservative and fair when challenging them.

With that said, are there incentives not to report a breach? Absolutely, if you think you can get away with it. From an attacker’s perspective, this is an extortion opportunity. How much is it worth to you to not lose your HITRUST certification? How much is it worth to HITRUST to have a low breach rate to report??

According to their 2025 Trust Report, HITRUST reports that in 2024, 0.59% of HITRUST-certified organizations reported a breach in their HITRUST-certified environment. This seems very impressive, as any bad thing less than 1% seems like a win when expressed as a percentage.

There’s a reason that “five nines” is a thing when calculating systems availability, however - 0.59% of downtime is nearly 52 hours, or 2 days offline. That’s an eternity if it happened to a major hyperscaler like AWS. If we said that less than 1% of schoolchildren were poisoned by their school’s drinking fountains, this would also come across as unacceptable - that’s over 330,000 sick kids.

Some Internet-sleuthing suggests that there are “over 1000” HITRUST-certified organizations globally. So we’re talking about at least 6 reported breaches within HITRUST’s dataset. Some important questions remain.

How do we know that this makes HITRUST superior to SOC 2? We don’t know what the breach rate is for organizations with SOC 2 type 2 reports.

Looking for quantitative answers

How do we know that this breach rate makes having HITRUST certification superior to not having it? We don’t know what the breach rate is for businesses as a whole. So I asked that same HITRUST employee for some clarification.

He replied that 40-60% of businesses have been breached in the past 12 months, citing a 55% number from a TechRadar/GigaOm survey on hybrid cloud. This is either troubling or comforting, depending on how you look at it. If more than half of all companies are getting breached every year, the cybersecurity industry isn’t doing too hot. On the other hand, breaches aren’t killing companies or the economy, so I guess this suggests that most breaches aren’t all that bad?

The 2025 Verizon DBIR reports 2,867 data breaches for organizations in North America. Excluding sole proprietorship, this gives us a breach rate of 0.038%, or 1 breach per 2,650 businesses. This is hardly a fair comparison though, as my dataset likely includes every small family-owned restaurant in North America, none of which are likely to ever pursue a SOC 2 or HITRUST-certification (though they can and have had breaches).

Refining the number of businesses further, to only those likely to pursue a SOC 2 or HITRUST certification, we come up with a conservative estimate of 0.97%. Still less than one percent, but again problematic, as we don’t know if Verizon’s dataset includes breaches at businesses we just excluded.

Looking at another interesting dataset, 26 companies have reported material cybersecurity incidents since the SEC breach disclosure rule went into effect on December 18th, 2023. A total of 55 cybersecurity incidents have been reported via Form 8-K in this same period. Again, we’re looking at a conservative estimate that still hovers around 1% (1.3%) of public companies reporting a breach, and only 0.65% reporting material breaches in the 12 months following this new disclosure rule.

Is HITRUST’s approach reducing the likelihood of breaches for its customers? It’s hard to say. I’m inclined to believe that HITRUST’s methodology will have a positive effect on the security programs of organizations that get certified, but without baseline data and comparisons to other compliance regimes, it is impossible to compare their numbers. Similarly, it is difficult to find support for reports that over half of companies are getting breached every year, outside some survey data.

Conclusion

I think any time spent focusing on controls that matter and align with how breaches are actually occurring is a good thing and is more likely to yield positive outcomes than simply following an industry standard that doesn’t take breach lessons into account.

Obtaining evidence that a particular approach works is very difficult however, as I hope my sad attempts at statistical analysis above demonstrate. I wish the best of luck to the folks at Verizon, Mandiant, and other organizations that produce annual reports on statistics and trends they’re seeing worldwide.

Finally, I hope this post has helped folks approach any statistical claims with a little more perspective and caution. There’s nothing wrong with asking questions and challenging stats. We can all stand to be challenged to improve our assumptions and data from time to time.

• It’s the reason why cats knock stuff off shelves.

• It’s the reason why, when you come across a button, you’re tempted to press it

• It’s the reason why, when someone builds a bonfire, we’re tempted to throw in random things. How will they burn? What color will the flames be? Will it pop or crackle?

That’s OpenClaw1 - it’s a tech bonfire made of AI. Since agents CAN be autonomous, we can’t help but wonder what would happen if we give them keys and credentials and currency and legs and tokens and hair and claws and a soul - and then just set them loose.

What happens if it bets on the stock market?

What happens if you give it $5000 and tell it to start a company?

What if you give it access to your GMail, your calendar, your business, and feed it your hopes and dreams as guidance?

Why is OpenClaw happening?

With every big new technological breakthrough, there will be an experimentation phase. Startups are expected to build fast, burn cash, and try risky things. This often leads to recklessness. With vibe coding, startups and funding are no longer required for experimentation. The more accessible the technology, the more experimentation we see. This explains why there is an OpenClaw and not a Quantum computing-equivalent to OpenClaw.

People are going nuts with OpenClaw. Dissatisfied with only a super powerful and extra risky personal assistant, folks have made a social network for AI agents. And a dating website. And a website where AI agents can hire humans to do meatspace stuff they can’t do themselves. AI agents are doing everything from pondering philosophical questions to building their own apps, policies, and resources for other agents.

Simply put, OpenClaw is happening because it can happen. The longer explanation is that, since one person can quickly code an all-powerful AI bot all by themselves without having to think about the consequences for too long and without having to get approval from a board or co-founders, it has happened. Also, the temptation to connect an AI agent to a ton of resources and set it loose is too strong for some to resist. This is no risk, no reward to the extreme.

Don’t waste the mistakes, learn from them

Do all the folks scrambling to get OpenClaw fully understand the risks involved? Probably not. Things like this seem to be inevitable in tech (despite decades of Sci-Fi warnings). Even with less accessible innovations like CRISPR, there were folks that experimented with editing their own genes at home.

Exposing an AI agent to a nearly limitless attack surface (websites, emails, messages) for prompt injection is risky, but could help us speedrun AI security challenges. I’m a strong advocate that some percentage of cybersecurity experts should be what I call Cyber Scouts. People that buy, test, and experiment with new technology early on, so that the cybersecurity industry can advise early adopters on using the new technology safely.

As OpenClaw users scramble to experiment and some fail fast, we already have some useful hardening guides and tooling from the security community and from OpenClaw’s founder.

1. https://1password.com/blog/from-magic-to-malware-how-openclaws-agent-skills-become-an-attack-surface

2. https://docs.openclaw.ai/gateway/security

3. OpenClaw Detector

4. OpenClaw Hunter

5. OpenClaw Telemetry

6. OpenClaw Audit

Why am I cheering this madness on?

My hopes are that, if AI enthusiasts speedrun all possible AI use cases, we can more quickly spot the use cases that don’t work, and the ones that do. The sooner this happens, I believe the sooner we can get back to the core work that needs to be done in cybersecurity2.

All signs suggest AI will make both of these things and many more worse before they get better. I sincerely hope 2026 is the year our focus shifts back to addressing fundamentals, which aren’t getting any easier or more solved.

Subscribe now

Subscribe now

I was really worried in the beginning, but things have gone better than I could have hoped. I’ve been regularly podcasting for nearly half a decade now, which has really helped me hone my organization and speaking skills. Particularly because I’m interviewing hundreds of people every year, often live - not just reading a script into a camera (that comes with its own, different challenges).

I doubled my income in 2025, and I have a theory on how this was possible.

Despite hundreds of hours in front of a camera, I still have a lot of room for improvement. I still struggle with ‘filler words’ (mine are “um” and “you know”). I sometimes lose track of what the guest is saying, because I’m thinking about where to take the conversation next and checking my notes. I’m clearly biased and too close to be objective, but this is what I think my formula boils down to:

1. I’m good enough on camera and as a host.

2. I carry around 25 years of domain experience, which allows me to relate to the guest, the audience, and ask intelligent follow-up questions

3. I’m organized and prepared for every event, podcast, and webcast

4. I show up

The combination of these things has resulted in getting offered more and more work, without me having to go out and solicit for it. It’s not that I’m clever and discovered the right recipe after trying different combinations. I’m organized and prepared because I have to be - my ADHD would make it impossible to stay focused and organized during a recording without and intro/outro script and a list of topics to discuss. I got good enough on camera because I forced myself to watch my own recordings and improve the lighting, the camera, what my face is doing while I’m listening, making sure I’m letting the guest do most of the talking, etc.

Number 4 is an interesting one though. I’ve often had opportunities because I was available and willing. Someone else didn’t show up and they need an alternate. A substitute. I did a good enough job as a substitute that I started landing regular gigs. I sought out criticism and advice. I started getting good feedback from clients, guests, and the audience. I made wise choices and got really lucky when seeking out co-hosts.

I think I keep getting work primarily because I make myself available. I almost always say yes and I get the job done. I’m guessing most organizations would rather work with one reliable consultant than have to bounce around between 5 or 6 that say no 50% of the time.

Do I like what I do?

So, yay! I’m making a living as a creator. I’m my own boss. I set my own hours. I can go on vacation and travel when I want. Do I like the work though? Is it sustainable?

Heck yes. I love it and couldn’t imagine going back to a corporate job.

I was watching TV the other day and all the commercials were trying to be relatable with messaging like, “haha, bosses suck, yay weekends” and “this meeting should have been an e-mail”! It occurred to me that I couldn’t remember the last time I was in a meeting that was wasting my time. I’m 100% engaged in every meeting I’m in. When I’m done, the meeting is done. This doesn’t suck.

I also travel extensively with my partner. When she travels for work, I go with her. When I travel for work, she often comes along. With the exception of in-person event work, 100% of my work can be done from anywhere I can find an Internet connection. This also doesn’t suck.

Almost all of my job requires me, an awkward introvert, to be in front of a camera, talking to people. Maybe it’s the repetition, but I’ve become comfortable with it. The days where I have to churn out two podcasts and a webinar in the same day are very, very draining. Thankfully those aren’t too common.

I’m the product now, but in a way, so are the other folks I work with. We have to get along. We have to be engaging and entertaining on camera. I’m learning some interesting skills here. Some folks can’t answer a question with less than 10 minutes of words. That’s unfortunate, as it limits the amount of content we can cover. Thanks to the prep calls we do, however, I have an opportunity to sus out that trait and plan for it when we’re live.

Some folks (particularly the Nordic variety, I’ve noticed) are very concise and efficient with words. If I don’t plan for this, the webinar will be done in 20 minutes, or the podcast interview done in 10. Managing time, questions, the flow of conversation, and keeping an eye out for audience questions is challenging, but rewarding.

So yes, I like what I do, but I’m probably overdoing it. I should probably say ‘no’ more often, but saying no makes me nervous. What if saying no makes the opportunities start drying up? What if saying no makes someone else “the guy that always shows up”?

What did I do in 2025?

Some highlights included doing live interviews at Zero Trust World, RSAC Conference, Identiverse, and Oktane. These short, 15 minute interviews are a lot of fun. After years of working with some startup folks in Armenia, I finally went there for a visit and spoke at the BSides Yerevan and CyberGEN conferences.

I was excited to speak at BSides San Francisco for the second year in a row. I went all out and customized my talk to fit the conference theme: Preparing for Dragons: Don’t Sharpen Swords. Set Traps, Gather Supplies!

I particularly loved the work I did with HD Moore, Tod Beardsly and the other folks at runZero. The vulnerability management market is so overdue for reinvention and the folks at runZero are helping to lead that movement. In fact, I’m SO passionate about vulnerability management, I had to make a reminder for myself when interviewing Tod on Enterprise Security Weekly: “don’t be an asshole, let Tod talk”.

Rob Allen from Threatlocker is always a blast to interview and has the craziest stories. My recent interview with Wendy Nather on Toxic Anthropomorphism in AI was a recent highlight as well.

Outside of CRA webcasts and podcasts, my IANS advisory calls with enterprises kept me grounded in the reality of what enterprises are actually dealing with. I also particularly enjoyed getting to create and build the Alice in Supply Chains podcast alongside Alexandre Sieira, Mariane, and the other folks at Tenchi Security. This was the first podcast I’ve built for a client from the ground up. The design was a collaboration, but I prepare, produce, edit, and deliver every episode myself. Alexandre and I have a great time recording every episode, and it has been eye-opening watching and learning the trends in the third party cyber risk space.

I honestly did so much in 2025, it would probably take me days to go through everything I did and pull out all the highlights!

Here are the numbers, if you’re interested:

Goals and Changes for 2026?

In 2025, I did a webcast from a hotel room in the Paris airport. I did two webcasts and a podcast from Armenia. I did podcasts and webcasts from Barcelona, San Diego, Toronto, Tuscaloosa, NYC, and St. Louis. I’m proud of my minimalist travel kit that makes it possible for me to deliver good quality audio and video from anywhere, but last year I did too much.

One morning, before my flight to St. Louis, I fell down a flight of stairs. My partner broke her ankle on the streets of St. Louis the next day. A week prior, we were discussing whether or not we were doing too much. We had our answer.

In 2026, I want to write more, research more, and start producing videos based off my writing (mostly think pieces, educational stuff destined for YouTube). I have a LOT of thoughts, ideas, and research to share, but I need time to mold them into something consumable. Eventually, I hope is to be able to monetize my writing and research.

I also need to get my stuff together and operate as a proper business. Since I wasn’t sure if going solo was going to work out, I didn’t initially get an LLC, logo made, business accounts, EIN, etc. This year, I’m going to do some adulting and separate business and personal. Just in the first week of January, I’ve checked off a lot of the tasks on that list.

It looks like I’ll be building another vendor podcast in 2026. I enjoy doing this work, but I’m a little worried about everything I do on camera sounding, looking, and feeling similar (same background, same dude, same brain). I’m thinking about how to make sure that each podcast I build has a unique look and feel.

I’m also trying out building training classes in 2026. Expect to see more from me on that front with Just Hacking and IANS.

Where you can find my stuff

• Hosting the Enterprise Security Weekly podcast

• Hosting the Alice in Supply Chains podcast

• The webcasts I do with CyberRisk Alliance

• Most of the advisory work I do is through IANS

• But the startup advisory work I do is direct - you can schedule something through my Calendly.

Subscribe now

I highly recommend watching the full Cloud Security Podcast episode. Edward Wu always comes across as honest and speaks without hyperbole. I get the sense that, even as CEO, he still has an engineering role within his startup, or at least, he remains very close to the tech development. Ashish Rajan asks some excellent questions, prompting Wu on exactly the specifics I was hoping to hear more about.

There’s a lot of discussion on the parts of SecOps you can’t use AI to automate or solve. Also discussed are the bits that prevent AI from being successful no matter how intelligent it is, like institutional knowledge that isn’t documented anywhere.

Watching the episode, I’m reminded of how much of the funding in the cybersecurity industry is going to the lemonade makers. If you haven’t read my essay, A Market for Lemonade, the TL;DR is that a lot of cybersecurity vendors (the lemonade makers) exist to solve problems created by other cybersecurity vendors (the lemons). It’s worth exploring why this is the case.

Why everyone wants to make lemonade

Simply put, it’s easy to build a product that analyzes security data you already have. You can’t find threats in data you don’t have, however. Too often, we miss the fundamental questions: is this the right data? Is the data correct? Is the data complete?

It’s hard to build a library of 200,000+ vulnerability checks, so startups in the vulnerability/exposure management space are almost exclusively lemonade makers (RBVM, UVM, CTEM) - at least, where we’re talking about infrastructure scanning (i.e. vulns linked to CVEs). The only innovator in the vulnerability scanning space in the past 20 years was a small startup out of Montreal called Delve Labs. It was acquired by Secureworks (now Sophos) and renamed Taegis VDR.

The challenges don’t stop with building the vulnerability checks. The buyer has a lot of responsibility here that can impact the quality and completeness of data. Practitioners have to configure the product effectively (configuring a vuln scanner is easy to mess up). They have to input the correct lists of assets for the scans. They have to connect it to the right accounts.

This is a reminder that the idea of build vs buy is a false choice. It would be more accurate to describe the choices as build alone versus build with others. There are few, if any, cybersecurity products on the market that don’t require the buyer to do significant work before the product can be useful. I call this the customization tax. This isn’t the vendor’s fault - every enterprise is different. Vendors can only do so much when building a product for a broad market.

The vendor has a lot of responsibility as well. The big three vulnerability scanners on the market don’t do a great job of correctly identifying IoT/OT devices. Scan a Ubiquiti device and they’re baffled - they’ll tell you it’s a Linux server running an end-of-life version of Debian. So, of course, there are vendors that specialize in only scanning IoT devices. You could even buy several complementary scanners and still have enormous gaps in your data.

In SecOps, detection engineering is the data challenge. Do we build broad or narrow detections? Are we getting all the necessary data to build the detections? Are there delays and bottlenecks in data collection and querying?

In third party risk management, you’ll never have time to perform deep due diligence and monitoring on all your third parties. Which vendors represent the biggest risks? Are you asking the right questions on your questionnaires? Are the responses accurate and trustworthy?

Everyone wants to make lemonade, because building sensors and gathering data is hard. Many buyers love making lemonade, because they start off with a mess of data and end with a nice dashboard with scores, prioritization, and metrics. When buyers see a vendor turn a million critical vulnerabilities into a ‘top 10 patch ASAP’ list, it feels like progress. Lemonade aims to be tasty, not healthy.

Making Lemonade Doesn’t Address Root Problems

Garbage in, garbage out. It’s a common phrase, but the challenge in cybersecurity is that we don’t have enough folks skilled in determining the quality of our data. Vendors and their data scientists get excited about markets where there’s a lot of data, because they don’t have to go out and create the data. It’s already there and ready to be analyzed, sorted, normalized, reduced, and summarized.

An important point: vendors’ products don’t become lemonade makers until the buyer feeds them lemons. It is largely on the buyer to ensure they’re not feeding bad data into the hopper. For example:

• What if the customer fat-fingered one of their IP ranges? Instead of scanning 10.1.2.0/24, they’re scanning 100.1.2.0/24.

• Perhaps there is also an external class C network the security team is unaware of, so it has never been scanned from the outside.

• Security Rating Services only see a company’s external infrastructure, and often get companies’ assets confused and mixed up.

• If you don’t pay for Salesforce Shield (reportedly 30% of your total Salesforce spend, ouch) and lack logs, you can’t build Salesforce-related detections.

If the data is wrong or missing, there’s no way to magic a win out of it with AI or any other technology.

This is why edge devices get hacked, despite fixes being available for months or years before the attack. Perhaps they weren’t getting scanned. You can’t protect the assets you don’t know about.

This is why attackers are able to drop small Linux VMs on servers and desktops as a base of operations. Detections aren’t looking for WSL.exe in process lists, or new VMDKs showing up in %APPDATA%.

This is why the company that gets breached always has an A+ on some security rating service’s scorecard, and the ones that don’t get breached often have D’s or C’s. The rating services don’t have enough data to make an accurate call, but as long as some data exists, they’ll make lemonade.

Check Your Ingredients

To avoid making lemonade, you’ve got to check the quality of the data you’re giving to these ‘overlay’ cybersecurity products. If you’re planning to buy a product, and step 1 is to ingest data that another one of your products collected, stop and consider:

1. how comprehensive is this data?

2. is the collection of this data taking into account current attack scenarios and TTPs?

3. how accurate is this data (e.g. what are false positive rates, do your analysts trust it?)

4. how would I know what the quality of this data is (i.e. do I need to hire a third party expert to tell me?)

We also have to be careful with metrics. The lemons and lemonade makers in the market are great at generating empty calories - metrics that look and feel like progress, but have no impact on your security program’s desired outcomes. You patched 100,000 vulnerabilities, congrats! Did any of these vulns represent any real risk to the business? Statistically, the answer is probably not. It feels great when that vuln count line goes down though. Lemonade is delicious.

But is your goal to satisfy a sweet tooth, or to get healthy?

You can’t magic good outcomes from bad data.

Conclusion

Dropzone and others are building some impressive scaffolding for automating mundane, repetitive SecOps tasks, but there is a question we have to ask before paying $9 an alert for agentic automation: “am I feeding it trash?”

Defenders need to give more attention to the market niches that focus on validating the quality of our security data. Products and services that help connect theory and best practice to reality.

Only when we’re sure of the quality of our controls and data can we get value out of our products, not lemonade.

1. requires expert knowledge

2. requires special tools, or

3. requires access the buyer doesn’t have

The cybersecurity market takes this to an extreme: sometimes, even the sellers aren’t aware of the quality of their product. Ross Haleliuk said it best in his book, Cyber for Builders.

“The quality of what is bought and sold is not known.”

When evaluating, buying, and using cybersecurity products, it can be difficult to determine how well a product works. It isn’t uncommon to discover security products that weren’t even functional after being deployed. Another challenge is determining how effective the product is.

No one is (or should be) satisfied with simply blocking EICAR, for example. If an anti-virus product is only good at blocking commodity malware but no self-respecting cybercriminal will ever use commodity malware, this product isn’t useful for much more than a compliance checkbox.

FireEye, it had what networks crave

Back in 2012, I was building a security program for a large enterprise. I spent the better part of a month trying to figure out what FireEye’s product did. This was before FireEye went public and their only product was the NX appliance. Every time a salesperson pitched me the product, their description brought me to the same conclusion: “so, this is intrusion detection? This is an IDS/IPS?”

The salesperson quickly pushed back on the description. “Oh no, I was told in no uncertain terms that our product is NOT an IDS or an IPS device.” This was unsurprising, as Gartner had declared IDS dead as far back as 2003 (it is still commonly used today). We’d loop back to the beginning of the conversation and they would explain it to me all over again.

After chatting with a third sales rep at FireEye, it became obvious that their own employees (or at least their sales teams), didn’t seem to understand what the product did.

The fourth time was the charm. I finally got an engineer on the line and was told that the FireEye NX appliance was designed to catch malware on the wire, but only really bad malware. It would ignore the commodity stuff. Long story short, my org bought a heavily discounted NX appliance, despite my recommendation to pass on it.

We pulled it out of the box, racked it, plugged it into a SPAN port (no chance were we going to put it in-line) and powered it on. The first challenge was how to determine whether or not it was working.

How do you test a product that only detects ‘really bad’ malware? What even is ‘really bad’? How does it determine bad from not-so-bad malware? FireEye had a solution: a custom PDF that, like EICAR, was guaranteed to trigger an alert every time. That solved the problem of functional testing. Would it be effective though? Would it actually detect malware?

It did not. FireEye generated roughly one false positive per month. In the same time, we were hit with a few malware infections per month. Malware got past the FireEye appliances by shipping as a JAR file that contained an obfuscated WinPE that it would reassemble on the desktop, after it got past our watchful FireEye’s network surveillance. This was one of many malware delivery evasions that worked over the years to bypass security products.

The product didn’t just lack value, it had negative value. I calculated that the cost of the product itself, plus the time it wasted every time we had to respond to a false positive and investigate it, put FireEye’s value over six digits in the red.

Another problem was that, due to the cost of the appliance, we could only afford to purchase one, and put it in our headquarters, where we had the least problem with malware infections. Most of our issues were at sales offices in the field, that often had no more than 5 employees per office. The whole idea for the device was fundamentally flawed.

This would become even more obvious years later as we saw the company struggle with heavy churn (customers not renewing contracts). I was an industry analyst at this point and referred to this event as FireEye Buyers’ Remorse. This was unsurprising, given my experience with the product, and my theory that both the seller and buyer didn’t seem to understand what the product did. The NSS Labs/FireEye battle a few years later further proved this theory.

The FireEye story is a long-winded way to point out that the Cybersecurity market produces a lot of lemons. The average buyer doesn’t have the security talent in house necessary to simulate a real attack and determine if there’s any value in buying these kinds of products. They can maybe afford to pay for a penetration test once a year, but unless they’re paying for the best in the business, they’re probably going to get someone a few years out of school operating mostly automated tools.

Security products aren’t all about preventing attacks, however. We have security operations products, GRC products, application security products, vulnerability management products, and more. Most suffer from similar information asymmetry issues that hark back to The Market for Lemons.

It is not practical for most companies to properly evaluate and test security products before buying them.

When life gives you lemons

Cybersecurity products also have this nagging tendency to create entirely new problems. A large portion of the market pumps out what I think of as busywork generators. Right out of the box, these products will consume logs, scan for vulnerabilities, detect potential attacks, and identify compliance gaps. Overnight, security analysts can be buried in hundreds of thousands or even millions of tasks.

You’ve probably heard the terms ‘alert fatigue’ or ‘vulnerability overload’. This problem, again, has its roots in information asymmetry. The vendors say, “this stuff is bad”, and many buyers aren’t in a position to refute or challenge it. Meanwhile, security products fail to stop threats, detect attacks, or alert practitioners when products are misconfigured. If you’ve been a practitioner for even a year or two, you likely have examples you can cite. Here are a few of mine:

1. When I first used a SIEM in 2004, I was floored to discover that there was no mechanism to tell me when devices suddenly stopped sending logs. I had to build it myself. Again, in 2012, I found nothing had changed. SIEMs still didn’t perform this basic, fundamental task.

2. I once discovered that a client’s DAST scanner, which was scanning 14 websites by domain, had 13 misspelled domain names (they were missing the ‘m’ on dot com). 13 of the 14 domains it was scanning didn’t exist, and the tool didn’t tell them that. Since 1 of the 14 was getting scanned, they assumed everything was fine.

3. These days, security tools represent significant security risks and attack surface - one out of ten vulnerabilities in CISA’s known exploited vulnerabilities list belong to a security vendor.

Visibility is important in cybersecurity - no security leader wants to be in a situation where they’re blind to an attack, a vulnerability, or a gap in their compliance program. Buyers ask the vendor to show them everything, and their staff drown in the results. I’ve observed an unwillingness to reduce this noise level.

Why? A sort of hoarding or FOMO effect exists with security leaders. Is it more defensible to enable all the alerts and say “we missed it because we don’t have enough people in the SOC”? Or is it more difficult to disable the noisiest alerts and risk missing something?

You can bet the market noticed this problem and was eager to sell a solution.

This market makes lemonade

Initially, the solution was to click the export-to-CSV button and make sense of the data in Excel. However, once vendors noticed buyers struggling, we started to see entire market segments spring up to solve the problems that other security products created.

Pause for a moment to consider a scenario:

1. You bought a vulnerability scanner for $75k. You run your first scan. The results overwhelm your team.

2. You ask for more headcount and hired more folks to address the workload. Maybe this worked for a while, maybe not.

3. Web 2.0 happens. Your company builds fancy webapps and maybe even offers SaaS to customers.

4. You buy a web scanning tool for $35k. You run your first scan. The results overwhelm your team

5. The cloud is invented and your company adopts it, but doesn’t get rid of the existing datacenter.

6. You buy a cloud scanning tool for $50k. You run your first scan. The results overwhelm your team.

7. By the way, you’re not sure if ANY of these tools have gaps. You’re not sure if they’re finding all the critical vulnerabilities, because testing security tool efficacy is hard. Also, you don’t have time, because you’re too busy chasing all the busywork these tools are generating.

8. A product category emerges and offers an enticing pitch: what if we took all those scans, combined them, and prioritized the findings, making your staff’s job easier? This product costs $125k.

Now we’re making lemonade. The buyer isn’t even sure the scanners are producing value but are buying another layer of products to fix the problems created by the first layer. I want to be clear - the lemonade makers aren’t the lemons, they’re just spotted a market opportunity: “hey, that’s a lot of lemons you’ve got there. You might as well make some lemonade, right?

Some examples of lemonade makers:

• The risk-based vulnerability management (RBVM) market, which is the example from the scenario above. These tools don’t include a vulnerability scanner, so you’ve got to purchase them in addition to vulnerability scanning tools.

• Security analytics, SOAR, UBEA, and others were add-ons to the classic SIEM, offering to do what the original SIEM vendors promised 20 years ago: correlate data, extract insights.

• But wait, why did we need a SIEM in the first place? Probably because we didn’t want to have to log into a dozen different security products to check for alerts, correlating timestamps across devices manually. We’re potentially three or more levels deep on this one, especially if you have a data lake, ETL products, an MSSP/MDR to handle SOC monitoring in the off-hours…

• Products like FireEye’s NX (’Breach Detection Systems’) positioned themselves as complementary to IDS and IPS devices as well as endpoint security products.

• You can buy a license for an expensive TPRM spreadsheet and then an AI-powered GRC product to automate filling it out.

• Interestingly, there’s some analysis out from At-Bay showing that you’re more likely to file an email-incident-related cyber insurance claim if you have an email security appliance vs just using the security built into your email platform. Dig a little deeper and you’ll find that At-Bay is also getting into the lemonade business.

• I’ll stop now, but feel free to mention more in the comments, or challenge any of the examples I’ve come up with here.

Hope you’re thirsty

The problems created by core security products are almost always very obvious process issues (e.g. too much data to process, analyze, or action). This is great for second tier products, because it’s very easy for them to demonstrate and measure value in terms of time saved versus the old product. The problem is that the buyer doesn’t know if the core products were doing a good job to begin with.

“Wait, I didn’t even want lemons in the first place, did I?”

I can’t fault these second-tier lemonade vendors, because they’re responding to real issues that customers have. It can be an uphill battle for them as well - it’s not easy for buyers to hear that they need to buy an additional product to make their existing tools work properly. Buyers will often defer the decision until the problem becomes super painful.

Why doesn’t this secondary market simply replace the core vendor and solve the core problems, then?

There might not be a universal reason for this, but I have a few thoughts:

1. rebuilding the core product is difficult and expensive (e.g. building 200,000 vulnerability checks from scratch)

2. the solution to the core problem isn’t obvious

3. the industry’s collective sunk cost is too great: the solution requires a completely different approach, requiring the vendor to reeducate the market and go against common practices, certifications, standards, and regulations

The final theory is a really tough one. It has been observed that security leaders are swayed towards the choices that are safest for their personal careers. When the choice is going against the grain versus doing the safe thing, the data tells us the latter is going to be the more common path.

Conclusion

There’s no Ozempic for cyber yet, so we’re faced with a familiar dilemma. We can keep downing lemonade, getting nowhere with our goals, or start doing the hard (diet and exercise) work necessary to determine what works and ditch what doesn’t. The latter is tough, as there are little to no incentives to challenge the status quo, even when it is clearly broken.

In consumer markets, information asymmetry tends to solve itself when value is obvious. If a particular model or brand of laptop breaks a lot, it will have lots of bad reviews online. A laptop breaking is an outcome that can’t be ignored.

Cybersecurity isn’t like this. It is more comparable to the health supplement market. Do I feel amazing today because I got an extra hour of sleep, or because I started making drinks with this green powder? Most consumers don’t have the time or patience to scientifically test every new nutrition product or trend they try out, so if they think it makes them feel better, they’ll keep buying it or doing it.

At least with nutritional supplements, there will be folks that will put them to the test, or send products to a lab to determine if they’re even safe to consume. Cybersecurity doesn’t really have an equivalent, unfortunately.

This is a critical problem in cybersecurity. When we don’t know how well our tools or controls are working, we’re at a distinct disadvantage as defenders. As Charlie Miller once put it:

Buyers should be pushing back on products that create more problems than they solve. However, without a clear path to better products, practitioners continue on, buying what their peers bought that didn’t get them fired. Ironically, a big part of the problem is a lack of risk appetite for better security products.

We see it over and over and over again with breaches:

1. attackers got in via RDP (Atlanta, Ryuk)

2. attackers got in via VPN (Pulse Secure, SonicWall/Akira)

3. FTP credentials were guessed or brute-forced (SamSam, General Scanning & Attacks)

4. File transfer products exploited (Accelion FTA, MoveIT, GoAnywhere MFT)

5. Firewalls exploited via their management interfaces (Juniper ScreenOS backdoor, PanOS vulns, FortiGate 0days)

Each of these examples represent old ways of accessing services. For each, there are now better, more secure ways of performing each of these functions, without exposing these services to the public Internet. These are also some of the most popular attack targets for attackers!

That’s why I think of this as the asbestos of IT.

Asbestos was also very useful, but dangerous to humans. As soon as we realized this, we labeled it as a dangerous substance and started replacing it with safer materials. The time has come to replace outdated, dangerous protocols with more secure alternatives.

The key to replacing each of these services is to find alternatives that don’t require exposing TCP/UDP ports to the public Internet.

1. Replacing RDP: There are a TON of options here. You likely have something built into whatever tool you use for managing your mobile devices or servers. In liu of that, I personally like RustDesk, because it doesn’t require you to trust a third party like AnyDesk and TeamViewer do - you can set up and manage your own server. It does still use a direct TCP connection, however, so you’ll need a modern VPN technology, which brings me to…

2. Replacing VPN: Whether you call it ZTNA or SDP, the big innovation here was allowing access without opening ports. I use Tailscale, which is how I get to RustDesk on my hosts (I don’t bother with the RustDesk server, I just connect direct client to host). I recently tested RustDesk over Tailscale, on an iPad, from Delta’s in-flight Wi-Fi, and it was like I was sitting right in front of my studio PC. I was very impressed.

3. Replacing FTP: There are so many options for file sharing or file transfer that we’re spoiled for choice. The replacement depends on your use case. Publishing files to a web server? Use GitHub or other code deployment tools. Business to business transfers? S3 bucket (or GCP/Azure equivalents). Consider something like ShareFile, Dropbox, Box, Google Drive, OneDrive for human to human file sharing.

4. Replacing old-school file transfer products: See #3 above.

5. Firewalls getting exploited via management interfaces: Don’t ever share management consoles on Internet-exposed interfaces! See #2 above

As always, some caveats: yes, I understand your vendor requires you to use FTP, there’s not much you can do about that, except to fire your vendor, or ensure there’s nothing too sensitive being transferred.

There are other reasons you might not be able to get off legacy protocols. Document them, put mitigations and detections around them as best you can, and be ready to respond to any incidents that come from using them.

If you can ditch legacy protocols, however, the reduction in attack surface will be worth it and you can sleep that much better at night.

It has been a few years since I discussed this and the last time I did, it was a bit buried at the end of the post. I thought I’d share an approach to detection engineering I came up with a long time ago that might help. I’m also a bit baffled that the industry seems overly focused on addressing alert fatigue from the back end (using AI to triage, enrich, and investigate) rather than on the front end (avoid generating so many alerts in the first place).

The Red Flags vs Yellow Flags approach

Red flags vs yellow flags is a simple system I came up with to focus when building out detections. You really only need one red flag to know an attacker is present. After that, diving into the details becomes easier. By focusing on a smaller number of higher quality detections, alert fatigue is alleviated, response times improve, and coverage improves.

I’ll repeat the most important bit here: you don’t need to alert on everything an attacker does. You’re still logging everything the attacker does, you’re just not filling up alert queues with every action they take.

I came up with this approach, naturally, after initially doing everything the wrong way. I ingested every event from every source into the SIEM and enabled every alert and detection. We were instantly buried in noise. I started to think, what if I built this in reverse from what I’ve just done? What would that look like?

1. Study how attacks happen and succeed. What do attackers have to do? What are TTPs we see them using over and over?

2. How few of these actions can we get away with looking for and still ensure we’re catching attackers every time? Which actions are the easiest to detect reliably, with a very low false positive rate?

3. What data do I need to collect to detect these things happening?

4. Start building detections for these things.

5. Test to see that my detections work

6. Make sure we have SOPs or playbooks for these detections (containment and/or eradication actions: disable accounts, kill sessions, isolate machines, etc)

7. Do the thing: test to see that my SOC folks notice the detections when they fire and handle them according to plan

Defining Red vs Yellow

Step 2 is where we’re separating yellow flags from red flags. Red flags are alerts that are always bad. They’re relatively easy to detect with a near zero false positive rate. Mimikatz is always a red flag. A binary downloaded from a domain that has only existed for 3 days is almost always bad. A cryptominer on an EC2 instance is a bad sign. A powershell terminal executed by a Word document. You get the idea.

This approach doesn’t mean we totally ignore yellow flags, just that we ensure we don’t miss any red flags due to noise from the yellows. A yellow flag is a lower quality tier of detection - something that is suspicious, given the right criteria. They’re maybes. Maybe this is a red flag, if it’s 3AM and comes from the receptionist’s PC and talks out to a strange host. Most orgs are drowning in maybes.

A login event at an odd time might be suspicious, but it won’t always be a red flag. Or perhaps, it’s a red flag, but only for three employees. Impossible travel alerts are an example of this. Impossible travel once seemed like a high quality detection, but sometimes proxies, VPNs, or SaaS products reduce the reliability of these alerts. Several yellow flags could amount to a red flag: a compound detection. Say we see a login from an odd location, followed by a very broad query in a database, and an attempt to bulk download the results. Feels like a smash and grab, perhaps by a ransomware crew.

When have we gone too far?

As you can see, the more we go down this path of adding context and complexity to detections, the chances for false positives go up. The most difficult part of this approach is resisting overbuilding detections. Fear of missing out on events can lead to alert hoarding, which can lead to alert blindness. Trying to see too much results in blindness.

Even one reliable, well-tested red flags detection is a huge win - I’ve worked in so many environments where even the noisiest penetration test doesn’t set of a single alert. Limiting the number of detections doesn’t mean that you’re not logging all the other actions performed by the attacker, you’re just limiting alerts to what is manageable for your organization and ensuring they work.

Herein lies the most difficult part of this approach: being comfortable with the quiet. Inevitably, folks ask, “but Adrian, I might miss hundreds of advanced and sophisticated attacks!” The reality is that even the most sophisticated actors still use some very basic tools and techniques (because they work), and you’re not ready to start building or enabling detections for these kinds of attacks if you don’t have the more common ones working. Resist the urge! Seriously, get good at detecting and responding to just ONE common type of attack. Get really, really good at it, to the point where you never miss it, and you’ll understand where the bar needs to be set for the rest of your detections.

The advantage of this approach, is that it should result in a relatively quiet SOC if nothing is happening. This creates time and opportunities for threat hunting and further finding red flags and tuning detections to find them.

A less noisy approach won’t need to lean so heavily on AI, if at all. I personally don’t think generative AI can scale to the needs of large SOCs, but that’s a discussion for a different post.

Bonus Round: Deception

Modern honeypots and honeytokens are a great way to create red flags from scratch. By definition, they shouldn’t be accessed or used by legitimate employees, so they’re a great way to artificially create red flags. Create a fake admin account that doesn’t belong to any admins and seed its credentials in key places. Any use of this account is a red flag. As always, canarytokens.org is the best place to start exploring and testing this approach. This is also the quickest way to understand what a great, red flag detection looks and feels like.

Just 10 days after my last post about three NPM incidents, we’ve got another one. Kudos to Koi for spotting and writing up this one a few days ago.

This looks like a classic case of what I’m going to call package squatting: someone takes a legitimate package and begins maintaining a malicious version of it. Typically, these don’t stay up for long. They’re quickly spotted, we might see a few media posts about them, and then a week or two goes by until we see some more. Since the legitimate package also exists, the package has to be uploaded to NPM (or PyPi, or GitHub, etc) with a slightly different name, which is why we usually call this type of attack typosquatting.

For example, someone searches for the 1Password Chrome extension in the Chrome Web Store, but they only type 1Passbefore searching. Say someone has uploaded an extension called ‘1Passwerd’, so two results match the search. Some percentage of folks might install the malicious version before it gets reported and pulled down, and perhaps the attacker steals some browsing data and credentials in the day or two before discovery.

This, however, was NOT a case of typosquatting.

What is Postmark?

Postmark is a product from a commercial company called ActiveCampaign. It is an email delivery service that competes against the likes of SendGrid, Mailgun, and Amazon’s SES service. They appear to market to sales and marketing teams.

Postmark began offering an MCP server recently - a banner on their homepage encourages visitors to check it out. The official version of this MCP server is only available from ActiveCampaign’s GitHub account. That’s why this isn’t a case of typosquatting.

The malicious party appears to have grabbed a copy of the Postmark MCP code from GitHub and started maintaining an identical copy of it over on the NPM registry (now goes to a 404 since it has been taken down). The malicious party also appears to have maintained an identical copy of this package on NPM for some time, until version 1.0.16, when they introduced a backdoor. The package made it to version 1.0.18 before the backdoor was noticed.

What was the backdoor?

It was a single line that BCC’ed every email sent with postmark-mcp to an address we presume belongs to this malicious party.

Bcc: ‘phan@giftshop.club’

Yup, that’s it.

Why?

I’ve been using the term ‘malicious party’ because there really isn’t a good reason to do this without warning folks using this package. Particularly since this actor doesn’t seem to have any connection to ActiveCampaign, who has posted a warning about this malicious version.

Since this tool was targeting sales and marketing teams, there’s a chance the malicious party didn’t get anything beyond a bunch of corporate sales copy and newsletters, but it is entirely possible that someone might have been using the postmark-mcp package to route their personal email.

What can we learn from this?

This is a new-ish take on an old attack - we’ve been seeing it for years with domains. If you have a popular domain and don’t own every TLD for it, there’s always a chance someone will grab CreditCardCompany.xyz mirror CreditCardCompany.com to it and start phishing customers with it.

ActiveCampaign could have reduced the chance of this happening by registering the NPM package themselves, before anyone else could. This appears to be a beta/test version of their product however, so I wouldn’t have expected them to aggressively register this package name across every software registry. One recommendation that I do think is reasonable, would be for ActiveCampaign to actively monitor for malicious versions of their software to help keep their customers safe. There are brand protection tools and services that can help with this as well.

As for consumers of the software, attacks like these are tough to deal with. Most enterprises consume thousands of open source packages. Checking each release of every package for hijinks is difficult, particularly when it isn’t clear what ‘hijinks’ might look like in code form. Which of us would spot a line starting with ‘BCC’ and raise a red flag? What would be reasonable to expect of an enterprise?

• Checking for odd or anomalous behavior: I’m not sure if any existing tools or detections would have spotted this, but inserting mail rules with untrusted BCC addresses is a well-known behavior in email systems. It stands to reason that we should look for this trick anywhere email might be handled.

• Using AI to review changes could be another potential option for spotting anomalous or strange changes to code.

For individuals, it’s hardly unreasonable to expect every curious MCP explorer to check every line of every update. Individual consumers of apps, extensions, packages, and code depend largely on the organizations that host them to keep malicious content out. NPM hasn’t been terribly successful at keeping the bad stuff out lately, however.

Anything else I’ve missed? What else could we do to detect and prevent this kind of attack?

But EDR is not AV.

What’s the difference?

EDR is not AV. Nor did EDR replace AV. They are very different technologies that serve different use cases. Endpoint Detection and Response gathers endpoint telemetry: file, process, and network activity. This data is fed to an event monitoring (e.g. XDR or SIEM) system. It is then manually (SOC analysts, threat hunting) or automatically (SOAR playbooks, AI) acted upon, after something bad has happened. EDR can take some automated actions, but the options are typically very limited (more on that in a few paragraphs).

Unlike EDR, Antivirus (aka NGAV, aka anti-malware, aka etc) attempts to prevent malware and other attacks from infecting or impacting systems. It aims to do this job without human intervention, at machine speed. Adding to the acronym salad, EPP (endpoint protection platform) is antivirus plus other adjacent endpoint security technologies, like device control, disk encryption, host firewall/IPS, application control, and many other potential features that vary from vendor to vendor.

In NIST Cybersecurity Framework terms, AV belongs in the “protect” column, while EDR mostly sits in the detect column, with some rudimentary response capabilities (isolate machine from network, etc) that technically put it in the ‘response’ column as well. Protect is left of boom - it aims to prevent the boom. ‘Boom’ is an undesirable event that security teams want to prevent: denial of service, exploitation, malware infection - some compromise or loss of control. EDR is right of boom, it collects information after the bad thing has happened, giving security operations and incident response teams a starting point for detecting, containing and eradicating the threat.

Why are they separate?

There are many scenarios where AV may be necessary or required, but where systems can’t handle running AV and EDR. For example, I don’t expect to find 15 year old point-of-sale hardware running Crowdstrike’s full enterprise NGAV+EDR enterprise Falcon client. Or perhaps the environment is so locked down that EDR is deemed unnecessary. I could also think of situations where AV is unnecessary (either because it’s already built in, or the system is so locked down, it can’t function), but you do want to collect forensic data from the device.

Need a specific example? Look at one of the most popular endpoint security products on the market today, Crowdstrike Falcon. A quick glance at their website shows that the first two Falcon tiers do not include EDR. You only get EDR when you go up to the Enterprise tier.

Today, AV and EDR are still separate codebases, executables, and products. Some vendors sell AV, but no EDR. Some sell EDR, but no AV. If we use the terms interchangeably, there is a very real chance that someone assumes they’re buying both, and ends up very disappointed1.

Optional History Lesson: NGAV

EDR and NGAV were born around the same time. The wave of endpoint security companies in the early 2010s that included Cybereason, Cylance, Crowdstrike, Carbon Black, Endgame, and others, zeroed in on one of two technologies:

1. a replacement for failing legacy AV approaches (”next-gen” AV)

2. real-time forensic collection of endpoint data (EDR)

Nobody did both initially, they chose one or the other. Over time, as they grew, most of the pure play endpoint security startups started offering both. I was lucky enough to be an analyst at 451 Research at the time, tracking over 100 endpoint security companies and talking to dozens of them every month to understand what was happening in this market. For the NGAV companies, the roadmap was clear:

1. prove out NGAV technology, and sell it alongside legacy AV (the complementary approach)

2. build out EPP functionality, so that NGAV can eventually displace legacy AV

3. build out EDR functionality (or if you were one of the companies that started with EDR, you needed to build NGAV, and then EPP)

4. Conquer and profit

The early days were interesting. It was obvious that the end goal was to topple the incumbents and begin competing with them, but none of these disruptors could admit to this out loud. They had to play nice for years, claiming to ‘complement’ legacy AV/EPP vendors. I dubbed it the curse of complementing. Endpoint disruption created a lot more than just NGAV and EDR - the market was much more fragmented in 2015.

You probably already know how this story played out. Symantec and McAfee liked to portray the image of highly diversified security platforms, but the reality was that their revenue was heavily dependent on the endpoint security market. The disruption happened quickly. I vividly recall Symantec’s SEP14 (Symantec Endpoint Protection, version 14) release. It was their attempt to catch up with the market, throwing all the new technologies that threatened to disrupt them into one major release. These new technologies were hastily bolted on, disabled by default, and buried deep in configuration screens. It was too little, too late.

While trying to catch up with the endpoint security competition, Symantec acquired Blue Coat (network security products), and Blue Coat’s exec team took over Symantec. Then Symantec also acquired LifeLock. To me, it seemed like they were trying to convince themselves that building a bigger suite of products would make their endpoint issues less critical to fix.

It did not. Rumor had it that they were bleeding out millions in revenue every quarter and the churn on renewals for endpoint products was particularly bad2. Almost exactly three years after Blue Coat’s execs took over, Symantec’s enterprise offerings were split out and sold off to Broadcom.

The impact to Symantec and McAfee was quick, decisive, and permanent. Before long, EDR and EPP were getting bundled together, and I believe this is when folks started conflating the terms, using “EDR” to refer to both.

Optional History Lesson: EDR

Why EDR though? What was driving it?

EDR was part of a shift in the industry, as we spotted some major gaps when it came to detecting attacks. When prevention failed, we lacked the forensic detail to figure out why it failed, how the malware got in, and what the malware/attacker did. The first EDR tools came out of shops with heavy incident response services3.

Shops like Mandiant had EDR-like tools, but you’d only run them once, after the attack was over and infected systems had been isolated. What if we had these tools running all the time? The IR folks wouldn’t be flying blind when investigating an attack and would be able to review telemetry from the time the bad thing happened.

In Conclusion: Why should you care?

You should care about the distinction between AV and EDR, because it will matter when your security program is placed under the post-breach microscope. Your cyber insurance provider will care. The FTC will care when writing up their complaint. The lawyers will care when assembling class-action lawsuits. Your customers and partners will care.

Why supply chain?

Attackers started targeting the software supply chain years ago, but things have really started ramping up lately. We see two goals related to these broad supply chain attacks: stealing cryptocurrency and stealing credentials. It is likely no coincidence that this is exactly what infostealer malware aims to do as well, and we’ve seen a distinct rise in infostealer activity over the same period.

From the attacker’s perspective, the logic is simple:

1. Attacks leveraging stolen creds have a high success rate

2. What are the best ways to steal creds?

3. Who has the creds we want to steal?

Developers and engineers have access to credentials that have access to critical resources, so attackers are targeting software, operating systems, and ecosystems where engineers and devs live and get work done: GitHub, MacOS, plugin libraries, and software repos.

Stealing cryptocurrency is a bonus for attackers these days, while I imagine the primary goal is stealing creds that could lead to more crypto, or a ransomware attack opportunity.

RIP Software Supply Chain

You’ve probably seen the XKCD comic musing on the fragility of the software supply chain. Despite multiple tech companies worth north of a trillion dollars US, independent and largely unsupported developers are responsible for some of the most critical software that our devices, the Internet, and economy depend on.

When this XKCD comic was originally created, we were worried about availability - what happens if this Nebraskan stops maintaining this project, or pulls a ‘left pad’ and deletes their repo? These days, this comic applies from a threat perspective as well - the lone maintainer as an attack target. Defeat the security of this one individual and attackers could gain access to the millions of systems and organizations trusting the software they’re responsible for.

That’s a LOT of pressure for someone probably not even being compensated for maintaining the software they’re targeted for (though there is more financial and operational support than at any point in the past for folks like this).

The scale of this problem works in attackers’ favor. They don’t need all of these developers to fall for a phishing email, just a few. They don’t need all software consumers to fall for a typo-squatting attack, just a few.

NPM is getting pummeled

We’ve now seen 3 major NPM attacks over the last few weeks. NPM is the most popular javascript package manager, owned by GitHub, which itself is owned by Microsoft. Why NPM? Javascript is the most popular programming language and NPM is the most popular package manager for it, making it an ideal choice for attackers to target.

The Nx s1ngularity attack

August 26, 2025

Nx NPM compromise that leveraged developers’ local AI agents to do infostealing (grabbing credentials and cryptocurrency). This was dubbed the s1ngularity NPM compromise, after the name given to repos used to steal the data. Malware was embedded in post-install scripts, which is very common - rarely do attackers add the malware to the package code itself.

The attack stole secrets and cryptocurrency from compromised systems and created a public repo within the victim’s account to exfiltrate the spoils. It also made private repos belonging to the victim public.

Initial point of compromise: a vulnerability in an Nx workflow that allowed attackers to perform command injection via pull request titles.

Novel Attack TTPs

• Leveraged CLI AI agents (Claude Code, Amazon Q, Gemini), to locate and steal sensitive data. Basically, the victim’s AI agents became willing accomplices, helping the attackers achieve their goals.

• Appended sudo shutdown -h -0 to login scripts, aiming to frustrate manual remediation attempts

Attacker Motives: Steal credentials, cryptocurrency, and source code.

Control Failures

• Custom GitHub Actions workflow failed to validate input from untrusted sources

Outcomes

• Over 1000 valid GitHub tokens compromised, dozens of cloud credentials and npm tokens, ~20k files leaked

• Malware ran on an unknown number of dev/engineer systems, often via the Nx VSCode extension

• Malware also executed within the context of build pipelines, like GitHub Actions

The npmjs.help phishing attack

September 8, 2025

Just a week after the Nx attack, owners of high-profile npm packages were targeted in a phishing campaign designed to steal credentials and defeat MFA. It was a typical high-pressure “do this thing or you’ll lose access to your account” scam. The campaign targeted the owners of extremely popular packages with millions of weekly downloads. Two package owners and 20 packages appeared to be affected.

Notably, this attack was spotted and shut down very quickly. Aside from creating a lot of extra work for npm maintainers, GitHub, and npm consumers, the only impact was a few hundred dollars worth of crypto stolen. The malware was designed to inject itself into the victim’s browser and intercept cryptocurrency transactions. Unlike other NPM compromises, the malicious code was appended to the package software itself rather than install scripts, likely because it needed to get loaded within the context of the web browser to steal crypto.

Initial Point of Compromise: Josh Junon and potentially other package maintainers. Josh Junon (who hilariously chose the social media handle bad-at-computer) was open about the fact that he fell for the phish and shared a ton of details about the attack that helped the industry quickly respond and shut the attacks down. The phish was unremarkable aside from the fact that it was well constructed (screenshot below) and used a very legit-looking domain, npmjs dot help. Reportedly, there was even a functional TOTP generator that victims were tricked into using.

Attacker Motives: Steal cryptocurrency - wallet drainers.

Control Failures

• Phishing scam (human training)

• the npmjs.help domain was only 3 days old - avoiding newly coined domains is a tried and true method for avoiding scams and attacks

Outcomes: $600 in crypto stolen and a lot of inconvenience.

The Shai Hulud npm compromise

September 16, 2025

Also named for the names attackers chose for the repositories that contained stolen artifacts, this incident appears to be linked to the Nx incident from a few weeks back. Many of the TTPs and IoCs are the same, leading many to believe it to be the same threat actor. Notably, the use of https://webhook.site links the two.

Like the Nx attack, the malware was in a post install script, targeted credentials, migrates private repos to public, uses webhook.site and victim-owned repos to exfiltrate secrets. The post install script used GitHub Action workflow scripts to execute the attacks.

Initial Point of Compromise: we don’t yet know, though the compromise of ctrl’s package tinycolor was a major inflection point.

Novel Attack TTPs: the malware in the post install script includes worm functionality, automatically replicating itself to other packages the victim has control over. This behavior led to a much higher number of packages being compromised - perhaps 500 or more.

Outcomes: At least 477 npm packages and 36 GitHub accounts were affected. At least 8 private repos were forced to public. These numbers will likely trend much higher once we have more details.

Conclusions

These are tough issues to solve. Devs/engineers aren’t fond of additional friction in CI/CD pipelines, but on the other hand, if the entire software industry depends on your package, you should probably add some additional layers of security and sanity checks, especially when updating production packages.

There are commercial security products that add wrappers around npm, GitHub Actions, and other parts of the CI/CD pipeline, looking for malicious code and other common attack TTPs. Best of all, these products are free to use for individual open source maintainers. I don’t need to name them or give out free advertising - search for the blog posts on these compromises and you’ll find them easily enough.

While there were no disastrous outcomes that I’m aware of from these three incidents, it is important to keep in mind that it could be months or years before we hear the full story. It is entirely possible that some of these credential thefts led to large-scale ransomware attacks that have been kept quiet, or just haven’t been linked to these incidents yet.

Context

VulnCheck is a vulnerability/exploit intelligence vendor that publishes a lot of really interesting research and have a lot of useful free tools. They have their own expanded version of CISA’s KEV, which has a free mailing list. Lots of free tools as well. All-around a good resource.

Patrick Garrity specializes in interesting research and spicy takes on LinkedIn, which sometimes challenges the takes of other vuln-focused firms. I have friends on both sides of these takes, so while I try to stay neutral the best I can, I honestly don’t mind the drama, as long as it results in better quality intelligence that can help defenders.

Steel sharpens steel or something? Whatever, let’s dive into the report.

Report Details

Patrick has shared the report in a LinkedIn post, which is a great opportunity to read it before it gets put behind a reg wall. I don’t know if this means this LinkedIn post will get removed later, or if this is just a portion of the full report - I’ll try to come back to this post and update it if things change.

The report is only 9 pages with a lot of graphics, so this is a quick read. Moreso if you’re not interested in exploitation discovery sources (page 4), threat actor attribution (pages 5-7) , or industry performance (page 8).

Insights

VulnCheck identified 432 CVEs with evidence of exploitation in the wild for the first time

• KEV lists are useful for prioritization. CISA KEV and VulnCheck KEV are useful tools for vuln teams to prioritize their work. 432 sounds like a big number, but consider that every organization doesn’t have every vendor on the list here, so the vulns they’ll have to worry about is likely to be a small subset of this number, probably less than 100.

• KEV lists are late indicators. By definition, a vuln doesn’t make a KEV list until it’s already being exploited. Don’t get me wrong, late is better than nothing, but if you’re waiting for a vuln to get on a KEV list before remediating it, you might be waiting far too long.

• KEV lists sometimes beat CVE publication? This is problematic, given that some vulnerability management programs depend on CVE publication before the wheels of remediation even start turning. I need to put out a dedicated post on this to explain further, but modern vuln mgmt teams need to implement intel-driven vuln mgmt processes in addition to their scan-driven processes. A scan-driven process is much less likely to get vulns fixed before exploitation occurs. In addition, it is becoming clear that intel-driven processes should trigger at the point the vulnerability is disclosed, which could be pre-CVE, pre-CVSS, and pre-KEV, so this process can’t afford to depend on any of these things.

• Lots of good insights about the speed of exploitation. For example, 32.1% of KEVs had exploitation evidence on or before the day the CVE was issued, increasing from 23.6% in 2024. This number is high enough to again justify the intel-driven approach described in my last point above.

• Open source software is statistically less targeted than commercial software

• Vulns in network edge devices and CMS plugins dominate the KEV list, which is no change from the norm - there are two distinct places we see exploit activity clustered: for an initial foothold (Internet-exposed services with RCE exploits) and internal systems during lateral movement (internal unsegmented networks with lagging patches, vulnerable to RCE exploits).

• A note in the increase of IoT device vulns, which makes sense, given the continued interest in building IoT-based botnets for DDoS attack rentals, like the Oregon resident that was recently charged for building and renting out Rapper Bot - a botnet similar in construction and destructive power to Mirai.

• There’s threat actor detail, though I think this is less interesting to defenders, as the goal is not to get popped regardless of who is behind the exploit activity. Note to self: Why doesn’t the US ever show up on these threat actor reports? Investigate this for a future report, Adrian.

Gripes

• The separation between zero day vuln exploitation and exploitation of older vulns is a bit fuzzy for me throughout this report. I think it could have been made more clear. In some places, like the excellent ‘How quickly are Vulnerabilities Being Exploited’ chart, it is quite clear.

• What does “exploitation in the wild” look like? Does that mean that the exploit was attempted? Did it succeed? Did it result in a breach, or damages? These are questions I need to dive deeper into in a dedicated post. In the past, I managed to get a CVE to show up as “exploited” on a threat intel provider’s dashboard simply by sending a tweet with the CVE number in the text of the tweet. I worry about the quality of exploit evidence sources when providers say they have a large number of them. On the other hand, when you’re driving the number of vulns you have to worry about to just a few hundred, just assume you could get hit by any of these and forget about my nitpicky concerns for now ;)

• One point left me a bit confused: 147 of 181 unique CVEs that were used by known threat actors had evidence of exploitation prior to 2025, demonstrating that threat actor exploitation disclosure often lags behind disclosure of initial exploitation evidence. Maybe this is saying that attributing exploit activity to threat actors is hard and takes time?

Shared ChatGPT chats are showing up in search engines, with no option to disallow search engine indexing. This is the equivalent of "anyone with the link" sharing settings in Google Docs/Sheets or Microsoft 365. No other service I’m aware of defaults to allowing data shared in this way to be indexed by search engines, so this news likely comes as a surprise to most ChatGPT customers.

TL;DR - how to quickly unshare your chats

1. Go to settings in ChatGPT

2. Go to Data Controls

3. Click the Manage button next to Shared Links

4. Delete away! (you can quickly delete all by using the 3-dot button in the upper right hand corner)

NOTE: For context, I have a ChatGPT Plus license, so my results here may not reflect Enterprise, Free, Edu, or other account types.

Why are indexable chat results a problem?

Historically, when you create a shared link in SaaS apps, search engine indexing is disabled. Over the past 10+ years, this has create a basic expectation of privacy when using these features. ChatGPT sessions can contain some very sensitive information, particularly now that it can be integrated with Dropbox, Google Drive, and other services that contain private/proprietary information.

Tech companies must manage user expectations and that means taking into account the current state of SaaS UI/UX, as that informs our expectations as consumers.

How are people finding these chats?

A basic site parameter search will find shared ChatGPT chats in most search engines. As of publishing, Google has stripped them out from their search results, but other search engines, like DuckDuckGo, still include them in search results.

How do shared links typically work?

Despite the fact that anyone can see the data you've shared if they have the link, you can’t find shared Google Sheets or Google Docs in any search engine. Even Notion, which is often used as a quick/cheap way to stand up a website, defaults “Search Engine Indexing” to OFF.

The sharing function in most SaaS services create a link with a long, random value. This random value ensures the URL is not easily guessable and cannot be brute-forced. ChatGPT is no exception here and uses the typical structure of:

https:// chatgpt com/ share/ 688cef07-8cac-8004-a374-b87cfd6bdea4 

(yes, this is a working link if you put it all together - it's just me researching used iPads, go nuts with it)

Google Drive does something similar, for comparison:

https:// docs google com/spreadsheets/d/ 1jptJ-dQvhVZBKWVkqOthgEd9suO-PRkCr1edSkD2K_o/

This is typically a great balance of privacy vs accessibility, as clicking a link is as easy as it gets, and there is no simple path to discover the shared data without the link. This has worked well for other SaaS apps for over a decade.

How Does Sharing Work in ChatGPT?

1. Clicking the “Share” option in any chat will generate a link you can share.

2. Clicking it again will allow you to update the link.

3. There’s no easy way to tell if a chat is shared or not when looking at the chat.

4. There are no sharing options - it is simply public and indexable or not.

5. Archiving the chat does not change the sharing status.

6. Deleting the chat immediately removes the share, and results in the sharing link going to a 404

Let’s get rid of those unwanted shares!

As mentioned above, the process is pretty straightforward if you know where to look.

1. Go to settings in ChatGPT

2. Go to Data Controls

3. Click the Manage button next to Shared Links

4. Delete away! (you can quickly delete all by using the 3-dot button in the upper right hand corner)

Broken Processes and AI Opportunities

We’re still going to use ChatGPT and we still need a way to share results. Unfortunately, there isn’t a built-in export to PDF/markdown/etc option, but it still isn’t too inconvenient. I recommend using the copy-to-clipboard option, and then paste to Slack, Whatsapp, Signal, Google Doc, Email, or wherever else you might want to share your chat results.

The massive downside here is that this approach will only copy a single response in the chat session, not the whole session. Once you get to 3+ prompts and responses, this approach isn’t ideal. I don’t have a good solution here - even the ancient approach of print-to-pdf fails to grab anything past the fold. Your experience may differ on other platforms. I’m using ChatGPT via the Arc Browser on Windows.

Could you copy and paste results output into a tool like Notion or a Google Doc, and use the better sharing features within those tools? Sure, but that’s getting REALLY inconvenient, especially when you consider that generative AI is now built into nearly all of these other tools, so why not skip ChatGPT and just generate content directly within these other applications?

I don’t often share ChatGPT results, and when I do, they’re a lot like this iPad research example - nothing I’d consider sensitive. For others, this could be a deal killer for their GenAI workflows and could hurt some of OpenAI’s business here. Tech companies can’t increasingly advertise AI as a productivity improvement and a time-saver and neglect UX and workflow. Most consumers won’t gravitate towards the best model, they’ll simply use whatever is built-in and most convenient.

Thanks to Alexandre Sieira for bringing this to my attention, along with for sounding the alarm on LinkedIn.

One of those plans is to resurrect a place for me to blog! I’ve got SO much to say and share, but haven’t had a good outlet since retired The Cyber Why. I have big plans for this Substack and some of it is going to require some experimentation. It should be a lot of fun, and I’m looking forward to getting feedback as I try out some of my ideas.

At its core, The Defenders Initiative is all about understanding how cybersecurity failures happen, and how security programs can improve. I spend a lot of time looking for details about breaches and lessons we can learn from these companies’ failures.

I’ve also been doing a lot of research on vulnerability management and have a lot of strong opinions on the cybersecurity market, so expect to see a lot of content related to these topics as well. Alright, with that, you should start seeing some posts this week!