22 October, 2026
There has been a lot of debate recently about using AI to augment or even completely replace manual threat modeling. A recent case study reported that manual threat modeling on a system took roughly sixty hours over two weeks, while AI-assisted analysis of a comparable scope reduced that to about twenty-two hours (Paidy, 2023). Findings like this motivated us to conduct a controlled experiment.
Looking at threat modeling from a slightly different perspective, we also note that the core values of the process as defined by the Threat Modeling Manifesto are focused on people and collaboration. Brainstorming together to find threats allows the team to align and create a shared understanding of risks, architecture, and design. When we use AI to completely automate threat modeling, these values are no longer achievable.
However, the question is no longer whether teams use AI, but who is in the driver’s seat. Our experiment results show that the AI-driven approach is unlikely to save time. The human-led approach produced more valid threats (8 versus 5), a higher share of valid threats (26% versus 20%), and twice as many valid threats that the other approach missed (6 versus 3).
Controlled experiment
Two teams of engineers threat modeled two case studies. Once with humans driving the process and AI assisting (human-led, AI-assisted) and once with AI driving the process and humans triaging (AI-driven, human-reviewed).
In the human-led approach, which we refer to as the “manual approach”, the team drives the process. They draw the DFD themselves and identify threats through group discussion. Individual team members were allowed to consult a general-purpose LLM as an assistant for a second opinion, and all six participants did. However we did not allow access to the codebase by AI. In the AI-driven approach, which we refer to as the “AI approach”, a purpose-built AI skill generates the DFD and the threats from the specifications and the codebase. The teams provide context and triage the AI output.
The two case studies were real Codific products, not toy case studies:
- Videolab CVR, a mobile app that allows healthcare professionals to record patient consultations. The change that requires threat modeling is allowing the app to play recordings from the cloud.
- SAMMY MCP, the MCP server of our application security platform SAMMY. The change that requires threat modeling is expanding the MCP server to allow writing data back to SAMMY.
Team GBH consisted of three developers. Team 7 Pandas consisted of two developers and one QA engineer. Everyone across both teams had been taking part in the regular threat modeling sessions for at least two years. Each team threat modeled one system manually and the other with AI. Â Each session was time-boxed to 120 minutes, in line with how the teams regularly threat model changes in practice.
An expert reviewer with over 15 years of experience in application security scored every submitted threat for validity, specificity, severity and novelty.
Quantitative results
Threats identified per session
In total, the teams produced 56 threats: 25 with AI and 31 manually. The manual approach produced more threats (31 versus 25) and more valid threats (8 versus 5). The total number of threats is arguably the less meaningful finding, as one could prompt the AI to generate more threats ad nauseam.
The manual approach found a higher share of valid threats overall (26% versus 20%). However, this was not consistent across systems. On SAMMY MCP, the AI approach had the higher share of valid threats (31% versus 23%). On Videolab CVR the manual approach found more valid threats (40% versus 8%). The weakest result was the AI session on Videolab CVR, where only 1 of 12 threats was valid and none was novel.
Causes for invalid threat classification
We also looked into the key causes why threats would be classified as invalid. Most invalid manual threats (10 of 23) were already prevented by the existing architecture. Nearly all of these stemmed from a single misunderstanding in one team, namely that the MCP server could bypass the API. In contrast, the invalid AI threats were mostly implementation-level issues (6 of 20) or outside the scope of the change (5 of 20). Only the AI approach produced threats that would materialize only with an unrealistic implementation.
Overlap between manual and AI findings
Although the AI sessions produced fewer valid threats, all five valid AI threats were rated highly specific. In contrast, three of the eight valid manual threats were rated low in specificity. Both approaches found exactly one high-severity threat, and in both cases it was the same threat, namely prompt injection against the SAMMY MCP.
The two methods were largely complementary. Only 2 of the 11 distinct valid threats were found by both, and both methods found threats the other missed.
Qualitative results
Every participant rated the value of the manual session at least as high as the AI session. All participants considered 120 minutes sufficient for the manual session, whereas two participants reported they could have found more threats with more time in the AI session.
Plausible is not valid
The most striking result of the survey is the gap between how the teams perceived the AI output and how the expert rated it. One of the teams used the AI approach on Videolab CVR, where the expert rejected 11 of the 12 AI-generated threats. Yet all three participants reported rejecting or rewriting less than 20% of the AI threats, rated their meaningfulness at 4.3 out of 5 on average, and recommended making the AI skill part of regular threat modeling sessions. The other team used the AI approach on SAMMY MCP, where the expert rejected 9 of the 13 AI threats. Team participants reported rejecting about half or more of the AI threats, rated their meaningfulness at 3.0, and were undecided or against adopting the AI skill.
AI-generated threats are not only often invalid, they are convincingly invalid. AI output sounds authoritative, and only someone with sufficient expertise can tell when it is wrong. This has an uncomfortable implication. AI-driven threat modeling is often promoted as a way to compensate for a lack of security expertise, yet using it correctly requires precisely that expertise. In the hands of less experienced engineers, it may well produce more noise rather than less.
Diagramming should be a human task
Remarkably, in the group discussions both teams mentioned that the case study they threat modeled with the AI approach was perceived as more complex. Both teams also mentioned that drawing a data flow diagram increases the understanding of the architecture immensely. AI seems to generate detailed and polished data flow diagrams, but they lack focus on the components that matter for the change under review. More importantly, participants mentioned that the key added value of the whole threat modeling process is exactly the drawing of the diagram and aligning on the architecture and risk.
Triaging is tedious
Despite the initial impression that the AI approach would be less mentally taxing, one of the teams perceived the triaging process as tedious. Understanding the AI-generated threats took time and additional prompts and investigations. The other team, on the other hand, accepted most of the AI output with little triaging, which, as discussed above, is a concern in itself.
As one participant put it: “AI Threat modelling skill is a good idea, but needs to be scoped only to suggesting threats not doing the DFD.”
So what should you do?
We strongly believe that threat modeling should remain a human-led activity, but keeping AI in the loop seems to provide additional value.
Diagramming builds understanding. The most valuable outcome of a threat modeling session is not the list of the threats. It is the shared understanding of the system that the team builds while drawing the DFD together. The drawing process forces the team to agree on components, data flows and trust boundaries. Furthermore, it often exposes misunderstandings before a single threat is discussed.
AI is most useful as an additional participant. AI performed best as a second opinion. In the manual sessions, participants gave their own DFD to an LLM and asked for threats they might have missed. One participant reported that the LLM suggested a missed threat and sparked a discussion that led to two new ones. Our data also suggests that AI and humans tend to find different threats, as only 2 of the 11 distinct valid threats were found by both approaches.
Treat AI threats as hypotheses. AI-generated threats sound convincing, which makes them easy to accept. This is consistent with automation bias, where plausible output from an automated aid is accepted without sufficient scrutiny (Parasuraman & Manzey, 2010). Teams should therefore treat every AI-suggested threat as a hypothesis to be verified against the architecture, not as a finding.
Confounding factors
The sample size is small and none of our results is statistically significant. The AI skill we created may have been suboptimal. Both teams included engineers with substantial security awareness and knowledge. And the order of the sessions was not counterbalanced: both teams used the manual approach first and the AI approach second, so they may simply have been more tired.
Conclusion
Teams that let AI drive the process lost what they valued most, namely building a shared understanding of the architecture while drawing the diagram together. Moreover, plausible AI output was easily accepted without sufficient scrutiny. Our conclusion is therefore that threat modeling should remain primarily a human-led activity. Humans should draw the diagram and lead the analysis, and AI should review the result for gaps.
We expect to publish the full paper, including the methodology, all threats and the survey data soon. We encourage others to repeat this experiment with more teams.
References
- Braiterman, Z., Shostack, A., Marcil, J., de Vries, S., Michlin, I., Wuyts, K., Hurlbut, R., Schoenfield, B. S. E., Scott, F., Coles, M., Romeo, C., Miller, A., Tarandach, I., Douglen, A., & French, M. (2020). Threat Modeling Manifesto. threatmodelingmanifesto.org
- Paidy, P. (2023). Leveraging AI in Threat Modeling for Enhanced Application Security. International Journal of Artificial Intelligence, Data Science, and Machine Learning, 4(2), 57–66. doi:10.63282/3050-9262.IJAIDSML-V4I2P107
- Parasuraman, R., & Manzey, D. H. (2010). Complacency and bias in human use of automation: An attentional integration. Human Factors, 52(3), 381–410. doi:10.1177/0018720810376055
