2 November, 2024
OWASP Software Assurance Maturity Model (SAMM) is one of the only comprehensive frameworks available for application security program management. Aside from BSIMM, there’s not much else around. Moreover, SAMM is open-source, making it accessible to everyone with zero barriers to entry. However there’s a catch: implementing OWASP SAMM comes with a learning curve. Beginner and even intermediate users seem to struggle with certain aspects of the model. Based on my experience, users have the hardest time figuring out:
- How to deal with quality criteria (or the “definition of done”) and what they mean;
- How to come up with a meaningful prioritization for the improvement roadmap;
- What type of evidence is required for demonstrating “compliance”;
- How to interpret the model for domains other than web application development;
- How can SAMM help for a smaller company.
In this blog, I will offer 10 foolproof ways to ensure your SAMM assessments and improvement roadmap remain completely useless and inefficient. So, if you’re looking to turn your SAMM journey into an exercise in frustration and futility, you’re in the right place! For everyone else, these “suggestions” will serve as a guide for what to avoid and how to make the most out of SAMM.
Preparation is overrated
Start your SAMM implementation with a quick glance at the model
If you want to ensure your SAMM implementation journey is off to a rocky start, here’s a foolproof tip: glance briefly at the SAMM model and assume you’re ready to go. Just skimming it should be enough, right? Dive straight into becoming the SAMM champion for your organization. You already know quite a bit about application security, you don’t really need to know the model that well. What could go wrong?
As the self-declared SAMM expert, you’ll be primed to frustrate everyone when it’s time for assessments and improvement planning—especially when those pesky details come up.
Pro tips for success
- Take the full SAMM Fundamentals course: It’s a solid starting point for understanding the ins and outs of SAMM.
- Join the SAMM monthly community calls: These sessions offer insights, best practices, and updates from the SAMM community. You can find details on the OWASP SAMM Community page.
- Hop into the SAMM Slack channel: Connect with other SAMM users, ask questions, and share insights. You can join through the SAMM Slack invitation link.
Following these steps will give you the knowledge and support you need to guide your organization effectively.
Assessment chaos
Ignore quality criteria, you don’t need them
If you’re looking to make your SAMM assessment as pointless as possible, here’s a tip: ignore those pesky quality criteria the SAMM core team put together. They’re weird, confusing, and, honestly, who really needs them? Because you’ve never read SAMM in depth, you can’t possibly know that these criteria define what “done” really means for each activity. So just skip them entirely. You will be blissfully clueless and ready for a spectacular failure.
Pro tips for success
- Stick to quality criteria: These criteria are mandatory for a reason—they define what “done” means. Now, here’s a secret (shh, don’t quote me!): it’s okay in exceptional situations to put a pin on some of them, but only if you’re consistent about it. At that point, though, you’re not implementing OWASP SAMM, but rather a SAMM variation.
Failing miserably in your first SAMM assessment and panic
Find a “volunteer”—preferably your buddy in the organization who is convinced his team will ace the assessment. After all, they have an ISO27001 certification. Confident in his success, you send him a link to fill out a self-assessment in the SAMMY tool. He scores nearly a perfect 3 on everything, leaving you puzzled why his score ended up at 2.92 instead of a full 3.0. Just kidding—what is most likely to happen is him scoring well below 1.0, and both of you going into panic mode. Cue the frantic realization that certification doesn’t exactly equate to security.
Pro tips for success
- View assessment as a starting point: Honesty is key. If in doubt, give yourself a lower score. Forget about trying to boost your score—most small companies would likely average less than 0.5 anyway. Larger organizations may do better in governance, but design, implementation, and validation scores will still be very low.
Prepare for the SAMM “Audit” and call in external experts
Your buddy blames you for failing the “audit” (as you’ve now taken to calling it). He’s on a mission now: using ChatGPT, he generates all the documentation he thinks he needs. Then, in a grand gesture, he calls in the Codific team for an external assessment. Surely, this will teach him a lesson! Congratulations, your organization has now wasted even more resources to realize…you’re back at square one. Time to start improving!
Pro tips for success
- Call in external assessors when ready: Only bring in external experts if you have something solid to assess. It’s a waste of time and money to call in professionals just to confirm there’s nothing there.
Chase scores when setting the targets
Focus on the score
If you want to really miss the mark with SAMM, here’s a great tip: focus solely on boosting your score as quickly as possible and ignore risk altogether. Instead of considering which improvements will reduce actual risk, just go for full scores in governance. Need documentation? Easy! Use ChatGPT to generate stacks of documents that no one, including yourself, can understand. It’s a foolproof way to ensure that the only thing you’re improving is your frustration level.
Pro tips for success
- Forget the scores! Focus on addressing real risks and prioritizing improvements that will provide the greatest impact on security. Planning around risk, not scores, is the most effective way to build lasting maturity.
Zero to hero in one quarter
Everyone knows that getting a perfect 3.0 in SAMM can be done in no time—one quarter should be plenty, right? Set the expectation that you’ll go from zero to hero at lightning speed. Spoiler alert: you’ll fail spectacularly. But if you’re determined, you could always lie about your scoring to achieve “instant” success.
Pro Tip for Success
- Realistically, improving SAMM scores takes time. A good rule of thumb is to aim for about 0.5 points of improvement per year. Any faster, and you’re setting up for burnout and unsustainable changes.
Botching the SAMM implementation work
Invest in Tools, you don’t need people nor processes
Who needs people and processes when you have tools? Even better, AI tools! With the power of modern LLMs, security is practically a solved issue. Just buy as many tools as possible—especially the ones that promise AI-driven magic.
Pro tips for success
- People first! Start by ensuring you have the right people in the right roles and that they understand their responsibilities. Build smooth processes with as few gates as possible to minimize bottlenecks. Once that’s running smoothly, introduce tools to make work more efficient.
- Tune your tools! Tools rarely work optimally out of the box. Plan for a regular investment of time and resources to fine-tune your tools for your specific context.
Threat model once and claim victory
Who has time for ongoing threat modeling? Just schedule one session, and you’re set! Surely, one session will take your scores from 0 to 2 (or even 3!) in Threat Modeling, Architecture Validation, and Architecture Mitigation. Better yet, hire a consultant to do it in one sitting and watch those scores soar.
Pro tips for success
- Threat modeling is a team process. Effective threat modeling takes time and collaboration. It’s a team exercise, not a one-off task. Make it an integral part of your SDLC to keep evolving your security posture.
Label defects with a “Security” tag and show it on a dashboard
Metrics are critical in SAMM, so let’s go straight for the easy points. Just tag your security-related defects with a “Security” label, throw those numbers onto a dashboard, and voilà—you’re scoring in Strategy and Metrics and Defect Management! Who needs deeper insights when you have a graph that looks good?
Pro tips for Success
- Labeling security-related defects is a start: Tagging defects with a “Security” label is helpful, but it’s far from the full picture. At the very least you need to know where the defects are coming from (e.g., tool generated, pen testing, bug bounty platform).
- Use Goal-Question-Metric (GQM) to reason about metrics: An effective metric should answer a question that supports a high-level goal. The GQM framework helps by starting with a goal, formulating questions that define that goal, and then identifying metrics to answer those questions. This approach leads to metrics that inform your decision-making rather than just populating a dashboard.
- Efficient metrics are unicorns: Metrics in SAMM are often misunderstood and underestimated. Start by reading Caroline Wong’s “Security Metrics: A Beginner’s Guide” and “Guide to Advanced Empirical Software Engineering”, which cover effective metrics and measurement. The latter is a collection of excellent research papers (most of them are available for free download) and I’d recommend to start reading chapters 1 and 6.
Roll out improvements and call it a day
One and done: feedback is for losers
Once you’ve rolled out your SAMM improvements, congratulations! You’re now officially the security hero of your organization. No need to re-evaluate or check if the changes are actually making a difference. Surely, the teams are following all the new practices and making full use of the new tools. And don’t bother with any feedback from them—they’re the ones who didn’t know what to do in the first place, right?
Pro tips for Success
- Implement feedback loops: Regularly assess how improvements are working and adapt as needed. Schedule routine check-ins with your teams to ensure the changes are effective and aligned with their needs. Since there are often interdependencies between SAMM activities, feedback loops help increase efficiency and cohesion across tasks.
- Expire scores: The SAMMY tool lets you set expiration dates for SAMM practices, so make use of this feature! It prompts you to revisit and re-evaluate how well teams are performing in specific security activities, ensuring continuous progress rather than complacency.
Conclusion: turning your SAMM implementation into a success
If you’ve been nodding along with these satirical “how-to-fail” tips, you’re not alone. Many teams face these hurdles. However, with the right approach, SAMM can help you build a structured, risk-based application security program that grows with your organization’s needs.
Here’s a quick recap of pro tips to set your SAMM journey on the right path:
- Preparation: Take the SAMM Fundamentals course, join community calls, and hop into the Slack channel to connect with other SAMM users. Understanding the model deeply is your foundation.
- Assessments: View assessments as a starting point for improvement. Stick to quality criteria as your “definition of done,” and be honest in scoring—SAMM is about growth, not perfection.
- Setting targets: Don’t chase scores; focus on risk. Prioritize improvements that address the most urgent security needs, aiming for a manageable 0.5 points of maturity growth per year.
- Implementation: Start with people and processes. Tools can help, but only after responsibilities and workflows are in place. Keep threat modeling ongoing, and remember that metrics need to support specific goals.
- Feedback and follow-through: Implement feedback loops, and use SAMMY’s score expiration feature to prompt regular re-evaluations. Check in with your teams to make sure improvements are effective and aligned with real needs.
Following these steps will help you avoid the common pitfalls and build a SAMM program that strengthens your organization’s security posture over time. Good luck, and remember: OWASP SAMM implementation is a journey, not a sprint!
