Updated: 11 December, 2024
24 January, 2022
Cybersecurity is currently a “hype.” It was a “hype” in 2001 and has remained so every year since, with the number of cybersecurity professionals, experts, and mentors exploding as everyone hops on the “hype train.” Yet, cybersecurity isn’t some new technology, methodology, or philosophy—it’s a quality of a software system, defined by ensuring Confidentiality, Integrity, and Availability (CIA). Simply put, cybersecurity means guaranteeing these principles, but the reality is that security remains a pervasive problem most organizations struggle to address systematically. Thankfully, systematic approaches like application security programs exist. In this blog, I’ll introduce what an application security program is and specifically compare BSIMM vs SAMM, two of the most promising frameworks for building such programs, focusing on application security as a critical subset of cybersecurity.
Disclaimer: While I am SAMM Core Team member I have never been involved in a BSIMM assessment. My analysis is based on the online resources and conversations with BSIMM practitioners. Hence, this article might contain factual inconsistencies about BSIMM and I welcome any feedback (especially from Synopsys).
Update: Since the initial publication of this blog both the US and the EU have actually introduced regulations targeting to improve the security posture of organizations. For the interested readers we refer to our blog on How to implement NIST SSDF with SAMM and SAMMY.
Traditional approach to software security
Security is a very hard quality for organizations to deal with. First of all, security is intangible. You can’t really see nor perceive whether a software system is secure. Secondly, you can’t really measure security either. It is impossible to check how secure a software product is and objectively compare it to an alternative offering. Finally, security only seems to incur costs without a clear upside. Sure, you could use security as a sales argument. Everybody does! However your customers cannot compare your product’s security to that of your competitors. Hence many organizations keep security investments to a bare minimum, which brings us to today’s realities.
Data breaches create jobs
Unfortunately bad guys sometimes hack into corporate networks and then start bragging around. That isn’t the worst thing to happen actually. Remember Petya ransomware which completely smashed FedEx for weeks and made them go back to the previous century? Major data breaches are weekly news nowadays (Uber and Optus for the past 2 weeks). So in the past 5 years organizations have started taking security more “seriously” simply because the lack of it can have a massive impact on the bottom line.
Note that regulatory pressures, such as the GDPR are also a factor organizations have to take into account when dealing with security. However we have all lawyered up. Even the least security aware firms out there can easily claim GDPR conformity.
Best practices in software security
So far the “best practices” in software security are typically limited to the following three activities:
- Security incident and event management (so basically advanced low-level monitoring and control)
- Regular patching
- Periodic penetration testing
While all of that is necessary, it is far from sufficient. By practicing these activities you mainly focus on finding bugs, not flaws. The latter are 1000 times more expensive to fix once the system is in production. Moreover, there are no guarantees that you have identified all bugs. Your bug-fix might also introduce a new vulnerability.
Security Tooling in the DevSecOps Pipeline
DevSecOps is a buzzword that typically refers to automation in building and deploying software systems. Adding security tooling in the build / deploy pipelines of your software systems is obviously a great idea. However there are at least 2 essential challenges in the industry with using security automation tools. First of all, the number of false positives identified by the tools can be mind-bending. Secondly, the tools typically rank the findings using a Common Vulnerability Scoring System (CVSS) score. CVSS score measures impact and not risk. As an organization you are only interested in the latter. All that eventually results in a non-trivial challenge on how to select the appropriate security tooling for your DevSecOps pipeline.
Shifting left
The more security-aware firms are using the “shift left” philosophy. Shifting left is a recent buzzword that promotes dealing with security properties during the early design phase of a software system. Threat modeling is a methodology enabling “security by design” and it is one of the most fundamental security activities to have in your application security program. However threat modeling is still not enough and far from mainstream. Furthermore I am more than convinced that most organizations are unaware what threat modeling is.
Application Security Program
The problem of a systematic approach to security has been solved quite some time ago. Microsoft has introduced the first security assurance program in 2004. Named Microsoft Security Development Lifecycle (MSDL), it was Microsoft’s response to reduce software maintenance costs and increase reliability of software concerning software security related bugs. Since then a number of security assurance initiatives have been launched, such as OpenSAMM, BSIMM, SAMM, SSE CMM, SafeCode, NIST SSDF, etc. Unfortunately, up till now, they never really got any grip in the community. The main reasons are that they are not simple and challenging to introduce to large organizations. Small organizations have a lower risk exposure and the management typically isn’t really aware of the problem until it’s too late.
Here is the really great news though. More regulatory pressure is on its way to mandate the use of an application security program for organizations. Memorandum M-22-18 mandates all US Federal agency suppliers to conform to NIST SSDF. European regulation is in the making. Furthermore, leading security-minded organizations are starting to set the bar higher for the rest.
OpenSAMM and Its Forks
The remainder of this blog focuses on two maturity models that originate from the same source, i.e., OpenSAMM. OpenSAMM was created by Pravir Chandra and sponsored by Fortify. Fortify has then donated OpenSAMM to the OWASP community. Both BSIMM and SAMM originate from OpenSAMM. Therefore, both models still contain some similarities, but follow different approaches to application security.
Building Security In Maturity Model (BSIMM)
BSIMM is a maturity model that helps organizations plan, implement and measure their software security assurance program. BSIMM consists of 4 domains split in 12 practices and containing a total of 125 security activities. So think of pen testing, patching, monitoring tools and threat modeling as some of these 125 activities you could (but not always should) do in your application security program. Here is a structural overview of the BSIMM13 domains and practices.
BSIMM is not only the framework, but is also a measuring stick in the industry. BSIMM comes with an objective assessment of the different activities in 130 organizations from 8 industry verticals (financial services, independent software vendors, technology, healthcare, cloud, Internet of Things, insurance, and retail).
If you would like to learn more about BSIMM specifically, you can go to the following article: “BSIMM (Building Maturity In Maturity Model: A Complete Guide”
Software Assurance Maturity Model (SAMM)
SAMM is a maturity model that provides an effective and measurable way for all types of organizations to analyze and improve their software security posture. SAMM consists of 5 business functions split over 15 security practices and containing a total of 90 security activities. Here is a structural overview of SAMM functions and practices.
SAMM already has successfully completed the first phase of the Benchmarking initiative. The project is currently collecting datasets and we expect to publish the initial benchmark in Q1 2024.
BSIMM vs SAMM: a Quick Overview
| BSIMM | SAMM |
|---|---|
| Descriptive The goal of the model is to observe and report. BSIMM doesn’t say what you should be doing, rather what other firms out there are doing. |
Prescriptive SAMM provides prescriptive advice on how to implement particular activities. |
| Binary scoring The scoring is binary, hence you get a yes or no. |
Scale scoring The scoring per activity is on a scale from 0 to 1 so you will get partial credit. |
| Proprietary BSIMM is proprietary to Synopsys. For an assessment you should reach out to them for cost and logistics. |
Open source SAMM is an open source OWASP flagship project. Hence, you can easily do a self-assessment of SAMM. |
| Activity levels Levels refer to how frequent a certain activity is and are not necessarily related to maturity. |
Maturity levels SAMM defines three maturity levels. Each level has a successively more sophisticated objective with specific activities, and more strict success metrics. |
| Industry comparison You can measure your own posture against someone else in the industry |
No comparison possible At the moment SAMM does not support any comparisons in terms of scores. The Benchmarking project has been completed though. We are currently collecting data to create the first dataset. |
| Annual updates Updates are qualitative and provide valuable insights on what’s going on in the industry. |
2-3 year updates SAMM model gets updated every 2-3 years. |
| Tool support The tooling for BSIMM is provided by Synopsis, however it is not publicly available. |
Tool support Various tools support SAMM including an open source tool SAMMWise, a free tool SAMMY and Excel/Google Doc spreadsheets. |
| Industry-based prioritization BSIMM provides a clear picture which security activities are most commonly implemented throughout the industry. |
Risk-based prioritization SAMM recommends picking your improvement roadmap based on your risk profile and driven by metrics. |
BSIMM vs SAMM: Which model is better?
I have to disappoint some readers at this point and say that I don’t have a black-and-white conclusion that states that one of the models is better. In this section I discuss the main advantages and disadvantages of both models from the perspective of an organization. My own subjective preference is with SAMM.
Industry Benchmark
One of the main strengths of BSIMM lies in its adoption and faster feedback loops to Synopsys. Synopsys updates BSIMM yearly based on a systematically collected set of qualitative and quantitative measurements from the participating organizations. This information is also shared with the participating organizations. Hence, an organization using BSIMM can more easily create a roadmap for improvements based on the industry standard set by BSIMM.
On the other hand, SAMM currently lacks this capability. We do expect the Benchmarking project to provide some improvements in the near future. However one of the most commonly asked questions by the SAMM users is where should they start? While the answer might be obvious (i.e., start from risk) for an organization which is just booting up their application security program it might be really hard to figure that out. Risk is everywhere and risk prioritization is not as straightforward as it might seem. Data from the industry can clearly help with that.
On the contrary, the fact that BSIMM observes certain activities in the participating organizations does not necessarily mean that copying those to your organization will have the best returns on investment. Moreover the list of participating organizations typically consists of very large enterprises (median number of developers 800). Thus it is unclear if BSIMM is a good fit for SMEs. From experience I know that SAMM is suitable for any size of organization.
Free and Open Source
BSIMM is proprietary and only Synopsys can do a real assessment. In fact, some details regarding the assessment are not disclosed (e.g., quality criteria to achieve an activity). A small organization is unlikely to be able to afford BSIMM. Conversely, SAMM is open source. SAMM welcomes community collaboration and contributions that has lead to a more robust and widely adopted framework. We have a Slack channel where you are welcome to ask any questions you might have. Moreover, the Core Team regularly organizes SAMM trainings that allow anyone interested to get a deep dive into the framework.
Even from a larger organization perspective SAMM is a great fit. BSIMM seems more of an all-in choice. SAMM allows you to experiment with a small scope and see what works for the organization. We know that for a fact as we have recently onboarded Zebra Technologies in the SAMMY tool.
Learning Curve
SAMM flattens the learning curve by leveraging maturity levels. Each security practice is defined in terms of successively more sophisticated (and more rigorous) objectives. BSIMM claims to be a maturity level, however the levels are largely related to how pervasive a certain activity is.
One of the greatest advantages of SAMM lies in its prescriptive guidance. SAMM provides advice on how to implement each activity. Many security initiatives fail due to the poor details.
SAMM allows you to score a security practice that is only partially implemented. As a result your improvement roadmap can be more granular as you will get partial credit for certain activities. BSIMM scoring seems to be binary.
Both SAMM and BSIMM could help you reduce the cost of CRA compliance find out how in this article.
SAMM also has a Slack channel where you are welcome to ask any questions you might have. There are also monthly calls that you can join and discuss, e.g., a specific quality criteria.
Finally, if you want to jumpstart with OWASP SAMM there is a free SAMM Fundamentals Course that has received excellent reviews.
Tooling
Finally, SAMM comes with a number of supporting tools including the open source SAMMWise, Codific’s free SAMMY tool and Google / Microsoft Spreadsheets published by SAMM. These tools can help you with an assessment, setting the improvement targets and demonstrating measurable improvements. We are not aware of any similar BSIMM tools available to the public.
Conclusion
I believe in a holistic approach to security where organizations practice security throughout the entire software development lifecycle. Unfortunately for the industry (and fortunately for Codific), security is a tough nut to crack. It is not a new technology we could learn and forget, nor is it a product or device we could buy and forget. Thus security is a quality that requires a systematic approach, and application security programs, such as BSIMM or SAMM, provide the necessary and sufficient solution to the problem.
Introducing a security program is challenging. Firstly, an organization’s behavior changes slowly over time, so changes need to be smaller and iterative to take hold and make a difference. Second, there is no single recipe that works for all organizations. Third, guidance related to security activities must be prescriptive. Too often, security initiatives fail due to poor details, lack of communication, or invalid assumptions. Coincidentally (or not really), these are the core principles behind SAMM, which is simple, well-defined, and measurable.
What do we build with SAMM and SAMMY?
Codific is a team of security software engineers that leverage privacy by design principles to build secure cloud solutions. We build applications in different verticals such as HR-tech, Ed-Tech and Med-Tech. Secure collaboration and secure sharing are at the core of our solutions.
Videolab is used by top universities, academies and hospitals to put the care in healthcare. Communication skills, empathy and other soft skills are trained by sharing patient interview recordings for feedback.
SARA is used by top HR-Consultants to deliver team assessments, psychometric tests, 360 degree feedback, cultural analysis and other analytical HR tools.
SAMMY Is a Software Assurance Maturity Model management tool. It enables companies to formulate and implement a security assurance program tuned to the risks they are facing. That way other companies can help us build a simple and safe digital future. Obviously our AppSec program and SAMMY itself is built on top of it.
We believe in collaboration and open innovation, we would love to hear about your projects an see how we can contribute in developing secure software and privacy by design architecture. Contact us.
