BSIMM vs SAMM: which is the better AppSec programme

5 October, 2022

Cybersecurity is currently a “hype”. It was a “hype” in the year 2001 and every year after that. The number of cybersecurity professionals, experts, gurus, <replace by a fancy title> is currently exploding as everyone is trying to hop on the “hype train”. BUT – cybersecurity is not really a new technology, methodology, philosophy, approach. It’s a quality property of the software system. There is an actual definition for it and CIA can help us all with that. CIA – as in short for Confidentiality, Integrity and Availability. Thus cybersecurity is making sure your software system guarantees confidentiality, integrity and availability. Simple, isn’t it? Here is a better definition for cybersecurity we all know. Security is a pervasive problem we have no idea how to deal with in a systematic way. Well, actually we (the real security experts) do. However it’s not simple, cheap nor painless. It’s called a security assurance programme.

In this blog, I’ll tell you more about what a security assurance programme is. I will focus on two most promising frameworks, namely BSIMM and SAMM and provide a comparison of both.

Traditional approach to software security

Security is a very hard quality for organizations to deal with. First of all, security is intangible. You can’t really see nor perceive whether a software system is secure. Secondly, you can’t really measure security either. It is impossible to check how secure a software product is and objectively compare it to an alternative offering. Finally, security only seems to incur costs without a clear upside. Sure, you could use security as a sales argument. Everybody does! However your customers cannot compare your product’s security to that of your competitors. Hence many organizations keep security investments to a bare minimum, which brings us to today’s realities.

Data breaches create jobs

Data breach at UberUnfortunately bad guys sometimes hack into corporate networks and then start bragging around. That isn’t the worst thing to happen actually. Remember Petya ransomware which completely smashed FedEx for weeks and made them go back to the previous century? Major data breaches are weekly news nowadays (Uber and Optus for the past 2 weeks). So in the past 5 years organizations have started taking security more “seriously” simply because the lack of it can have a massive impact on the bottom line.

Note that regulatory pressures, such as the GDPR are also a factor organizations have to take into account when dealing with security. However we have all lawyered up. Even the least security aware firms out there can easily claim GDPR conformity.

Best practices in software security

So far the “best practices” in software security are typically limited to the following three activities:

  • Security incident and event management (so basically advanced low-level monitoring and control)
  • Regular patching
  • Periodic penetration testing

While all of that is necessary, it is far from sufficient. By practicing these activities you mainly focus on finding bugs, not flaws. The latter are 1000 times more expensive to fix once the system is in production. Moreover, there are no guarantees that you have identified all bugs. Your bug-fix might also introduce a new vulnerability.

DevSecOps

DevSecOps is a buzzword that typically refers to automation in building and deploying software systems. Adding security tooling in the build / deploy pipelines of your software systems is obviously a great idea. However there are 2 essential challenges in the industry with using security automation tools. First of all, the number of false positives identified by the tools can be mind-bending. Secondly, the tools typically rank the findings using a Common Vulnerability Scoring System (CVSS) score. CVSS score measures impact and not risk. As an organization you are only interested in the latter.

Shifting left

The more security-aware firms are using the “shift left” philosophy. Shifting left is a recent buzzword that promotes dealing with security properties during the early design phase of a software system. Threat modeling is a methodology enabling “security by design” and it is one of the most fundamental security activities to have in your security assurance programme. However threat modeling is still not enough and far from mainstream (I bet 9 out of 10 companies have no idea what TM is).

Shifting everywhere

Security assurance programme sounds too “researchy” and we clearly need a buzzword for it. So shift everywhere (as opposed to shift left) is the one. It’s about dealing with security throughout the complete software development lifecycle.

The problem of a systematic approach to security has been solved quite some time ago. Microsoft has introduced the first security assurance programme in 2004. Named Microsoft Security Development Lifecycle (MSDL), it was Microsoft’s response to reduce software maintenance costs and increase reliability of software concerning software security related bugs. Since then a number of security assurance initiatives have been launched, such as OpenSAMM, BSIMM, SAMM, SSE CMM, SafeCode, NIST SSDF, etc. Unfortunately, up till now, they never really got any grip in the community. The main reasons are that they are not simple and challenging to introduce to large organizations. Small organizations have a lower risk exposure and the management typically isn’t really aware of the problem until it’s too late.

Here is the really great news though. More regulatory pressure is on its way to mandate the use of a security assurance programme for organizations. Memorandum M-22-18 mandates all US Federal agency suppliers to conform to NIST SSDF. European regulation is in the making. Furthermore, leading security-minded organizations are paving the road for the rest.

Security assurance programmes

The remainder of this blog focuses on two maturity models that originate from the same source, i.e., OpenSAMM. OpenSAMM was created by Pravir Chandra and sponsored by Fortify. Fortify has then donated OpenSAMM to the OWASP community. Both BSIMM and SAMM originate from OpenSAMM.

Building Security In Maturity Model (BSIMM)

BSIMM is a maturity model that helps organizations plan, implement and measure their software security assurance programme. BSIMM consists of 4 domains split in 12 practices and containing a total of 125 security activities. So think of pen testing, patching, monitoring tools and threat modeling as some of these 125 activities you could (but not always should) do in your security assurance programme. Here is a structural overview of the BSIMM13 domains and practices.

BSIMM
Building Security In Maturity Model

BSIMM is not only the framework, but is also a measuring stick in the industry. BSIMM comes with an objective assessment of the different activities in 130 organizations from 8 industry verticals (financial services, independent software vendors, technology, healthcare, cloud, Internet of Things, insurance, and retail).

Software Assurance Maturity Model (SAMM)

SAMM is a maturity model that provides an effective and measurable way for all types of organizations to analyze and improve their software security posture. SAMM consists of 5 business functions split over 15 security practices and containing a total of 90 security activities. Here is a structural overview of SAMM functions and practices.

OWASP SAMM
Software Assurance Maturity Model

SAMM is planned to include the upcoming Benchmarking project that will also allow one to compare your own security posture with the rest of the industry.

BSIMM vs SAMM

BSIMM SAMM
Descriptive
The goal of the model is to observe and report. BSIMM doesn’t say what you should be doing, rather what other firms out there are doing.
Prescriptive
SAMM provides clear (yet high-level) prescriptive advice on how to implement particular activities.
Binary scoring
The scoring is binary, hence you get a yes or no.
Scale scoring
The scoring per activity is on a scale from 0 to 1 so you will get partial credit.
Scored by Synopsys and proprietary
BSIMM is proprietary to Synopsys. For an assessment you should reach out to them for cost and logistics.
Self-scoring and open source
SAMM is an open source OWASP flagship project. Hence, you can easily do a self-assessment of SAMM. There are third parties that can help as well.
Activity levels
Levels refer to how frequent a certain activity is and are not necessarily related to maturity.
Maturity levels
SAMM defines three maturity levels. Each level has a successively more sophisticated objective with specific activities, and more strict success metrics.
Industry comparison
You can measure your own posture against someone else in the industry
No comparison possible
At the moment SAMM does not support any comparisons in terms of scores, however Benchmarking project is in the making.
Annual updates
Updates are qualitative and provide valuable insights on what’s going on in the industry.
2-3 year updates
SAMM model gets updated every 2-3 years.
No tool support
I am not aware of any tool support for BSIMM.
Tool support
Various tools support SAMM including an open source tool SAMMWise, a free tool SAMMY and Excel/Google Doc spreadsheets.
Industry-based prioritization
BSIMM provides a clear picture which security activities are most commonly implemented throughout the industry.
Risk-based prioritization
SAMM recommends picking your improvement roadmap based on your risk profile and driven by metrics.

Which model is better: SAMM or BSIMM?

The main strength of BSIMM currently lies in its adoption and faster feedback loops to Synopsys. Synopsys updates BSIMM yearly based on a systematically collected set of qualitative and quantitative measurements from the participating organizations. Thus you could create a roadmap for improvements based on the industry standard set by BSIMM. SAMM currently lacks this capability, however the Benchmarking project will provide some improvements.

To use BSIMM you should contact Synopsys to get an assessment for your own company. You could then compare your organization’s security posture to those participating in BSIMM. SAMM comes with a clear set of quality criteria. Hence you can easily conduct a SAMM assessment yourself. However currently it remains impossible to check how well you perform in terms of SAMM scores.

The fact that BSIMM observes certain activities in the participating organizations does not necessarily mean that copying those to your organization will have the best returns on investment. Moreover the list of participating organizations typically consists of very large enterprises (median number of developers 800). Thus it is unclear if BSIMM is a good fit for SMEs. SAMM on the other hand supports any organization type by design. Risk tolerance and metrics are the factors that guide the improvement strategy.

SAMM supports maturity levels that are successively more sophisticated allowing for a flatter learning curve. BSIMM levels are largely related to how pervasive a certain activity is. SAMM also provides prescriptive advice on how to implement each activity.

In SAMM you can get a scale score for a security practice that is partially implemented. As a result your improvement roadmap can be more granular as you will get partial credit for certain activities. BSIMM scoring seems to be binary (the details of the scoring are not published in the model).

Finally, SAMM comes with several supporting tools including the open source SAMMWise, Codific’s free SAMMY tool and Google / Microsoft Spreadsheets published by SAMM. These tools can help you with an assessment, setting the improvement targets and demonstrating measurable improvements. We are not aware of any similar BSIMM tools available to the public.

Conclusion

I believe in a holistic approach to security where organizations practice security throughout the entire software development lifecycle. Unfortunately for the industry (and fortunately for Codific) security is a really tough nut to crack. It is not a new technology we could learn and forget. It is not a product nor device we could buy and forget. Security is a quality that requires a systematic approach. Security assurance programmes provide the necessary and sufficient solution to the problem.

Unfortunately introducing a security programme is extremely challenging. First of all, an organization’s behaviour changes slowly over time. Hence changes need to be smaller and iterative to really take hold and make a difference. Secondly, there is no single recipe that works for all organizations. Thirdly, guidance related to security activities must be prescriptive. Too often, security initiatives fail due to poor details, lack of communication, or invalid assumptions. Coincidentally (or not really), these are the core principles behind SAMM that is simple, well-defined, and measurable.

What do we build with SAMM and SAMMY?

Codific is a team of security software engineers that leverage privacy by design principles to build secure cloud solutions. We build applications in different verticals such as HR-tech, Ed-Tech and Med-Tech. Secure collaboration and secure sharing are at the core of our solutions.

Videolab is used by top universities, academies and hospitals to put the care in healthcare. Communication skills, empathy and other soft skills are trained by sharing patient interview recordings for feedback.

SARA is used by top HR-Consultants to deliver team assessments, psychometric tests, 360 degree feedback, cultural analysis and other analytical HR tools.

SAMMY Is a Software Assurance Maturity Model management tool. It enables companies to formulate and implement a security assurance program tuned to the risks they are facing. That way other companies can help us build a simple and safe digital future. Obviously our AppSec program and SAMMY itself is built on top of it.

We believe in collaboration and open innovation, we would love to hear about your projects an see how we can contribute in developing secure software and privacy by design architecture. Contact us.