CISO Forum Summary – Establishing Meaningful Metrics in your Security Program
Every month, Fortify Experts holds CISO Round Tables discussing the latest trends and topics. Recently, we discussed how these successful security leaders have been able to create higher-performing security teams. Here is a summary of their top tips or suggestions on how to create better metrics to move your program forward.
Below are perspectives from 18 Security Leaders who provided input on the following questions:
What metric has helped drive your program forward the most?
- Understanding the Audience: The metric that has helped drive his program the most is understanding the audience and getting the metrics they want. Thinks that is one of the critical differentiators so that both parties will speak the same language and are on the same page.
- MTTD (Mean time to Detect) & MTTR (Mean Time to Resolve/Respond): The ones that he has always gone back to are MTTD (Mean time to Detect) & MTTR (Mean Time to Resolve/Respond). This is useful in the operational realm because it shows your responsivity and how quickly you can get back up. Another metric is your level of preparedness and patchwork for your vulnerabilities. These aren’t the metrics and the measures that go with them; it is more the ability to tell the story and how it will impact others. These metrics will mean different things to others in the company, and that is why it is helpful to understand what story you are trying to tell.
- Impact Analysis: The metric that has helped him the most is a quality metric which is an impact analysis that he does after every widespread cyber-attack that comes to the news. He looks at the impact of that attack and analyzes that attack for different companies and his own. If that attack did not affect his organization the same way it did others, he will examine what works for him and see the differences between the other companies and vice versa. This has been good for showing how his program works for people on his board.
- Risk-Based Metrics: The metrics that work best with her programs are risk-based Metrics. Metrics that share the risks exceeding the agreed mitigation timeframe with the enterprise. Also, extending chances that the owner’s request since extensions are always requested is mitigation of a problem that has not been resolved. Another metric is the measurement of risks they are being accepted. These are usually of value to executives by bringing these to the surface to be discussed.
- Readiness Metrics: The metric that has worked for him is reviewing all of the big hacks and presenting them to the executive committees, explaining what has happened, and showing their readiness for that to potentially happen at his organization. It feels like we get too technical with terms that many people, especially executives, do not understand, and he feels it is best to keep things as simple as possible for people to all understand.
- NIST CSF Maturity Score: The metric that has worked best for him is reporting his company’s maturity score, as measured by the NIST cybersecurity framework. He knows it is subjective but used his proper funding to hire a third party to analyze the maturity score.
- Tracking Against a Baseline: You’ve got a board or an executive leadership team that only thinks of risk after getting a poor or fair assessment. Establishing a baseline foundation and starting tracking against it has been effective for him over the years.
- NIST maturity assessment: He completed a NIST maturity assessment which has given him leverage to talk to the board about focusing on tracking metrics that focus on vulnerabilities and patchwork. His company bought Tenable and scans their devices every week, showing that things were old and needed to be patched. His goal is to get where he can detect in 1 minute, contain in 10 minutes, and recover fully in 60 minutes.
- Measuring against a Framework: He found a lot of success by starting with the simple things that people can wrap their arms around, such as project status. NCSF (NIST Cybersecurity Framework) is always at the top of the list of customers he has worked with. They are working with key stakeholders and internal auditors to define agreed-upon attributes that encompass a maturity level capability, which allows the maturity level to be their own.
- TOP 4 Metrics: 4 metrics have helped him along the way, one being visible grading systems available on the internet (BitSight, Recon, etc.) because it shows what the world thinks when they look at his company. The other three are % of completed commitments planned, the % of the operationalized controls, and the maturity of those implemented controls.
- Qualitative Metrics: His new company focuses on the qualitative side and operates in a no-blame culture.
- IAM Metrics: Use metrics around identity and access management. Who has access to customer data is getting a lot of attention from the executives.
- Top 10 Most Asked Questions: Building a program from the ground up, they went straight to the business. From there, they would take the top 10 most asked questions from prospective clients and compare them to their existing controls environment and map them out to missed opportunities/missed revenue. This is what he dubbed the Security Blitz and has helped gain executive support and drive a lot of change.
- Question Provoking Metrics: Impactful metrics are taking credit for success and showing where the achievements are. Metrics should drive more questions for the executives, especially before asking for more resources.
- Connecting to Organizational Strategy: The metrics that have helped him are the ones that are related to risk. It was understanding the risks related to the IT environment and the risk posed to the business environment. The key to his success is aligning the metrics that he is presenting to the overall strategic plan for the organization and making that connection solid.
- Financial Impact Metrics: The metric that got the most attention from his board and President was when they reported the number of records they have and the potential financial impact of the organization if those records were breached. This helps the conversations start moving forward so that they can get additional resources going.
- Gamification Scoring Metrics: We are hiring analysts who have a technical background but who also have a gaming background because they are competitive. They do gamification of the SOC internal by finding remediating against the metrics they have. By having a points system that will be rewarded each month. So, keeping them motivated and gamification are helpful for his team.
- Business Aligned Outcome-Driven Metrics: By getting with the business leaders we have them identify what value they see in their investment for security. We also have them define an acceptable baseline. We developed Protection Level Agreements to help businesses understand the value we are giving to them. We developed metrics to give regular status updates on those business objectives.
What technologies are used to help drive better metrics?
- Solutions like RiskLens or SecurityStudio bring a lot of visibility to risk managers presenting on that front. In enterprise environments, Looker and PowerBI take data out of they’re data dumping lake to help make sense of all of it and eventually dashboard it.
- One leader just relies on their ticketing system. They put a lot of effort into getting everybody in the university to use it. This helped his university understand where the issues are coming from and where they need to put their resources. The ticketing system is probably their most significant resource in gaining metrics for their ticketing system.
- API Connections to a cloud-based tool for our compliance. For their SOC, it has API connections in all the various devices that they feed data so that it knows based on the controls what the evidence should be, and it pulls it monthly.
What are the most useless metrics to track?
- Number of phishing emails! Rather have the number that did not get clicked.
- Billions of attacks on the firewall.
In Summary:
Thanks to the input of the security leaders who joined our CISO Forum which provided such insightful advice on how to improve your Red Teaming activities.
If you are a security leader and would like to participate in our monthly CISO Forums where we discuss valuable and actionable information as well as best practices and challenges, please register here.
About Tim Howard
Tim Howard is the founder of 4 technology firms including Fortify Experts which helps companies hire the Best Cyber Talent on the Planet as well as provides expert consulting and NIST-based security assessments.
In addition, he has a passion for helping CISOs develop Higher Performing Teams through coaching, by creating interactive CISO Forums, and by helping them create highly-effective team cultures.
With each new hire, his firm produces an Employee Operating Manual to help clients understand how to motivate and maximize productivity while meeting the needs of each employee.
He also teamed up with Lyndrel Downs to launch Cybersecurity DIVAS to help promote the most influential women in cybersecurity and provide a mentoring program to help encourage and support more diversity within the industry.
Tim is married with 3 kids. He is an avid runner and has completed two IRONMAN Texas events. He is also a graduate of Texas A&M University.
Invite Tim to connect: www.linkedin.com/in/timhoward