Fortify Experts holds CISO Round Tables monthly to discuss the latest trends and topics in security. Recently, we repeated the topic of Leaverging and Securing Generative AI in Cybersecurity that we covered last September. For comparison, we asked leaders some of the same questions from last year, and here are the comparative results:
From these charts, it appears that worry about Generative AI in 2024 is a little more tempered than it was a year ago and that some progress has been made in implementing policies and controls to manage it.
This year’s discussion highlighted a mix of successes and cautions as security leaders explore both the transformative power of AI and the inherent risks it carries. Below is a detailed summary of the key points, with specific examples of how AI can enhance security efficiencies.
1. AI Adoption and Risk Perspectives
The participants opened the discussion by assessing the risks and benefits of adopting generative AI. Opinions were mixed—some viewed AI as a necessary tool to improve efficiencies, while others expressed concerns about its disruptive potential and associated risks.
One participant pointed out, “AI has been the backbone of fraud detection in the credit card industry for 30 years,” indicating that AI has long been embedded in financial systems.
There was some discussion about whether “AI hallucinations” should more accurately be called “AI confabulations.”
As AI generates incorrect or misleading information it often is not random but instead, a continuation of the topic that may sound true even if it is not. Hence, ‘confabulation’ is more accurate instead of ‘hallucination.’ Read more Here: It’s a Confabulation, Not a Hallucination.
“AI will provide pretty good answers, but we have to validate what it’s saying is true.”
2. AI in Financial Services
In financial services, the use of AI is growing, but institutions remain cautious due to strict regulatory frameworks. Leaders discussed the potential for AI to revolutionize backend operations, but they also highlighted the risks of bias and regulatory compliance, particularly in decision-making processes.
One Top 5 bank is using AI to enhance wealth management but they had to carefully navigate compliance and legal concerns.
One participant added, “The regulators have told us explicitly that using AI for loan decisions could lead to significant penalties,” emphasizing the importance of compliance with regulatory guidelines when employing AI for tasks that directly affect customers.
A CISO shared that the FTC has become particularly aggressive in scrutinizing AI implementations, stating, “The FTC has made it clear that if a company misuses AI, they might be required to destroy the model entirely if it leads to biased outcomes.”
Financial institutions, especially banks, are proactively working with regulators to ensure that their AI models meet compliance standards. “We’ve adopted a cautious approach to deploying AI, particularly in decisions that impact customers directly, like lending or wealth management,” said one leader from the banking sector.
3. Enhancing Security Efficiencies with AI
Several examples were shared on how AI can be used to enhance cybersecurity efficiencies. These included using AI to automate tasks that previously required significant human intervention, enabling teams to focus on higher-level strategic priorities.
One key example was how AI is being used to generate exploit payloads based on vulnerability patches. A participant explained, “We feed patches into AI to quickly generate exploit payloads, which helps identify security risks faster and allows us to act before malicious actors exploit vulnerabilities.”
Another practical use case involved using AI to automate policy development and compliance.. One participant shared, “I uploaded all of our policies and procedures into an LLM (Large Language Model) so that I can auto-generate answers to vendor questionnaires, significantly reducing the time spent on manual tasks.”
AI is also improving threat detection capabilities. One CISO noted that they are employing AI-driven analytics to continuously monitor for anomalous behavior across networks, “allowing us to respond faster to incidents and reducing response times significantly.”
“AI is allowing us to respond faster to incidents and reducing response times significantly.”
4. Ethics and Bias in AI
The conversation turned to the ethical concerns surrounding AI, especially regarding bias. Leaders were particularly concerned about AI models reinforcing historical biases in decision-making processes.
In healthcare, a participant raised concerns about bias affecting medical treatment decisions, remarking, “Once the model is trained on biased data, there’s no way to truly remove that bias.”
A CISO in the financial services industry pointed out that AI could perpetuate discrimination if not managed carefully, saying, “If our models are trained on data favoring white men, that bias could perpetuate itself without proper checks and balances.” See this specific example of GenAI bias here:
5. Challenges with Data Privacy
The privacy risks associated with AI adoption were discussed, particularly in industries like healthcare and financial services, where sensitive data is frequently involved.
A participant highlighted the risk of data leakage when using AI models, stating, “Once your data is used in an AI model, it’s hard to guarantee it can be fully removed, even if the AI company claims they don’t store it.”
The issue of data ownership was also emphasized, particularly when using third-party AI services. One university CISO warned, “Ensuring we retain ownership of our data and that it’s not used for model training by third parties is a priority for us.”
“AI is a tool, but like any tool, it can cut both ways—efficient or harmful depending on how it’s wielded.”
6. AI Applications Across Industries
Participants shared creative applications of AI that go beyond automating routine tasks. These included more advanced uses of AI to drive business efficiencies and improve operations.
AI is being used to automate coding processes and streamline compliance reviews, making backend operations more efficient.
AI is being leveraged to automate threat detection and response in cybersecurity. One CISO said, “We’ve integrated AI tools into our SIEM (Security Information and Event Management) systems to automatically detect and flag unusual patterns in network traffic, greatly improving our ability to preemptively address security threats.”
Additionally, AI is being deployed in identity and access management (IAM) to enhance authentication processes. AI models are used to analyze user behavior and flag deviations from normal activity patterns, thereby improving access security. “AI helps us continuously verify identities in real-time, reducing the risk of credential compromise,” noted one participant.
8. Best Practices for AI Adoption
To mitigate risks while maximizing AI’s benefits, participants suggested a number of best practices:
Implementing robust monitoring of AI systems to detect bias and inaccuracies.
Limiting access to GenAI and using DLP (Data Loss Prevention) solutions to monitor data inputs into models, ensuring sensitive data isn’t leaked or misused.
One tool mentioned was Biri
Establishing Ethics Committees to review AI-driven decisions, especially in sensitive sectors like healthcare and finance.
Where possible, employ open-source models and local instances to maintain control over data rather than relying on third-party services.
9. GenAI Tools:
Consider 3rd Party tools like GRAICE (www.graice.com) to provide enterprise-level GenAI management & DLP. It provides more control, more LLMs, and a cost savings over Copilot. It enables any LLM internal and external plus built-in DLP & monitoring.
This CISO Forum illuminated the growing role of Generative AI in cybersecurity. While the benefits of enhanced efficiencies and automation were acknowledged, the participants expressed caution, particularly around ethical concerns, bias, and regulatory compliance. Best practices such as monitoring AI systems, restricting data access, and engaging with regulators were highlighted as key strategies to mitigate these risks while fully harnessing AI’s potential.
About Tim Howard
Tim Howard is the founder of 5 technology firms including Fortify Experts which helps companies create higher-performing teams through People (Executive Search and vCISO/Advisory consulting), Process (NIST-based 3rd party security assessments and Leadership Coaching), and Technology (security simplifying solutions).
