Mitigating Technological Risks

"this toolkit shows you how to identify, monitor and mitigate the ‘hidden’ behavioural and organisational risks associated with AI roll-outs. These are the unintended consequences that can arise from how well-intentioned people, teams and organisations interact with AI solutions. Who is this toolkit for? This toolkit is designed for individuals and teams responsible for implementing AI tools and services within organisations and those involved in AI governance. It is intended to be used once you have identified a clear business need for an AI tool and want to ensure that your tool is set up for success. If an AI solution has already been implemented within your organisation, you can use this toolkit to assess risks posed and design a holistic risk management approach. You can use the Mitigating Hidden AI Risks Toolkit to: • Assess the barriers your target users and organisation may experience to using your tool safely and responsibly • Pre-empt the behavioural and organisational risks that could emerge from scaling your AI tools • Develop robust risk management approaches and mitigation strategies to support users, teams and organisations to use your tool safely and responsibly • Design effective AI safety training programmes for your users • Monitor and evaluate the effectiveness of your risk mitigations to ensure you not only minimise risk, but maximise the positive impact of your tool for your organisation" A very practical guide to behavioural considerations in managing risk by Dr Moira Nicolson and others at the UK Cabinet Office, which builds on the MIT AI Risk Repository.

As organizations transition from pilots to enterprise-wide deployment of Generative and Agentic AI, it's crucial to recognize that GAI risks differ significantly from traditional software risks. Towards that, it is important to go back to basics and this publication from 2024 by National Institute of Standards and Technology (NIST)'s Generative AI Profile does a great job! 🌐 Here are the four highest-impact risks and the mitigation actions every organization should implement:- 1. Systemic Risk: Algorithmic Monocultures & Ecosystem-Level Failures When multiple industries depend on the same foundation models, a single unexpected model behavior can lead to correlated failures across the ecosystem. ⚡ Mitigation: - - Build model diversity and avoid single-model dependencies. - Maintain fallback systems and contingency workflows. - Apply stress tests that simulate sector-wide shocks. 2. Human-Originating Risks (Misuse, Over-Trust, Manipulation) Many GAI incidents stem from human behavior, including misuse, over-reliance, indirect prompt injection, and flawed assumptions. ⚡ Mitigation:- - Implement continuous user education on limitations and safe use. - Enforce access controls, privilege separation, and plugin vetting. - Maintain audit trails and logging to identify misuse early. 3. Content Integrity Risks (Hallucinations, Synthetic Media, Provenance Failure) GAI increases the scale and believability of fabricated content, from medical misinformation to deepfake-enabled harms. ⚡ Mitigation:- - Invest in content provenance, watermarking, and metadata tracking. - Require pre-deployment testing for hallucination profiles across contexts. - Use cross-model verification before high-stakes outputs are acted upon. 4. Security Risks (Prompt Injection, Data Leakage, Model Extraction) NIST highlights increasingly sophisticated attack surfaces unique to LLMs: indirect prompt injection, data extraction, and plugin-initiated compromise. ⚡ Mitigation:- - Apply secure-by-design reviews for all LLM integration points. - Red-team regularly using GAI-specific attack methods. - Log inputs/outputs via incident-ready documentation so breaches can be traced. 🔐 The bottom line:- AI risk management is not a technical afterthought, it is now a core capability. Organizations that operationalize governance, provenance, testing, and incident disclosure (NIST’s four focus pillars) will be the ones that deploy AI safely and at scale. 💬 If you’d like to explore Gen AI and Agentic AI risks, practical mitigation strategies, or how to operationalize the NIST AI RMF for your organization, feel free to comment or DM. Let’s build safer AI systems together! #AI #GenAI #AIGovernance #NIST #AIRMF #RiskManagement #AITrust #ResponsibleAI #AILeadership

Understanding IT Risk Management In today's digital landscape, managing risks in IT is crucial for the stability and security of organizations. The diagram shared outlines the key components of IT Risk Management, providing a structured approach to identifying and mitigating risks. Key Components: 1. Context Establishment: - This initial step involves understanding the environment in which the organization operates. It sets the stage for effective risk management by identifying stakeholders, regulatory requirements, and the organization's objectives. 2. Risk Assessment: This is divided into several phases: - Risk Identification: Recognizing potential risks that could impact services, functions, or systems. - Risk Analysis: Evaluating identified risks by examining threats and vulnerabilities to understand their potential impact. - Risk Estimation: Assessing the likelihood and impact of risks to prioritize them effectively. 3. Risk Evaluation: - This step involves comparing the estimated risks against the organization's risk criteria to determine their significance and decide on the appropriate actions. 4. Risk Treatment: Organizations must decide how to address identified risks through: - Reduction: Implementing measures to decrease the likelihood or impact of risks. - Avoidance: Altering plans to sidestep risks entirely. - Retention: Accepting the risk when the benefits outweigh the potential consequences. - Transfer: Shifting the risk to another party, often through insurance. 5. Risk Acceptance: - After evaluating and treating risks, organizations must decide which risks they are willing to accept based on their risk appetite and tolerance. 6. Risk Monitoring and Review: - Continuous monitoring of risks and the effectiveness of risk management strategies is essential. Regular reviews ensure that the organization remains prepared for emerging threats and changes in the IT landscape. 7. Risk Communication and Consultation: - Effective communication with stakeholders about risks and the strategies in place to manage them fosters transparency and trust. By systematically addressing IT risks through this framework, organizations can better safeguard their assets, enhance decision-making, and ensure compliance with regulatory requirements. Embracing a proactive approach to IT Risk Management is not just about avoiding threats—it's about enabling the organization to thrive in an increasingly complex digital world.

The EDPB recently published a report on AI Privacy Risks and Mitigations in LLMs.   This is one of the most practical and detailed resources I've seen from the EDPB, with extensive guidance for developers and deployers. The report walks through privacy risks associated with LLMs across the AI lifecycle, from data collection and training to deployment and retirement, and offers practical tips for identifying, measuring, and mitigating risks.   Here's a quick summary of some of the key mitigations mentioned in the report:   For providers: • Fine-tune LLMs on curated, high-quality datasets and limit the scope of model outputs to relevant and up-to-date information. • Use robust anonymisation techniques and automated tools to detect and remove personal data from training data. • Apply input filters and user warnings during deployment to discourage users from entering personal data, as well as automated detection methods to flag or anonymise sensitive input data before it is processed. • Clearly inform users about how their data will be processed through privacy policies, instructions, warning or disclaimers in the user interface. • Encrypt user inputs and outputs during transmission and storage to protect data from unauthorized access. • Protect against prompt injection and jailbreaking by validating inputs, monitoring LLMs for abnormal input behaviour, and limiting the amount of text a user can input. • Apply content filtering and human review processes to flag sensitive or inappropriate outputs. • Limit data logging and provide configurable options to deployers regarding log retention. • Offer easy-to-use opt-in/opt-out options for users whose feedback data might be used for retraining.   For deployers: • Enforce strong authentication to restrict access to the input interface and protect session data. • Mitigate adversarial attacks by adding a layer for input sanitization and filtering, monitoring and logging user queries to detect unusual patterns. • Work with providers to ensure they do not retain or misuse sensitive input data. • Guide users to avoid sharing unnecessary personal data through clear instructions, training and warnings. • Educate employees and end users on proper usage, including the appropriate use of outputs and phishing techniques that could trick individuals into revealing sensitive information. • Ensure employees and end users avoid overreliance on LLMs for critical or high-stakes decisions without verification, and ensure outputs are reviewed by humans before implementation or dissemination. • Securely store outputs and restrict access to authorised personnel and systems.   This is a rare example where the EDPB strikes a good balance between practical safeguards and legal expectations. Link to the report included in the comments.   #AIprivacy #LLMs #dataprotection #AIgovernance #EDPB #privacybydesign #GDPR

🧩 AI Risk Oversight: Connecting Compliance, Strategy, and Board Responsibilities🧩 Corporate boards have a duty to align all initiatives, including those involving AI, with the organization’s mission, financial health, and enterprise risk management. While AI offers significant opportunities, its risks demand careful governance. Directors must move beyond compliance-driven oversight to adopt a strategic, integrated approach that safeguards organizational priorities. ➡️Linking AI to Mission and Values AI systems can amplify your organization’s mission by driving efficiency, improving decision-making, and creating value for your stakeholders, but poorly governed AI can do just the opposite. For example: 🔹AI missteps, like biased decision-making, can damage reputations and undermine commitments to fairness and inclusivity. 🔹A lack of oversight may lead to AI systems failing to serve the organization’s core purpose or violating stakeholder expectations. Boards can ensure alignment by embedding ethical AI principles, such as those found in #ISO42001, into governance frameworks. ➡️AI’s Financial Implications AI impacts the bottom line through potential cost savings, revenue generation, and risk exposure. Boards must weigh: 🔹Cost Savings: Automation and data-driven insights can reduce inefficiencies and improve margins. 🔹Revenue Opportunities: New products and services powered by AI can create competitive advantages. 🔹Risk Management: Financial losses due to AI failures, regulatory penalties, or legal actions from misuse can be significant. Tools like #ISO42005 (DIS) can help you assess and mitigate risks, enabling informed decisions that protect financial interests while maximizing returns. ➡️Managing AI within Enterprise Risk Frameworks AI introduces new dimensions of enterprise risk. You must integrate AI governance into the broader enterprise risk management strategy, considering risks like: 🔹Operational Disruptions: Failures in AI systems can impact core operations or supply chains. 🔹Regulatory Compliance: Laws governing AI are evolving, and non-compliance could lead to penalties. 🔹Reputational Risk: Public trust can erode if AI systems are perceived as unfair, opaque, or harmful. Standards like #ISO23894 provide actionable guidance for managing AI risks throughout its lifecycle, aligning with existing enterprise risk frameworks. ➡️A Balanced Approach: AI Oversight as a Strategic Imperative Boards must ensure AI strategies align with mission goals, drive financial performance, and mitigate enterprise risks. A balanced approach includes: 🔹Adopting Standards: Use #ISO42001 to establish an AI management system (#AIMS) and ISO42005 (DIS) to assess potential impacts. 🔹Prioritizing Risks: Leverage ISO23894 to identify and address AI-specific risks effectively. 🔹Integrating Oversight: Embed AI governance into broader strategic and risk discussions to ensure alignment with the organization’s mission. A-LIGN #TheBusinessofCompliance

I'm very excited to share a new paper I've been working on for the last few months titled Effective Mitigations for Systemic Risks from General-Purpose AI. This paper would not have been possible without my brilliant and hard-working co-authors Annemieke Brouwer, Tim Schreier, Noemi Dreksler, Valeria Pulignano, and Rishi Bommasani. The paper is long, but the executive summary is only a few pages. We surveyed 76 domain experts across five key systemic risk areas – AI safety; critical infrastructure; democratic processes; chemical, biological, radiological, and nuclear risks (CBRN); and discrimination and bias – about the perceived effectiveness of 27 proposed risk mitigation measures. Key findings of the study: • A wide-range of measures were judged to be both effective and technically feasible. • An overwhelming majority of experts (91% or above) thought all of the measures were technically feasible. • Safety incident reporting and security information sharing emerged as the most effective measure across various risk domains by the experts (70-91% agreement). • Pre-deployment risk assessments and third-party pre-deployment audits were also among the highest ranked, emphasising the importance of external scrutiny (70-86% and 62-87% agreement, respectively). These top-rated measures underscore the importance of external scrutiny, proactive evaluation, and transparency in mitigating systemic risks from general-purpose AI. These findings have immediate policy relevance. In particular, the EU AI Act, which requires the development of compliance guidance for providers of general-purpose AI by May 2025. For one, the results suggest that experts think there are a wide-range of technically feasible and effective risk mitigation measures that could be implemented and legally mandated to reduce systemic risks. You can read the full paper below. Please send us your feedback, we're happy to improve the paper for the next version.

AI Risk Management Framework from the Cloud Security Alliance. Here are the concepts I found actionable from the paper... 1) Comprehensive MRM Framework: Example: Establish a governance committee that oversees AI development, ensuring compliance with industry standards and regulatory requirements. 2) Model Cards: Example: To enhance transparency, create detailed documentation for each AI model outlining its purpose, design, training data, and performance metrics. 3) Data Sheets: Example: Document the sources, quality, and preprocessing steps of training data used for a model to identify potential biases. 4) Risk Cards: Example: Develop risk cards that identify and mitigate potential issues, such as data bias, in hiring models by implementing fairness constraints and diverse training datasets. 5) Scenario Planning: Example: Conduct scenario planning for an AI-powered chatbot to explore how it might handle offensive language or misinformation and develop mitigation strategies. 6) Continuous Monitoring: Example: Set up automated monitoring for a fraud detection model to track its performance and accuracy over time and identify any drifts or anomalies. 7) Prioritize Mitigation: Example: First, focus on high-impact risks, such as implementing strong encryption and access controls for AI systems handling sensitive financial data. 8) Transparency and Trust: Example: Regularly update stakeholders on AI model performance and risk mitigation efforts through transparent reporting and open communication channels. By implementing these steps, you can harness AI’s full potential while minimizing risks. There is no tool you can buy that will do this for you (yet). It’s good old-fashioned process.  💡🔒  #AI #RiskManagement #AIGovernance #ModelRisk #Innovation #CyberSecurity Cloud Security Alliance Caleb Sima

“Mapping Cybersecurity Threats to Defenses: A Strategic Approach to Risk Mitigation” Most of the time we talk about reducing risk by implementing controls, but we don’t talk about if the implemented controls will reduce the Probability or Impact of the Risk. The below matrix helps organizations build a robust, prioritized, and strategic cybersecurity posture while ensuring risks are managed comprehensively by implementing controls that reduces the probability while minimising the impact. Key Takeaways from the Matrix 1. Multi-layered Security: Many controls address multiple attack types, emphasizing the importance of defense in depth. 2. Balance Between Probability and Impact: Controls like patch management and EDR reduce both the likelihood of attacks (probability) and the harm they can cause (impact). 3. Tailored Controls: Some attacks (e.g., DDoS) require specific solutions like DDoS protection, while broader threats (e.g., phishing) are countered by multiple layers like email security, IAM, and training. 4. Holistic Approach: Combining technical measures (e.g., WAF) with process controls (e.g., training, third-party risk management) creates a comprehensive security posture. This matrix can be a powerful tool for understanding how individual security controls align with specific threats, helping organizations prioritize investments and optimize their cybersecurity strategy. Cyber Security News ®The Cyber Security Hub™

Are you curious how others are managing AI risks? 🤔 I recently came across the International AI Safety Report, a collaborative effort between the Department for Science, Innovation and Technology and the AI Security Institute. This report showcases techniques employed by developers and risk management professionals worldwide to enhance the reliability of AI models and systems, while also mitigating the risk of misuse. The report covers various topics, including defense-in-depth strategies, limiting undesired behaviors, deploying safeguards, post-deployment monitoring, and early governance approaches, among others. At the end of the report, they highlight how developers are using "assurance tools to substantiate claims" about the capabilities of their AI products. As someone with an assurance background, I appreciate the good-faith effort that technical resources are making not only to build safety into these technologies but also to measure their level of safety and report on it periodically. I believe this space is ripe for further clarity and standardization in the coming months and years. #ai #airisk #aisafety #riskmanagement

The MIT AI Risk Mitigation Taxonomy consolidates 831 safeguards from major AI safety reports into four categories: Governance, Technical, Operational, and Transparency. It exposes gaps in technical safeguards and a lack of shared terminology in areas like red teaming and risk management. The framework serves as a strategic blueprint for aligning stakeholders and building robust, accountable AI systems. #AIRiskMitigation #MITAI #AISafety #AIRegulation #ResponsibleAI #TechGovernance #AILeadership #RiskManagement #AICompliance #AIStandards #ArtificialIntelligence #AIFramework #RedTeaming #AIEthics #AIAccountability Post link: https://lnkd.in/guqv8ZmJ Source: https://lnkd.in/g3ix-z9T AND PPT deck: https://lnkd.in/g8qqYR34