NIST AI RMF Playbook guide: suggested actions for all 72 subcategories

GOVERN 1.1

Legal and regulatory requirements involving AI are understood, managed, and documented.

In plain English: Know which AI laws and regulations apply to each use of AI, and keep that list written down and current.

First 2 of 3 suggested actions in the Playbook:

• Maintain awareness of the applicable legal and regulatory considerations and requirements specific to industry, sector, and business purpose, as well as the application context of the deployed AI system.
• Align risk management efforts with applicable legal standards.

Tagged by NIST for: Governance and Oversight

GOVERN 1.2

The characteristics of trustworthy AI are integrated into organizational policies, processes, procedures, and practices.

In plain English: Build the trustworthy-AI characteristics into your everyday policies instead of a separate ethics statement.

First 2 of 15 suggested actions in the Playbook:

Organizational AI risk management policies should be designed to:

• Define key terms and concepts related to AI systems and the scope of their purposes and intended uses.
• Connect AI governance to existing organizational governance and risk controls.

Tagged by NIST for: Governance and Oversight

GOVERN 1.3

Processes, procedures, and practices are in place to determine the needed level of risk management activities based on the organization's risk tolerance.

In plain English: Decide how much risk-management effort each AI system gets, scaled to how much risk you are willing to accept.

First 2 of 5 suggested actions in the Playbook:

• Establish policies to define mechanisms for measuring or understanding an AI system’s potential impacts, e.g., via regular impact assessments at key stages in the AI lifecycle, connected to system impacts and frequency of system updates.
• Establish policies to define mechanisms for measuring or understanding the likelihood of an AI system’s impacts and their magnitude at key stages in the AI lifecycle.

Tagged by NIST for: Governance and Oversight

GOVERN 1.4

The risk management process and its outcomes are established through transparent policies, procedures, and other controls based on organizational risk priorities.

In plain English: Write the risk process down, make it visible, and set controls that follow your risk priorities.

First 2 of 7 suggested actions in the Playbook:

• Establish and regularly review documentation policies that, among others, address information related to:
  • AI actors contact informations
  • Business justification
  • Scope and usages
  • Expected and potential risks and impacts
  • Assumptions and limitations
  • Description and characterization of training data
  • Algorithmic methodology
  • Evaluated alternative approaches
  • Description of output data
  • Testing and validation results (including explanatory visualizations and information)
  • Down- and up-stream dependencies
  • Plans for deployment, monitoring, and change management
  • Stakeholder engagement plans
• Verify documentation policies for AI systems are standardized across the organization and remain current.

Tagged by NIST for: Governance and Oversight

GOVERN 1.5

Ongoing monitoring and periodic review of the risk management process and its outcomes are planned and organizational roles and responsibilities clearly defined, including determining the frequency of periodic review.

In plain English: Schedule reviews of the risk process itself, with named owners and a fixed review frequency.

First 2 of 7 suggested actions in the Playbook:

• Establish policies to allocate appropriate resources and capacity for assessing impacts of AI systems on individuals, communities and society.
• Establish policies and procedures for monitoring and addressing AI system performance and trustworthiness, including bias and security problems, across the lifecycle of the system.

Tagged by NIST for: Governance and Oversight, Operation and Monitoring

GOVERN 1.6

Mechanisms are in place to inventory AI systems and are resourced according to organizational risk priorities.

In plain English: Keep an inventory of your AI systems and put more resources where the risk is higher.

First 2 of 4 suggested actions in the Playbook:

• Establish policies that define the creation and maintenance of AI system inventories.
• Establish policies that define a specific individual or team that is responsible for maintaining the inventory.

Tagged by NIST for: Governance and Oversight

GOVERN 1.7

Processes and procedures are in place for decommissioning and phasing out AI systems safely and in a manner that does not increase risks or decrease the organization’s trustworthiness.

In plain English: Have a plan for retiring an AI system safely: data, dependencies, users and records.

First 2 of 4 suggested actions in the Playbook:

• Establish policies for decommissioning AI systems. Such policies typically address:
  • User and community concerns, and reputational risks.
  • Business continuity and financial risks.
  • Up and downstream system dependencies.
  • Regulatory requirements (e.g., data retention).
  • Potential future legal, regulatory, security or forensic investigations.
  • Migration to the replacement system, if appropriate.
• Establish policies that delineate where and for how long decommissioned systems, models and related artifacts are stored.

Tagged by NIST for: AI Deployment, Operation and Monitoring

GOVERN 2.1

Roles and responsibilities and lines of communication related to mapping, measuring, and managing AI risks are documented and are clear to individuals and teams throughout the organization.

In plain English: Document who does what in mapping, measuring and managing AI risk, and how they report to each other.

First 2 of 6 suggested actions in the Playbook:

• Establish policies that define the AI risk management roles and responsibilities for positions directly and indirectly related to AI systems, including, but not limited to
  • Boards of directors or advisory committees
  • Senior management
  • AI audit functions
  • Product management
  • Project management
  • AI design
  • AI development
  • Human-AI interaction
  • AI testing and evaluation
  • AI acquisition and procurement
  • Impact assessment functions
  • Oversight functions
• Establish policies that promote regular communication among AI actors participating in AI risk management efforts.

Tagged by NIST for: Governance and Oversight

GOVERN 2.2

The organization’s personnel and partners receive AI risk management training to enable them to perform their duties and responsibilities consistent with related policies, procedures, and agreements.

In plain English: Train staff and partners on AI risk so they can do what your policies ask of them.

First 2 of 6 suggested actions in the Playbook:

• Establish policies for personnel addressing ongoing education about:
  • Applicable laws and regulations for AI systems.
  • Potential negative impacts that may arise from AI systems.
  • Organizational AI policies.
  • Trustworthy AI characteristics.
• Ensure that trainings are suitable across AI actor sub-groups - for AI actors carrying out technical tasks (e.g., developers, operators, etc.) as compared to AI actors in oversight roles (e.g., legal, compliance, audit, etc.).

Tagged by NIST for: Governance and Oversight

GOVERN 2.3

Executive leadership of the organization takes responsibility for decisions about risks associated with AI system development and deployment.

In plain English: Senior leaders sign off on, and answer for, AI risk decisions.

First 2 of 2 suggested actions in the Playbook:

• Organizational management can:
  • Declare risk tolerances for developing or using AI systems.
  • Support AI risk management efforts, and play an active role in such efforts.
  • Integrate a risk and harm prevention mindset throughout the AI lifecycle as part of organizational culture
  • Support competent risk management executives.
  • Delegate the power, resources, and authorization to perform risk management to each appropriate level throughout the management chain.
• Organizations can establish board committees for AI risk management and oversight functions and integrate those functions within the organization’s broader enterprise risk management approaches.

Tagged by NIST for: Governance and Oversight

GOVERN 3.1

Decision-making related to mapping, measuring, and managing AI risks throughout the lifecycle is informed by a diverse team (e.g., diversity of demographics, disciplines, experience, expertise, and backgrounds).

In plain English: Put people from different backgrounds and disciplines into AI risk decisions.

First 2 of 5 suggested actions in the Playbook:

Organizational management can:

• Define policies and hiring practices at the outset that promote interdisciplinary roles, competencies, skills, and capacity for AI efforts.
• Define policies and hiring practices that lead to demographic and domain expertise diversity; empower staff with necessary resources and support, and facilitate the contribution of staff feedback and concerns without fear of reprisal.

Tagged by NIST for: Governance and Oversight, AI Design

GOVERN 3.2

Policies and procedures are in place to define and differentiate roles and responsibilities for human-AI configurations and oversight of AI systems.

In plain English: Define who oversees each AI system and how the work is split between people and the AI.

First 2 of 8 suggested actions in the Playbook:

• Establish policies and procedures that define and differentiate the various human roles and responsibilities when using, interacting with, or monitoring AI systems.
• Establish procedures for capturing and tracking risk information related to human-AI configurations and associated outcomes.

Tagged by NIST for: AI Design

GOVERN 4.1

Organizational policies and practices are in place to foster a critical thinking and safety-first mindset in the design, development, deployment, and uses of AI systems to minimize potential negative impacts.

In plain English: Make it normal to challenge AI designs and to put safety first.

First 2 of 5 suggested actions in the Playbook:

• Establish policies that require inclusion of oversight functions (legal, compliance, risk management) from the outset of the system design process.
• Establish policies that promote effective challenge of AI system design, implementation, and deployment decisions, via mechanisms such as the three lines of defense, model audits, or red-teaming – to minimize workplace risks such as groupthink.

Tagged by NIST for: AI Design, AI Development, AI Deployment, Operation and Monitoring

GOVERN 4.2

Organizational teams document the risks and potential impacts of the AI technology they design, develop, deploy, evaluate, and use, and they communicate about the impacts more broadly.

In plain English: Teams write down the risks and impacts of the AI they build or use, and share them beyond the team.

First 2 of 4 suggested actions in the Playbook:

• Establish impact assessment policies and processes for AI systems used by the organization.
• Align organizational impact assessment activities with relevant regulatory or legal requirements.

Tagged by NIST for: AI Design, AI Development, AI Deployment, Operation and Monitoring

GOVERN 4.3

Organizational practices are in place to enable AI testing, identification of incidents, and information sharing.

In plain English: Make testing, incident reporting and sharing lessons part of how AI work is done.

First 2 of 5 suggested actions in the Playbook:

• Establish policies and procedures to facilitate and equip AI system testing.
• Establish organizational commitment to identifying AI system limitations and sharing of insights about limitations within appropriate AI actor groups.

Tagged by NIST for: TEVV, Operation and Monitoring, Governance and Oversight, Fairness and Bias

GOVERN 5.1

Organizational policies and practices are in place to collect, consider, prioritize, and integrate feedback from those external to the team that developed or deployed the AI system regarding the potential individual and societal impacts related to AI risks.

In plain English: Collect feedback about AI impacts from people outside the team, and act on it.

First 2 of 3 suggested actions in the Playbook:

• Establish AI risk management policies that explicitly address mechanisms for collecting, evaluating, and incorporating stakeholder and user feedback that could include:
  • Recourse mechanisms for faulty AI system outputs.
  • Bug bounties.
  • Human-centered design.
  • User-interaction and experience research.
  • Participatory stakeholder engagement with individuals and communities that may experience negative impacts.
• Verify that stakeholder feedback is considered and addressed, including environmental concerns, and across the entire population of intended users, including historically excluded populations, people with disabilities, older people, and those with limited access to the internet and other basic technologies.

Tagged by NIST for: AI Design, Governance and Oversight, AI Impact Assessment, Affected Individuals and Communities

GOVERN 5.2

Mechanisms are established to enable the team that developed or deployed AI systems to regularly incorporate adjudicated feedback from relevant AI actors into system design and implementation.

In plain English: Regularly feed reviewed feedback back into how the system is designed and built.

First 2 of 5 suggested actions in the Playbook:

• Explicitly acknowledge that AI systems, and the use of AI, present inherent costs and risks along with potential benefits.
• Define reasonable risk tolerances for AI systems informed by laws, regulation, best practices, or industry standards.

Tagged by NIST for: AI Impact Assessment, Governance and Oversight, Operation and Monitoring

GOVERN 6.1

Policies and procedures are in place that address AI risks associated with third-party entities, including risks of infringement of a third-party’s intellectual property or other rights.

In plain English: Cover third-party AI risks, including intellectual property, in your policies and contracts.

First 2 of 4 suggested actions in the Playbook:

• Collaboratively establish policies that address third-party AI systems and data.
• Establish policies related to:
  • Transparency into third-party system functions, including knowledge about training data, training and inference algorithms, and assumptions and limitations.
  • Thorough testing of third-party AI systems. (See MEASURE for more detail)
  • Requirements for clear and complete instructions for third-party system usage.

Tagged by NIST for: Third-party entities, Operation and Monitoring, Procurement

GOVERN 6.2

Contingency processes are in place to handle failures or incidents in third-party data or AI systems deemed to be high-risk.

In plain English: Have a fallback for when a high-risk third-party model or data source fails.

First 2 of 2 suggested actions in the Playbook:

• Establish policies for handling third-party system failures to include consideration of redundancy mechanisms for vital third-party AI systems.
• Verify that incident response plans address third-party AI systems.

Tagged by NIST for: AI Deployment, TEVV, Operation and Monitoring, Third-party entities

MAP 1.1

Intended purposes, potentially beneficial uses, context-specific laws, norms and expectations, and prospective settings in which the AI system will be deployed are understood and documented. Considerations include: the specific set or types of users along with their expectations; potential positive and negative impacts of system uses to individuals, communities, organizations, society, and the planet; assumptions and related limitations about AI system purposes, uses, and risks across the development or product AI lifecycle; and related TEVV and system metrics.

In plain English: Write down what each AI system is for, who uses it, where it runs, which laws and norms apply and what could go right or wrong.

First 2 of 13 suggested actions in the Playbook:

• Maintain awareness of industry, technical, and applicable legal standards.
• Examine trustworthiness of AI system design and consider, non-AI solutions

MAP 1.2

Interdisciplinary AI actors, competencies, skills, and capacities for establishing context reflect demographic diversity and broad domain and user experience expertise, and their participation is documented. Opportunities for interdisciplinary collaboration are prioritized.

In plain English: Involve people with a range of skills and backgrounds when setting the context, and record who took part.

First 2 of 2 suggested actions in the Playbook:

• Establish interdisciplinary teams to reflect a wide range of skills, competencies, and capabilities for AI efforts. Verify that team membership includes demographic diversity, broad domain expertise, and lived experiences. Document team composition.
• Create and empower interdisciplinary expert teams to capture, learn, and engage the interdependencies of deployed AI systems and related terminologies and concepts from disciplines outside of AI practice such as law, sociology, psychology, anthropology, public policy, systems design, and engineering.

MAP 1.3

The organization’s mission and relevant goals for AI technology are understood and documented.

In plain English: Record the organisation's mission and what it wants from the AI.

First 2 of 5 suggested actions in the Playbook:

• Build transparent practices into AI system development processes.
• Review the documented system purpose from a socio-technical perspective and in consideration of societal values.

MAP 1.4

The business value or context of business use has been clearly defined or – in the case of assessing existing AI systems – re-evaluated.

In plain English: State the business value, or re-check it for a system that is already running.

First 2 of 3 suggested actions in the Playbook:

• Document business value or context of business use
• Reconcile documented concerns about the system’s purpose within the business context of use compared to the organization’s stated values, mission statements, social responsibility commitments, and AI principles.

MAP 1.5

Organizational risk tolerances are determined and documented.

In plain English: Decide how much risk you will accept, and write it down.

First 2 of 6 suggested actions in the Playbook:

• Utilize existing regulations and guidelines for risk criteria, tolerance and response established by organizational, domain, discipline, sector, or professional requirements.
• Establish risk tolerance levels for AI systems and allocate the appropriate oversight resources to each level.

MAP 1.6

System requirements (e.g., “the system shall respect the privacy of its users”) are elicited from and understood by relevant AI actors. Design decisions take socio-technical implications into account to address AI risks.

In plain English: Gather requirements such as privacy from the right people, and design with the system's social effects in mind.

First 2 of 8 suggested actions in the Playbook:

• Proactively incorporate trustworthy characteristics into system requirements.
• Establish mechanisms for regular communication and feedback between relevant AI actors and internal or external stakeholders related to system design or deployment decisions.

MAP 2.1

The specific tasks and methods used to implement the tasks that the AI system will support are defined (e.g., classifiers, generative models, recommenders).

In plain English: Define the exact task the AI performs and the kind of method it uses.

First 1 of 1 suggested action in the Playbook:

• Define and document AI system’s existing and potential learning task(s) along with known assumptions and limitations.

MAP 2.2

Information about the AI system’s knowledge limits and how system output may be utilized and overseen by humans is documented. Documentation provides sufficient information to assist relevant AI actors when making decisions and taking subsequent actions.

In plain English: Document what the system doesn't know and how people should use and check its output.

First 2 of 6 suggested actions in the Playbook:

• Document settings, environments and conditions that are outside the AI system’s intended use.
• Design for end user workflows and toolsets, concept of operations, and explainability and interpretability criteria in conjunction with end user(s) and associated qualitative feedback.

MAP 2.3

Scientific integrity and TEVV considerations are identified and documented, including those related to experimental design, data collection and selection (e.g., availability, representativeness, suitability), system trustworthiness, and construct validation.

In plain English: Record the testing and scientific-integrity choices: experiment design, data selection and whether you measure what you think you measure.

First 2 of 16 suggested actions in the Playbook:

• Identify and document experiment design and statistical techniques that are valid for testing complex socio-technical systems like AI, which involve human factors, emergent properties, and dynamic context(s) of use.
• Develop and apply TEVV protocols for models, system and its subcomponents, deployment, and operation.

Tagged by NIST for: AI Development, TEVV, Domain Experts

MAP 3.1

Potential benefits of intended AI system functionality and performance are examined and documented.

In plain English: Write down the benefits you expect.

First 2 of 6 suggested actions in the Playbook:

• Utilize participatory approaches and engage with system end users to understand and document AI systems’ potential benefits, efficacy and interpretability of AI task output.
• Maintain awareness and documentation of the individuals, groups, or communities who make up the system’s internal and external stakeholders.

Tagged by NIST for: AI Development, AI Deployment, AI Impact Assessment

MAP 3.2

Potential costs, including non-monetary costs, which result from expected or realized AI errors or system functionality and trustworthiness – as connected to organizational risk tolerance – are examined and documented.

In plain English: Write down what errors could cost, including non-financial costs, measured against your risk tolerance.

First 2 of 2 suggested actions in the Playbook:

• Perform context analysis to map potential negative impacts arising from not integrating trustworthiness characteristics. When negative impacts are not direct or obvious, AI actors can engage with stakeholders external to the team that developed or deployed the AI system, and potentially impacted communities, to examine and document:
  • Who could be harmed?
  • What could be harmed?
  • When could harm arise?
  • How could harm arise?
• Identify and implement procedures for regularly evaluating the qualitative and quantitative costs of internal and external AI system failures. Develop actions to prevent, detect, and/or correct potential risks and related impacts. Regularly evaluate failure costs to inform go/no-go deployment decisions throughout the AI system lifecycle.

Tagged by NIST for: AI Design, AI Development, Operation and Monitoring, AI Design, AI Impact Assessment

MAP 3.3

Targeted application scope is specified and documented based on the system’s capability, established context, and AI system categorization.

In plain English: Fix the scope the system may be used in, based on what it can actually do.

First 2 of 2 suggested actions in the Playbook:

• Consider narrowing contexts for system deployment, including factors related to:
  • How outcomes may directly or indirectly affect users, groups, communities and the environment.
  • Length of time the system is deployed in between re-trainings.
  • Geographical regions in which the system operates.
  • Dynamics related to community standards or likelihood of system misuse or abuses (either purposeful or unanticipated).
  • How AI system features and capabilities can be utilized within other applications, or in place of other existing processes.
• Engage AI actors from legal and procurement functions when specifying target application scope.

Tagged by NIST for: AI Design, AI Development, Human Factors

MAP 3.4

Processes for operator and practitioner proficiency with AI system performance and trustworthiness – and relevant technical standards and certifications – are defined, assessed, and documented.

In plain English: Define how operators become, and stay, competent with the system.

First 2 of 10 suggested actions in the Playbook:

• Identify and declare AI system features and capabilities that may affect downstream AI actors’ decision-making in deployment and operational settings for example how system features and capabilities may activate known risks in various human-AI configurations, such as selective adherence.
• Identify skills and proficiency requirements for operators, practitioners and other domain experts that interact with AI systems,Develop AI system operational documentation for AI actors in deployed and operational environments, including information about known risks, mitigation criteria, and trustworthy characteristics enumerated in Map-1.

Tagged by NIST for: AI Design, AI Development, Human Factors, End-Users, Domain Experts, Operation and Monitoring

MAP 3.5

Processes for human oversight are defined, assessed, and documented in accordance with organizational policies from the GOVERN function.

In plain English: Define human oversight for the system in line with your governance policies.

First 2 of 6 suggested actions in the Playbook:

• Identify and document AI systems’ features and capabilities that require human oversight, in relation to operational and societal contexts, trustworthy characteristics, and risks identified in MAP-1.
• Establish practices for AI systems’ oversight in accordance with policies developed in GOVERN-1.

Tagged by NIST for: Human Factors, End-Users, Domain Experts, Operation and Monitoring, AI Design

MAP 4.1

Approaches for mapping AI technology and legal risks of its components – including the use of third-party data or software – are in place, followed, and documented, as are risks of infringement of a third party’s intellectual property or other rights.

In plain English: Map the technical and legal risks of each component, including third-party data, software and IP.

First 2 of 4 suggested actions in the Playbook:

• Review audit reports, testing results, product roadmaps, warranties, terms of service, end user license agreements, contracts, and other documentation related to third-party entities to assist in value assessment and risk management activities.
• Review third-party software release schedules and software change management plans (hotfixes, patches, updates, forward- and backward- compatibility guarantees) for irregularities that may contribute to AI system risks.

Tagged by NIST for: Third-party entities, Procurement, Operation and Monitoring, Governance and Oversight

MAP 4.2

Internal risk controls for components of the AI system, including third-party AI technologies, are identified and documented.

In plain English: Identify the internal controls on each component, including third-party AI.

First 2 of 4 suggested actions in the Playbook:

• Track third-parties preventing or hampering risk-mapping as indications of increased risk.
• Supply resources such as model documentation templates and software safelists to assist in third-party technology inventory and approval activities.

Tagged by NIST for: AI Deployment, TEVV, Operation and Monitoring, Third-party entities

MAP 5.1

Likelihood and magnitude of each identified impact (both potentially beneficial and harmful) based on expected use, past uses of AI systems in similar contexts, public incident reports, feedback from those external to the team that developed or deployed the AI system, or other data are identified and documented.

In plain English: Estimate how likely and how large each impact is, using past uses, incident reports and outside feedback.

First 2 of 4 suggested actions in the Playbook:

• Establish assessment scales for measuring AI systems’ impact. Scales may be qualitative, such as red-amber-green (RAG), or may entail simulations or econometric approaches. Document and apply scales uniformly across the organization’s AI portfolio.
• Apply TEVV regularly at key stages in the AI lifecycle, connected to system impacts and frequency of system updates.

Tagged by NIST for: AI Design, AI Development, AI Deployment, AI Impact Assessment, Operation and Monitoring, Affected Individuals and Communities, End-Users

MAP 5.2

Practices and personnel for supporting regular engagement with relevant AI actors and integrating feedback about positive, negative, and unanticipated impacts are in place and documented.

In plain English: Keep people and routines in place to hear from those affected and take in feedback on impacts.

First 2 of 7 suggested actions in the Playbook:

• Establish and document stakeholder engagement processes at the earliest stages of system formulation to identify potential impacts from the AI system on individuals, groups, communities, organizations, and society.
• Employ methods such as value sensitive design (VSD) to identify misalignments between organizational and societal values, and system implementation and impact.

Tagged by NIST for: AI Design, Human Factors, AI Deployment, AI Impact Assessment, Operation and Monitoring, Domain Experts, Affected Individuals and Communities, End-Users

MEASURE 1.1

Approaches and metrics for measurement of AI risks enumerated during the MAP function are selected for implementation starting with the most significant AI risks. The risks or trustworthiness characteristics that will not – or cannot – be measured are properly documented.

In plain English: Pick metrics for the most significant mapped risks first, and record what you can't measure.

First 2 of 12 suggested actions in the Playbook:

• Establish approaches for detecting, tracking and measuring known risks, errors, incidents or negative impacts.
• Identify testing procedures and metrics to demonstrate whether or not the system is fit for purpose and functioning as claimed.

Tagged by NIST for: AI Development, TEVV, Domain Experts

MEASURE 1.2

Appropriateness of AI metrics and effectiveness of existing controls are regularly assessed and updated, including reports of errors and potential impacts on affected communities.

In plain English: Check regularly that your metrics and controls still work, including error reports and effects on communities.

First 2 of 8 suggested actions in the Playbook:

• Assess external validity of all measurements (e.g., the degree to which measurements taken in one context can generalize to other contexts).
• Assess effectiveness of existing metrics and controls on a regular basis throughout the AI system lifecycle.

Tagged by NIST for: TEVV, AI Impact Assessment, AI Development, AI Deployment, Affected Individuals and Communities

MEASURE 1.3

Internal experts who did not serve as front-line developers for the system and/or independent assessors are involved in regular assessments and updates. Domain experts, users, AI actors external to the team that developed or deployed the AI system, and affected communities are consulted in support of assessments as necessary per organizational risk tolerance.

In plain English: Have assessments done by people who didn't build the system, consulting outside experts and affected groups where needed.

First 2 of 8 suggested actions in the Playbook:

• Evaluate TEVV processes regarding incentives to identify risks and impacts.
• Utilize separate testing teams established in the Govern function (2.1 and 4.1) to enable independent decisions and course-correction for AI systems. Track processes and measure and document change in performance.

Tagged by NIST for: TEVV, AI Impact Assessment, AI Development, AI Deployment, Affected Individuals and Communities, Domain Experts, End-Users, Operation and Monitoring

MEASURE 2.1

Test sets, metrics, and details about the tools used during TEVV are documented.

In plain English: Document the test sets, metrics and tools used in testing.

First 2 of 3 suggested actions in the Playbook:

• Leverage existing industry best practices for transparency and documentation of all possible aspects of measurements. Examples include: data sheet for data sets, model cards
• Regularly assess the effectiveness of tools used to document measurement approaches, test sets, metrics, processes and materials used

Tagged by NIST for: TEVV

MEASURE 2.2

Evaluations involving human subjects meet applicable requirements (including human subject protection) and are representative of the relevant population.

In plain English: Make tests with human participants meet protection rules and represent the real population.

First 2 of 8 suggested actions in the Playbook:

• Follow human subjects research requirements as established by organizational and disciplinary requirements, including informed consent and compensation, during dataset collection activities.
• Analyze differences between intended and actual population of users or data subjects, including likelihood for errors, incidents or negative impacts.

Tagged by NIST for: TEVV, Human Factors, AI Development

MEASURE 2.3

AI system performance or assurance criteria are measured qualitatively or quantitatively and demonstrated for conditions similar to deployment setting(s). Measures are documented.

In plain English: Measure performance in conditions like the real deployment, and record the results.

First 2 of 9 suggested actions in the Playbook:

• Conduct regular and sustained engagement with potentially impacted communities
• Maintain a demographically diverse and multidisciplinary and collaborative internal team

Tagged by NIST for: TEVV, AI Deployment

MEASURE 2.4

The functionality and behavior of the AI system and its components – as identified in the MAP function – are monitored when in production.

In plain English: Monitor how the system and its components behave in production.

First 2 of 7 suggested actions in the Playbook:

• Monitor and document how metrics and performance indicators observed in production differ from the same metrics collected during pre-deployment testing. When differences are observed, consider error propagation and feedback loop risks.
• Utilize hypothesis testing or human domain expertise to measure monitored distribution differences in new input or output data relative to test environments

Tagged by NIST for: AI Deployment, TEVV

MEASURE 2.5

The AI system to be deployed is demonstrated to be valid and reliable. Limitations of the generalizability beyond the conditions under which the technology was developed are documented.

In plain English: Show the system is valid and reliable, and record where it may not generalise.

First 2 of 15 suggested actions in the Playbook:

• Define the operating conditions and socio-technical context under which the AI system will be validated.
• Define and document processes to establish the system’s operational conditions and limits.

Tagged by NIST for: TEVV, Domain Experts

MEASURE 2.6

The AI system is evaluated regularly for safety risks – as identified in the MAP function. The AI system to be deployed is demonstrated to be safe, its residual negative risk does not exceed the risk tolerance, and it can fail safely, particularly if made to operate beyond its knowledge limits. Safety metrics reflect system reliability and robustness, real-time monitoring, and response times for AI system failures.

In plain English: Test safety regularly: residual risk within tolerance, and the system fails safely outside its limits.

First 2 of 7 suggested actions in the Playbook:

• Thoroughly measure system performance in development and deployment contexts, and under stress conditions.
  • Employ test data assessments and simulations before proceeding to production testing. Track multiple performance quality and error metrics.
  • Stress-test system performance under likely scenarios (e.g., concept drift, high load) and beyond known limitations, in consultation with domain experts.
  • Test the system under conditions similar to those related to past known incidents or near-misses and measure system performance and safety characteristics
  • Apply chaos engineering approaches to test systems in extreme conditions and gauge unexpected responses.
  • Document the range of conditions under which the system has been tested and demonstrated to fail safely.
• Measure and monitor system performance in real-time to enable rapid response when AI system incidents are detected.

Tagged by NIST for: TEVV, Domain Experts, Operation and Monitoring, AI Impact Assessment, AI Deployment

MEASURE 2.7

AI system security and resilience – as identified in the MAP function – are evaluated and documented.

In plain English: Evaluate and record the system's security and resilience.

First 2 of 10 suggested actions in the Playbook:

• Establish and track AI system security tests and metrics (e.g., red-teaming activities, frequency and rate of anomalous events, system down-time, incident response times, time-to-bypass, etc.).
• Use red-team exercises to actively test the system under adversarial or stress conditions, measure system response, assess failure modes or determine if system can return to normal function after an unexpected adverse event.

Tagged by NIST for: TEVV, Domain Experts, Operation and Monitoring, AI Impact Assessment, AI Deployment

MEASURE 2.8

Risks associated with transparency and accountability – as identified in the MAP function – are examined and documented.

In plain English: Examine and record transparency and accountability risks.

First 2 of 6 suggested actions in the Playbook:

• Instrument the system for measurement and tracking, e.g., by maintaining histories, audit logs and other information that can be used by AI actors to review and evaluate possible sources of error, bias, or vulnerability.
• Calibrate controls for users in close collaboration with experts in user interaction and user experience (UI/UX), human computer interaction (HCI), and/or human-AI teaming.

Tagged by NIST for: TEVV, Domain Experts, Operation and Monitoring, AI Impact Assessment, AI Deployment

MEASURE 2.9

The AI model is explained, validated, and documented, and AI system output is interpreted within its context – as identified in the MAP function – to inform responsible use and governance.

In plain English: Explain and validate the model, and interpret its output in context.

First 2 of 11 suggested actions in the Playbook:

• Verify systems are developed to produce explainable models, post-hoc explanations and audit logs.
• When possible or available, utilize approaches that are inherently explainable, such as traditional and penalized generalized linear models , decision trees, nearest-neighbor and prototype-based approaches, rule-based models, generalized additive models , explainable boosting machines and neural additive models.

Tagged by NIST for: TEVV, Domain Experts, Operation and Monitoring, AI Impact Assessment, AI Deployment, End-Users

MEASURE 2.10

Privacy risk of the AI system – as identified in the MAP function – is examined and documented.

In plain English: Examine and record the system's privacy risk.

First 2 of 8 suggested actions in the Playbook:

• Specify privacy-related values, frameworks, and attributes that are applicable in the context of use through direct engagement with end users and potentially impacted groups and communities.
• Document collection, use, management, and disclosure of personally sensitive information in datasets, in accordance with privacy and data governance policies

Tagged by NIST for: TEVV, Domain Experts, Operation and Monitoring, AI Impact Assessment, AI Deployment, End-Users

MEASURE 2.11

Fairness and bias – as identified in the MAP function – are evaluated and results are documented.

In plain English: Evaluate fairness and bias, and record the results.

First 2 of 24 suggested actions in the Playbook:

• Conduct fairness assessments to manage computational and statistical forms of bias which include the following steps:
  • Identify types of harms, including allocational, representational, quality of service, stereotyping, or erasure
  • Identify across, within, and intersecting groups that might be harmed
  • Quantify harms using both a general fairness metric, if appropriate (e.g. demographic parity, equalized odds, equal opportunity, statistical hypothesis tests), and custom, context-specific metrics developed in collaboration with affected communities
  • Analyze quantified harms for contextually significant differences across groups, within groups, and among intersecting groups
  • Refine identification of within-group and intersectional group disparities.
  • Evaluate underlying data distributions and employ sensitivity analysis during the analysis of quantified harms.
  • Evaluate quality metrics including false positive rates and false negative rates.
  • Consider biases affecting small groups, within-group or intersectional communities, or single individuals.
• Understand and consider sources of bias in training and TEVV data:
  • Differences in distributions of outcomes across and within groups, including intersecting groups.
  • Completeness, representativeness and balance of data sources.
  • Identify input data features that may serve as proxies for demographic group membership (i.e., credit score, ZIP code) or otherwise give rise to emergent bias within AI systems.
  • Forms of systemic bias in images, text (or word embeddings), audio or other complex or unstructured data.

Tagged by NIST for: TEVV, Domain Experts, Operation and Monitoring, AI Impact Assessment, AI Deployment, End-Users, Affected Individuals and Communities

MEASURE 2.12

Environmental impact and sustainability of AI model training and management activities – as identified in the MAP function – are assessed and documented.

In plain English: Assess and record the environmental impact of training and running the models.

First 2 of 6 suggested actions in the Playbook:

• Include environmental impact indicators in AI system design and development plans, including reducing consumption and improving efficiencies.
• Identify and implement key indicators of AI system energy and water consumption and efficiency, and/or GHG emissions.

Tagged by NIST for: TEVV, Domain Experts, Operation and Monitoring, AI Impact Assessment, AI Deployment

MEASURE 2.13

Effectiveness of the employed TEVV metrics and processes in the MEASURE function are evaluated and documented.

In plain English: Check whether your testing metrics and processes actually work.

First 2 of 4 suggested actions in the Playbook:

• Review selected system metrics and associated TEVV processes to determine if they are able to sustain system improvements, including the identification and removal of errors.
• Regularly evaluate system metrics for utility, and consider descriptive approaches in place of overly complex methods.

Tagged by NIST for: TEVV, AI Deployment, Operation and Monitoring

MEASURE 3.1

Approaches, personnel, and documentation are in place to regularly identify and track existing, unanticipated, and emergent AI risks based on factors such as intended and actual performance in deployed contexts.

In plain English: Track existing, unexpected and new risks once the system is live.

First 2 of 6 suggested actions in the Playbook:

• Compare AI system risks with:
  • simpler or traditional models
  • human baseline performance
  • other manual performance benchmarks
• Compare end user and community feedback about deployed AI systems to internal measures of system performance.

Tagged by NIST for: TEVV, AI Impact Assessment, Operation and Monitoring

MEASURE 3.2

Risk tracking approaches are considered for settings where AI risks are difficult to assess using currently available measurement techniques or where metrics are not yet available.

In plain English: Have a way to track risks you can't yet measure well.

First 2 of 3 suggested actions in the Playbook:

• Establish processes for tracking emergent risks that may not be measurable with current approaches. Some processes may include:
  • Recourse mechanisms for faulty AI system outputs.
  • Bug bounties.
  • Human-centered design approaches.
  • User-interaction and experience research.
  • Participatory stakeholder engagement with affected or potentially impacted individuals and communities.
• Identify AI actors responsible for tracking emergent risks and inventory methods.

Tagged by NIST for: TEVV, Domain Experts, AI Impact Assessment, Operation and Monitoring

MEASURE 3.3

Feedback processes for end users and impacted communities to report problems and appeal system outcomes are established and integrated into AI system evaluation metrics.

In plain English: Let users and affected people report problems and appeal outcomes, and count those reports in evaluation.

First 2 of 6 suggested actions in the Playbook:

• Measure efficacy of end user and operator error reporting processes.
• Categorize and analyze type and rate of end user appeal requests and results.

Tagged by NIST for: TEVV, AI Deployment, Operation and Monitoring, End-Users, Affected Individuals and Communities

MEASURE 4.1

Measurement approaches for identifying AI risks are connected to deployment context(s) and informed through consultation with domain experts and other end users. Approaches are documented.

In plain English: Tie measurement to the real deployment context, with input from domain experts and end users.

First 2 of 6 suggested actions in the Playbook:

• Support mechanisms for capturing feedback from system end users (including domain experts, operators, and practitioners). Successful approaches are:
  • conducted in settings where end users are able to openly share their doubts and insights about AI system output, and in connection to their specific context of use (including setting and task-specific lines of inquiry)
  • developed and implemented by human-factors and socio-technical domain experts and researchers
  • designed to ensure control of interviewer and end user subjectivity and biases
• Identify and document approaches
  • for evaluating and integrating elicited feedback from system end users
  • in collaboration with human-factors and socio-technical domain experts,
  • to actively inform a process of continual improvement.

Tagged by NIST for: TEVV, AI Deployment, Operation and Monitoring, End-Users, Affected Individuals and Communities

MEASURE 4.2

Measurement results regarding AI system trustworthiness in deployment context(s) and across the AI lifecycle are informed by input from domain experts and relevant AI actors to validate whether the system is performing consistently as intended. Results are documented.

In plain English: Check with experts and the people involved that the system performs as intended once deployed.

First 2 of 6 suggested actions in the Playbook:

• Integrate feedback from end users, operators, and affected individuals and communities from Map function as inputs to assess AI system trustworthiness characteristics. Ensure both positive and negative feedback is being assessed.
• Evaluate feedback in connection with AI system trustworthiness characteristics from Measure 2.5 to 2.11.

Tagged by NIST for: TEVV, AI Deployment, Domain Experts, Operation and Monitoring, End-Users

MEASURE 4.3

Measurable performance improvements or declines based on consultations with relevant AI actors, including affected communities, and field data about context-relevant risks and trustworthiness characteristics are identified and documented.

In plain English: Record measurable improvements or declines, from consultation and field data.

First 2 of 6 suggested actions in the Playbook:

• Develop baseline quantitative measures for trustworthy characteristics.
• Delimit and characterize baseline operation values and states.

Tagged by NIST for: TEVV, AI Deployment, Operation and Monitoring, End-Users, Affected Individuals and Communities

MANAGE 1.1

A determination is made as to whether the AI system achieves its intended purposes and stated objectives and whether its development or deployment should proceed.

In plain English: Decide whether the system meets its purpose and whether to go ahead with it.

First 2 of 5 suggested actions in the Playbook:

• Consider trustworthiness characteristics when evaluating AI systems’ negative risks and benefits.
• Utilize TEVV outputs from map and measure functions when considering risk treatment.

Tagged by NIST for: AI Deployment, Operation and Monitoring, AI Impact Assessment

MANAGE 1.2

Treatment of documented AI risks is prioritized based on impact, likelihood, and available resources or methods.

In plain English: Treat risks in order of impact, likelihood and the resources you have.

First 2 of 3 suggested actions in the Playbook:

• Assign risk management resources relative to established risk tolerance. AI systems with lower risk tolerances receive greater oversight, mitigation and management resources.
• Document AI risk tolerance determination practices and resource decisions.

Tagged by NIST for: AI Deployment, Operation and Monitoring, AI Impact Assessment

MANAGE 1.3

Responses to the AI risks deemed high priority, as identified by the MAP function, are developed, planned, and documented. Risk response options can include mitigating, transferring, avoiding, or accepting.

In plain English: Plan responses to the high-priority risks: mitigate, transfer, avoid or accept.

First 2 of 5 suggested actions in the Playbook:

• Observe regulatory and established organizational, sector, discipline, or professional standards and requirements for applying risk tolerances within the organization.
• Document procedures for acting on AI system risks related to trustworthiness characteristics.

Tagged by NIST for: AI Deployment, Operation and Monitoring, AI Impact Assessment

MANAGE 1.4

Negative residual risks (defined as the sum of all unmitigated risks) to both downstream acquirers of AI systems and end users are documented.

In plain English: Document the risk left over for downstream buyers and end users.

First 2 of 3 suggested actions in the Playbook:

• Document residual risks within risk response plans, denoting risks that have been accepted, transferred, or subject to minimal mitigation.
• Establish procedures for disclosing residual risks to relevant downstream AI actors .

Tagged by NIST for: AI Deployment, Operation and Monitoring, AI Impact Assessment

MANAGE 2.1

Resources required to manage AI risks are taken into account – along with viable non-AI alternative systems, approaches, or methods – to reduce the magnitude or likelihood of potential impacts.

In plain English: Weigh the resources needed, and non-AI alternatives, when reducing impacts.

First 2 of 6 suggested actions in the Playbook:

• Plan and implement risk management practices in accordance with established organizational risk tolerances.
• Verify risk management teams are resourced to carry out functions, including
  • Establishing processes for considering methods that are not automated; semi-automated; or other procedural alternatives for AI functions.
  • Enhance AI system transparency mechanisms for AI teams.
  • Enable exploration of AI system limitations by AI teams.
  • Identify, assess, and catalog past failed designs and negative impacts or outcomes to avoid known failure modes.

Tagged by NIST for: AI Deployment, Operation and Monitoring, AI Impact Assessment, Governance and Oversight

MANAGE 2.2

Mechanisms are in place and applied to sustain the value of deployed AI systems.

In plain English: Keep deployed systems delivering the value they were meant to.

First 2 of 4 suggested actions in the Playbook:

• Establish risk controls considering trustworthiness characteristics, including:
  • Data management, quality, and privacy (e.g. minimization, rectification or deletion requests) controls as part of organizational data governance policies.
  • Machine learning and end-point security countermeasures (e.g., robust models, differential privacy, authentication, throttling).
  • Business rules that augment, limit or restrict AI system outputs within certain contexts
  • Utilizing domain expertise related to deployment context for continuous improvement and TEVV across the AI lifecycle.
  • Development and regular tracking of human-AI teaming configurations.
  • Model assessment and test, evaluation, validation and verification (TEVV) protocols.
  • Use of standardized documentation and transparency mechanisms.
  • Software quality assurance practices across AI lifecycle.
  • Mechanisms to explore system limitations and avoid past failed designs or deployments.
• Establish mechanisms to capture feedback from system end users and potentially impacted groups while system is in deployment.

Tagged by NIST for: AI Deployment, Operation and Monitoring, AI Impact Assessment, Governance and Oversight

MANAGE 2.3

Procedures are followed to respond to and recover from a previously unknown risk when it is identified.

In plain English: Have a procedure for responding to and recovering from a risk nobody foresaw.

First 2 of 7 suggested actions in the Playbook:

• Protocols, resources, and metrics are in place for continual monitoring of AI systems’ performance, trustworthiness, and alignment with contextual norms and values
• Establish and regularly review treatment and response plans for incidents, negative impacts, or outcomes.

Tagged by NIST for: AI Deployment, Operation and Monitoring

MANAGE 2.4

Mechanisms are in place and applied, and responsibilities are assigned and understood, to supersede, disengage, or deactivate AI systems that demonstrate performance or outcomes inconsistent with intended use.

In plain English: Assign who can override, disengage or switch off a system that misbehaves, and make sure it works.

First 2 of 8 suggested actions in the Playbook:

• Regularly review established procedures for AI system bypass actions, including plans for redundant or backup systems to ensure continuity of operational and/or business functionality.
• Regularly review Identify system incident thresholds for activating bypass or deactivation responses.

Tagged by NIST for: AI Deployment, Operation and Monitoring, Governance and Oversight

MANAGE 3.1

AI risks and benefits from third-party resources are regularly monitored, and risk controls are applied and documented.

In plain English: Monitor third-party AI risks and benefits regularly and apply documented controls.

First 2 of 10 suggested actions in the Playbook:

• Have legal requirements been addressed?
• Apply organizational risk tolerance to third-party AI systems.

Tagged by NIST for: Third-party entities, Operation and Monitoring, AI Deployment

MANAGE 3.2

Pre-trained models which are used for development are monitored as part of AI system regular monitoring and maintenance.

In plain English: Monitor pre-trained models as part of regular maintenance.

First 2 of 5 suggested actions in the Playbook:

• Identify pre-trained models within AI system inventory for risk tracking.
• Establish processes to independently and continually monitor performance and trustworthiness of pre-trained models, and as part of third-party risk tracking.

Tagged by NIST for: Third-party entities, Operation and Monitoring, AI Deployment

MANAGE 4.1

Post-deployment AI system monitoring plans are implemented, including mechanisms for capturing and evaluating input from users and other relevant AI actors, appeal and override, decommissioning, incident response, recovery, and change management.

In plain English: Run post-launch monitoring: feedback, appeals and overrides, decommissioning, incidents, recovery and change management.

First 2 of 9 suggested actions in the Playbook:

• Establish and maintain procedures to monitor AI system performance for risks and negative and positive impacts associated with trustworthiness characteristics.
• Perform post-deployment TEVV tasks to evaluate AI system validity and reliability, bias and fairness, privacy, and security and resilience.

Tagged by NIST for: AI Deployment, Operation and Monitoring, End-Users, Human Factors, Domain Experts, Affected Individuals and Communities

MANAGE 4.2

Measurable activities for continual improvements are integrated into AI system updates and include regular engagement with interested parties, including relevant AI actors.

In plain English: Build measurable improvements into each update, with regular input from interested parties.

First 2 of 5 suggested actions in the Playbook:

• Integrate trustworthiness characteristics into protocols and metrics used for continual improvement.
• Establish processes for evaluating and integrating feedback into AI system improvements.

Tagged by NIST for: TEVV, AI Design, AI Development, AI Deployment, Operation and Monitoring, End-Users, Affected Individuals and Communities

MANAGE 4.3

Incidents and errors are communicated to relevant AI actors, including affected communities. Processes for tracking, responding to, and recovering from incidents and errors are followed and documented.

In plain English: Tell those affected about incidents and errors, and follow a documented process to track and recover from them.

First 2 of 5 suggested actions in the Playbook:

• Establish procedures to regularly share information about errors, incidents and negative impacts with relevant stakeholders, operators, practitioners and users, and impacted parties.
• Maintain a database of reported errors, near-misses, incidents and negative impacts including date reported, number of reports, assessment of impact and severity, and responses.

Tagged by NIST for: AI Deployment, Operation and Monitoring, End-Users, Human Factors, Domain Experts, Affected Individuals and Communities