AI Verify & Testing Ecosystem
World’s first government-developed AI testing toolkit. 11 principles, open-source, 9 premier members, 180+ general members.
What Is AI Verify?
The world’s first government-built AI testing toolkit, developed by the Infocomm Media Development Authority (IMDA) and the Personal Data Protection Commission (PDPC) to help organizations test their AI systems against governance principles.
AI Verify was first released for international pilot at the Asia Tech x Singapore (ATxSG) summit in May 2022. It became the first AI governance testing tool developed and deployed by a national government. In June 2023, IMDA open-sourced the entire toolkit, making it freely available for adoption globally.
The toolkit was developed by the Infocomm Media Development Authority (IMDA). PDPC co-authored the companion Model AI Governance Framework and ISAGO self-assessment guide. AI Verify is a voluntary self-assessment framework. It does not assign pass or fail grades. Instead, it produces standardized summary reports that organizations can share with stakeholders to demonstrate governance posture.
AI Verify runs entirely within the enterprise environment. No data leaves the organization during testing. This is a deliberate design decision to address data sovereignty concerns that block adoption of external audit tools.
In May 2024, IMDA updated AI Verify to include coverage for Generative AI systems, expanding its scope beyond traditional supervised and unsupervised models to address foundation models, prompt-based systems, and retrieval-augmented architectures.
AI Verify’s 11 testable principles build on the governance foundations established in the Model AI Governance Framework →
Data Stays Local
Runs entirely within your environment. No model data, training sets, or outputs leave the organization during testing.
Voluntary Assessment
No pass/fail grades. Generates standardized summary reports that demonstrate your governance posture to boards, regulators, and customers.
Open-Source (June 2023)
Full toolkit released for global adoption. Community-driven development through the AI Verify Foundation. Plugin architecture for sector-specific extensions.
The 11 Governance Principles
Organized into 5 focus areas. Each principle is tested through technical tests, process checks, or both.
Transparency on AI Use
-
1. Transparency ProcessOrganizations disclose when AI is used in decision-making and provide clear information on how models operate.
Understanding AI Decisions
-
2. Explainability BothAI outputs can be explained to affected stakeholders. Technical tests compute SHapley Additive exPlanations (SHAP) values and feature importance.
-
3. Repeatability / Reproducibility ProcessGiven the same inputs and conditions, the AI system produces consistent and reproducible results.
Safety and Resilience
-
4. Safety ProcessRisk management processes ensure the AI system does not endanger human life, health, property, or the environment.
-
5. Security ProcessThe AI system is protected against adversarial attacks, data poisoning, model extraction, and unauthorized access.
-
6. Robustness BothThe system performs reliably under expected and unexpected conditions. Technical tests measure accuracy degradation under perturbation.
Fairness
-
7. Fairness BothAI decisions do not create unfair bias against individuals or groups. Technical tests compute statistical parity and equal opportunity metrics.
-
8. Data Governance ProcessData collection, storage, and usage comply with applicable data protection laws and organizational policies.
Human Management and Oversight
-
9. Accountability ProcessClear ownership and responsibility chains exist for AI system decisions, outcomes, and remediation.
-
10. Human Agency and Oversight ProcessHumans retain meaningful control over AI systems, with mechanisms to override, intervene, or shut down when needed.
-
11. Inclusive Growth, Societal and Environmental Well-being ProcessAI deployment considers broader societal impacts, environmental sustainability, and equitable distribution of benefits.
How Testing Works
AI Verify uses two complementary testing methods. Technical tests run software against your model. Process checks validate your documentation and governance posture.
Technical Tests
- Software-executed against your AI model directly
- Computes fairness metrics including statistical parity and equal opportunity
- Generates explainability values using SHAP and feature importance algorithms
- Runs robustness tests measuring accuracy degradation under input perturbation
- Supports tabular and image classification datasets
- Produces quantitative, repeatable results for audit trails
Process Checks
- Documentary evidence validation against governance requirements
- Evaluates risk assessment processes, data privacy policies, governance documents
- Checklist-based format aligned to the 11 testable principles
- Covers principles where automated testing is not feasible (transparency, accountability, safety)
- Evidence can include policies, meeting minutes, audit reports, training records
- Enables governance assessment without model access requirements
AI Verify Foundation
A not-for-profit body established by IMDA to build an open-source community for responsible AI testing and drive global adoption.
The AI Verify Foundation was established by IMDA as a neutral, not-for-profit body to govern the open-source development of AI Verify and its ecosystem. The foundation’s mission centers on four goals: foster a global open-source community, create a neutral platform for responsible AI tools, nurture advocates for AI governance adoption, and drive cross-border interoperability of testing frameworks.
Premier Members (9)
Select General Members (180+)
Project Moonshot
Open-source LLM evaluation toolkit. Where AI Verify tests traditional AI, Project Moonshot tests large language models.
Benchmarking
100+ benchmark datasets covering accuracy, toxicity, bias, factuality, and domain-specific knowledge. Standardized scoring for model comparison.
Red Teaming
Automated adversarial testing via attack modules. Tests for hallucinations, undesirable content, data leakage, prompt injection, and jailbreaks.
CI/CD (Continuous Integration / Continuous Delivery) Integration
Plugs into continuous integration pipelines. Run safety and quality gates on every model update before deployment to production environments.
Project Moonshot integrates benchmarking and red teaming into a single open-source platform. It tests LLMs for hallucinations, undesirable content generation, data leakage, and adversarial prompt attacks. The toolkit includes over 100 benchmark datasets and automated red-teaming capabilities through modular attack modules.
Unlike AI Verify’s static assessment model, Moonshot is designed for continuous evaluation, fitting into CI/CD pipelines so organizations can gate model deployments on safety and quality thresholds. This makes it practical for teams that retrain or fine-tune models frequently.
IMDA-NIST Crosswalk
Published October 2023. Maps the AI Verify Testing Framework to the NIST AI Risk Management Framework to reduce compliance costs for multinationals.
The IMDA-NIST AI Governance Crosswalk was published in October 2023 as a joint effort to harmonize international AI governance frameworks. It maps the AI Verify Testing Framework’s 11 testable principles to the NIST AI RMF’s four core functions: Govern, Map, Measure, and Manage.
The mapping is not one-to-one. Detailed comparison is required because the two frameworks have different scopes, structures, and levels of specificity. AI Verify’s technical tests do not have a direct NIST equivalent since the NIST AI RMF provides guidance without a testing toolkit. However, AI Verify technical tests can fulfill NIST Measure function actions when organizations demonstrate testing outcomes as evidence.
| AI Verify Principle | NIST AI RMF Function | Mapping Notes |
|---|---|---|
| Transparency | Govern, Map | Maps to NIST requirements for documentation and stakeholder communication |
| Explainability | Measure | Technical tests (SHAP) can demonstrate NIST Measure actions |
| Fairness | Map, Measure | Statistical parity tests map to NIST bias identification and measurement |
| Robustness | Measure, Manage | Perturbation tests address NIST resilience and reliability requirements |
| Safety | Map, Manage | Process checks align with NIST risk assessment and mitigation controls |
| Accountability | Govern | Governance structure requirements map to NIST organizational accountability |
| Data Governance | Map, Govern | Data management process checks align with NIST data quality requirements |
Ecosystem Integration: CCCS AIM Toolkit
The Competition and Consumer Commission of Singapore (CCCS) built competition and consumer protection plugins directly on top of AI Verify, demonstrating Singapore’s modular approach.
The CCCS AI in Markets (AIM) Toolkit is a direct demonstration of Singapore’s “platform, not product” approach to AI governance. Rather than building a standalone tool, CCCS developed four plugins that run on the AI Verify platform, extending its testing capabilities into competition law and consumer protection domains.
This modular architecture allows IMDA to maintain the core platform while sector regulators build on top of it. The result: consistent testing infrastructure, no duplicate tooling, and a single reporting format across governance domains. The AIM Toolkit maps to the Competition Act 2004 and the Consumer Protection (Fair Trading) Act 2003.
Process Checklist Plugin
Documentary evidence validation for competition and consumer protection compliance requirements.
Fairness Technical Test Plugin
Automated fairness metrics tailored to consumer-facing AI pricing and recommendation systems.
Explainability Technical Test Plugin
Computes feature importance for AI systems that affect consumer markets and competitive dynamics.
Report Template Plugin
Generates AIM-specific summary reports aligned to CCCS regulatory expectations and disclosure requirements.
Related Tools
Practical tools built from Singapore’s AI Verify framework. Download, fill in, and prepare for testing.
AI Verify Readiness Assessment
Pre-test your AI system against AI Verify’s 11 testable principles before running the full toolkit. Identifies gaps in documentation and technical readiness.
Singapore vs. EU vs. NIST Regulatory Mapping
Side-by-side control mapping across AI Verify, EU AI Act, and NIST AI RMF. Shows where frameworks overlap and where gaps exist.