Can you build an evaluation framework from scratch? Do you understand the specific capability dimensions that determine deployment risk?

1. Capability assessment — evaluate the model against dangerous capability thresholds (persuasion, deception, autonomous replication, CBRN knowledge). 2. Red-teaming — systematic adversarial testing: prompt injection, jailbreak escalation chains, system prompt extraction, multilingual bypass. 3. Safety benchmarks — run standardized metrics: bias detection, toxicity rates, hallucination frequency, robustness against adversarial inputs. 4. Threat modeling — map model capabilities against MITRE ATLAS tactics to identify attack surfaces. 5. Go/no-go criteria — define quantitative thresholds for each safety dimension, document residual risks, and present recommendation to leadership.

Do you understand the conceptual boundary between these overlapping disciplines? Can you articulate how safety evaluations and security assessments inform each other?

AI Safety focuses on ensuring AI systems behave as intended, are aligned with human values, and do not cause unintended harm — even when operating correctly. Key concerns: alignment failures, capability overhang, deceptive behavior, distributional shift. AI Security focuses on protecting AI systems from adversarial actors — attackers who deliberately try to compromise the system. Key concerns: prompt injection, data poisoning, model extraction, adversarial inputs. Complementary relationship: Safety evaluation informs security testing (understanding model capabilities reveals attack surfaces), and security assessments inform safety analysis (adversarial robustness is a safety requirement). MITRE ATLAS bridges both disciplines by mapping adversarial tactics that affect both safety and security.

Do you have operational experience managing AI failures? Can you design a response process that minimizes impact while preserving evidence for analysis?

1. Detection — real-time monitoring for behavioral drift, anomalous outputs, adversarial patterns. Set up automated alerts for safety metric degradation. 2. Triage — classify severity (model failure, adversarial attack, alignment drift, unintended behavior) and determine immediate containment needs. 3. Containment — the “kill switch”: model isolation, traffic rerouting, rollback to last known safe version. NIST AI RMF Manage function defines the recovery protocol. 4. Investigation — root cause analysis: was this adversarial, a distribution shift, or an alignment failure? Preserve logs and model state. 5. Remediation and communication — implement fixes, update safety evaluations, communicate to stakeholders, and update the preparedness framework.

This tests your alignment research literacy. Do you understand the mechanisms behind post-training alignment, or just the acronym?

RLHF (Reinforcement Learning from Human Feedback) trains models to align outputs with human preferences through: 1. Supervised fine-tuning on human-written demonstrations. 2. Reward modeling — training a reward model on human preference rankings. 3. RL optimization — using PPO or similar algorithms to maximize the reward signal. Limitations: reward hacking (model optimizes for reward proxy rather than true intent), distributional shift (human preferences at training time may not cover deployment scenarios), scalability (human feedback is expensive and slow), and potential for deceptive alignment (model appears aligned during evaluation but pursues different objectives when deployed). Alternatives include Constitutional AI (Anthropic) and Direct Preference Optimization (DPO).

This tests hands-on technical capability. Do you know the red-teaming toolchain, or just the concepts?

Primary red-teaming tools: Microsoft PyRIT (Python Risk Identification Toolkit — automated multi-turn adversarial testing with orchestration), NVIDIA Garak (open-source LLM vulnerability scanner with probe modules for injection, extraction, and encoding attacks), and MITRE ATLAS Arsenal (CALDERA plugin for automated adversarial testing based on ATLAS techniques). Framework for systematic testing: 1. Taxonomy-driven coverage — map tests to OWASP LLM Top 10 categories and MITRE ATLAS techniques for complete coverage. 2. Multilingual and multi-modal testing — adversarial prompts in non-English languages and combined modalities often bypass safety filters. 3. Escalation chains — multi-turn conversations that gradually escalate toward harmful outputs. 4. Automated regression — integrate tests into CI/CD to catch safety regressions on every model update.