Do you know the specific legal requirements, or just the general concept of bias testing? NYC LL 144 is an enforceable mandate with precise calculation requirements — this tests your operational knowledge.

NYC LL 144 applies to automated employment decision tools (AEDTs) used in hiring or promotion decisions. Structure: 1. Scope definition — identify the AEDT, confirm it meets the legal definition (used to substantially assist or replace discretionary decision-making). 2. Data collection — gather selection rate or scoring rate data by race/ethnicity and sex categories as they appear in historical records. 3. Impact ratio calculation — apply the four-fifths (80%) rule; compare each group’s rate against the most-selected group. 4. Intersectional analysis — LL 144 requires intersectional race/sex combinations, not just individual categories. 5. Public disclosure — results must be publicly posted on the employer’s website. Non-compliance: $500–$1,500 per violation per day.

Can you articulate the full mitigation toolkit and the decision logic for selecting between approaches? Hiring managers want practitioners who know what each technique does and what it costs.

Three intervention stages: 1. Pre-processing (data level) — techniques like reweighing (adjust sample weights to reduce bias), disparate impact remover (edit feature values while preserving rank order), and data augmentation. Choose when you have access to training data and want to address bias at the source. Lower implementation risk, but requires data access. 2. In-processing (training level) — techniques like adversarial debiasing (adds fairness objective to the training loss function) and prejudice remover. Choose when you control the model training process. Can improve fairness and accuracy simultaneously in some cases. 3. Post-processing (output level) — techniques like equalized odds post-processing and calibrated equalized odds (adjusting decision thresholds per group). Choose when you don’t have access to model internals, or as a final step after other techniques. Each stage involves accuracy-fairness trade-offs that must be documented.

This is a real and frequent challenge. Do you understand why metrics conflict, and can you make principled decisions about which one governs in a given context?

Fairness metric conflicts are mathematically expected — researchers have formally proved that demographic parity, equalized odds, and calibration cannot all be satisfied simultaneously except in degenerate cases (this is known as the impossibility theorem of fairness). Your response framework: 1. Establish the decision context — what is the harm of a false positive vs. a false negative for each group? High-stakes decisions (criminal justice, medical triage) require different metric priorities than lower-stakes ones. 2. Identify the regulatory standard — NYC LL 144 specifies impact ratios; fair lending uses the four-fifths rule. External mandates narrow the metric choice. 3. Document the trade-off explicitly — no metric is neutral; your job is to make the trade-off visible and defensible, not to find a conflict-free answer. 4. Involve legal and ethics stakeholders — the choice of governing metric is a policy decision, not purely technical.

This tests your cross-functional communication and influence-without-authority skills. The ability to navigate organizational resistance to fairness findings is often what separates effective practitioners from technically competent ones.

Framework for presenting contested findings: 1. Lead with methodology, not verdict — walk through what you measured, how you measured it, and why the metric is appropriate before stating the conclusion. Makes it harder to dismiss as subjective. 2. Use SHAP to show the mechanism — feature importance visualization makes proxy discrimination concrete. When engineers can see which features are driving disparate outcomes, skepticism often converts to problem-solving mode. 3. Quantify the risk, not just the harm — frame in terms engineers respond to: regulatory exposure ($500–$1,500/day for LL 144 non-compliance), reputational risk, potential downstream liability. 4. Propose a joint mitigation experiment — rather than issuing a finding, invite the team to test a mitigation approach together. Collaborative investigation is more effective than adversarial auditing.

Do you understand the specific regulatory framework you’ll be implementing, or just the general concept? Employers in regulated industries need practitioners with operational EU AI Act knowledge.

The EU AI Act designates certain AI applications as “high-risk” based on use case and sector (Annex III: employment, critical infrastructure, education, law enforcement, etc.). For high-risk systems: 1. Data governance requirements (Article 10) — training, validation, and testing datasets must be examined for possible biases; data must be relevant, representative, free of errors, and complete to the extent possible. 2. Technical documentation (Article 11) — providers must document bias testing methodology and results. 3. Transparency and human oversight (Articles 13–14) — systems must be designed to allow human oversight, particularly where outcomes affect individuals. 4. Fundamental rights impact assessment — deployers of high-risk AI in certain contexts must conduct assessments covering bias and discrimination risks. In practice: your bias audit reports become compliance documentation that could be reviewed by national supervisory authorities.