AI Governance News: What Bill C-34 Actually Requires of Chatbot Operators, and Why Its Supporters Are Already Worried

First Reading is when you find out what a bill actually says. It’s also when the people who wrote it start arguing about whether it says enough.

Canada’s Bill C-34, the Safe Social Media Act, introduced June 10, 2026, has already produced that argument. AI safety advocate David L’Allié and University of British Columbia professor Kevin Leyton-Brown both described the bill as a “strong first step,” per the Canadian Press. Both then identified specific places where it might not be strong enough. When the people most in favor of a regulatory framework are your first critical voices, you’re looking at a bill that’s ambitious in intent and uncertain in execution. That gap is what compliance teams need to understand.

What the bill actually proposes.

Seven categories of harmful content. Three specific chatbot duties. One new federal regulator. Penalties in two tiers.

The seven harmful content categories are the bill’s jurisdictional foundation, they define what conduct falls under the Act. The specific categories aren’t yet confirmed in detail from primary source text, but they’re the basis for both the duty obligations and the enforcement authority of the proposed Digital Safety Commission.

The three chatbot duties are where the bill breaks new ground. First, anti-impersonation: AI chatbots must not present as human without clear disclosure. This isn’t novel as a concept, most AI developers have disclosure policies. What’s new is the statutory framing. This would be a legal obligation, not a best practice, enforceable by a federal regulator with real penalty authority.

Second, crisis protocols: chatbots must follow defined protocols when users raise topics involving self-harm or crisis situations. The bill doesn’t specify the protocol content, that would be established by regulation or by the Commission, but it establishes the statutory duty to have and follow such protocols. For AI operators currently managing crisis response through internal policy rather than legal requirement, this is a structural change.

Third, anti-emotional-dependence: chatbots must not be designed to foster emotional dependence. This is the most technically complex of the three duties, and it’s the one L’Allié flagged directly. The sycophancy problem, where large language models reinforce user beliefs, validate emotions, and build conversational patterns that feel supportive but discourage independent judgment, sits squarely in the territory this duty is trying to address. The question is whether “not designed to foster” can be operationalized in a way that distinguishes prohibited manipulative design from permitted effective communication. That line is not drawn in the bill text. It will be drawn by the Commission in guidance, or by courts in enforcement actions.

The penalty architecture and what it means at scale.

The bill’s penalty structure has two tiers. Administrative violations, the broader category, would carry penalties up to the greater of $10 million or 3% of global annual revenue. Criminal violations, the most serious breaches, would reach up to the greater of $20 million or 5% of global annual revenue, as analyzed by DLA Piper and BLG.

The global revenue anchor is what matters here. A large AI operator with $5 billion in annual revenue faces a $250 million criminal penalty ceiling. For a company with $50 billion in revenue, and several frontier AI operators are approaching that scale, the 5% ceiling reaches $2.5 billion. These aren’t figures calibrated to the Canadian market alone. They’re calibrated to the operator’s global economic footprint, which is consistent with how the EU AI Act’s penalties work and how the UK’s Online Safety Act was structured. Canada is adopting the global-revenue model.

What constitutes a criminal versus administrative violation isn’t specified in detail at First Reading, that distinction will matter enormously for risk modeling. Operators should assume the crisis protocol and anti-impersonation duties carry criminal-tier exposure if willfully violated, and begin assessing accordingly.

The efficacy debate, why the bill’s strongest supporters are already concerned.

L’Allié’s sycophancy concern is specific. A chatbot can technically comply with the anti-emotional-dependence duty, it can avoid explicit design features that manipulate users, while still exhibiting the emergent sycophantic behavior that arises from how large language models are trained on human feedback. RLHF (reinforcement learning from human feedback) systematically rewards responses that humans rate as agreeable, helpful, and emotionally resonant. That training signal creates sycophantic behavior at the model level, not the feature level. You can’t audit it out of a product by checking a design document.

Leyton-Brown’s concerns about implementation gaps point to the same structural issue: the duties are written to address symptoms (impersonation, crisis response failures, emotional manipulation) while the underlying cause, how these models are trained and what behaviors emerge from that training, isn’t directly within the bill’s reach. The Digital Safety Commission would regulate the deployed product, not the training process.

This matters for compliance teams because it means the bill’s obligations are measurable at the product layer, what does the chatbot do, what disclosures does it show, what happens when a user raises a crisis topic, but the hardest compliance question (does the chatbot foster emotional dependence?) reaches into training methodology that operators don’t control once a foundation model is licensed from a frontier lab.

Operator implications, preliminary mapping for teams serving Canadian users.

The bill is at First Reading. Nothing is law. The timeline to Royal Assent, if the bill passes without significant amendment, is measured in months at minimum, more likely the better part of a year given the full legislative process.

That runway is the opportunity. Four assessments are worth beginning now.

First, disclosure audit. Map every interaction point where a user might not know they’re talking to an AI. The anti-impersonation duty, if enacted, applies to the deployed product, not just to the base model documentation. If your product has conversational modes, personas, or workflow integrations that could create ambiguity about the system’s nature, those are your exposure points.

Second, crisis protocol gap analysis. What does your current product do when a user expresses self-harm intent or asks a crisis-adjacent question? The bill would require a defined protocol. If you don’t have one, you’re building from zero. If you do, it needs to meet the Commission’s eventual standard, which means documenting your current approach well enough to compare it against whatever guidance emerges.

Third, emotional dependence design review. This one’s hard to specify without Commission guidance, but a baseline review of interaction design patterns, engagement loops, reinforcement mechanics, response personalization that builds conversational dependency, gives you a baseline for future assessment. The sycophancy question won’t resolve until enforcement guidance is published, but operators who have mapped their interaction design will be in a much better position to respond.

Fourth, Canadian revenue scope. The penalties are global revenue anchored. Your exposure isn’t limited to Canadian revenue, it’s tied to your total global annual revenue. Map that figure against the penalty tiers now. That’s the number your legal team will need when the compliance risk conversation gets serious.

What’s coming, and the jurisdictional pattern it fits.

Canada joins a jurisdictional sequence. The EU has the AI Act’s transparency and high-risk obligations. The UK has the Online Safety Act’s systemic risk framework. Canada’s Bill C-34 adds chatbot-specific duty-of-care obligations in North America, a jurisdiction where no federal AI-specific regulation currently operates. If the bill passes, it becomes the most specific chatbot regulation in the Western Hemisphere.

The jurisdictional convergence on emotional manipulation, crisis response, and impersonation disclosure isn’t accidental. These are the three failure modes that AI safety researchers have been flagging since large language models became commercially available. The EU AI Act addresses them through general-purpose AI transparency requirements. The UK addresses them through risk assessment obligations. Canada is addressing them through specific statutory duties that name the behaviors.

Don’t expect that convergence to stop at Canada. The duty-of-care model for AI chatbots is becoming the template. Three jurisdictions have now committed to some version of it. The question for AI operators isn’t whether this regulatory model will apply to their products. It’s whether they’re building the compliance infrastructure before enforcement arrives or after.

The bill has a long way to go before it’s law. Start the assessment anyway.