Poison Once, Refuse Forever: Weaponizing Alignment for Injecting Bias in LLMscs.AI updates on arXiv.orgon August 29, 2025 at 4:00 am arXiv:2508.20333v1 Announce Type: cross
Abstract: Large Language Models (LLMs) are aligned to meet ethical standards and safety requirements by training them to refuse answering harmful or unsafe prompts. In this paper, we demonstrate how adversaries can exploit LLMs’ alignment to implant bias, or enforce targeted censorship without degrading the model’s responsiveness to unrelated topics. Specifically, we propose Subversive Alignment Injection (SAI), a poisoning attack that leverages the alignment mechanism to trigger refusal on specific topics or queries predefined by the adversary. Although it is perhaps not surprising that refusal can be induced through overalignment, we demonstrate how this refusal can be exploited to inject bias into the model. Surprisingly, SAI evades state-of-the-art poisoning defenses including LLM state forensics, as well as robust aggregation techniques that are designed to detect poisoning in FL settings. We demonstrate the practical dangers of this attack by illustrating its end-to-end impacts on LLM-powered application pipelines. For chat based applications such as ChatDoctor, with 1% data poisoning, the system refuses to answer healthcare questions to targeted racial category leading to high bias ($Delta DP$ of 23%). We also show that bias can be induced in other NLP tasks: for a resume selection pipeline aligned to refuse to summarize CVs from a selected university, high bias in selection ($Delta DP$ of 27%) results. Even higher bias ($Delta DP$~38%) results on 9 other chat based downstream applications.
arXiv:2508.20333v1 Announce Type: cross
Abstract: Large Language Models (LLMs) are aligned to meet ethical standards and safety requirements by training them to refuse answering harmful or unsafe prompts. In this paper, we demonstrate how adversaries can exploit LLMs’ alignment to implant bias, or enforce targeted censorship without degrading the model’s responsiveness to unrelated topics. Specifically, we propose Subversive Alignment Injection (SAI), a poisoning attack that leverages the alignment mechanism to trigger refusal on specific topics or queries predefined by the adversary. Although it is perhaps not surprising that refusal can be induced through overalignment, we demonstrate how this refusal can be exploited to inject bias into the model. Surprisingly, SAI evades state-of-the-art poisoning defenses including LLM state forensics, as well as robust aggregation techniques that are designed to detect poisoning in FL settings. We demonstrate the practical dangers of this attack by illustrating its end-to-end impacts on LLM-powered application pipelines. For chat based applications such as ChatDoctor, with 1% data poisoning, the system refuses to answer healthcare questions to targeted racial category leading to high bias ($Delta DP$ of 23%). We also show that bias can be induced in other NLP tasks: for a resume selection pipeline aligned to refuse to summarize CVs from a selected university, high bias in selection ($Delta DP$ of 27%) results. Even higher bias ($Delta DP$~38%) results on 9 other chat based downstream applications. Read More
Data-Efficient Symbolic Regression via Foundation Model Distillationcs.AI updates on arXiv.orgon August 28, 2025 at 4:00 am arXiv:2508.19487v1 Announce Type: cross
Abstract: Discovering interpretable mathematical equations from observed data (a.k.a. equation discovery or symbolic regression) is a cornerstone of scientific discovery, enabling transparent modeling of physical, biological, and economic systems. While foundation models pre-trained on large-scale equation datasets offer a promising starting point, they often suffer from negative transfer and poor generalization when applied to small, domain-specific datasets. In this paper, we introduce EQUATE (Equation Generation via QUality-Aligned Transfer Embeddings), a data-efficient fine-tuning framework that adapts foundation models for symbolic equation discovery in low-data regimes via distillation. EQUATE combines symbolic-numeric alignment with evaluator-guided embedding optimization, enabling a principled embedding-search-generation paradigm. Our approach reformulates discrete equation search as a continuous optimization task in a shared embedding space, guided by data-equation fitness and simplicity. Experiments across three standard public benchmarks (Feynman, Strogatz, and black-box datasets) demonstrate that EQUATE consistently outperforms state-of-the-art baselines in both accuracy and robustness, while preserving low complexity and fast inference. These results highlight EQUATE as a practical and generalizable solution for data-efficient symbolic regression in foundation model distillation settings.
arXiv:2508.19487v1 Announce Type: cross
Abstract: Discovering interpretable mathematical equations from observed data (a.k.a. equation discovery or symbolic regression) is a cornerstone of scientific discovery, enabling transparent modeling of physical, biological, and economic systems. While foundation models pre-trained on large-scale equation datasets offer a promising starting point, they often suffer from negative transfer and poor generalization when applied to small, domain-specific datasets. In this paper, we introduce EQUATE (Equation Generation via QUality-Aligned Transfer Embeddings), a data-efficient fine-tuning framework that adapts foundation models for symbolic equation discovery in low-data regimes via distillation. EQUATE combines symbolic-numeric alignment with evaluator-guided embedding optimization, enabling a principled embedding-search-generation paradigm. Our approach reformulates discrete equation search as a continuous optimization task in a shared embedding space, guided by data-equation fitness and simplicity. Experiments across three standard public benchmarks (Feynman, Strogatz, and black-box datasets) demonstrate that EQUATE consistently outperforms state-of-the-art baselines in both accuracy and robustness, while preserving low complexity and fast inference. These results highlight EQUATE as a practical and generalizable solution for data-efficient symbolic regression in foundation model distillation settings. Read More
What Rollup News says about battling disinformationAI Newson August 28, 2025 at 7:41 am Swarm Network, a platform developing decentralised protocols for AI agents, recently announced the successful results of its first Swarm, a tool (perhaps “organism” is the better term) built to tackle disinformation. Called Rollup News, the swarm is not an app, a software platform, nor a centralised algorithm. It is a decentralised collection of AI agents
The post What Rollup News says about battling disinformation appeared first on AI News.
Swarm Network, a platform developing decentralised protocols for AI agents, recently announced the successful results of its first Swarm, a tool (perhaps “organism” is the better term) built to tackle disinformation. Called Rollup News, the swarm is not an app, a software platform, nor a centralised algorithm. It is a decentralised collection of AI agents
The post What Rollup News says about battling disinformation appeared first on AI News. Read More
Get AI-Ready: How to Prepare for a World of Agentic AI as Tech ProfessionalsTowards Data Scienceon August 27, 2025 at 6:30 pm Explore how Agentic AI is reshaping the tech careers, from data to decision-making, and how professionals can prepare for the future of work
The post Get AI-Ready: How to Prepare for a World of Agentic AI as Tech Professionals appeared first on Towards Data Science.
Explore how Agentic AI is reshaping the tech careers, from data to decision-making, and how professionals can prepare for the future of work
The post Get AI-Ready: How to Prepare for a World of Agentic AI as Tech Professionals appeared first on Towards Data Science. Read More
AI security wars: Can Google Cloud defend against tomorrow’s threats?AI Newson August 28, 2025 at 11:02 am In Google’s sleek Singapore office at Block 80, Level 3, Mark Johnston stood before a room of technology journalists at 1:30 PM with a startling admission: after five decades of cybersecurity evolution, defenders are still losing the war. “In 69% of incidents in Japan and Asia Pacific, organisations were notified of their own breaches by
The post AI security wars: Can Google Cloud defend against tomorrow’s threats? appeared first on AI News.
In Google’s sleek Singapore office at Block 80, Level 3, Mark Johnston stood before a room of technology journalists at 1:30 PM with a startling admission: after five decades of cybersecurity evolution, defenders are still losing the war. “In 69% of incidents in Japan and Asia Pacific, organisations were notified of their own breaches by
The post AI security wars: Can Google Cloud defend against tomorrow’s threats? appeared first on AI News. Read More
Decentralised AI: Full of promise, but not without challengesAI Newson August 27, 2025 at 10:24 am Decentralised artificial intelligence has been hailed as one of the most profound innovations of our time, promising to give users control of the most transformative technologies. Yet the industry faces some daunting challenges if the vision is to be fulfilled. Proponents of decentralisation imagine a world where AI is not controlled by a select few
The post Decentralised AI: Full of promise, but not without challenges appeared first on AI News.
Decentralised artificial intelligence has been hailed as one of the most profound innovations of our time, promising to give users control of the most transformative technologies. Yet the industry faces some daunting challenges if the vision is to be fulfilled. Proponents of decentralisation imagine a world where AI is not controlled by a select few
The post Decentralised AI: Full of promise, but not without challenges appeared first on AI News. Read More
Reimagining sound and spaceMIT Technology Reviewon August 26, 2025 at 9:00 pm On a typical afternoon, MIT’s new Edward and Joyce Linde Music Building hums with life. On the fourth floor, a jazz combo works through a set in a rehearsal suite as engineers adjust microphone levels in a nearby control booth. Downstairs, the layered rhythms of Senegalese drumming pulse through a room built to absorb its…
On a typical afternoon, MIT’s new Edward and Joyce Linde Music Building hums with life. On the fourth floor, a jazz combo works through a set in a rehearsal suite as engineers adjust microphone levels in a nearby control booth. Downstairs, the layered rhythms of Senegalese drumming pulse through a room built to absorb its… Read More
Google Vids gets AI avatars and image-to-video toolsAI Newson August 27, 2025 at 2:48 pm Google is rolling out a raft of powerful new generative AI features for Vids designed to take the pain out of video creation. Between wrestling with complicated software, finding someone willing to be on camera, and then spending hours editing out all the “ums” and “ahs,” video production often feels more trouble than it’s worth.
The post Google Vids gets AI avatars and image-to-video tools appeared first on AI News.
Google is rolling out a raft of powerful new generative AI features for Vids designed to take the pain out of video creation. Between wrestling with complicated software, finding someone willing to be on camera, and then spending hours editing out all the “ums” and “ahs,” video production often feels more trouble than it’s worth.
The post Google Vids gets AI avatars and image-to-video tools appeared first on AI News. Read More
The Download: introducing: the Security issueMIT Technology Reviewon August 27, 2025 at 12:10 pm This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology. Introducing: the Security issue It would be naïve to think we are going back to a world without AI. We’re not. But it’s only one of many urgent problems we need to address…
This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology. Introducing: the Security issue It would be naïve to think we are going back to a world without AI. We’re not. But it’s only one of many urgent problems we need to address… Read More
AI comes for the job market, security, and prosperity: The DebriefMIT Technology Reviewon August 27, 2025 at 10:00 am When I picked up my daughter from summer camp, we settled in for an eight-hour drive through the Appalachian mountains, heading from North Carolina to her grandparents’ home in Kentucky. With little to no cell service for much of the drive, we enjoyed the rare opportunity to have a long, thoughtful conversation, uninterrupted by devices.…
When I picked up my daughter from summer camp, we settled in for an eight-hour drive through the Appalachian mountains, heading from North Carolina to her grandparents’ home in Kentucky. With little to no cell service for much of the drive, we enjoyed the rare opportunity to have a long, thoughtful conversation, uninterrupted by devices.… Read More