TJS Articles & News

Panacea: Mitigating Harmful Fine-tuning for Large Language Models via Post-fine-tuning Perturbationcs.AI updates on arXiv.org arXiv:2501.18100v2 Announce Type: replace-cross
Abstract: Harmful fine-tuning attack introduces significant security risks to the fine-tuning services. Main-stream defenses aim to vaccinate the model such that the later harmful fine-tuning attack is less effective. However, our evaluation results show that such defenses are fragile–with a few fine-tuning steps, the model still can learn the harmful knowledge. To this end, we do further experiment and find that an embarrassingly simple solution–adding purely random perturbations to the fine-tuned model, can recover the model from harmful behaviors, though it leads to a degradation in the model’s fine-tuning performance. To address the degradation of fine-tuning performance, we further propose Panacea, which optimizes an adaptive perturbation that will be applied to the model after fine-tuning. Panacea maintains model’s safety alignment performance without compromising downstream fine-tuning performance. Comprehensive experiments are conducted on different harmful ratios, fine-tuning tasks and mainstream LLMs, where the average harmful scores are reduced by up-to 21.2%, while maintaining fine-tuning performance. As a by-product, we analyze the adaptive perturbation and show that different layers in various LLMs have distinct safety affinity, which coincide with finding from several previous study. Source code available at https://github.com/w-yibo/Panacea.

 arXiv:2501.18100v2 Announce Type: replace-cross
Abstract: Harmful fine-tuning attack introduces significant security risks to the fine-tuning services. Main-stream defenses aim to vaccinate the model such that the later harmful fine-tuning attack is less effective. However, our evaluation results show that such defenses are fragile–with a few fine-tuning steps, the model still can learn the harmful knowledge. To this end, we do further experiment and find that an embarrassingly simple solution–adding purely random perturbations to the fine-tuned model, can recover the model from harmful behaviors, though it leads to a degradation in the model’s fine-tuning performance. To address the degradation of fine-tuning performance, we further propose Panacea, which optimizes an adaptive perturbation that will be applied to the model after fine-tuning. Panacea maintains model’s safety alignment performance without compromising downstream fine-tuning performance. Comprehensive experiments are conducted on different harmful ratios, fine-tuning tasks and mainstream LLMs, where the average harmful scores are reduced by up-to 21.2%, while maintaining fine-tuning performance. As a by-product, we analyze the adaptive perturbation and show that different layers in various LLMs have distinct safety affinity, which coincide with finding from several previous study. Source code available at https://github.com/w-yibo/Panacea. Read More  

Robot-R1: Reinforcement Learning for Enhanced Embodied Reasoning in Roboticscs.AI updates on arXiv.org arXiv:2506.00070v3 Announce Type: replace-cross
Abstract: Large Vision-Language Models (LVLMs) have recently shown great promise in advancing robotics by combining embodied reasoning with robot control. A common approach involves training on embodied reasoning tasks related to robot control using Supervised Fine-Tuning (SFT). However, SFT datasets are often heuristically constructed and not explicitly optimized for improving robot control. Furthermore, SFT often leads to issues such as catastrophic forgetting and reduced generalization performance. To address these limitations, we introduce Robot-R1, a novel framework that leverages reinforcement learning to enhance embodied reasoning specifically for robot control. Robot-R1 learns to predict the next keypoint state required for task completion, conditioned on the current scene image and environment metadata derived from expert demonstrations. Inspired by the DeepSeek-R1 learning approach, Robot-R1 samples reasoning-based responses and reinforces those that lead to more accurate predictions. To rigorously evaluate Robot-R1, we also introduce a new benchmark that demands the diverse embodied reasoning capabilities for the task. Our experiments show that models trained with Robot-R1 outperform SFT methods on embodied reasoning tasks. Despite having only 7B parameters, Robot-R1 even surpasses GPT-4o on reasoning tasks related to low-level action control, such as spatial and movement reasoning.

 arXiv:2506.00070v3 Announce Type: replace-cross
Abstract: Large Vision-Language Models (LVLMs) have recently shown great promise in advancing robotics by combining embodied reasoning with robot control. A common approach involves training on embodied reasoning tasks related to robot control using Supervised Fine-Tuning (SFT). However, SFT datasets are often heuristically constructed and not explicitly optimized for improving robot control. Furthermore, SFT often leads to issues such as catastrophic forgetting and reduced generalization performance. To address these limitations, we introduce Robot-R1, a novel framework that leverages reinforcement learning to enhance embodied reasoning specifically for robot control. Robot-R1 learns to predict the next keypoint state required for task completion, conditioned on the current scene image and environment metadata derived from expert demonstrations. Inspired by the DeepSeek-R1 learning approach, Robot-R1 samples reasoning-based responses and reinforces those that lead to more accurate predictions. To rigorously evaluate Robot-R1, we also introduce a new benchmark that demands the diverse embodied reasoning capabilities for the task. Our experiments show that models trained with Robot-R1 outperform SFT methods on embodied reasoning tasks. Despite having only 7B parameters, Robot-R1 even surpasses GPT-4o on reasoning tasks related to low-level action control, such as spatial and movement reasoning. Read More  

Using Local LLMs to Discover High-Performance AlgorithmsTowards Data Science How I used open-source models to explore new frontiers in efficient code generation, using my MacBook and local LLMs.
The post Using Local LLMs to Discover High-Performance Algorithms appeared first on Towards Data Science.

 How I used open-source models to explore new frontiers in efficient code generation, using my MacBook and local LLMs.
The post Using Local LLMs to Discover High-Performance Algorithms appeared first on Towards Data Science. Read More  

Scaling AI value beyond pilot phase purgatoryAI News Scaling AI value from isolated pilots to enterprise-wide adoption remains a primary hurdle for many organisations. While experimentation with generative models has become ubiquitous, industrialising these tools (i.e. wrapping them in necessary governance, security, and integration layers) often stalls. Addressing the gap between investment and operational return, IBM has introduced a new service model designed
The post Scaling AI value beyond pilot phase purgatory appeared first on AI News.

 Scaling AI value from isolated pilots to enterprise-wide adoption remains a primary hurdle for many organisations. While experimentation with generative models has become ubiquitous, industrialising these tools (i.e. wrapping them in necessary governance, security, and integration layers) often stalls. Addressing the gap between investment and operational return, IBM has introduced a new service model designed
The post Scaling AI value beyond pilot phase purgatory appeared first on AI News. Read More  

Bridging the Gap Between Research and Readability with Marco Hening TallaricoTowards Data Science Diluting complex research, spotting silent data leaks, and why the best way to learn is often backwards.
The post Bridging the Gap Between Research and Readability with Marco Hening Tallarico appeared first on Towards Data Science.

 Diluting complex research, spotting silent data leaks, and why the best way to learn is often backwards.
The post Bridging the Gap Between Research and Readability with Marco Hening Tallarico appeared first on Towards Data Science. Read More  

SAP and Fresenius to build sovereign AI backbone for healthcareAI News SAP and Fresenius are building a sovereign AI platform for healthcare that brings secure data processing to clinical settings. For data leaders in the medical sector, deploying AI requires strict governance that public cloud solutions often lack. This collaboration addresses that gap by creating a “controlled environment” where AI models can operate without compromising data
The post SAP and Fresenius to build sovereign AI backbone for healthcare appeared first on AI News.

 SAP and Fresenius are building a sovereign AI platform for healthcare that brings secure data processing to clinical settings. For data leaders in the medical sector, deploying AI requires strict governance that public cloud solutions often lack. This collaboration addresses that gap by creating a “controlled environment” where AI models can operate without compromising data
The post SAP and Fresenius to build sovereign AI backbone for healthcare appeared first on AI News. Read More