Addressing the Ecological Fallacy in Larger LMs with Human Contextcs.AI updates on arXiv.org arXiv:2603.05928v1 Announce Type: cross
Abstract: Language model training and inference ignore a fundamental linguistic fact — there is a dependence between multiple sequences of text written by the same person. Prior work has shown that addressing this form of textit{ecological fallacy} can greatly improve the performance of multiple smaller (~124M) GPT-based models. In this work, we ask if addressing the ecological fallacy by modeling the author’s language context with a specific LM task (called HuLM) can provide similar benefits for a larger-scale model, an 8B Llama model. To this end, we explore variants that process an author’s language in the context of their other temporally ordered texts. We study the effect of pre-training with this author context using the HuLM objective, as well as using it during fine-tuning with author context (textit{HuFT:Human-aware Fine-Tuning}). Empirical comparisons show that addressing the ecological fallacy during fine-tuning alone using QLoRA improves the performance of the larger 8B model over standard fine-tuning. Additionally, QLoRA-based continued HuLM pre-training results in a human-aware model generalizable for improved performance over eight downstream tasks with linear task classifier training alone. These results indicate the utility and importance of modeling language in the context of its original generators, the authors.
arXiv:2603.05928v1 Announce Type: cross
Abstract: Language model training and inference ignore a fundamental linguistic fact — there is a dependence between multiple sequences of text written by the same person. Prior work has shown that addressing this form of textit{ecological fallacy} can greatly improve the performance of multiple smaller (~124M) GPT-based models. In this work, we ask if addressing the ecological fallacy by modeling the author’s language context with a specific LM task (called HuLM) can provide similar benefits for a larger-scale model, an 8B Llama model. To this end, we explore variants that process an author’s language in the context of their other temporally ordered texts. We study the effect of pre-training with this author context using the HuLM objective, as well as using it during fine-tuning with author context (textit{HuFT:Human-aware Fine-Tuning}). Empirical comparisons show that addressing the ecological fallacy during fine-tuning alone using QLoRA improves the performance of the larger 8B model over standard fine-tuning. Additionally, QLoRA-based continued HuLM pre-training results in a human-aware model generalizable for improved performance over eight downstream tasks with linear task classifier training alone. These results indicate the utility and importance of modeling language in the context of its original generators, the authors. Read More
An Interactive Multi-Agent System for Evaluation of New Product Conceptscs.AI updates on arXiv.org arXiv:2603.05980v1 Announce Type: new
Abstract: Product concept evaluation is a critical stage that determines strategic resource allocation and project success in enterprises. However, traditional expert-led approaches face limitations such as subjective bias and high time and cost requirements. To support this process, this study proposes an automated approach utilizing a large language model (LLM)-based multi-agent system (MAS). Through a systematic analysis of previous research on product development and team collaboration, this study established two primary evaluation dimensions, namely technical feasibility and market feasibility. The proposed system consists of a team of eight virtual agents representing specialized domains such as R&D and marketing. These agents use retrieval-augmented generation (RAG) and real-time search tools to gather objective evidence and validate concepts through structured deliberations based on the established criteria. The agents were further fine-tuned using professional product review data to enhance their judgment accuracy. A case study involving professional display monitor concepts demonstrated that the system’s evaluation rankings were consistent with those of senior industry experts. These results confirm the usability of the proposed multi-agent-based evaluation approach for supporting product development decisions.
arXiv:2603.05980v1 Announce Type: new
Abstract: Product concept evaluation is a critical stage that determines strategic resource allocation and project success in enterprises. However, traditional expert-led approaches face limitations such as subjective bias and high time and cost requirements. To support this process, this study proposes an automated approach utilizing a large language model (LLM)-based multi-agent system (MAS). Through a systematic analysis of previous research on product development and team collaboration, this study established two primary evaluation dimensions, namely technical feasibility and market feasibility. The proposed system consists of a team of eight virtual agents representing specialized domains such as R&D and marketing. These agents use retrieval-augmented generation (RAG) and real-time search tools to gather objective evidence and validate concepts through structured deliberations based on the established criteria. The agents were further fine-tuned using professional product review data to enhance their judgment accuracy. A case study involving professional display monitor concepts demonstrated that the system’s evaluation rankings were consistent with those of senior industry experts. These results confirm the usability of the proposed multi-agent-based evaluation approach for supporting product development decisions. Read More
A practitioner’s guide to the AI-driven development paradigm reshaping how software is built. Introduction: Vibe Code Meaning Software development has always evolved in step with the tools available to engineers. From assembly language to high-level languages, from waterfall to agile, each wave of change has redefined what it means to “write code.” The arrival of […]
Understanding Context and Contextual Retrieval in RAGTowards Data Science Why traditional RAG loses context and how contextual retrieval dramatically improves retrieval accuracy
The post Understanding Context and Contextual Retrieval in RAG appeared first on Towards Data Science.
Why traditional RAG loses context and how contextual retrieval dramatically improves retrieval accuracy
The post Understanding Context and Contextual Retrieval in RAG appeared first on Towards Data Science. Read More
GPT-5.4 Thinking System CardOpenAI News
Pandas vs. Polars: A Complete Comparison of Syntax, Speed, and MemoryKDnuggets Need help choosing the right Python dataframe library? This article compares Pandas and Polars to help you decide.
Need help choosing the right Python dataframe library? This article compares Pandas and Polars to help you decide. Read More
10 GitHub Repositories to Master System DesignKDnuggets Want to move beyond drawing boxes and arrows and actually understand how scalable systems are built? These GitHub repositories break down the concepts, patterns, and real-world trade-offs that make great system design possible.
Want to move beyond drawing boxes and arrows and actually understand how scalable systems are built? These GitHub repositories break down the concepts, patterns, and real-world trade-offs that make great system design possible. Read More
How Human Work Will Remain Valuable in an AI WorldTowards Data Science The Road to Reality — Episode 1
The post How Human Work Will Remain Valuable in an AI World appeared first on Towards Data Science.
The Road to Reality — Episode 1
The post How Human Work Will Remain Valuable in an AI World appeared first on Towards Data Science. Read More
OpenAI Releases Symphony: An Open Source Agentic Framework for Orchestrating Autonomous AI Agents through Structured, Scalable Implementation RunsMarkTechPost OpenAI has released Symphony, an open-source framework designed to manage autonomous AI coding agents through structured ‘implementation runs.’ The project provides a system for automating software development tasks by connecting issue trackers to LLM-based agents. System Architecture: Elixir and the BEAM Symphony is built using Elixir and the Erlang/BEAM runtime. The choice of stack focuses
The post OpenAI Releases Symphony: An Open Source Agentic Framework for Orchestrating Autonomous AI Agents through Structured, Scalable Implementation Runs appeared first on MarkTechPost.
OpenAI has released Symphony, an open-source framework designed to manage autonomous AI coding agents through structured ‘implementation runs.’ The project provides a system for automating software development tasks by connecting issue trackers to LLM-based agents. System Architecture: Elixir and the BEAM Symphony is built using Elixir and the Erlang/BEAM runtime. The choice of stack focuses
The post OpenAI Releases Symphony: An Open Source Agentic Framework for Orchestrating Autonomous AI Agents through Structured, Scalable Implementation Runs appeared first on MarkTechPost. Read More
How to Design an Advanced Tree-of-Thoughts Multi-Branch Reasoning Agent with Beam Search, Heuristic Scoring, and Depth-Limited PruningMarkTechPost In this tutorial, we build an advanced Tree-of-Thoughts (ToT) multi-branch reasoning agent from scratch. Instead of relying on linear chain-of-thought reasoning, we design a system that generates multiple reasoning branches, scores each branch using a heuristic evaluation function, prunes weak candidates, and continues expanding only the strongest paths. We combine an instruction-tuned transformer model with
The post How to Design an Advanced Tree-of-Thoughts Multi-Branch Reasoning Agent with Beam Search, Heuristic Scoring, and Depth-Limited Pruning appeared first on MarkTechPost.
In this tutorial, we build an advanced Tree-of-Thoughts (ToT) multi-branch reasoning agent from scratch. Instead of relying on linear chain-of-thought reasoning, we design a system that generates multiple reasoning branches, scores each branch using a heuristic evaluation function, prunes weak candidates, and continues expanding only the strongest paths. We combine an instruction-tuned transformer model with
The post How to Design an Advanced Tree-of-Thoughts Multi-Branch Reasoning Agent with Beam Search, Heuristic Scoring, and Depth-Limited Pruning appeared first on MarkTechPost. Read More