Title:	Towards Scalable Inter-LLM Communication
DNr:	Berzelius-2025-363
Project Type:	LiU Berzelius
Principal Investigator:	Dejan Manojlo Kostic <dmk@kth.se>
Affiliation:	Kungliga Tekniska högskolan
Duration:	2025-10-28 – 2026-05-01
Classification:	10201
Homepage:	https://www.kth.se/blogs/dai/
Keywords:

Abstract

The rapid emergence of large language models (LLMs) as autonomous and collaborative agents introduces new challenges in how such models exchange and integrate information. Existing communication protocols mostly rely on natural language, which incurs high decoding computational cost and information loss. In this project, we start from our previous work where we demonstrated the effectiveness of sharing selective key–value (KV) pairs as an effective communication medium. We will develop mechanisms for efficient communication that optimize the level and structure of information exchange. The research will focus on three key directions: (1) Communication among models with different parameters/architectures. Finetuning models to achieve specialization on specific tasks provides agents with different capabilities and allows them to collaborate on a shared task. However, finetuning changes the model's internal representations, which can be detrimental to communication. We will develop mechanisms to enable communication between finetuned models with different internal representations, or even models with distinct architectures. (2) Efficient and effective information exchange. Despite natural language communication being easy to implement, it suffers from high decoding computational cost and information loss. Following our previous work, we will optimize the information exchange between sender and receiver models to lower the communication cost and improve the effectiveness. (3) Multi-agent interaction and cooperation within the communication framework. Built on the previous two directions, we will develop mechanisms to enable multi-agent interaction and cooperation. As a part of Prof. Kostic’s Wallenberg Scholar Project "Sustainable and Adaptive Inferencing for Democratizing AI", we plan to achieve our goals by advancing the ability of language models to communicate through their internal representations rather than text, paving the way for scalable, low-latency, and energy-efficient AI ecosystems.