Abstract
This paper presents the development and evaluation of a Large Language Model(LLM), also known as foundation models, based multi-agent system framework forcomplex event processing (CEP) with a focus on video query processing usecases. The primary goal is to create a proof-of-concept (POC) that integratesstate-of-the-art LLM orchestration frameworks with publish/subscribe (pub/sub)tools to address the integration of LLMs with current CEP systems. Utilizingthe Autogen framework in conjunction with Kafka message brokers, the systemdemonstrates an autonomous CEP pipeline capable of handling complex workflows.Extensive experiments evaluate the system's performance across varyingconfigurations, complexities, and video resolutions, revealing the trade-offsbetween functionality and latency. The results show that while higher agentcount and video complexities increase latency, the system maintains highconsistency in narrative coherence. This research builds upon and contributesto, existing novel approaches to distributed AI systems, offering detailedinsights into integrating such systems into existing infrastructures.