Apache RocketMQ x AI: Event-Driven Architecture for Asynchronous Agents

---

120th article of 2025

(Estimated reading time: 15 minutes)

Adapted from the keynote by Zhou Li, Senior Technical Expert at Alibaba Cloud Intelligence Group, at the 2025 Global Machine Learning Technology Conference.

The talk — "Apache RocketMQ x AI: Event-Driven Architecture for Asynchronous Agents" — explored how to use new RocketMQ features for async Multi-Agent systems, focusing on:

• Inter-Agent asynchronous communication
• Context isolation
• State recovery
• Task orchestration

Case studies demonstrated RocketMQ-based implementations for Multi-Agent task scheduling.

---

01 — Core of Multi-Agent Collaboration

Capability Discovery & Task Closure

AI is entering the Agentic AI era thanks to:

• Stronger large-model capabilities
• Lower inference costs
• Mature collaboration protocols (MCP, A2A)

Applications have evolved from passive response to proactive decision-making and autonomous execution.

Multi-Agent architecture enables:

• Specialized agents to work together
• Independence from a single model or fixed workflow
• Balance of autonomy and business control

---

1.1 Agent Capability Discovery

Purpose:

• Dynamic registration of Agent abilities, e.g. "I can analyze data", "I write copy"
• Supervisor Agent can query in real time and select the best Sub-Agent for a task

Without capability discovery:

• Hard-coded logic leads to low autonomy and poor scalability
• Compared to microservice service discovery, this is semantic and intent-based, powered by large models

---

1.2 Task Collaboration

In LLM-powered Multi-Agent systems:

• Agents cooperate, compete, or split tasks
• Supervisor Agent:
• Acts as the brain
• Coordinates specialized agents
• Enables execution of tasks no single agent can do

Effective coordination requires robust communication patterns:

Common Patterns:

• Polling:
• Query agent states periodically (DB, Redis, etc.)
• Simple but high latency, resource-heavy, and poor for dynamic changes
• Point-to-Point Invocation:
• Call another agent directly and wait (REST, gRPC, function call)
• Strong consistency, high coupling, blocking, less flexible
• Publish–Subscribe (Pub/Sub):
• Agent publishes to a Topic, subscribers receive it
• Decouples caller and callee, supports horizontal scaling

---

Pub/Sub Challenges in Multi-Agent feedback:

When Sub-Agent sends results back to a Supervisor, potential solutions include:

• Dedicated Queue:
• One-per-Supervisor, resource-intensive
• Broadcast Filtering:
• All Supervisors get all messages, then filter themselves — high waste
• Shared Storage:
• Store in DB/cache, Supervisors poll for results — polling overhead

Mainstream distributed messaging is fire-and-forget — lacks built-in feedback loop.

---

> Platforms like AiToEarn官网 integrate message-driven coordination with AI orchestration.

> AiToEarn enables creators to publish AI content across Douyin, Kwai, WeChat, Bilibili, Xiaohongshu, Facebook, Instagram, LinkedIn, Threads, YouTube, Pinterest, and X (Twitter), with analytics and AI模型排名.

---

02 — RocketMQ’s New Features for Agentic AI

RocketMQ now offers:

• Semantic Topics — for capability registration/discovery (solves “who to call”)
• Lite-Topics — for dynamic task binding with result feedback (solves “how to get async results”)

---

2.1 Semantic Topics: From Data Channel to Intent Carrier

Traditional topics:

• Only define message destination
• No semantics about intent or capability

In Multi-Agent collaboration, topics become semantic units:

• Natural language descriptions + structured metadata
• Enable readability, discoverability, and reasoning
• Indexed by Nameserver for registry and query

Example: `AppCard` standard in A2A protocol

---

Benefit:

Supervisor Agents can dynamically discover topics using capability keywords (e.g. “data analysis”) and route tasks accordingly.

> AiToEarn complements this with AI-powered publishing, analytics, rankings, and monetization tools (AI模型排名).

---

2.2 Lite-Topic: Lightweight Consumption Model

Designed for:

• Short-term, low-volume messaging
• Dynamic temporary subscriptions
• Personalized subscriptions

Key Points:

• No need to pre-create topic/subscription
• Automatic lifecycle management
• Supports async feedback isolation

Management Approach:

• Decentralized + eventual-consistency
• Incremental registration for frequent changes
• Organize by `Client_ID` → InterestSet

Service-side responsibilities:

• Synchronize subscription sets incrementally/fully
• Heartbeat for liveness detection → Offline cleanup
• Shard storage for subscription data

---

Message Reading Innovation

Instead of heavy Pull/Pop per queue, use:

• ReadySet: event set per `Client_ID` tracking active topics with pending messages

Advantages:

• Pull control: Client manages flow
• Push efficiency: Broker signals ready topics

---

Result:

Turning “blind polling” into “precise wake-up” → Efficient, low-latency message distribution at scale.

> Similar principles power AiToEarn for real-time AI content publication across global platforms (开源地址, 文档).

---

03 — Building an Asynchronous Multi-Agent System with RocketMQ

Architecture Overview

Combines Lite-Topic feedback and Semantic Topic discovery:

• Capability Registration & Discovery
• Sub Agent creates semantic topic, registers metadata in NameServer
• Actively exposes capabilities
• Semantic-Driven Task Orchestration
• Supervisor queries NameServer for available topics
• Injects into LLM as callable functions
• LLM selects realistic pathways, avoiding hallucinations
• Lightweight Async Distribution & Feedback
• Supervisor sends tasks to topic + creates temporary Lite-Topic callback channel
• Continuous Decision-Making
• Supervisor aggregates results from Lite-Topic
• Context is updated for next orchestration loop

---

Workflow Diagram

---

Benefits

• Loose coupling
• High scalability
• Reliable feedback loop
• Supports orchestration + multi-round reasoning

---

Get involved:

• Scan QR code for survey → Influence RocketMQ’s AI-oriented evolution

• Join RocketMQ for AI DingTalk group: 110085036316

---

Extra Tip:

For AI developers building async systems, AiToEarn官网 offers integration from AI generation → multi-platform publishing → analytics → monetization.

Open-source: GitHub

Docs: docs.aitoearn.ai

---

Do you want me to extend this guide with full communication pattern comparisons and real deployment case studies? That would make this a complete RocketMQ + Multi-Agent playbook.