# Which LLM is Best for Your SDLC? *(Sponsored)*
[Read the full report »](https://bit.ly/Sonar_102825CTA)
[](https://bit.ly/Sonar_102825CTA)
---
## Why the Newest LLM Isn’t Always the Best
Do you assume the latest LLM is automatically the best choice for coding? **Sonar** tested cutting‑edge models — GPT‑5, Claude Sonnet 4, and Llama 3 — in an expanded analysis. The results may surprise you.
**Explore insights on:**
- **Shared strengths and weaknesses of LLMs**
- **Coding archetypes for leading models**
- **Hidden quality and security risks in code generation**
- **Selecting the right model for your needs**
- **A “trust and verify” blueprint for safe AI integration**
📥 [Download the report](https://bit.ly/Sonar_102825Report)
▶️ [Watch the on‑demand webinar](https://bit.ly/Sonar_102825Video)
📊 [Get the facts](https://bit.ly/Sonar_102825CTA)
---
> *Thanks to Karthik Ramgopal, Distinguished Engineer at LinkedIn, for explaining LinkedIn’s GenAI architecture. All technical details and diagrams credited to the LinkedIn Engineering Team.*
---
# LinkedIn’s Rapid GenAI Evolution
Over two years, LinkedIn transformed its approach to building and delivering AI‑powered products.
### Early GenAI Features
- Collaborative articles
- AI-assisted recruiter capabilities
- AI‑powered insights for members and customers
**Flagship example:** **Hiring Assistant** — LinkedIn’s first large‑scale AI agent for recruiters, streamlining candidate sourcing and engagement.
### The Need for a Unified Stack
Early experiments were valuable but siloed:
- Separate prompt handling, model calls, and memory systems per product team
- Led to inconsistency, slower iteration, duplicated work
**Solution:** Create a **unified GenAI application stack** to:
1. Move quickly to capture GenAI momentum
2. Allow engineers to build features without reinventing infrastructure
3. Embed trust, privacy, and safety guardrails
---
# Beyond LinkedIn – Industry Trends
Open ecosystems like [AiToEarn官网](https://aitoearn.ai/) bring similar principles to global content monetization:
- AI content generation
- Cross-platform publishing
- Performance analytics
- Platforms supported: Douyin, Kwai, WeChat, Bilibili, Xiaohongshu, Facebook, Instagram, LinkedIn, Threads, YouTube, Pinterest, X/Twitter
---
# Foundations of the GenAI Application Stack
Between 2023 and 2024, LinkedIn built a **stable, scalable, unified GenAI foundation** enabling rapid experimentation. GenAI generates text, images, code based on input — sitting within broader AI/ML domains.
[](https://substackcdn.com/image/fetch/$s_!Gmcq!...)
---
## 1. Language & Framework Shift
### Problem
- Java dominated online production systems
- Python used for offline experimentation (prompt engineering, model evaluation)
- High friction moving prototypes from Python to Java
### Decision: Make Python a First-Class Language
**Drivers:**
- **Ecosystem:** Open-source AI tools evolve faster in Python
- **Developer familiarity:** AI engineers preferred Python
- **Avoid duplication:** One language end‑to‑end
**Incremental migration:**
- gRPC support for Python clients
- REST proxy for Espresso DB
- Partial request context implementation
**Developer tooling added:**
- Native Python builds
- Automation for importing/maintaining libraries
- Support for newest Python versions
### Framework Choice: LangChain
**Evaluation criteria:**
- Ecosystem maturity
- Dev velocity
- Integration flexibility
- Production reliability
- Adaptability to change
LangChain won due to robust ecosystem, strong abstractions, fine‑grained control.
Maintained a **thin LinkedIn-specific layer** for easy future migrations.
---
## Prompt Management
### Early Approach
Hardcoded prompt strings in code → chaotic maintenance.
### Solution: Prompt Source of Truth
- Centralized prompt storage
- Jinja templating
- **Benefits:**
- Version control & gradual rollouts
- Modular prompt fragments
- Consistency across apps
**Multi-turn conversations:** Adopted **OpenAI Chat Completions API**.
---
## 3. Skills Abstraction
LLMs often need API/data access.
### Steps:
1. Introduced a **Skills** abstraction — LLM-usable capabilities
2. Built a **Skill Registry** with schema, docs, central discovery
3. Added build-phase plugins, governance layers, duplicate checks
Emerging protocols:
- **MCP** – standardized tool discovery
- **A2A** – agent collaboration
---
## 4. Memory
### Two Types:
- **Conversational** – chat history, semantic search, summarization
- **Experiential** – structured insights (tone, preferences) from past use
**Privacy:** Strict governance on sensitive info.
---
## 5. Model Inference & Fine-Tuning
- Initially: Azure OpenAI models via central proxy (RAI checks, streaming, quotas)
- Expanded to LinkedIn’s own PyTorch/DeepSpeed/vLLM platform
- Fine-tuned open-source Llama models → matched/exceeded proprietary models
- Exposed all via single OpenAI-compatible API
---
## 6. Migration at Scale
Strategy:
- Depth-first prototypes to fix gaps early
- Breadth-first rollout after proven stable
- A/B testing to ramp traffic safely
- Paired senior Java engineers with experienced Python devs
---
# From Assistants to AI Agents
By 2025, the stack supported:
- Modular AI agents with multi-step reasoning
- Human-in-the-loop checkpoints
- Agent discoverability via gRPC schemas and agent registry
- Workflow orchestration via LinkedIn Messaging backbone
---
## Observability & Tooling
**Pre-production:** LangSmith tracing with LangChain/LangGraph
**Production:** OpenTelemetry spans, correlated traces, regression testing datasets
**Playground:** Internal sandbox to prototype agents/skills, inspect memory, test auth, view real-time traces
---
## Emerging Agent Patterns
- **Chat-native UX**
- **Background agents**
- **Context engineering** (RAG, knowledge graphs)
- **LangGraph for complex flows**
---
## Platform Guardrails & Interoperability
- Separation of client data, memories, agents
- Policy-driven interfaces, strong auth
- Synchronous/asynchronous paths for latency vs. long-running tasks
- Exploration of open protocols MCP & A2A
---
# Conclusion
LinkedIn’s progression from isolated GenAI scripts to a mature agentic platform shows:
- **Standardize prompts, skills, memory infra**
- **Loose coupling of frameworks/models**
- **Reuse reliable messaging infra for orchestration**
- **Blend LangSmith dev tracing with OTel production spans**
- **Governance in skill management prevents chaos without slowing devs**
**Key Insight:** AI augments engineers — success relies on technical skill *and* leadership/communication.
---
**References:**
- [Behind the platform: the journey to create the LinkedIn GenAI application tech stack](https://www.linkedin.com/blog/engineering/generative-ai/behind-the-platform-the-journey-to-create-the-linkedin-genai-application-tech-stack)
- [Extending to Build AI Agents](https://www.linkedin.com/blog/engineering/generative-ai/the-linkedin-generative-ai-application-tech-stack-extending-to-build-ai-agents)
- [The Agent-to-Agent Protocol](https://developers.googleblog.com/en/a2a-a-new-era-of-agent-interoperability/)
