semantic caching
## Key Takeaways - **Semantic caching** — a **Retrieval-Augmented Generation (RAG)** technique — stores queries and responses as **vector embeddings** for reuse. - Improves **efficiency** by avoiding repeated large language model (LLM) calls. - Case study: failure ➡ production success through **7 bi-encoder models**, **4 setups**, **1,000 banking queries**. - **Three model types*
LLM optimization
🏆 Foundational Models ① Project: HiPO HiPO-8B is a novel reinforcement learning framework based on Hybrid Policy Optimization, enabling dynamic reasoning capabilities in large language models (LLMs). Key Highlights: * Developed by KwaiKAT team at Kuaishou in collaboration with NJU-LINK Laboratory (Nanjing University) and ARiSE Laboratory. * Features “think-on” and “think-off” mode switching to
AI testing
# Bridging AI Evaluation with Real-World Business Impact *Magdalena Picariello reframes AI conversations around measurable business value, iterative development, and feedback-driven optimization.* --- ## Introduction Magdalena shifts focus from **algorithms and metrics** to **tangible business outcomes**, advocating for evaluation systems that go beyond accuracy. Her approach: - **Continuous feedback loops** - **Iterative
Fast-dLLM
Autoregressive (AR) LLMs vs. Diffusion LLMs (dLLM) Autoregressive (AR) large language models generate output sequentially, token-by-token, which limits inference efficiency. Diffusion-type LLMs (dLLM) allow parallel generation, but traditionally struggle with: * KV cache reuse * Variable-length generation * Consistently outperforming AR in quality --- Fast-dLLM v2 — Pragmatic Parallel Decoding Fast-dLLM v2 adapts a
Java inference
# OpenAI **gpt-oss** — Java CPU Inference Implementation & Performance Optimization In **August 2025**, OpenAI released **gpt-oss**, marking its first **open-weights model** since GPT‑2 — with **120B** and **20B** parameter reasoning models.  
context engineering
# 2025-10-24 · Zhejiang   --- ## Introduction With the rapid growth of AI Agents, a new term — **Context Engineering** — has emerged. Many are asking: - How does it differ
Tencent AI
Training-Free GRPO: A Cost-Effective Breakthrough in LLM Optimization Only 120 RMB — outperforming fine-tuning that costs 70,000 RMB! Tencent has introduced a new method for upgrading large-model agents: Training-Free Group Relative Policy Optimization (Training-Free GRPO). Key idea: No parameter adjustment required — the method leverages brief experiential learning within prompts to
AI research
Farewell to Traditional Fine-Tuning: Introducing ACE A groundbreaking study from Stanford University, SambaNova Systems, and the University of California, Berkeley has demonstrated a transformative approach to improving AI models — without adjusting a single weight. The method, called Agent Contextual Engineering (ACE), relies on context engineering rather than retraining. It autonomously