distributed inference

distributed inference

Efficient Distributed Inference Framework: Optimized for Generative AI Throughput and Latency | Open Source Daily No.757

Dynamo – Distributed Inference Framework for Data Centers Repository: ai-dynamo/dynamo Stars: 5.1k License: Apache-2.0 Dynamo is an open-source distributed inference service framework designed for data center-scale generative AI and large inference models. It prioritizes high-throughput and low-latency operations while supporting multi-GPU and multi-server collaboration across diverse inference engines.