BentoML

BentoML began as an open-source Python framework that packages machine learning models from frameworks such as PyTorch, TensorFlow and scikit-learn into containerized API services. Its commercial layer, the Bento Inference Platform (Bento Cloud), runs those services on managed NVIDIA and AMD GPUs, including H100, B200 and MI300X. It handles autoscaling, scale-to-zero, faster cold starts, cross-region and multi-cloud orchestration, and distributed inference across several GPUs. Teams can also deploy into their own cloud account (BYOC), on-premises, or onto Kubernetes. The platform supports real-time endpoints, async jobs, batch inference and multi-model pipelines for RAG. It also offers canary, shadow and A/B releases with rollbacks, plus an LLM gateway and LLM-specific monitoring. It fits ML and platform engineering teams that want to serve open or custom models in production without building their own serving stack. Modular acquired BentoML in February 2026. The framework stays under Apache 2.0, and Modular has said Bento Cloud will remain supported while it merges into Modular Cloud over time. Pricing is no longer published on bentoml.com, which links to Modular's pricing page. That page lists hosted endpoints billed per token or per GPU minute, BYOC billed per minute of reserved GPU capacity, and enterprise contracts with custom terms. The self-hosted open-source option is free. The main limitation is uncertainty: buyers face an evolving roadmap after the acquisition, few public reviews, and opaque GPU pricing that requires a sales conversation. Note also that the product covers serving only and has no experiment tracking or pipeline orchestration.