發生什麼事
AWS Machine Learning Blog introduced KV cache-aware routing for SageMaker AI real-time endpoints. The feature selects an instance based on how much of a prompt it has cached. The article explains the routing behavior, compares when to use it with prefix-aware routing, and describes how to enable it.
01Sources
Evidence: 🗣 Official statement; not independently tested.
- Amazon SageMaker AI real-time endpoints now support KV cache aware routing: Published by AWS (AWS Machine Learning Blog) on 2026-11-17.