Briefs · Taipei time (GMT+8)

AWS Adds NVIDIA Dynamo Support to SageMaker HyperPod Inference

一句重點:AWS has brought NVIDIA Dynamo into SageMaker’s managed inference service for large models.

發生什麼事

AWS says SageMaker HyperPod Inference now supports NVIDIA Dynamo. With this setup, the service can recover automatically from failures and direct requests based on KV cache state, while inference for large models can span multiple nodes (Managed NVIDIA Dynamo on Amazon SageMaker HyperPod Inference).

01Sources

Evidence: 🗣 Official statement; not independently tested.