What changed
AWS has introduced guidance on minimizing the costs of generative AI systems by identifying when deterministic rules or constrained decision calls can replace model calls. The company highlights that many tasks, such as input validation and routing, can be handled by rules or structured decision-making components rather than large language models (LLMs).
What this means in practice
The guidance suggests that teams should evaluate each workflow step to determine if a rule can handle the task. For example, checking an order number's format can be done with a rule, while routing a message by intent may involve a fast path with a keyword match followed by a constrained decision call. AWS recommends using tools like AWS Step Functions Choice states and AWS Lambda for implementing these deterministic checks. For tasks that require a model, AWS advises using the smallest right-sized model on Amazon Bedrock to minimize costs.
Limitations and availability
The approach emphasizes that deterministic code is more cost-effective and has lower latency than model calls. However, for open-ended tasks like drafting a reply, a model is still necessary. AWS provides several options for structured decision-making, including small Amazon Bedrock models with structured outputs, embedding similarity, and experimental open-source models like Strands Decider. These options require validation against labeled data before deployment. The guidance is available through AWS resources such as the Well-Architected Generative AI Lens and the Orchestration models: From rule-based to AI-native in AWS Prescriptive Guidance.
