What changed

AWS has introduced guidance on minimizing the costs of generative AI systems by identifying when deterministic rules or constrained decision calls can replace model calls. The company highlights that many tasks, such as input validation and routing, can be handled by rules or structured decision-making components rather than large language models (LLMs).

What this means in practice

The guidance suggests that teams should evaluate each workflow step to determine if a rule can handle the task. For example, checking an order number's format can be done with a rule, while routing a message by intent may involve a fast path with a keyword match followed by a constrained decision call. AWS recommends using tools like AWS Step Functions Choice states and AWS Lambda for implementing these deterministic checks. For tasks that require a model, AWS advises using the smallest right-sized model on Amazon Bedrock to minimize costs.

Limitations and availability

The approach emphasizes that deterministic code is more cost-effective and has lower latency than model calls. However, for open-ended tasks like drafting a reply, a model is still necessary. AWS provides several options for structured decision-making, including small Amazon Bedrock models with structured outputs, embedding similarity, and experimental open-source models like Strands Decider. These options require validation against labeled data before deployment. The guidance is available through AWS resources such as the Well-Architected Generative AI Lens and the Orchestration models: From rule-based to AI-native in AWS Prescriptive Guidance.

Source facts for AWS recommends using rules over models for cost-effective AI workflows: Cut generative AI costs: Code for rules, models for judgment The biggest cost lever in a generative AI system is whether you call a model at all. Each call is cheap in isolation, but at production scale these small costs become one of the largest line items in the architecture. Wherever a deterministic rule can do the job, skipping the model saves money, cuts latency, and removes a source of non-determinism.
Original AINewsTrend explainer based on the cited source. This graphic summarizes source statements; it is not an independent benchmark.

Sources and further reading

  1. awsml: Cut generative AI costs: Code for rules, models for judgment