Amazon Nova Forge enables users to create custom reward functions for multi-turn reinforcement learning, addressing the challenge of designing effective rewards for complex, multi-step tasks. The platform allows teams to focus on defining what constitutes a good outcome while handling the orchestration of rollouts, message passing, and conversation state. Nova Forge offers a serverless multi-turn RL option, now generally available, for teams that prefer not to manage the environment directly. This post uses the Bring Your Own Orchestration (BYOO) path, where users run their reward logic in their own environment.
The core of reinforcement fine-tuning (RFT) lies in the reward function, which guides the model's learning. Unlike supervised fine-tuning (SFT), RFT learns from evaluation signals on the model's own outputs rather than curated examples. Multi-turn RFT extends this to agents that act over a sequence of steps, such as calling tools, executing code, or recovering from a mistake. It optimizes cumulative reward across the whole trajectory rather than grading a single response. The reward function is a grader written in code, which can be a rule-based check or an LLM-as-Judge approach.
The source explains how the reward function runs with Nova Forge, depending on the task. For single-turn RFT, the reward is registered as an AWS Lambda function, while multi-turn tasks use BYOO. Nova Forge delegates each rollout to the user's environment container, which runs the user simulator, executes code, and calls a verifier. It returns an aggregate reward score per sample, along with optional per-component scores. The training job generates candidate rollouts from the Nova model for each prompt, and the reward function receives each rollout to perform three steps: running the task logic, scoring the completed trajectory, and returning an aggregate reward per rollout.
Source: awsml
