Amazon Nova Forge enables users to build and customize large language models for specialized tasks while preserving general capabilities. The platform allows developers to blend proprietary data with curated training datasets, ensuring models retain broad reasoning and instruction-following abilities. This approach helps prevent catastrophic forgetting, which can undermine domain-specific performance. The blog post outlines strategies for balancing hyperparameter tuning to achieve optimal results without degrading model performance on general tasks. It emphasizes the importance of careful tuning to avoid costly failed training runs. The platform also supports multiple customization techniques, including continued pre-training, supervised fine-tuning, and reinforcement fine-tuning, each tailored to different stages of model development. These techniques help users achieve the best possible outcomes for their specific use cases. Source: awsml
Hyperparameter tuning on domain-specialized models presents three main challenges. Catastrophic forgetting occurs when a model trained on narrow domain data overwrites its general capabilities, leading to degraded performance on tasks outside the training domain. This results in a stability-flexibility tradeoff, where the model becomes highly specialized but loses reasoning and instruction-following abilities. The blog post highlights how Nova Forge addresses this through data mixing and checkpoint selection, which help preserve general capabilities while adapting to domain-specific data. Another challenge is finding the right learning rate, which controls how much the model’s weights change during training. A learning rate that is too high can destabilize training or cause rapid forgetting of base capabilities, while a rate that is too low wastes compute on slow convergence. Nova Forge provides calibrated service defaults for different training techniques to account for these interactions, especially when mixing Nova data with user data. The third challenge involves baseline performance constraints, where reinforcement fine-tuning (RFT) works best within a specific range of baseline task accuracy. If the model’s baseline accuracy is too low, there are insufficient examples for reward-guided learning, and if it is already very high, additional training yields diminishing returns. The blog post explains how Nova Forge addresses these constraints by using supervised fine-tuning (SFT) for low-baseline scenarios and ensuring reward functions have discriminative power for high-baseline tasks. Source: awsml
Amazon Nova Forge offers three complementary customization techniques: continued pre-training (CPT), supervised fine-tuning (SFT), and reinforcement fine-tuning (RFT). CPT expands foundational model knowledge through self-supervised learning on domain-specific data, making it suitable for tasks requiring specialized vocabulary or industry concepts. SFT customizes model behavior using input-output pairs for specific tasks, ensuring the model follows required response formats or performs structured tasks. RFT optimizes model output using reward signals, pushing performance beyond what SFT alone can achieve. The blog post emphasizes that while all three stages can be used together for the strongest results, each can be optional depending on data availability and task type. Source: awsml