Preparing data for supervised fine-tuning (SFT) involves more than just ensuring the dataset is clean and formatted correctly. The next steps focus on optimizing the data to enhance training outcomes. AWS outlines four advanced strategies to achieve this, including evaluating data readiness through learning curve analysis and selecting high-quality subsets of data. These methods aim to improve model performance while reducing unnecessary computational costs. The guidance is applicable to any model, with specific references to Amazon Nova customization findings.
A key aspect of data readiness evaluation is determining the appropriate dataset size. AWS recommends starting with around 2,000 high-quality samples for a typical SFT task, though this can vary depending on task difficulty and the model’s current performance. A learning curve analysis is suggested to assess how performance improves with increasing data volume. This involves training a model on the full dataset, saving checkpoints at regular intervals, and evaluating each checkpoint on a held-out evaluation set. The goal is to identify the saturation point, where further data adds minimal improvement.
The source emphasizes that data volume alone does not guarantee proportional gains in SFT. Instead, the coverage and depth of the instruction set are more critical. Research shows that a smaller, high-quality dataset can outperform a larger one if trained to full memorization. Training token accuracy serves as a practical stopping criterion, as gains tend to plateau once the model achieves near-perfect accuracy. These insights highlight the importance of quality over quantity in data preparation for supervised fine-tuning.
Source: awsml
