AWS said in an August 2026 notice that SageMaker AI Serverless Model Customization supports full fine-tuning (FFT) for more than 25 open-source models. A technical post AWS published on November 18 reports its later coding experiments comparing FFT with LoRA; the results differ by evaluation metric.

Choose evaluation measures that fit the task: AWS assessed coding correctness, code similarity and retention of general knowledge as separate dimensions.

When did AWS announce serverless FFT?

AWS’s August 2026 notice said SageMaker AI Serverless Model Customization supports FFT across the gpt-oss, Gemma, Llama, Nemotron and Qwen model families. It named four Regions: US East (N. Virginia), US West (Oregon), Asia Pacific (Tokyo) and Europe (Ireland). The November 18 post came later and focused on AWS’s experiments. AWS’s August notice

What serverless changes—and what FFT changes

Dark SageMaker interface showing a completed Qwen3-4B training job, its details and training- and validation-loss charts.

Serverless describes how SageMaker manages the training job; FFT describes which model parameters the training updates. AWS says SageMaker manages compute allocation and checkpoints. Customers specify the base model, dataset and hyperparameters. AWS’s technical post

The two methods change different parts of the model:

MethodWeight updatesAdaptation structure
Full fine-tuning (FFT)Updates all model parametersTrains the full set of model weights
LoRAKeeps pretrained weights frozenTrains low-rank adaptation matrices

How AWS evaluated its coding experiments

AWS compared supervised fine-tuning of Qwen3-4B using the same prompts, targets and evaluation code for FFT and LoRA. The Magicoder-Evol-Instruct-110K dataset was split into 99,000 training records, 5,500 validation records and 5,500 held-out test records. AWS’s experiment report

The evaluation measures addressed different questions. AWS used Amazon Nova Pro as an LLM-as-Judge for logical correctness. PRIMECODE assessed code similarity with BLEU, ROUGE-1, ROUGE-2, ROUGE-L and F1. MMLU measured performance across 57 academic subjects as an indicator of retained general knowledge.

What results did AWS report?

AWS reported 90.0% semantic correctness at three epochs. In a separate LLM-as-Judge comparison, LoRA scored 81.5% and FFT scored 79.5%. Those are results from different comparisons, not a single set of scores.

AWS also reported that both methods scored below the base model on MATH exact match in its experiments. In FFT trials with learning rates of at least 5e-5, AWS reported MATH scores of 2–3% and MMLU accuracy of 3.6%. Among the configurations it tested, a learning rate of 2e-5, global batch size of 8 and 5% warmup best preserved MMLU; AWS says useful settings vary by task.