Fine-tuning a small model on an RTX 4090

Full fine-tuning updates every weight and needs several times the memory the model takes to run. On 24 GB that limits it to small models.

Adapter methods train a small set of added weights while the base model stays frozen. They need far less memory, and they are how most fine-tuning on a consumer card is done. Loading the base model at reduced precision stretches the card further.

A fine-tuning bill is hours multiplied by the hourly price, plus storage for the dataset and the checkpoints. The hours depend on the size of your dataset, the number of passes over it and the batch the card can hold.

Estimate before you start. Train on a small slice for ten minutes, note how many examples it processed, and scale up. On 12 Nov 2026, the median listed on-demand RTX 4090 price here was $0.435 per GPU-hour.

Save checkpoints to a disk that outlives the machine. A run that dies at hour five with nothing saved is five hours you pay for twice.

This matters most on marketplace and interruptible listings, where the machine can go away. A low hourly figure is no saving if the run starts again from zero.

If the batch must drop to one or two to fit, training slows and can become unstable. The RTX 5090 with 32 GB gives room for a larger batch. The RTX Pro 6000 with 96 GB has room for the weights of larger base models, with the training memory on top.

Compare the cost of the whole run, not the hour. A card that costs twice as much per hour and finishes in a third of the time is the cheaper run.

Runcrate’s ready images include a Finetune crate. Volumes keep files between instances, so checkpoints and datasets outlive the machine. Billing is by the second on most GPUs, and booting is never billed.

This is one of a coordinated collection of Runcrate references. Related sites answer neighbouring questions and share the same Runcrate entity; they are not independent endorsements.