nairadithya

← all notes

Fine tuning

concept

Taking a pre-trained model and updating its weights after the fact usually for cases where the task isn’t exactly what it’s designed for.

Say you have weights
π‘Š
 in
ℝ
𝑑
Γ—
π‘˜
 for a given model,

Updating the weights is just:

π‘Š
new
=
π‘Š
+
Ξ”
π‘Š

The most naive way of doing this is just making the gradients flow again through the entire learned network, but that’s expensive and sometimes unnecessary.

LoRA

Making the reasonable assumption that the model has learned something useful from the pre-training step and the domain we’re adapting to is a minimal change from the domain it was trained for… it’s fair to say that we don’t need to update all the weights of the network.

So let’s learn a factorization for
Ξ”
π‘Š
 into two matrices of size
𝑑
Γ—
π‘Ÿ
 and
π‘Ÿ
Γ—
π‘˜
, A and B where
π‘Ÿ
β‰ͺ
min
(
𝑑
,
π‘˜
)

π‘Š
new
=
π‘Š
+
𝐴
𝐡
where
Ξ”
π‘Š
β‰ˆ
𝐴
𝐡

Update only
𝐴
 and
𝐡
 and keep W frozen.

The parameter savings are pretty clear.

  1. Complete fine-tuning -
    𝑑
    Γ—
    π‘˜
     parameter updates
  2. LoRA -
    π‘Ÿ
    (
    𝑑
    +
    π‘˜
    )
     parameter updates

After training, it’s straightforward to add
𝐴
𝐡
 directly to
π‘Š
 and not maintain a separate pair of matrices during inference.

What’s the tradeoff? Reasoning about it, your choice of
π‘Ÿ
 constrains the search down to a subspace of rank(
Ξ”
π‘Š
)
≀
π‘Ÿ
 within the weight space.
Ξ”
π‘Š
 is now a low rank matrix. Given the assumption that we don’t need to explore the entire weight space because we’re working from a solid pre-trained model, this works.

Original LoRA paper - https://arxiv.org/abs/2106.09685