The geometry of a rank-one update

Rank-one matrices form a saddle

Set W₀ to zero. A rank-one LoRA update is an outer product AB; fix its top-left entry c and the other three entries form a surface.

AB = = =
1.4