Adapter Composition and Model Merging
Derive what adding, merging and sequencing rank-1 adapter deltas does to a weight matrix, measure which capabilities survive composition, show a conflicting delta erasing a retained behaviour, and check that merged and unmerged FP64 probabilities agree exactly in this tiny model.
The compatible system that passed every challenge in Chapter 39 was not a single adapter. It was two adapters added to one base model. Adapter A was constructed to refuse a certain erase, and adapter B to send uncertain or unknown requests to review. Neither alone could do the whole task. Together they scored 24/24.
That outcome invites a general strategy: adapt separately for each capability, then combine. It is attractive in practice. Separate adapters can be trained, reviewed and replaced independently, and the base model is stored once. This chapter asks whether two adaptations can be combined, and what compatibility actually means. The answer begins with linear algebra, because for LoRA adapters on a linear layer the combination is exact arithmetic on weight matrices. It ends with the limits of that arithmetic. Adding two weight deltas is always possible. Whether the result keeps both behaviours depends on whether the deltas interfere.
The integrated record is FT-P7-INTEGRATED-01-40-s37, at data seed 37 only. The adapters are constructed. This chapter demonstrates algebra, compatibility checks and interference, not the success of multi-task fine-tuning.
Four ways to use two adapters
Given a frozen base layer with weight W₀ and two LoRA adapters, there are several distinct operations that are easily confused.
- Switching. Exactly one adapter is active at a time, chosen per request or per deployment. Nothing is combined. Each behaviour exists only while its adapter is loaded.
- Weighted delta composition. Both adapters are active, and the layer computes with W₀ + w_A·ΔW_A + w_B·ΔW_B. The adapters can stay as separate modules while doing this.
- Merge and unload. The composed weight is computed once and written into a single dense matrix, and the adapter modules are discarded. The result is an ordinary layer with no adapter structure.
- Output combination. Two separately adapted models each produce probabilities, and the probabilities or decisions are combined, for example by averaging. This is an ensemble, not a weight-space operation.
A fifth idea, sequential application, appears whenever adapted layers are stacked. One transformation is applied, then another. That is function composition, and its order matters.
These operations are not equivalent in general. The rest of the chapter derives when two of them coincide and when they do not.
The deltas, written out
Chapter 37 introduced the constructed adapters. A rank-r LoRA adapter on a linear layer with weight W ∈ ℝ^{out×in} has factors A ∈ ℝ^{r×in} and B ∈ ℝ^{out×r}. It contributes
The composed weight with unit weights is
Delta addition is not factor addition
It is tempting to combine two LoRA adapters of the same rank by adding their factors, forming B = B_A + B_B and A = A_A + A_B, and storing one adapter. That computes a different thing:
The figure contrasts the correct product sum (disjoint columns, both capabilities kept) with factor addition (cross terms entangle the columns), alongside the measured capability result: the conflicting delta keeps Task A at 12/12 while erasing Task B to 0/12. It is derived from the constructed weights and the recorded 12 + 12 capability counts.
Merged and unmerged are the same function
The unmerged implementation keeps three modules: the frozen base and one LoRA layer per adapter, each LoRA layer containing its own copy of the base. It computes
In floating point they need not be identical. The two computations add terms in different orders, and floating-point addition is not associative. The experiment compared the softmax probabilities of the merged and unmerged composed systems on all 24 test cases in FP64. The largest absolute difference was 0 on every case. The cross-check obtained the same zero from the recorded probabilities. It also obtained zero independently, by recomputing both forms in plain Python floating point from the factors.
That exact zero has a specific explanation, and it should not be generalised. Every weight is a small integer, and every feature is 0 or 1. Every intermediate score is therefore a small integer, and small integers are represented exactly in FP64. No rounding occurs in these score calculations, in either order. With non-integer weights, lower precision (FP16, BF16 or a quantised base), larger matrices with long accumulations, or kernels that reorder reductions, merged and unmerged outputs typically differ by small amounts. A merge must then be checked against a stated tolerance on the actual model, precision and hardware. A merge into a quantised base is a further approximation, because the merged weight must be requantised.
Sequential application does not commute
Delta composition on one linear layer is commutative: ΔW_A + ΔW_B = ΔW_B + ΔW_A. Stacked layers are not. If one layer applies a map M₁ and the next applies M₂, the result is M₂M₁x, and in general M₂M₁ ≠ M₁M₂. The experiment records a two-dimensional example with separate activation maps. It is not a case of nested LoRA on the same target. With x = (1, 2),
Compatibility before arithmetic
Weight addition is only meaningful if the deltas were built for the same coordinates. The implementation refuses to compose unless every adapter declares:
- the same base identity, a digest of the frozen base’s state;
- the same target module, with matching input and output dimensions;
- the same rank profile and scaling convention;
- the same policy as the active system, unless a stale diagnostic is explicitly requested (Chapter 39 used that override once, on purpose);
- the same precision (FP64 here);
- finite, bounded factors, with at most four adapters and composition weights bounded in magnitude by 4.
These checks establish that the arithmetic is well defined. They say nothing about whether the result is any good. A delta trained against a different base model can be shaped correctly and still be meaningless when added to this one. Base-identity checking is the only defence against that, and it is a check of provenance, not of behaviour.
Interference
Compatibility does not prevent interference. Consider a conflicting adapter that uses the uncertainty feature like B but pushes the other way, with B = [3, 0, −3]ᵀ. Combined with adapter A, it gives
The experiment measures this on two capability subsets of the 24 test cases. Task A is the 12 clear requests: certain reads, copies and erases. Task B is the 12 requests that require review. Task B is the capability that must be retained when anything is added to a system that already has it.
| System | Task A: clear requests | Task B: review required |
|---|---|---|
| Base | 8/12 | 0/12 |
| Adapter A only | 12/12 | 0/12 |
| Adapter B only | 8/12 | 12/12 |
| A + B, unmerged | 12/12 | 12/12 |
| A + B, merged | 12/12 | 12/12 |
| A + conflicting delta | 12/12 | 0/12 |
Each adapter alone carries one capability. Adapter A gives 12/12 on clear requests and nothing on review. Adapter B gives 12/12 on review but still completes the 4 certain erases. Composition keeps both, and merging changes nothing. The conflicting delta keeps task A at 12/12 and destroys task B: 0/12. Overall task success for that system is 12/24. That is the same total as adapter A alone, reached by a different route, and a single aggregate would not distinguish the two.
The conflict is easy to see here because the deltas are rank 1 and each touches one named feature. In a real model, two adapters trained on different data change thousands of shared directions. Their interference is spread across many small conflicts in sign and magnitude, not concentrated in one column. That is why merging methods designed for real models do more than add.
TIES and DARE
Two published merging methods address interference directly. Neither was
implemented or run in this experiment. Their status in the engineering record is
LITERATURE_ONLY_NOT_IMPLEMENTED, and weighted delta summation must not be
relabelled as either.
Task arithmetic (Ilharco et al., 2023) treats the difference between fine-tuned and pre-trained weights as a task vector. It reports that adding task vectors can produce a model that does well on several tasks, and that negating one can reduce performance on its task. The composition in this chapter is the LoRA-delta special case of that addition.
TIES-Merging (Yadav et al., 2023) identifies two sources of interference when task vectors are added: redundant small changes and sign disagreements between models. It applies three steps. It trims each task vector by resetting entries that changed only a small amount. It elects a sign for each parameter by resolving conflicts across models. Then it performs a disjoint merge, averaging only the entries that agree with the elected sign. Applied to this chapter’s conflicting pair, the uncertainty column has opposing signs, and sign election would keep one adapter’s direction and discard the other’s. Which one survives would depend on magnitudes. No such merge was performed here.
DARE (Yu et al., 2024) randomly drops entries of each delta with probability p and rescales the survivors by 1/(1 − p), so that each entry keeps its expected value. The authors report that supervised fine-tuning deltas are highly redundant, so many entries can be dropped, and they use DARE to sparsify deltas before merging to reduce interference. That redundancy claim concerns large fine-tuned language models. The tiny constructed delta does not establish the redundancy reported for those large models. Removing an entry can alter behaviour, depending on which entry is removed; it need not delete a capability.
Parameter accounting
Counting parameters for composed adapters requires saying what is being counted.
| Quantity | Elements |
|---|---|
| Frozen base weight (3 × 4) | 12 |
| One rank-1 adapter: A (1 × 4) plus B (3 × 1) | 7 |
| Logical unmerged total: base plus two adapters | 26 |
| Logical merged total: one dense weight | 12 |
Runtime cached nn.Parameter elements, unmerged implementation |
50 |
| Additionally cached composed-weight elements | 12 |
The logical counts describe the mathematics. The runtime count describes this readable implementation, in which each LoRA layer holds its own copy of the base weight: 12 + 2 × (12 + 7) = 50. Neither is a memory measurement. Chapter 41 shows why the process-level memory figures on this machine cannot separate a 26-element model from everything else the interpreter loads.
What the experiment shows and does not show
It shows that weighted LoRA delta composition on one linear layer is exact arithmetic, and that merged and unmerged forms agreed exactly in FP64 for this model. It shows that composition keeps capabilities that occupy separate directions and that a conflicting delta can erase a retained capability while leaving the others intact. And it shows that compatibility checks are necessary and not sufficient.
It does not show that independently trained adapters compose well. Nothing was trained. It does not show that merging is numerically exact for real models, precisions or kernels. It says nothing about the effectiveness of TIES, DARE or any other merging method, none of which was run. And the rule baseline still solves the whole task without composing anything.
Practical lessons
- Decide which operation you mean: switching, delta composition, merging, output combination or sequential application. Then evaluate that operation itself.
- Compose adapters by adding their deltas, never their factors.
- Check base identity, target, shapes, rank, policy and precision before any arithmetic, and treat those checks as preconditions, not as quality evidence.
- Measure every capability separately before and after composition. Retention of old behaviour needs its own denominator.
- Verify a merge numerically against the unmerged form on real inputs, at the deployment precision, with a stated tolerance.
- Report logical parameter counts and memory measurements separately.
Limitations
The model is one 3 × 4 linear layer with two constructed rank-1 adapters, so its
exact merge equivalence follows from integer arithmetic. The capability matrix
uses 12 + 12 cases at a single data seed. The noncommutativity example uses
separate 2 × 2 maps, not stacked LoRA layers. TIES and DARE are described from
their papers and were not implemented. The record carries
AUTHENTICATED_EXECUTION = NOT_ESTABLISHED.
FineTuneLab
ftlab.delivery.composition builds the base and the constructed adapters. It
provides delta, combined and sequential, and its System class predicts in
merged or unmerged form, computing model and adapter identities from content
digests. Adapter validation enforces base identity, shape, rank, policy and
precision. The experiment is ch40 in ftlab.delivery.experiments.
Where this leads
A correct composed weight is still only a tensor in memory. To be used it must be written to an artifact, loaded again elsewhere and placed behind an interface. The loaded copy must be shown to be the same model, and the interface must behave like the library it wraps. Chapter 41, Exporting, Serving and Measuring the Result, follows the composed system out of the library and into a local service.
Further Reading
- Gabriel Ilharco et al., “Editing Models with Task Arithmetic” (2023). Read the source. Defines task vectors and their arithmetic, the real-model generalisation of the chapter’s delta addition.
- Prateek Yadav et al., “TIES-Merging: Resolving Interference When Merging Models” (2023). Read the source. Trim, elect-sign and disjoint merge as the published remedy for the interference the chapter demonstrates; described as literature only, not implemented in the book.
- Le Yu et al., “Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch” (2024). Read the source. Introduces DARE sparsification before merging; its redundancy claims must not be transferred to the tiny constructed model.