Adapter Merging And Unmerging

via “adapter merging and unmerging with weight fusion”

Parameter-Efficient Fine-Tuning (PEFT)

Unique: Implements reversible adapter merging through method-specific merge logic that fuses adapter weights into base weights mathematically (e.g., LoRA: W' = W + alpha/r * A @ B^T), enabling both merged and unmerged states from the same checkpoint. The unmerge operation recovers original weights by subtracting the adapter contribution.

vs others: More flexible than permanent merging because unmerge() enables recovery of original weights and adapter separation, while merged models achieve inference latency parity with non-adapter baselines. Supports both merged and adapter-based deployment strategies from the same training run.