Simulates whether merging fine-tuned experts preserves or destroys their individual capabilities. 864 configs across 6 methods including a documented non-monotonic failure mode (DARE-TIES is too aggressive on well-aligned models specifically). Ten sources (2022-2026) in merging_theory.py, including Model Soups, Task Arithmetic, TIES-Merging, DARE, DELLA, and a large "in-the-wild" study (4 base LLMs × 12 fine-tunes × 16 tasks).
© 2026 - Copyright AIKosh. All rights reserved.