ReFT outperforms PEFT methods in fine-tuning LLMs
In a new paper, researchers at Stanford University introduce Representation Fine-Tuning (ReFT), a technique that can customize large language models (LLM) for downstream tasks while making very small modifications.
ReFT rivals parameter-efficient fine-tuning (PEFT) methods, which are based on modifying a fraction of the weights. However, instead of modifying weights across all layers, ReFT seeks out representations of concepts that are relevant to the target task and can perform the fine-tuning much more efficiently.
LoReFT, a low-rank implementation of ReFT is 50-100x more efficient than LoRA, its PEFT equivalent. And it competes with the best fine-tuning methods on several key benchmarks.
The researchers have released the code for a Python-based ReFT library and plan to further explore its capacities.
Read about ReFT on TechTalks.
Read the original paper on Arxiv.
For more on AI research:
ncG1vNJzZmialKmypLTTmqOkq16owqO%2F05qapGaTpLpwvI6rnJ%2BsXaTCtbzEq52oqp2oerGxxa1kpp2knbylv4yipQ%3D%3D