Empirical Analysis of Efficient Fine-Tuning Methods for Large Pre-Trained Language Models

The advent of large pre-trained language models, such as BERT (Devlin et al., 2018) and GPT3 (Brown et al., 2020), has revolutionized the field of natural language processing (NLP). These models havedemonstrated remarkable performance across a wide range of language tasks. However, their effectiveness often hinges on the ability to fine-tune them for specific tasks. Traditional finetuning involves using the pre-trained model as an encoder to generate contextual representations, followed by adding a task-specific classification layer. This combined network is then trained endto-end to minimize the task-specific loss. While this approach has proven effective, it is not without its limitations.

One significant challenge, as highlighted in Zaken et al. (2021), is the sheer size and complexity of these models, which can obscure understanding of the modifications made during fine-tuning. Additionally, this method becomes cumbersome when scaling to a large number of tasks, as it requires retraining the entire model for each new task. Other notable drawbacks of full end-to-end f ine-tuning include the substantial computational resources and time required, which can be prohibitive. The risk of overfitting is also heightened with larger models, especially when fine-tuned on smaller datasets. The extensive resources required for such training raise concerns about energy consumption and environmental impact. Moreover, the large size of fully fine-tuned models presents challenges in deployment, particularly in resource-constrained environments. Maintaining and updating these models, especially in production environments, can be challenging due to the need for constant updates with new data or for new tasks. 


In response to these challenges, Zaken et al. (2021) proposes four criteria for an improved fine-tuning framework:

The evolution of machine learning has defied traditional statistical expectations, particularly with the advent of increasingly larger models. Contrary to the anticipated implications of the bias-variance trade-off, these larger models have consistently demonstrated superior performance (Belkin et al., 2019). This trend is especially evident in the realm of language models (LMs), where models like BERT(Devlin et al., 2018) and GPT-4 (Bubeck et al., 2023) have showcased remarkable capabilities that scale with the addition of more parameters. These models benefit from extensive pre-training on vast text corpora, which steers their learning towards optimal minima, thereby enhancing generalization (Erhan et al., 2010).

Fine-tuning is a critical step in harnessing the power of these language models. It involves additional optimization using a smaller subset of task-specific parameters. For instance, fine-tuning BERToften results in a model whose parameters remain closely aligned with the original, requiring adjustments only in the most crucial layers (Radiya-Dixit and Wang, 2020). Various strategies have been developed for fine-tuning, such as ULMFiT, which significantly reduced error rates by approximately 20% (Howard and Ruder, 2018). Another notable approach is the T5 (Text-to-Text Transfer Transformer) model by Raffel et al., which reframes different tasks as text-to-text problems, offering a unique perspective on fine-tuning (Raffel et al., 2019). Factors like weight initialization and the order of training data are crucial in fine-tuning, especially given the typically small size of training datasets in this phase (Dodge et al., 2020).

Concurrently, research is delving into how language models can more effectively interact with humans, a critical aspect as these models become more integrated into various applications and industries. There have been efforts to fine-tune language models towards human preferences, consisting of a system of rewards defined by human judgement (Ziegler et al., 2019). Later work in this line of inquiry has included interactive fine-tuning, with a conversational style interface allowing users to directly influence model behavior with natural language (Vuli´ c et al., 2021). Fine-tuning can often involve sensitive, personally identifiable data, so differential privacy has been employed to reduce the risks of utilizing such data while still tailoring the model to specific applications. (Yu et al., 2021).

Adapters, introduced by Rebuffi et al., aim to learn a single representation adaptable to various tasks, mirroring the objectives of pre-training (Rebuffi et al., 2017). Houlsby et al. further developed this concept, designing an adapter module that adds a few layers to a pre-trained model. This design facilitates rapid learning of minimal task-specific parameters (Houlsby et al., 2019). Both studies tested their methods on pre-trained BERT models, evaluating performance against the GLUE benchmarks (Wang et al., 2018). When evaluating adapter effectiveness it has been shown to be more robust to overfitting and in some circumstances adapter based tuning can outperform finetuning such as on low-resource and cross lingual tasks. (He et al., 2021) Work has been done to combine both adapter models and fine tuning by only tuning the adapter layers. One such method, Adamix, has demonstrated to be capable of outperforming models that were trained using each of these approaches separately (i.e. adapter models, or fine tuned) suggesting that improvements could be made by combining these techniques. (Wang et al., 2022) Recent advancements have also seen large language models (LLMs) being used to fine-tune other LMs, with the effectiveness varying depending on the task (Peng et al., 2023).

The scope of language models has broadened to encompass multimodal capabilities, as exemplified by CLIP (Contrastive Language-Image Pre-training) from Radford et al., which effectively learns joint representations of text and images (Radford et al., 2021). In specific domains, such as healthcare, tailored models like BioBERT have been developed for biomedical text, demonstrating the adaptability of language models to specialized fields (Lee et al., 2019). With the increasing size of these models, there is a growing focus on enhancing their efficiency. Techniques such as model distillation, knowledge distillation, and pruning are being explored to reduce the computational and memory demands of large models while maintaining their performance. There have also been efforts towards replicating the functionality of deep neural networks with more interpretable and less resource intensive architectures (Gorlla, 2023).

We utilize a subset of the General Language Understanding Evaluation (GLUE) benchmarks, specifically focusing on the MRPC, COLA, and STS-B datasets. These datasets have been curated as part of GLUE to serve as a comprehensive benchmark for classification and regression tasks in natural language understanding.

The MRPC (Microsoft Research Paraphrase Corpus) dataset is centered around a binary classification task that involves determining whether pairs of sentences are paraphrases of each other. Originating from online news sources, MRPC contains approximately 5,800 sentence pairs, and its evaluation is based on a combination of accuracy and F1 score, providing a measure of both precision and recall. The COLA (Corpus of Linguistic Acceptability) dataset, on the other hand, is designed for a binary classification task to assess grammatical correctness. It comprises sentences extracted from linguistic publications, totaling around 8,500 sentences. The key metric for COLA is the Matthews Correlation Coefficient (MCC), which offers a balanced measure of quality in binary classifications, especially useful in datasets with class imbalances. Lastly, the STS-B (Semantic Textual Similarity Benchmark) dataset involves scoring pairs of sentences based on their semantic similarity on a scale from 0 to 5. This dataset, which includes about 7,000 sentence pairs from sources like news headlines and image captions, is evaluated using the Pearson and Spearman correlation coefficients, reflecting the degree of semantic similarity captured by the model.

Our approach involves dividing these datasets into training and test groups and evaluating the performance using the corresponding GLUE task and metric. This methodology allows us to cover a diverse range of NLP tasks—from semantic equivalence in MRPC and STS-B to linguistic acceptability in COLA—providing a comprehensive evaluation of our models’ performance across different facets of language understanding. We use a non-modified BERT Base pre-trained model as a baseline to compare the effectiveness of the fine-tuning techniques we investigate, specifically focusing on BitFit and Adapter layers methods.

The BitFit methodology, as introduced in (Zaken et al., 2021) proposes a streamlined approach to f ine-tuning large language models by focusing primarily on the bias terms. This method, which stands for BIas-Term Fine-Tuning, involves freezing most of the transformer-encoder parameters and training only the bias terms and the task-specific classification layer. The key properties of BitFit are: (i) matching the results of a fully fine tuned model, (ii) enabling tasks to arrive in a stream without requiring simultaneous access to all datasets, and (iii) fine-tuning only a small portion of the model’s parameters.

The approach is highly parameter-efficient: for each new task, it requires storing only the bias term parameter vectors, which constitute less than 0.1% of the total number of parameters, in addition to the task-specific final linear classifier layer.

In our study, we aim to conduct a thorough comparison of three fine-tuning methodologies applied to the BERT Base model from HuggingFace: full end-to-end fine-tuning, BitFit, and Adapter layer methods. Our primary objective is to evaluate not only the performance of each method but also their efficiency and practicality concerning time and computational constraints. All experiments were performed on the Python 3 Google Compute Engine backend with the NVIDIA Tesla T4 GPU and 13 GB of system RAM.

Our approach begins with a sequential analysis to understand how data efficiency varies across these fine-tuning methods. We plan to train the BERT Base model for up to 20 epochs using varying proportions of the training data, starting from 30% and gradually increasing to 50%, 70%, and finally 100%. This incremental training process will allow us to observe the amount of data each method requires to achieve performance comparable to full fine-tuning. By comparing the performance metrics at each data level, we can identify the method that demonstrates the highest data efficiency.

Complementing this, we will conduct a time-constrained analysis where each fine-tuning method is applied to the BERT model for a fixed duration of 10 minutes. This experiment is designed to provide insights into the practicality of each method under time limitations. We will measure and compare the performance achieved by each method after this training period, offering a more detailed view of their efficiency.

To assess the tendency of each method to overfit, we introduce a validation set in addition to the training and test sets. We will monitor the performance of the models on both the validation and test sets after training for up to 20 epochs. This step will help us ascertain whether the models are merely optimizing for the test set or exhibiting more generalizable performance. For each of these experiments, standard metrics associated with each GLUE task dataset will be used: accuracy and F1 score for MRPC, Matthews Correlation Coefficient for COLA, and Pearson and Spearman correlation coefficients for STS-B. These metrics provide a comprehensive understanding of model performance across various aspects of language understanding.

In terms of implementation, the BERT Base model as implemented in the HuggingFace library will serve as the foundation for all our experiments. For BitFit, we will freeze all parameters except the bias terms and the classification layer, while for Adapter layers, we will integrate trainable modules into the pre-trained model while keeping the original weights fixed.

In our research, we explored the effectiveness of various fine-tuning methodologies, particularlyfocusing on the BitFit and Adapter Layer methods compared to traditional full fine-tuning. One ofour key takeaways is that despite the dramatic reduction in the number of parameters altered in theBitFit and Adapter methods, there was not a significant gain in performance over full fine-tuning.This outcome is intriguing as it suggests that the extensive parameter modifications in traditionalfine-tuning might not always be necessary for achieving comparable results.

BitFit, interestingly, showed a certain degree of stability with smaller data sets, although these findingswere not entirely conclusive. Our observations suggest that while BitFit fine-tunes fewer parameters,it may offer a balance between performance and parameter efficiency, particularly in dataconstrainedscenarios. This aspect makes BitFit an appealing alternative if the goal is to enhanceinterpretability by minimizing the number of changed weights. However, this should be weighedagainst other approaches to model interpretability, which might offer more direct insights into modelbehavior without altering the fine-tuning process.

The Adapter Layer method, on the other hand, presented notable challenges. We were unable to reproducethe stability and performance reported in the original paper, indicating potential deficienciesin this approach. Our results suggest that while the Adapter Layer method theoretically offers a flexibleand efficient fine-tuning mechanism, it may require more sophisticated optimization strategiesto ensure stable and reliable performance.

Looking ahead, a deeper comparative analysis between BitFit and full fine-tuning could providemore insights, particularly using a broader range of datasets. Such an analysis would help ascertainwhether BitFit consistently offers advantages in fitting models to smaller datasets. Additionally,examining the tendency of each method to overfit, especially in the context of full fine-tuning, would  be valuable. There exists a potential risk that allowing every model weight to change in a smalldataset downstream task might lead to overfitting, an aspect that deserves further investigation.

In conclusion, our findings point towards BitFit as a potential alternative to traditional full finetuning, especially when parameter efficiency and stability are considered. The Adapter Layer method, despite its promise, requires further exploration to overcome its training instability and achieve the touted benefits. Our results reiterate the need for efficient and stable model tuning approaches.