LoRA Fine-Tuning of English-Norwegian NMT for the Oil & Gas Industry

RQ1. How much training data do we need? RQ2. Which LoRA hyperparameters matter most? RQ3. How well does the final model perform?

We designed a three-stage experimental framework to answer these questions. Experiment 1 determines the training-data budget, Experiment 2 identifies and validates the LoRA configuration, and Experiment 3 evaluates the final model against strong baselines and through domain-aware human analysis.

Before training, I built a data diagnostic and cleaning pipeline to assess the quality of the raw English-Norwegian translation data. The pipeline checks sentence length, alignment, completeness, duplicates, and domain relevance, then normalizes the text and removes very short sentences and duplicate pairs. The corpus is finally divided into fixed training, development, and test sets using an 80/10/10 split. The raw data has a high overall quality score (OQS 2.0 = 0.976), while 30.9% of sentence pairs were removed as duplicates. The high level of duplication mainly comes from repeated standard text in regulatory documents, while Nynorsk sentences are deliberately retained.

We trained LoRA models with nine different subsets of training data, ranging from 100 to 13,935 sentence pairs, using three random seeds for each setting. The learning curve shows three stages: rapid gains up to 2,000 pairs, diminishing returns from 2,000 to 8,000, and little additional improvement beyond 8,000. The 8,000-pair subset used 57% of the full training data while retaining 96% of the maximum BLEU, so we selected it as the training budget for subsequent hyperparameter optimization.

We used two complementary search strategies to study LoRA rank, scaling factor, and dropout. Grid search mapped a fixed set of configurations, while Optuna with ASHA explored the space more flexibly and stopped weak trials early. The strongest candidates were then compared across BLEU and chrF and retrained with multiple random seeds.

We tested 27 LoRA configurations with different ranks, alpha values, and dropout rates. Higher alpha generally improved BLEU, while rank and dropout had smaller effects. The best setting was r = 8, alpha = 64, dropout = 0. Since grid search tests only fixed values, better settings may exist between or beyond them.

We ran 50 Optuna trials on a 2,000-pair subset to explore a wider range of LoRA configurations. Optuna proposed new parameter combinations, while ASHA stopped poorly performing trials early. The most promising configurations were then selected for further evaluation on the 8,000-pair subset. An fANOVA analysis attributed 97.3% of the observed performance variation to alpha, compared with 1.9% for rank and 0.8% for dropout.

We used Optuna to optimize both BLEU and chrF and identified Pareto-optimal configurations that performed well on both metrics. We selected the top three candidates and retrained them on 8,000 pairs with three random seeds. Based on their performance and stability, we selected r = 8, alpha = 64, dropout = 0 as the final configuration.

We evaluated the selected LoRA model through three comparisons: zero-shot and commercial baselines, full fine-tuning, and human analysis of translation errors.

We compared the final LoRA model with zero-shot NLLB and commercial MT systems. It achieved comparable performance on BLEU, chrF++, and COMET. However, these results are based on a limited dataset and may not fully reflect real-world petroleum-domain performance.

LoRA achieved similar BLEU to full fine-tuning across different training sizes, with fewer trainable parameters and less computation.

We conducted a human error analysis to complement BLEU, chrF, and COMET. We reviewed 50 held-out NPD test sentences from low-, mid-, and high-BLEU groups. Two reviewers annotated errors by type and severity. Agreement was substantial for error type (κ = 0.71) and moderate for severity (κ = 0.61).

We found 62 errors across 41 sentences. Most were minor, but some critical errors changed the meaning. Word choice and Norwegian variant mixing were the most common errors. Some critical errors appeared even in high-BLEU translations, showing that human evaluation is still important for domain-specific translation.