Beyond Routing: Diagnosing Modular LoRA Experts for Low-Resource Multilingual Petroleum-Domain Translation
Tech Stack
Description
This project is a diagnostic study of modular LoRA adaptation for multilingual petroleum-domain translation. We investigate how routing accuracy, expert specialisation, and source distribution affect the benefits of language-specific LoRA experts.
The experiments show that routing accuracy alone does not explain modular performance. Cross-expert and authentic-source evaluations reveal the roles of expert specialisation, overlap, and source distribution.
As first author and experimental lead, I designed the modular framework, synthetic-data pipeline, routing experiments, and authentic-source evaluation. I also led the cross-expert and terminology-aware analyses.
- Built a modular multilingual NMT system with language-specific LoRA experts and learned routing over a frozen NLLB backbone.
- Developed Target-Anchored Synthesis, an LLM-based pipeline for generating and filtering petroleum-domain parallel data when authentic training data were unavailable.
- Analysed expert specialisation and routing through cross-expert evaluation, routing ablations, linear probing, and controlled representation interventions.
- Found that suppressing language-related information in encoder representations sharply reduced routing accuracy while only modestly affecting translation quality; the routed system outperformed shared Multitask LoRA on authentic-source evaluation.
Research Questions
Can Target-Anchored Synthesis provide useful training data?
Do language-specific LoRA experts specialise, and how much do they overlap?
When does expert selection improve translation quality, and does more accurate routing lead to better translations?
System Overview
Figure 1. Expert training, router training, and top-1 inference.
The system uses a frozen NLLB-200-distilled-600M backbone with four language-specific LoRA experts. The EN expert is trained on authentic EN–NO data, while the DE, NL, and FR experts are trained on synthetic source sentences paired with Norwegian targets.
After training the experts, we train a gated MLP router on mixed multilingual data, while keeping the backbone and experts frozen. At inference, the router selects one expert for each input sentence and uses that expert to generate the Norwegian translation.
We compare three settings: Independent Experts, which directly use the expert corresponding to the source language; Multitask LoRA, which uses one shared LoRA adapter for all languages; and MoE, which uses the router to automatically select an expert.
Target-Anchored Synthesis
Figure 4. Target-Anchored Synthesis and filtering stages.
We start with authentic English–Norwegian petroleum data from the Norwegian Petroleum Directorate (NPD). The Norwegian sentences are kept unchanged as reliable target anchors.
Step one is generation. We scan the NPD glossary and inject 70 validated terminology pairs into the prompt as hard constraints. GPT-4o-mini then generates German, Dutch, and French source sentences from the corresponding English sources.
Step two is quality checking. We use LaBSE similarity to check whether the meaning is preserved, and FTA to check terminology accuracy.
Step three is a round-trip check. We back-translate the generated sources into English and use BLEU as a diagnostic only, without a hard threshold.
Step four is the final quality gate. A pair is discarded only when both the LaBSE and FTA checks fail. Otherwise, it is retained.
The final corpus contains 51,890 synthetic pairs across the training, development, and test splits, including 41,527 training pairs.
RQ1 · Can Target-Anchored Synthesis provide useful training data?
We first test whether models trained on the synthetic data learn a useful translation signal. We then use human evaluation to assess the quality of the generated source sentences.
Translation results — Does the synthetic data work for training?
| Language pair | BLEU | % of EN–NO BLEU | FTA gain vs Google Translate |
|---|---|---|---|
| DE–NO | 57.7 | 93.5% | +0.013 |
| NL–NO | 59.3 | 96.1% | +0.121* |
| FR–NO | 57.8 | 93.7% | +0.062* |
EN–NO benchmark: 61.7 BLEU. Synthetic-data experts reach 93.5–96.1% of the benchmark, with statistically significant terminology gains for NL–NO and FR–NO. * Statistically significant improvement over Google Translate.
Human validation — Are the generated sources reliable?
300 source sentences were manually evaluated—100 each in German, Dutch, and French.
4.86 / 5
Adequacy
4.70 / 5
Fluency
94.4%
Terminology accuracy
The generated sources generally preserve the intended meaning, remain fluent, and use the required petroleum terminology correctly.
RQ2 · Do language-specific LoRA experts specialise, and how much do they overlap?
Overall comparison
| System | BLEU (avg) | FTA (avg) |
|---|---|---|
| Independent Experts | 59.1 | .726 |
| Multitask LoRA | 61.0 | .713 |
| MoE hard routing | 58.4 | .711 |
Multitask LoRA gets the highest average BLEU, while Independent Experts get the highest average FTA. MoE hard routing is lower on both metrics.
Here, we use the MoE mainly to study what the experts learn and how routing works.
Cross-expert analysis
Bold cells show each language with its matching expert.
Specialisation
The matching expert gets the highest BLEU for all four languages.
→ The experts learn language-specific skills.
Overlap
Other experts can also perform quite well. For example, the DE expert gets 56.2 BLEU on the Dutch test data, compared with 59.5 BLEU from the NL expert.
→ The experts still share some capabilities.
Conclusion: The experts are specialised, but they are not completely separate. This helps explain why choosing the wrong expert does not always cause a large drop in BLEU.
RQ3 · When does expert selection improve translation quality, and does more accurate routing lead to better translations?
01 · Does better routing improve translation?
Hard-routing confusion matrix
True source → routed expert
Rows show source languages; columns show selected experts. Diagonal cells are correct routes.
The router gets 64.8% of routes correct overall. German is the hardest to route correctly (40.1%), while Dutch is the easiest (86.8%).
Learned router vs. LangID router
| Router | Routing accuracy | BLEU | FTA |
|---|---|---|---|
| Learned | 64.8% | 58.4 | .711 |
| LangID | 77.5% | 58.9 | .720 |
Replacing the learned router with LangID improves routing accuracy by 12.7 points, but BLEU increases by only 0.5 and FTA by .009.
More accurate routing does not automatically lead to much better translation.
02 · What information does the router use?
Controlled routing interventions averaged over three runs. The left plot removes language information from different encoder layers. The right plot removes important or random hidden units from the router. These runs use a separate 63.4% mean routing baseline.
Language information
We removed language information from different encoder layers and checked whether the router selected a different expert.
- Layer 12: routing accuracy −22.6 points
- 33.5% of routes change
- Random removal: only about 0.4 points
The router uses language information to choose an expert.
Important router units
We removed the 16 hidden units that were most important to the router and compared them with 16 random units.
- Top 16 units: routing accuracy −6.64 points
- Random 16 units: no clear reduction
A small number of hidden units are especially important for choosing the expert.
Does changing the expert change translation?
Routing accuracy
−22.6 points
BLEU
only −0.74
This result fits the expert overlap seen in RQ2.
The router uses language information to pick an expert. But a different expert often gives a similar translation.
03 · When does expert selection help?
| System | Synthetic-source BLEU | Authentic-source BLEU |
|---|---|---|
| Multitask LoRA | 61.0 | 39.30 |
| Independent Experts | 59.1 | 42.10 |
| MoE | 58.4 | 41.89 |
Independent Experts use the matching expert for each language. MoE chooses the expert automatically.
On the synthetic-source test, Multitask LoRA performs best.
On authentic petroleum text, Independent Experts perform best. MoE also performs better than Multitask LoRA and comes close to Independent Experts.
The benefit of expert selection depends on the source data.
Conclusion: Better routing does not always mean better translation. Different experts can give similar results. On authentic petroleum text, MoE performs better than Multitask LoRA and comes close to the matching expert.