The integration of large language models (LLMs) into translation practice has substantially reshaped translation workflows (Kornacki and Pietrzak, 2025). Since translation quality depends partly on how these models are prompted, prompt design deserves closer attention as a key stage of the LLM-augmented translation process. This study investigates Spanish<>Italian translation with GPT 5.1 in the advertising and biomedical domains. It examines whether prompt language affects the quality of translations generated with the GPT 5.1 model, and specifically whether prompts written in the target language outperform prompts written in English, the language most prevalent in the model’s training data (Armengol-Estapé et al., 2022). Three prompt templates, varying in complexity and informational content, were tested. The translations were first screened for textual similarity, and only the translations generated from the template that produced the greatest variation across outputs were subsequently selected for human evaluation. Human judgments were collected through a pairwise-comparison task. The findings indicate that prompts written in the target language tend to yield higher-quality translations compared to prompts written in English.
Evaluating the Effect of Prompt Language on LLM-based Translation: Evidence from Spanish<>Italian Translation
Bove Antonella
;
2026
Abstract
The integration of large language models (LLMs) into translation practice has substantially reshaped translation workflows (Kornacki and Pietrzak, 2025). Since translation quality depends partly on how these models are prompted, prompt design deserves closer attention as a key stage of the LLM-augmented translation process. This study investigates Spanish<>Italian translation with GPT 5.1 in the advertising and biomedical domains. It examines whether prompt language affects the quality of translations generated with the GPT 5.1 model, and specifically whether prompts written in the target language outperform prompts written in English, the language most prevalent in the model’s training data (Armengol-Estapé et al., 2022). Three prompt templates, varying in complexity and informational content, were tested. The translations were first screened for textual similarity, and only the translations generated from the template that produced the greatest variation across outputs were subsequently selected for human evaluation. Human judgments were collected through a pairwise-comparison task. The findings indicate that prompts written in the target language tend to yield higher-quality translations compared to prompts written in English.| File | Dimensione | Formato | |
|---|---|---|---|
|
2026.eamt-1.51.pdf
accesso aperto
Tipologia:
Versione dell'editore
Licenza:
Creative commons
Dimensione
399.45 kB
Formato
Adobe PDF
|
399.45 kB | Adobe PDF | Visualizza/Apri |
I documenti in ARCA sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.



