The integration of large language models (LLMs) into translation practice has substantially reshaped translation workflows (Kornacki and Pietrzak, 2025). Since translation quality depends partly on how these models are prompted, prompt design deserves closer attention as a key stage of the LLM-augmented translation process. This study investigates Spanish<>Italian translation with GPT 5.1 in the advertising and biomedical domains. It examines whether prompt language affects the quality of translations generated with the GPT 5.1 model, and specifically whether prompts written in the target language outperform prompts written in English, the language most prevalent in the model’s training data (Armengol-Estapé et al., 2022). Three prompt templates, varying in complexity and informational content, were tested. The translations were first screened for textual similarity, and only the translations generated from the template that produced the greatest variation across outputs were subsequently selected for human evaluation. Human judgments were collected through a pairwise-comparison task. The findings indicate that prompts written in the target language tend to yield higher-quality translations compared to prompts written in English.

Evaluating the Effect of Prompt Language on LLM-based Translation: Evidence from Spanish<>Italian Translation

Bove Antonella
;
2026

Abstract

The integration of large language models (LLMs) into translation practice has substantially reshaped translation workflows (Kornacki and Pietrzak, 2025). Since translation quality depends partly on how these models are prompted, prompt design deserves closer attention as a key stage of the LLM-augmented translation process. This study investigates Spanish<>Italian translation with GPT 5.1 in the advertising and biomedical domains. It examines whether prompt language affects the quality of translations generated with the GPT 5.1 model, and specifically whether prompts written in the target language outperform prompts written in English, the language most prevalent in the model’s training data (Armengol-Estapé et al., 2022). Three prompt templates, varying in complexity and informational content, were tested. The translations were first screened for textual similarity, and only the translations generated from the template that produced the greatest variation across outputs were subsequently selected for human evaluation. Human judgments were collected through a pairwise-comparison task. The findings indicate that prompts written in the target language tend to yield higher-quality translations compared to prompts written in English.
2026
Proceedings of the 26th Annual Conference of the European Association for Machine Translation (Volume 1)
File in questo prodotto:
File Dimensione Formato  
2026.eamt-1.51.pdf

accesso aperto

Tipologia: Versione dell'editore
Licenza: Creative commons
Dimensione 399.45 kB
Formato Adobe PDF
399.45 kB Adobe PDF Visualizza/Apri

I documenti in ARCA sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10278/5129730
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
social impact