Italian-based Large Language Models at the Syntax-Semantics Interface: the Case of Instrumental Role

The linguistic competence of Large Language Models (LLMs) has been the focus of extensive investigation in recent years. Yet, the syntax-semantics interface remains a relatively understudied aspect of LLMs’ linguistic abilities. This study aims to address this gap by focusing on the Instrumental role in Italian. In this language, Instruments can always be syntactically omitted, yet they remain semantically present, as they are recoverable either from the verb meaning alone (when the verb is presented in isolation) or from the interaction between the verb meaning and that of its internal argument (when the verb appears within a syntactic context). To assess the ability of LLMs to semantically determine the most appropriate Instrument(s) from the verb meaning, we conducted two experiments based on psycholinguistically inspired tasks, comparing the performance of GePpeTto and Minerva models (350M, 1B, 3B and 7B) to that of Italian speakers. In the first experiment, verbs were presented in isolation, while in the second, they were presented within a syntactic context. Our findings indicate that the performance of LLMs is influenced by the semantic selectivity of verbs, the presence or absence of a clausal context and model characteristics.