Do parameterisations matter?

A parametrisation of a probability distribution is a choice of parameters used to characterise it. For instance, the Gamma distribution is commonly parametrised either in terms of shape and scale $(\alpha,\theta)$ or shape and rate $(\alpha,\lambda)$. A natural question then arises: which one should we use? Classical works (Huzurbazar, 1956; Cox and Reid 1987) provide a framework for constructing orthogonal parametrisations, which yield a diagonal Fisher information matrix. % and can simplify inference. Such parametrisations have been studied in Bayesian and regression settings, but guidance on their practical relevance remains limited. In this work, we propose a new method to discriminate between parametrisations by studying how neighbourhoods in the space of distributions translate into regions in the parameter spaces. This topological viewpoint provides a criterion for choosing parametrisations depending on whether the goal is accurate estimation of the full distribution or separate estimation of individual parameters.