Design Analysis was introduced by Gelman &amp; Carlin (2014) as an extension of Power Analysis. Traditional power analysis has a narrow focus on statistical significance. Design analysis, instead, evaluates together with power levels also other inferential risks (i.e., Type M error and Type S error), to assess estimates uncertainty under hypothetical replications of a study. Given an hypothetical value of effect size and study characteristics (i.e., sample size, statistical test directionality, significance level), Type M error (Magnitude, also known as Exaggeration Ratio) indicates the factor by which a statistically significant effect is on average exaggerated. Type S error (Sign), instead, indicates the probability of finding a statistically significant result in the opposite direction to the hypothetical effect. Although Type M error and Type S error depend directly on power level, they underline valuable information regarding estimates uncertainty that would otherwise be overlooked. This enhances researchers awareness about the inferential risks related to their studies and helps them in the interpretation of their results. However, design analysis is rarely applied in real research settings also for the lack of dedicated software.

### PRDA: An R package for Prospective and Retrospective Design Analysis

#### Abstract

Design Analysis was introduced by Gelman & Carlin (2014) as an extension of Power Analysis. Traditional power analysis has a narrow focus on statistical significance. Design analysis, instead, evaluates together with power levels also other inferential risks (i.e., Type M error and Type S error), to assess estimates uncertainty under hypothetical replications of a study. Given an hypothetical value of effect size and study characteristics (i.e., sample size, statistical test directionality, significance level), Type M error (Magnitude, also known as Exaggeration Ratio) indicates the factor by which a statistically significant effect is on average exaggerated. Type S error (Sign), instead, indicates the probability of finding a statistically significant result in the opposite direction to the hypothetical effect. Although Type M error and Type S error depend directly on power level, they underline valuable information regarding estimates uncertainty that would otherwise be overlooked. This enhances researchers awareness about the inferential risks related to their studies and helps them in the interpretation of their results. However, design analysis is rarely applied in real research settings also for the lack of dedicated software.
##### Scheda breve Scheda completa Scheda completa (DC)
2021
6
File in questo prodotto:
File
joss.02810.pdf

accesso aperto

Tipologia: Versione dell'editore
Licenza: Accesso gratuito (solo visione)
Dimensione 141.02 kB
Utilizza questo identificativo per citare o creare un link a questo documento: `https://hdl.handle.net/10278/3753411`