Download TSA user manual - Copenhagen Trial Unit
Transcript
User Manual for TSA Copenhagen Trial Unit 2011 Pr(|Z|≥c | H0 is true) ≤ α (2) Pr(|Z|=c | H0 is true) = α (3) For the remaining theoretical sections on repeated significance testing (sections 2.2.2 to 2.2.5), we will assume that all statistical tests are two-sided. We will also assume that all test statistic values, Z, are absolute values. We assume the latter because the involved algebra becomes much simpler by doing so. For example, in defining two-sided thresholds for a non-absolute test statistic, one would need to consider the probability that Pr(Z≤-c or Z≥c | ... ) rather than Pr(|Z|≥c | ... ). Problems with repeated significance testing Conventional single significance tests can be considered reliable if ‘enough’ data has accumulated. In meta-analysis, a single significance test can be considered reliable once the required information size is surpassed.14;6;11;12;20;59 If we perform a single test for statistical significance at or after a meta-analysis has surpassed its required information size, statistical significance testing simply entails determining an appropriate threshold, c, that will make equations (2) and (3) true. For example, for α=5% we would consider c=1.96 appropriate if the meta-analysis data had not previously been subjected to significance testing. When a cumulative meta-analysis is subjected to significance testing more than once (before surpassing its required information size), the situation becomes more complex. Consider the example where a meta-analysis is updated once and where the conventional 5% maximum type I error is used. In this situation, the first meta-analysis yields a Z-value, Z1, and the metaanalysis update yields another, Z2. If the first meta-analysis yields a Z-value larger than 1.96, the two investigated interventions are declared significantly different. However, if the first meta-analysis is not significant (i.e., Z1<1.96), the two interventions can still be declared statistically significant if the metaanalysis update yields a Z-value larger than 1.96 (i.e., if Z2≥1.96). By the laws - 36 -