Can a significant effect remain if we re-analyze it in a subpopulation with lower N?

In the benefit assessment of drugs, it may happen that a benefit must be re-examined in a subpopulation, for example if the approval is restricted to a subpopulation which particularly benefits from the treatment. In these cases, a previously demonstrated significance can be lost because the sample size and thus the power of the statistical analysis are reduced.

What a pity - but there is a solution! The IQWiG has described how to proceed in such cases, in the working paper GA18-01 with the title “The investigation of statistical properties of procedures to transfer study results to subpopulations”.

The paper addresses the following question: Can we postulate a significant treatment effect that we have found for the entire study population also for the subpopulation, although the effect in this subpopulation does not become significant due to lower power?

What the IQWiG describes there is the lifebuoy for a statistical significance threatened by drowning: a 5-step flow chart that, through a combined consideration of treatment effects and interactions, leads to the answer as to whether the significant effect in the entire study population can also be assumed for the subpopulation.

The IQWiG working paper is a sportive experience for every statistician who has understood everything about null and alternative hypothesis so far - read it, enjoy it, and realize that you should read it again from the beginning.

You can access the IQWiG working paper here: (Deutsch)