There’s always a gap between theory and practice. This is especially true when it comes to sports science research. When you ask a scientist about good research, it comes down to statistical significance. But for coaches, what matters is coaching significance.
Statistical significance is no doubt important in the academic world, and important to sports science as a whole. Like any statistics, it can be manipulated via p-hacking, but it sets a standard for the field to live up to.
The problem comes when we translate it into practice. A study might have a statistically significant finding that some new training method improves training by .0001%. It is statistically significant, so the result likely isn’t by chance. But as a coach reading the research, what difference does it make? The athletes barely improved. Don’t get lured by a headline touting significance when it still won’t help your athletes get better.
On the other hand, there are results that might not be statistically significant because due to a small sample size or other variables, but show large 2-3% improvements from an intervention. This is significant for coaches. It’s not that coaches should drop everything and try the new methods, but it is significant enough for a closer look. 2-3% is the difference between being a champion and middle of the pack. That is what it is all about.
The best research has both statistical and coaching significance. But those are few and far between. In the meantime it helps to know the difference between statistical significance and coaching significance.