HARKing in a bachelor thesis - I feel tormented

[Carl-Richard]

That moment when the first sentence in the introduction of the most widely used measurement device for dispositional mindfulness in the scientific community is a reference to Ken Wilber 

[Michael569]

On 15/03/2023 at 11:34 PM, Carl-Richard said:

I ran the analyses, and none of the hypotheses had a statistically significant result (which means both hypotheses get scrapped).

Those are your results. There is a non-statistically significant association between mindfulness and physical activity. That's A result. You don't need to find a positive association to have a valid hypothesis and valid research. 

Post hoc hypothesis is a common practice due to publishing bias but science is not immune to this. It all depends on you. 

If you think conducting the original experiment would teach you more (despite result being non-statistically significant) then you should follow your original intuition, otherwise, you run the risk of opening yourself up to major biases even while doing the new paper. 

Also consider that statistically non-significant often indicates lack of evidence rather than the absence of an effect. Sometimes the statistical power is low so confidence intervals are barn door and you don't get what you wanted but then 5 years later the amount of research catches up and it becomes SS (or it doesn't, confidence intervals widen even further and then you will be able to say "yah there is definitely nothing there" 

Just out of curiosity, how did you assess the SS? Did you run a forest plot in Revman? 

Edited March 26, 2023 by Michael569

[Carl-Richard]

1 hour ago, Michael569 said:

Those are your results. There is a non-statistically significant association between mindfulness and physical activity. That's A result. You don't need to find a positive association to have a valid hypothesis and valid research. 

Post hoc hypothesis is a common practice due to publishing bias but science is not immune to this. It all depends on you. 

If you think conducting the original experiment would teach you more (despite result being non-statistically significant) then you should follow your original intuition, otherwise, you run the risk of opening yourself up to major biases even while doing the new paper. 

I'm fine with what I'm doing now

 

1 hour ago, Michael569 said:

Also consider that statistically non-significant often indicates lack of evidence rather than the absence of an effect. Sometimes the statistical power is low so confidence intervals are barn door and you don't get what you wanted but then 5 years later the amount of research catches up and it becomes SS (or it doesn't, confidence intervals widen even further and then you will be able to say "yah there is definitely nothing there" 

After I found a standardized data cleaning protocol for the survey that I used for measuring physical activity (Godin's Leisure-Time Questionnaire), the first hypothesis turned out to actually be statistically significant at p < 0.10, which is cool, but it's a really weak effect (the standardized regression coefficient is at 0.070). I could probably easily add another 50% to the sample size if I wanted to, but I would rather move on to more important stuff.

 

1 hour ago, Michael569 said:

Just out of curiosity, how did you assess the SS? Did you run a forest plot in Revman? 

I used a linear regression in JASP.