ARE THE RESULTS VALID.docx

ARE THE RESULTS VALID? Did intervention and control groups start with the same prognosis? Were patients randomized? Was group allocation concealed? Were patients in the study groups similar with respect to known prognostic variables? 1) Yes - participants were randomly assigned to one of the t...

ARE THE RESULTS VALID? Did intervention and control groups start with the same prognosis? Were patients randomized? Was group allocation concealed? Were patients in the study groups similar with respect to known prognostic variables? 1) Yes - participants were randomly assigned to one of the three intervention groups using computer generated block randomization 2) No - there is no mention of how randomization allocation sequence was concealed to those enrolling patients 3) No - there is no information provided in the article that the groups were balanced following randomization. We cannot assume that randomization was successful because we are unsure if randomization was concealed and no information on baseline characteristic (i.e., Table 1 is not provided) Did the intervention and control groups start with the same prognosis? This is ether a YES or NO answer based on how you answered the above questions. For me I would state: NO - there is insufficient information provided in the manuscript to convince me that these two groups were indeed balanced. Lack of allocation concealment (potentially) and lack of a baseline table leads me to error on the side of caution and indicate that the groups were not balanced. Was prognostic balance maintained as the study progressed? To what extent was the study blinded? NO -Although it is likely the patients were blinded because of the use of placebos, there is no information in the manuscript regarding the blinding of nurses, analysts etc. As a result, although placebo\'s were used, blinding was likely not maintained appropriately throughout the trial. As a result prognostic balance was not likely retained. Were the groups prognostically balanced at the study\'s completion? Was follow-up complete? Were patients analyzed in the groups to which they were first allocated? Was the trial stopped early? 1. Was follow-up complete -- No - only 87% completed follow-up. As a result, a substantial number of losses could clearly affect the study results as more than 5% were lost. There is also no mention of any differential losses to follow-up which are equally important in assessing losses to follow-up 2. Were patient analyzed in the groups assigned? NO - From table 1, we can see that only 1179 were included in the results for years 1-4 and 1085 in years 2- 4. Thus, an ITT was not completed as once randomized - analyze was not completed! 3. Was the trail stopped early? NO It does not appear that the trial was stopped early. Thus, the groups were not prognostically balanced at the end of the study because the substantial losses to follow-up and the fact that ITT analysis was not maintained. WHAT ARE THE RESULTS? Overall, although there were large statistically significant reductions, the clinical importance of this information is questionable. The confidence intervals were relatively imprecise, and a statistically significant effect was only observed for calcium and vitamin D compared to placebo. There\'s no evidence beyond chance that calcium alone improves cancer outcomes. How large was the treatment effect? What was the relative risk reduction? What was the absolute risk reduction? There was a large statistically significant but questionable clinically significant 60% reduction in the incidence of cancer for the vitamin D and calcium group compared to placebo. Although the 60% reduction in the incidence of cancer is clinically meaningful, the confidence intervals around this estimate were extremely large ranging from and 80% reduction to a 17% reduction. Because the conference intervals include our importance threshold of 20% reduction, it is possible the results are not as clinically significant as the point estimate would lead us to believe. Conversely, for the calcium only group there was a 47% relative risk reduction compared to placebo which is beyond our 20% threshold, however, the confidence intervals are again very wide leading to questionable importance. Moreover, these results were not statistically significant and as a result may have occurred due to chance. In terms of absolute risk reduction from table 1 we can determine that for the calcium vitamin D arm of the study there was a 4% absolute risk difference \[6.9- 2.9\]. The calcium only arm there is a 3.1% absolute risk reduction \[6.9-3.8\]. Again, both of these estimates would suggest a large absolute risk reduction that may be clinically relevant to most patients. However, for the calcium vitamin D group although a clinically important 47% reduction in the incidence of cancer was observed this was not statistically significant. How precise was the estimate of the treatment effect? What were the confidence intervals? As indicated, the precision around the estimates was poor. The confidence intervals for both estimates are extremely large. For the calcium and vitamin D arm the conference interval ranged from 0.2 upwards to 0.82, P=0.013 which was statistically significant but very imprecise. Conversely, for the calcium only group the conference interval range from 0.27 upwards to 1.03, P=0.063, indicating that again these results are relatively imprecise and in this case were not statistically significant