|Year : 2021 | Volume
| Issue : 3 | Page : 292-299
Reporting of sample size parameters for randomized controlled trials published in high impact factor endodontic journals in the last decade: A bibliometric analysis
Fahad Umer, Zainab Haji, Shizrah Jamal
Department of Surgery, Section of Dentistry, Aga Khan University Hospital, Karachi, Pakistan
|Date of Submission||19-Aug-2020|
|Date of Decision||29-Sep-2020|
|Date of Acceptance||30-Sep-2020|
|Date of Web Publication||3-Sep-2021|
Dr. Shizrah Jamal
Department of Surgery, Section of Dentistry, Aga Khan University Hospital, Stadium Road, Karachi
Source of Support: None, Conflict of Interest: None
Introduction: Analysis of journals helps in identifying areas of improvement in the respective field as an author/reviewer. The aim of the present study was to report the parameters used in sample size calculations for randomized controlled trials (RCTs) published in Journal of Endodontics (JOE) and International Endodontic Journal (IEJ) in the last decade.
Materials and Methods: A Medline search retrieved all published RCTs in JOE and IEJ from the last decade. The searche was limited to RCTs studies published in english language from January 01, 2010 to April 31, 2020 having enrolled human participants. Articles were reviewed for description of method used to calculate sample size and reporting of parameters i.e. (level of significance, power, effect size, and variance) used for sample size calculation. The data were statistically analyzed.
Results: Sample size was mentioned in 94.5% of all articles. Alpha was reported in 81.6% followed by power, effect size, and variance as 67.3%, 57.1%, and 11.5%, respectively. Only 8.7% articles explicitly reported all four parameters of sample size.
Conclusion: The reporting of sample size calculation parameters for RCTs was inconsistent. Authors, editors, and reviewers need to increase the reporting of adequate sample size parameters to make the results of RCTs more robust for clinical applications.
Keywords: Bibliometric analysis, endodontics, journal impact factor, randomized controlled trial, sample size
|How to cite this article:|
Umer F, Haji Z, Jamal S. Reporting of sample size parameters for randomized controlled trials published in high impact factor endodontic journals in the last decade: A bibliometric analysis. Saudi Endod J 2021;11:292-9
|How to cite this URL:|
Umer F, Haji Z, Jamal S. Reporting of sample size parameters for randomized controlled trials published in high impact factor endodontic journals in the last decade: A bibliometric analysis. Saudi Endod J [serial online] 2021 [cited 2021 Dec 7];11:292-9. Available from: https://www.saudiendodj.com/text.asp?2021/11/3/292/325399
| Introduction|| |
Randomized controlled trials (RCTs) are considered as the highest level of evidence for clinical decision-making in evidence-based dentistry. They are designed in such a way that potential risk of biases are reduced, by giving equal chance to all participants in receiving either of the treatments which ensures that all groups are alike. Therefore, it is of prime importance that the quality of these RCTs is evaluated before clinical application in order to practice evidence-based dentistry. Good quality RCTs should report the study design, methodology, implementation, analysis, and results with great accuracy and transparency.
Sample size determination is one of the key elements of a well-designed clinical experiment. The objective of sample size determination in RCTs is to provide a count of subjects required in order to identify a clinically relevant outcome for that specific research question or intervention. A well-conducted study with small sample size will fail to detect a difference even if it is present. Whereas, having a sample size larger than what is actually required will lead to unnecessary expenses and wastage of resources.
Sample size determination is a methodical process in which initially, the researcher has to understand and state the hypothesis in classic terms. It is then followed by the calculation of sample size for which the researcher has to take four parameters into consideration; alpha (level of significance), power, effect size, and variance. The threshold of level of significance alpha and power of study is identified first, followed by the estimated value of the effect size which is the minimum difference in outcome, between two test groups. A larger effect size would result in a smaller sample size and vice versa. Finally, variance is taken into account which is generally not known and therefore, investigator estimates it from a pilot study or a comparable study conducted previously.
According to Chan et al., RCTs published in medical literature have poorly reported sample size calculations Only 11 of 62 (18%) RCTs reported the sample size calculations completely and consistently in publications. Herman et al. reviewed RCTs published in prosthodontic journal and reported that only 50% of the studies reported sample size calculation with only 17% considering all four important parameters for accurate sample size estimation.
In the field of endodontics, two leading journals with highest impact factors include Journal of Endodontics (JOE) and International Endodontic Journal (IEJ), have maximum citable articles for RCTs in the specialized field of endodontics. The precision with which sample size calculation is reported in RCTs of endodontic journals in general is lacking in previously published literature. A study may fail to answer its research question if the sample size is inadequate, while a large enough sample size may be impractical to implement. Hence, the aim of this study is to assess the reported methodology with consideration to sample size parameters in RCTs published by two high impact factor endodontic journals in the last decade.
| Materials and Methods|| |
A MEDLINE search via PubMed using publication type was conducted for articles in two selected journals (IEJ and JOE). MESH terms included [randomized clinical trial] or [randomized controlled trial] [RCT] AND [Journal of endodontics] and [International endodontic journal] published in the last decade from January 01, 2010 to April 31, 2020. Each article retrieved by the search, was reviewed by two authors before inclusion. Both authors drew consensus on which article should be included. Any disagreement on inclusion was resolved after discussion by third author. Only RCTs involving human subjects with random allocation in groups were included. Pilot studies and in vitro studies were excluded [Figure 1].
|Figure 1: Identification of included randomized controlled trials studies|
Click here to view
The methodology section of each included article was reviewed for description of method used to calculate sample size and reporting of parameters used for sample size calculation, i.e., level of significance, power, effect size, and variance. Moreover, articles were further scrutinized for reporting hypothesis in classic terms, direction of the test, i.e., one-tailed or two-tailed, inflation of the sample size and reporting of the reference study used for sample size calculation. Descriptive statistics were used to cumulate the results. Frequencies were reported in numbers and percentages for each variable studied from both the journals included in this study.
| Results|| |
The initial search yielded a total of 386 articles. After screening them further as per inclusion criteria, 217 articles were extracted of which 171 articles belonged to JOE and 46 articles belonged to IEJ [Figure 1]. From all the studies included, only 76 studies (35%) reported hypothesis in the classical terms. JOE articles overall, had the lowest reporting rate of hypothesis, i.e., 28.6%, whereas 58.65% of articles in IEJ reported some type of hypothesis (null/alternate). However, there were 7 studies in JOE that specifically reported both, i.e., null as well as alternate hypothesis [Figure 2].
Sample size was mentioned in 94.5% of all articles with similar trends being followed in both the journals. Loss to follow-up was taken into consideration by 54 (24.9%) RCTs. Out of all the parameters required, alpha was the most reported parameter found in 177 (81.6%) articles followed by power in 146 articles (67.3%). However, variance was the least reported parameter in a handful of 25 (11.5%) articles [Table 1]. Only 55% of the articles reported more than two parameters for sample size calculation, while 28% articles presented two or less whereas 16.6% articles did not mention any parameters for sample size calculation. Only 19 articles (8.7%) explicitly reported all four parameters [Table 2].
|Table 1: Sample size variables evaluated in included randomized controlled trials|
Click here to view
A total of 15 authors published more than one article of which Aggarwal et al.,,,,,,,, had 9 articles, which were the highest number of publications, followed by Parirokh et al.,,,,, with 6 articles. Out of all the authors, 14 authors had multiple RCTs published in JOE while only two authors had published multiple RCTs in IEJ. Aggarwal et al.,,,,,,,, was noted to be the only author who published multiple RCTs in both IEJ and JOE [Table 3]. The topic of anesthesia 35.5% was found to be most prevalent among the published data from the past 10 years followed by pain management 21.2%.[Table 4]. Trials on less common topics by these journals included microbial studies, radiographic assessment, retreatment, and regenerative endodontics. A few trials,, also included topics related to infection control, file systems, apex locators, lasers, apical size, and pediatric dentistry [Table 5]. Maximum number of RCTs with reported sample size were published in 2012 (n=30) followed by 2018 (n=28) [Figure 3].
|Table 4: Trend of randomized controlled trials topics published in the last 10 years|
Click here to view
|Table 5: Uncommon randomized controlled trials topics published in the last 10 years|
Click here to view
| Discussion|| |
Bibliometric reviews provide useful insight into the dynamism of a particular scientific discipline with which literature can be analyzed for quality and rigor in order to to corroborate direction and formation of new policies to improve future research output and techniques for data assimilation.
In the context of evidence-based dentistry RCTs are considered as the highest level of evidence to establish causality and study intervention efficacy. However, the results of RCTs can be affected by various factors including poor statistical design, random errors, and confounders making it essential to assess the quality of RCTs. A recent study analyzed the quality of RCTs conducted between 1997 and 2012 in endodontics and rated it to be poor with inappropriate sample size calculation. Another study reported suboptimal CONSORT score (49.5%) despite the journal endorsing CONSORT guidelines. During the last decade, emphasis on evidenced-based dentistry has increased which necessitated the evaluation of sample size calculations over the course of time for transparent reporting of trials. The present study shows that during the last decade, 45% of the RCTs did not report adequate parameters (two or less) for appropriate sample size calculation.
Specifying the null and alternative hypothesis is the initial step of sample size calculation.This was found to be reported by only one-third of the RCTs, along with negligible reporting of framework description (superiority, equivalence, non-inferiority) which makes it difficult to choose appropriate power calculations as they differ for various types of RCTs. The most commonly reported sample size calculation parameter was alpha/level of significance) followed by beta (power [1-beta]. These two parameters are typically kept as 1% or 5% alpha with a power of 80%–90%. Variance and/or effect size however, were inconsistently reported and they are unique for the study and sample size calculations. These parameters are usually derived from the past experiments, pilot study or any previous literature, the reporting of which was also found to be unsatisfactory., In the current study, only 84 articles adhered to a previously conducted study. This lack of reporting is not only unique to dentistry but has also been observed in other medical literature.
Only about one-fourth of the RCTs mentioned contingency strategy for events like loss to follow-up and drop out by inflating their sample size. Planning for these unconventionalities, account for more pragmatic results. The accuracy of sample size calculation cannot be confirmed if all parameters are not reported. Large treatment effects low variance and low power will yield unrealistic sample size, these patterns have been highlighted in publications in high impact journals the results of which are based on grossly underpowered calculations. Similarly, replication of sample size estimation would not be possible without all the parameters being mentioned.This will jeopardize the ability of a researcher to recalculate proper sample size calculation therefore; not reporting these parameters will lead to ambiguity while evaluating the quality of RCTs.
A study by Pandis concluded that the sample size calculations in dental journals were adequately reported in 50% out of the 95 trials examined which was lower than that reported by Lucena et al. Another study focused on RCTs of eight high impact factor journals belonging to various disciplines in dentistry and reported that Journal of Clinical Periodontology (JCP) had the highest odds of adequately reporting sufficient data to permit sample size recalculation. This was followed by the American Journal of Orthodontics and Dentofacial Orthopedics and Journal of Dental Research. It is interesting to note that neither of the endodontic journals were part of the top three dental journals. The reason for better reporting in JCP could be that evidence of underpowered studies was highlighted in periodontal literature and appropriate steps were taken to curtail these practices within the specialty. With quality initiatives like Preferred Reporting Items For Study Design in Endodontology (PRIDE) (http://pride-endodonticguidelines.org/) we expect to see an improvement in all aspects of endodontic literature reporting, The Preferred Reporting Items for Randomized Trials in Endodontics (PRIRATE) can serve as an important tool for auditing quality of randomized control trials for researchers, reviewers, and editors alike which will ultimately benefit the quality of research in endodontics.
Although not a primary focus of this bibliographic review, it was noted that most common topic for RCTs were related to efficacy of anesthesia or pain management and anesthetic techniques, a similar trend was noted in previous studies., This may be attributed to the ease of conducting these trials because the sample size can be achieved without any difficulty and may not require long-term follow-ups. Vital pulp therapy and microsurgical endodontics were the less popular topics for randomized controlled trials. This can be attributed for the need of long term follow ups in such studies. In contrast, the hot topics, that need more trials were cone-beam computed tomography, upcoming state of art lasers and regenerative endodontics that have produced promising in vitro studies.,,
In the past 10 years, Parirokh et al. had six RCTs published,,,,, that focused on the efficacy of anesthetics and pain management, the same author was also a part of several reviews conducted on mineral trioxide aggregate and bioactive cements but did not conduct any RCTs on the same topic,,,, which could most likely be due to increased funding resource requirements, sample size requirements and follow-ups that are more challenging to maintain for randomized clinical trials on other topics.
One important limitation of our study is that we concentrated on only two journals to represent the endodontic profile for our bibliometric review. However, the premise of doing so was that these were the two highest impact factor journal and it represents 64% of the RCTs that are being published. Other top tier endodontic journals such as Restorative dentistry and endodontics, Iranian Endodontic Journal, Australian Endodontic Journal and Dental Traumatology might contain RCTs which were missed. Moreover, endodontic articles were or are still being published in general dental journals such as the Journal of Dental Research, Dental Materials, and Journal of Dentistry. Nevertheless, JOE and IEJ are leading journals of endodontology in terms of impact factor and the main representatives of the specialty in endodontic literature. In this perspective, conclusions drawn by the study of bibliometric indexes of these two journals can be considered relevant. Reporting of better quality in trials should therefore be encouraged to benefit the structure of future clinical trials.
| Conclusion|| |
This review highlights that the reporting of sample size calculation parameters for RCTs was inconsistent. Authors, editors, and reviewers need to increase the reporting of adequate sample size parameters to make the results of RCTs more robust for clinical applications. It is scientifically and ethically emphasized that the awareness of accurate determination of minimum required sample size and application and reporting of appropriate estimation methods are extremely important in realistically achieving more reliable, valid, and generalizable results.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Shaughnessy AF. Clinical epidemiology: A basic science for clinical medicine. BMJ 2007;335:777.
Dumbrigue HB, Dumbrigue EC, Dumbrigue DC, Chingbingyong MI. Reporting of sample size parameters in randomized controlled trials published in prosthodontic Journals. J Prosthodont 2019;28:159-62.
Machin D, Campbell MJ, Tan SB, Tan SH. Sample Sizes for Clinical, Laboratory and Epidemiology Studies. Hoboken, New Jersey: John Wiley & Sons; 2018.
Noordzij M, Tripepi G, Dekker FW, Zoccali C, Tanck MW, Jager KJ. Sample size calculations: Basic principles and common pitfalls. Nephrol Dial Transplant 2010;25:1388-93.
Chan AW, Hróbjartsson A, Jørgensen KJ, Gøtzsche PC, Altman DG. Discrepancies in sample size calculations and data analyses reported in randomised trials: Comparison of publications with protocols. BMJ 2008;337:a2299.
Aggarwal V, Singla M, Kabi D. Comparative evaluation of effect of preoperative oral medication of ibuprofen and ketorolac on anesthetic efficacy of inferior alveolar nerve block with lidocaine in patients with irreversible pulpitis: A prospective, double-blind, randomized clinical trial. J Endod 2010;36:375-8.
Aggarwal V, Singla M, Miglani S, Ansari I, Kohli S. A prospective, randomized, single-blind comparative evaluation of anesthetic efficacy of posterior superior alveolar nerve blocks, buccal infiltrations, and buccal plus palatal infiltrations in patients with irreversible pulpitis. J Endod 2011;37:1491-4.
Aggarwal V, Singla M, Miglani S, Kohli S. Comparative evaluation of mental incisal nerve block, inferior alveolar nerve block, and their combination on the anesthetic success rate in symptomatic mandibular premolars: A randomized double-blind clinical trial. J Endod 2016;42:843-5.
Aggarwal V, Singla M, Miglani S, Kohli S. Efficacy of articaine versus lidocaine administered as supplementary intraligamentary injection after a failed inferior alveolar nerve block: A randomized double-blind study. J Endod 2019;45:1-5.
Aggarwal V, Singla M, Miglani S, Kohli S, Irfan M. A prospective, randomized single-blind evaluation of effect of injection speed on anesthetic efficacy of inferior alveolar nerve block in patients with symptomatic irreversible pulpitis. J Endod 2012;38:1578-80.
Aggarwal V, Singla M, Miglani S, Kohli S, Singh S. Comparative evaluation of 1.8 mL and 3.6 mL of 2% lidocaine with 1:200,000 epinephrine for inferior alveolar nerve block in patients with irreversible pulpitis: A prospective, randomized single-blind study. J Endod 2012;38:753-6.
Aggarwal V, Singla M, Rizvi A, Miglani S. Comparative evaluation of local infiltration of articaine, articaine plus ketorolac, and dexamethasone on anesthetic efficacy of inferior alveolar nerve block with lidocaine in patients with irreversible pulpitis. J Endod 2011;37:445-9.
Aggarwal V, Singla M, Miglani S, Kohli S. Comparison of the anaesthetic efficacy of epinephrine concentrations (1: 80 000 and 1: 200 000) in 2% lidocaine for inferior alveolar nerve block in patients with symptomatic irreversible pulpitis: A randomized, double-blind clinical trial. Int Endod J 2014;47:373-9.
Aggarwal V, Singla M, Miglani S, Kohli S, Sharma V, Bhasin SS. Does the volume of supplemental intraligamentary injections affect the anaesthetic success rate after a failed primary inferior alveolar nerve block? A randomized-double blind clinical trial. Int Endod J 2018;51:5-11.
Parirokh M, Ashouri R, Rekabi AR, Nakhaee N, Pardakhti A, Askarifard S, et al
. The effect of premedication with ibuprofen and indomethacin on the success of inferior alveolar nerve block for teeth with irreversible pulpitis. J Endod 2010;36:1450-4.
Parirokh M, Jalali S, Haghdoost AA, Abbott PV. Comparison of the effect of various irrigants on apically extruded debris after root canal preparation. J Endod 2012;38:196-9.
Parirokh M, Rekabi AR, Ashouri R, Nakhaee N, Abbott PV, Gorjestani H. Effect of occlusal reduction on postoperative pain in teeth with irreversible pulpitis and mild tenderness to percussion. J Endod 2013;39:1-5.
Parirokh M, Sadeghi AS, Nakhaee N, Pardakhty A, Abbott PV, Yosefi MH. Effect of topical anesthesia on pain during infiltration injection and success of anesthesia for maxillary central incisors. J Endod 2012;38:1553-6.
Parirokh M, Sadr S, Nakhaee N, Abbott PV, Manochehrifar H. Comparison between prescription of regular or on-demand ibuprofen on postoperative pain after single-visit root canal treatment of teeth with irreversible pulpitis. J Endod 2014;40:151-4.
Parirokh M, Yosefi MH, Nakhaee N, Manochehrifar H, Abbott PV, Reza Forghani F. Effect of bupivacaine on postoperative pain for inferior alveolar nerve block anesthesia after single-visit root canal treatment in teeth with irreversible pulpitis. J Endod 2012;38:1035-9.
Pandis N. The evidence pyramid and introduction to randomized controlled trials. Am J Orthod Dentofacial Orthop 2011;140:446-7.
Lucena C, Souza EM, Voinea GC, Pulgar R, Valderrama MJ, De-Deus G. A quality assessment of randomized controlled trial reports in endodontics. Int Endod J 2017;50:237-50.
Aljarbou F, Alharbi F, Alamri H. Are endodontic abstracts of published randomized clinical trials reported adequately? Saudi Endod J 2020;10:152-6. [Full text]
Petrie A, Bulman JS, Osborn JF. Further statistics in dentistry. Part 4: Clinical trials 2. Br Dent J 2002;193:557-61.
Browne RH. On the use of a pilot sample for sample size determination. Stat Med 1995;14:1933-40.
Charles P, Giraudeau B, Dechartres A, Baron G, Ravaud P. Reporting of sample size calculation in randomised controlled trials: Review. BMJ 2009;338:b1732.
Pandis N, Polychronopoulou A, Eliades T. An assessment of quality characteristics of randomised control trials published in dental journals. J Dent 2010;38:713-21.
Vickers AJ. Underpowering in randomized trials reporting a sample size calculation. J Clin Epidemiol 2003;56:717-20.
Koletsi D, Fleming PS, Seehra J, Bagos PG, Pandis N. Are sample sizes clear and justified in RCTs published in dental journals? PLoS One 2014;9:e85949.
Hujoel PP, Baab DA, DeRouen TA. The power of tests to detect differences between periodontal treatments in published studies. J Clin Periodontol 1992;19:779-84.
Nagendrababu V, Duncan HF, Bjørndal L, Kvist T, Priya E, Pulikkotil SJ, et al
. Preferred Reporting Items for RAndomized Trials in Endodontics (PRIRATE) guidelines: A development protocol. Int Endod J 2019;52:974-8.
Ordinola-Zapata R, Peters OA, Nagendrababu V, Azevedo B, Dummer PMH, Neelakantan P. What is of interest in Endodontology? A bibliometric review of research published in the International Endodontic Journal and the Journal of Endodontics from 1980 to 2019. Int Endod J 2020;53:36-52.
Ahmad P, Elgamal HAM. Citation Classics in the Journal of Endodontics and a Comparative Bibliometric Analysis with the Most Downloaded Articles in 2017 and 2018. J Endod 2020;46:1042-51.
Alherabi A. Involved levels of the neck in head & neck cancer: A clinico-pathological correlation. Saudi Endod J 2019;9:1-7.
Mahfouze A, El Gendy A, Elsewify T. Bacterial reduction of mature Enterococcus faecalis
biofilm by different irrigants and activation techniques using confocal laser scanning microscopy. An in vitro
study. Saudi Endod J 2020;10:247-53. [Full text]
Parirokh M, Torabinejad M. Mineral trioxide aggregate: A comprehensive literature review Part III: Clinical applications, drawbacks, and mechanism of action. J Endod 2010;36:400-13.
Parirokh M, Torabinejad M. Mineral trioxide aggregate: A comprehensive literature review Part I: Chemical, physical, and antibacterial properties. J Endod 2010;36:16-27.
Parirokh M, Torabinejad M, Dummer PMH. Mineral trioxide aggregate and other bioactive endodontic cements: An updated overview Part I: Vital pulp therapy. Int Endod J 2018;51:177-205.
Torabinejad M, Parirokh M. Mineral trioxide aggregate: A comprehensive literature review Part II: Leakage and biocompatibility investigations. J Endod 2010;36:190-202.
Torabinejad M, Parirokh M, Dummer PMH. Mineral trioxide aggregate and other bioactive endodontic cements: An updated overview Part II: Other clinical applications and complications. Int Endod J 2018;51:284-317.
Tzanetakis GN, Stefopoulos S, Loizides AL, Kakavetsos VD, Kontakiotis EG. Evolving trends in endodontic research: An assessment of published articles in 2 leading endodontic journals. J Endod 2015;41:1962-8.
[Figure 1], [Figure 2], [Figure 3]
[Table 1], [Table 2], [Table 3], [Table 4], [Table 5]