Rating

The NeuroBITE Rating Scales

It is important to note that a high rating score obtained by using the PEDro-P or the RoBiNT scale indicates that a study has not been compromised by sources of bias. It does not necessarily mean that the study is a ‘good’ one. It is still necessary to consider:

The PEDro-P Scale

All RCT and non-RCT studies listed on NeuroBITE are rated using the PEDro-P scale. The PEDro-P scale consists of 11 criteria. The first item relates to the external validity (specifically the participant selection criteria). The remaining 10 items (criteria 2 - 10) assess the internal validity of each trial and whether the trial contains sufficient statistical information to make it interpretable. Thus, the internal validity of each trial is ranked based on a total score out of 10 (i.e., excluding criterion 1). NeuroBITE gratefully acknowledges the PEDro scale by the PEDro database team. The PEDro-P scale is a refinement of the original PEDro scale in order to rate studies on NeuroBITE. PEDro-P has adapted and re-worded the same 11 criteria as used on PEDro to ensure that they apply unambiguously to trials on NeuroBITE. The original PEDro scale is based on the Delphi list developed by Verhagen and colleagues. The Delphi list is a list of trial characteristics that was thought to be related to trial "quality" by a group of clinical trial experts (for details see Verhagen et al., Journal of Clinical Epidemiology 51: 1235-1241, 1998). The PEDro scale contains additional items on adequacy of follow-up and between-group statistical comparisons. One item on the Delphi list which translates to the first criterion on eligibility criteria is related to external validity, so it does not reflect the dimensions of quality assessed by the PEDro scale . This item is not used to calculate the method score that is displayed in the search results (which is why the 11 item scale gives a score out of 10). This item has, nevertheless, been retained so that all Delphi list items are represented on the PEDro scale. The reliability of the PEDro scale for rating quality of randomised controlled trials has been published in Physical Therapy 83(8): 713-21, 2003. You can download the PEDro-P Scale (PDF 93 Kb).

The RoBiNT Scale

All studies on NeuroBITE that report on experimentally controlled single-case designs (SCDs) are evaluated for their methodological strength. The most common experimental SCDs in the neurorehabilitation literature include the withdrawal/reversal design (e.g. A-B-A-B or variations thereof) and the multiple-baseline design. There are also other less frequently employed designs that allow for experimental control of study conditions. When designed rigorously, a SCD can draw valid conclusions about treatment efficacy. This feature makes these designs particularly relevant and applicable for clinical practice, as well as research studies in populations where it is difficult to achieve an adequately powered sample size due to infrequent occurrence of the condition or heterogeneous samples. The tool used to rate the methodological quality of an experimental SCD on NeuroBITE is the Risk of Bias in N-of-1 Trials (RoBiNT) Scale (Tate et al., 2013). The scale consists of 15 items in two subscales: the Internal Validity (IV) Subscale (7 items) and the External Validity and Interpretation (EVI) Subscale (8 items). For further information on the RoBiNT Scale, see the updated manual: The Risk of Bias in N-of-1 Trials (RoBiNT) Scale: An expanded manual for the critical appraisal of single-case reports. (Tate et al., 2015). This 100-page manual is a useful pedagogical resource on SCDs in general, as well as containing more than 100 examples of scoring the items of the RoBiNT Scale, along with a rationale for the scoring decisions.

The RoBiNT Scale: Relationship to the SCED Scale

The RoBiNT Scale, published in 2013, is a revision of and replaces the original scale developed by our team, the Single-Case Experimental Design (SCED) Scale (Tate et al., 2008). The RoBiNT Scale differs radically from the SCED Scale with respect to its structure, item content, and scoring procedures. During the course of using the SCED Scale to appraise single-case reports on NeuroBITE, we saw ways in which it could be improved, both in terms of content and scoring procedures, as well as its application in different fields. We aimed to revise the scale so that it was consistent in content and scoring with standards for SCDs reported in the literature of that time (e.g., Kratochwill et al., 2010), as well as making it applicable beyond the neurorehabilitation field (e.g. medicine, educational psychology, special education, sport psychology). One of the major changes in the RoBiNT Scale compared to the SCED Scale, is the move from a binary to three-point scoring system. This allows raters to differentiate among studies that fail to meet criteria for a given item (score 0), those meeting previously accepted standards (score 1), and those meeting the currently recommended criteria (score 2). This change has resulted in a much more rigorous and discriminating scale. The SCED Scale was previously used to rate SCDs on NeuroBITE. We are now working towards having all experimentally controlled SCDs rated using the RoBiNT as soon as possible. SCED Scale ratings have been removed and RoBiNT Scale ratings will be published as the articles are rated.