Title: Interrater Reliability of Techniques Used to Identify Somatic Dysfunction Related to the Abdomen and Viscera: A Scoping Review
Authors: Kristen P. W. Gavin, OMS4; Angie K. Maxson, OMS4; Tianfu Shang, OMS1; Phillip G. Munoz, OMS3; Crystal Martin, DO
Introduction
Interrater reliability (IRR) is commonly assessed for diagnostic tests and tools in medicine. There is interest in the IRR of techniques used to identify somatic dysfunction (SD). The Educational Council on Osteopathic Principles publishes A Teaching Guide for Osteopathic Manipulative Medicine (Guide) providing guidance to United States osteopathic medical schools regarding curriculum design, including teaching such techniques. This scoping review aims to identify the number and characteristics of published data regarding the IRR of techniques for identification of SD related to the abdomen and viscera using Guide as a reference.
Methods
A systematic search was performed via PubMed, the Journal of Osteopathic Medicine (JOM), OSTMED.DR®, and Cochrane Reviews, with no date restrictions. Queries were developed to generate results relevant to identification of SD pertaining to the abdomen and viscera, including viscerosomatic, somatovisceral, and Chapman reflexes. For the latter three databases, search terms were interrater, interobserver, interexaminer, and intertester and were limited to the title for OSTMED.DR® and Cochrane Reviews (option unavailable for JOM). For PubMed, all queries began (interrater[tiab] OR interobserver[tiab] OR interexaminer[tiab] OR intertester[tiab]) AND (humans[Filter]) and ended NOT (“diagnostic imaging”[mesh] OR radiography[mesh]). AND connected the former to search terms regarding asymmetry, tissue texture, restriction of motion, SD, and terms per Guide, including relevant synonyms, all with [tiab]. Query results were de-duplicated and assessed for eligibility using two primary reviewers and a third tiebreaker as needed. Query results were eligible if: available in English; IRR data was reported for at least one technique using observation or palpation relevant to identification of SD related to the abdomen or viscera; raters were healthcare professionals or students; and raters assessed human bodies or proxy. Exclusion criteria were opposite, including IRR of methods involving diagnostic imaging or devices.
Results
Searches yielded 50 unique results; zero were eligible for inclusion.
Discussion
A common reason for exclusion was that the assessment was not performed using observation or palpation. There is an apparent paucity of data regarding the IRR of techniques used to identify SD pertaining to the abdomen and viscera. Future research in this area and pertaining to other body regions would be valuable to support the use of SD diagnoses in clinical practice and in osteopathic medical education. This review was limited by databases used and query design. For example, despite being an indicator of SD, tenderness was not an inclusion criterion given its reliance on patient subjectivity.
I know how important IRR is for OMM so I appreciate your efforts on this project. As one of the judges on your poster, I was wondering what your thoughts are on whether you were too stringent in your exclusion criteria and if you had reason to believe a priori that there would be multiple articles that you could have included in your review.
Thank you for your engagement and questions, Dr. Rhodes.
Regarding your question about exclusion criteria: overall, we wanted the inclusion/exclusion criteria to facilitate isolating records that were relevant to the classic methods taught for identifying somatic dysfunction/that mimic how somatic dysfunction is identified in clinical settings using the provider’s hands and eyes and the patient’s intact body. For clarity, I included in the presentation some examples of techniques that would not be relevant and, thus, would be excluded. For example, our searches yielded records that reported IRR for techniques in which tissues were observed intraoperatively or histologically.
Also overall, we wanted the inclusion/exclusion criteria to facilitate finding records that were relevant to provider assessment (rather than patient assessment). We knew that tenderness is taught as one indicator of somatic dysfunction (along with tissue texture abnormality, asymmetry, and restriction of motion) and we did not want to include records that only investigated IRR of tenderness due to its heavy reliance on patient subjectivity rather than relying more heavily on provider assessment, which was our focus.
To address more explicitly your first question regarding stringency of exclusion criteria: because we would include records if they had at least one technique that was relevant/met inclusion criteria, I do not think that the exclusion criteria led us to exclude records that would have been relevant and otherwise included. For example, if a record reported the IRR of both palpating the tissue texture of the abdomen and visualizing the tissue under a microscope, the latter would not have caused the record to be excluded.
Regarding your question about expectations for articles to include in our review: we did not have prior knowledge of relevant records and were unsure whether there would be relevant records prior to conducting this review.
I hope this addresses and answers your questions. Thank you for asking!
Thank you for your bravery in presenting negative data! It is an important part of the scientific process. You mentioned that you may have lost some relevant material by excluding “imaging” papers. Did you go back and change your search criteria and see if that was true?
Thank you for your engagement and question, Dr. Habecker.
I’ll clarify that there were essentially two ways that we filtered out records regarding diagnostic imaging.
First, in our PubMed searches, a search term we used was: NOT (“diagnostic imaging”[mesh] OR radiography[mesh]). I did briefly run queries without that search term for multiple body regions in our study (including abdomen and viscera), and I did not see records that looked relevant. That said, I did not screen all of the records that were generated. By not using the aforementioned search term, there are many more records that were generated (five-fold or more). While it might have been feasible for the abdomen and viscera part of our study, it would have been a substantial undertaking for other body regions involved in the overall project, and, based on my cursory assessment, for likely little benefit.
Second, during our screening and review process, we excluded records if the techniques for which they assessed IRR were involving diagnostic imaging. That said, because we would include records if they had at least one technique that was relevant/met inclusion criteria, I do not think that this second way led us to exclude records that would have been relevant and otherwise included. For example, if a record reported the IRR of both palpating the tissue texture of the abdomen and an imaging study of the abdomen, the latter would not have caused the record to be excluded.
I hope this addresses and answers your question. Thank you for asking!
Thank you for your presentation and work in exploring this topic. I would like to follow in the above judges’ line of questioning about radiographic or imaging and other diagnostic technics. Are these (or other criteria) used to validate the SD findings in other studies? If so, perhaps excluded studies could have been comparing raters to the other diagnostic criteria while also looking at IRR (perhaps comparing expert vs. novice). Would your criteria have excluded those studies?
Thank you for your engagement and questions, Dr. Salido.
I’ll respond to your questions in parts.
I have seen some studies that use imaging and/or other tools to validate SD identified by visualization and/or palpation. Regarding whether our criteria would have excluded studies that compared raters to such diagnostic tools and to each other in their SD identification, I’ll start by addressing the imaging component. There were essentially two ways that we filtered out records regarding diagnostic imaging.
First, in our PubMed searches, a search term we used was: NOT (“diagnostic imaging”[mesh] OR radiography[mesh]). Using this search term would prevent records that have been manually tagged with mesh terms “diagnostic imaging” and “radiography” from being generated. (Note that some records pertaining to diagnostic imaging and radiography still get through if they are not manually tagged with the mesh terms). So, it is possible that using this search term filtered out studies like you mentioned. I will note that we found studies that used other tools to validate SD findings through non-PubMed databases and our searches through such databases did not filter out imaging studies. These studies did not pertain to the abdomen and viscera.
Second, during our screening and review process, we excluded records if the techniques for which they assessed IRR were involving diagnostic imaging. That said, because we would include records if they had at least one technique that was relevant/met inclusion criteria, I do not think that this second way led us to exclude records that would have been relevant and otherwise included.
To filter out records regarding the use of other diagnostic tools and techniques, the second way above was used (screen and review). Again, because we would include records if they had at least one technique that was relevant/met inclusion criteria, I do not think that this second way led us to exclude records that would have been relevant and otherwise included.
I hope this addresses and answers your questions. Thank you for asking!