ORIGINAL CONTRIBUTION
Outpatient Osteopathic Single Organ System
Musculoskeletal Exam Form: Training and Certification
Sandra L. Sleszynski, DO
Thomas Glonek, PhD
William A. Kuchera, DO
The Outpatient Osteopathic Single Organ System Musculoskeletal Exam Form (SOS form) is a standardized
examination data form. A standardized form is necessary
to ensure that essential quality data are collected during
osteopathic studies and that each submitted form is completed adequately and uniformly. Use of the standardized
form permits reliable statistical computations from the
collected data. The training process for the SOS form incorporated the following elements: (1) training investigators
to use the form; (2) having trainees transcribe three clinical
case examples onto SOS forms; (3) comparing each transcribed clinical case to a prepared key; (4) evaluating the
trainees’ work for accuracy; and (5) statistically evaluating
the trainees’ records for intraexaminer and interexaminer
reliability.
The success or failure of trainees to receive certification in their training process involved evaluating their
ability to accurately and appropriately record data collected from three case examples. These cases were designed
to evaluate recording accuracy and intraexaminer and
interexaminer reliability. All trainees scored 80% or better
for accuracy, and their work had good intraexaminer and
interexaminer reliability. As a result, all trainees were
awarded a numbered certificate for successful completion
of the training process. Having a pool of well-trained, certified investigators available and ready to participate in the
gathering of data through the use of the SOS form ensures
that necessary data are collected and that the resulting
databases are unified. It will also facilitate comparisons and
statistical analysis of osteopathic research projects. Standardized forms and certified investigators will improve the
quality of osteopathic research throughout the profession.
Dr Sleszynski is an affiliate clinical assistant professor at Midwestern University’s Chicago College of Osteopathic Medicine in Downer’s Grove, Illinois, as
well as a private practitioner at Crossroads Premiere Health Care in Kenosha,
Wisconsin. Dr Glonek is a research professor in the Osteopathic Manipulative
Medicine Department at Midwestern University’s Chicago College of Osteopathic Medicine, Downer’s Grove, Illinois. Dr Kuchera is emeritus professor at
Kirksville College of Osteopathic Medicine in Kirksville, Missouri.
This study was made possible by a grant from the American Osteopathic
Association.
Address correspondence to Sandra L. Sleszynski, DO, Crossroads Premiere
Health Care, 1010 35th St, Kenosha, WI 53140.
E-mail: [email protected]
R
eliable, high-quality methods for collecting data require
a standardized form and training in the use of standardized data collection tools. In response to the need for
acquiring quality data supporting osteopathic principles and
practice, the Louisa Burns Osteopathic Research Committee
of the American Academy of Osteopathy designed and validated the Outpatient Osteopathic SOAP Note Form (SNF).1
Use of the SNF in the clinic, however, illuminated the need
for a musculoskeletal specialty form to be used for documenting and evaluating patients’ initial outpatient visits.2 In
response to this, the Outpatient Osteopathic Single Organ
System Musculoskeletal Exam Form (SOS form) was
designed.
The name for the SOS form was based on the use of the
term, single organ system, referring to “an extensive exam in
one specialty or organ system” as it was used in the May
1997 Health Care Financing Administration Evaluation and
Management Documentation Guidelines.3 The SOS form
(Figure) provides a standardized instrument for recording
essential information required in the documentation guidelines for a comprehensive musculoskeletal examination.3
Use of this form by a physician, even if not an investigator, would be beneficial, as it documents the actual extent
of an examination and the treatment that was given, producing a complete record of the patient’s visit and justifying
an appropriate level of reimbursement. The form is structured so that if all indicated areas are examined, all boxes
will be filled in and the physician will have met all coding
requirements to earn a level 5 comprehensive examination
reimbursement. The Osteopathic SOS Musculoskeletal Exam
project (as with the SOAP note project that preceded it) uses
the standard glossary of osteopathic terminology that was
developed by the Educational Council on Osteopathic Principles. This glossary was subsequently incorporated into the
text, Foundations for Osteopathic Medicine.4 Although this glossary is standard within the osteopathic profession, it is not recognized as standard within the greater North American medical community. For this reason, a standard osteopathic
thesaurus is currently being prepared for submission to the
US National Library of Medicine’s Unified Medical LanFigure. Outpatient Osteopathic Single Organ System Musculoskeletal
Exam Form.
76 • JAOA • Vol 104 • No 2 • February 2004
Downloaded From: http://jaoa.org/pdfaccess.ashx?url=/data/journals/jaoa/932021/ on 06/16/2017
Sleszynski et al • Original Contribution
ORIGINAL CONTRIBUTION
Sleszynski et al • Original Contribution
Downloaded From: http://jaoa.org/pdfaccess.ashx?url=/data/journals/jaoa/932021/ on 06/16/2017
JAOA • Vol 104 • No 2 • February 2004 • 77
ORIGINAL CONTRIBUTION
guage System, the recognized medical lexicon authority in
North America.5 The terms and abbreviations presently used
in the SOS form conform, insofar as possible, with current definitions as given within the Unified Medical Language System
and the glossary.
As standard nomenclature for diagnosis is used on the
SNF and SOS forms, the responses of patients with certain diseases or conditions to various types of osteopathic manipulative treatment (OMT) may be centrally gathered, classified, and studied. The SOS form is intended to be one of a set
of four forms that will facilitate efficient collection and accurate recording of clinical data without interfering with a
physician’s medical practice. To facilitate research, it is suggested that practicing osteopathic physicians use both the
SOS form (initial visit note) and the SNF (follow-up visit
SOAP note) to gather data regarding osteopathic examinations, somatic dysfunctions found, treatment regimens used,
and responses obtained from treatments.
The purpose of this article is to provide a mechanism for
training individuals in the use of a standardized form to validate examination and treatment data gathered from research
projects. This method can also be used for standardizing
forms used in other studies, training individual investigators, determining their accuracy, and checking for intraexaminer and interexaminer reliability.
Methods
Before the training course, the investigators prepared two different patient prototype examples on SOS forms. These prototypes included hypothetical data to be used in generating the
answer keys against which the cases, transcribed by the
trainees, would be compared. These prototypes contained
multiple examples of the training objectives. From these two
prototypes, three written case studies were developed before
the instruction sessions. Two of the case studies had identical
scenarios, with the exception of the patient’s name, sex, age, and
date of visit. In addition, the narrative text was recast. Alterations in the narrative of the first study were introduced to give
the illusion that the first two visit scenarios were different.
Thus, a direct comparison of these two case narratives yielded
a measure of intraexaminer reliability in the use of the SOS
progress note. The third case study narrative had a completely
different scenario.
Cases were then placed into three groups, and trainees
were randomly assigned to one of the groups. Each group
had the same three cases presented in different order. This
randomization procedure allowed for interexaminer reliability testing, as well as testing whether each trainee completely filled out each of the SOS forms in similar fashion,
regardless of order or presentation. To summarize, the initial
setup for the cases tests whether trainees can fill out the same
case the same way every time (intraexaminer reliability) and
whether trainees can fill out the same case the same way as
other trainees (interexaminer reliability).
Initially, all participants in the study filled out a physician
demographic/participation form. Participants answered questions relative to his or her individual clinical practice, education, and personal information. The information on this form
was used to contact participants and send out certificates. It will
also be used to select investigators for future studies and to provide any demographic information that may be needed in the
future.
Fifteen investigators participated in the SOS note training
session—8 physicians (4 physicians doing more than 60%
family practice and 4 primarily OMT specialists), 3 osteopathic
manipulative medicine residents in the Plus-One Program, 1
osteopathic medicine resident, 1 family practice resident, and
2 osteopathic medicine undergraduate fellows. Ten of the
investigators were school-based, and 5 were in private practice. Five of the 8 physicians were board-certified in neuromusculoskeletal medicine. All of the trainees were trained in
the need for and use of the SOS form—12 by instruction at a
didactic lecture and 3 by telephone conference.
At the start of the training session, all participants received
a training and reference manual, the study protocol, and contact information. The developmental history of the SOS form
and its importance to the osteopathic medical profession for validating research endeavors were presented. The SOS form
was systematically reviewed in detail, and its use was demonstrated with an example. This was followed by having the
trainees transcribe each of their three written case narratives
onto SOS forms, one case per form. The trainees then labeled
each sheet with their group letter and case numbers as defined
in the personal packets they received and signed each SOS
form. Trainees then handed in their case packets and the three
completed SOS forms. The progress notes were tabulated and
evaluated for participant performance against the prepared reference keys to determine the accuracy of their recording ability.
Problems that seemed to result from instructions that were
given to the trainees were identified and corrected.
A trainee was certified as a study investigator only when
he or she achieved a performance score of 80% or better. This
minimal criterion was based on standard teaching criteria for
competency adopted by Midwestern University’s Chicago
College of Osteopathic Medicine. According to this standard,
a trainee who scored at least 80% was deemed proficient in
transcribing a case accurately onto a different format. This
testing procedure does not rate proficiency in a physician’s
ability to examine a patient. It only rates accuracy in documenting his or her examination findings. All investigators
meeting these criteria were awarded certificates of completion in transcription proficiency in the use of the SOS form. Each
certificate contained a serial number for easy tracking. The
certificate allows investigators to participate in any future
research study involving the SOS form, with the assurance to
the principal investigator that the certified investigator had
adequate training and would record generated data within
acceptable standard limits.
78 • JAOA • Vol 104 • No 2 • February 2004
Downloaded From: http://jaoa.org/pdfaccess.ashx?url=/data/journals/jaoa/932021/ on 06/16/2017
Sleszynski et al • Original Contribution
ORIGINAL CONTRIBUTION
Results
Table
Trainee Percentage-Correct Score Descriptive Statistics
Statistical Parameter
Value
N
15
Mean
90.30
SEM
Median
SD
1.04
88.50
4.03
Variance
16.23
Range
12.00
Minimum
83.5
Maximum
95.0
Once data were analyzed for intraexaminer, interexaminer, and data-entry reliability, as well as answer key accuracy,
it was grouped in a variety of ways to answer secondary
questions.
Data Reduction and Statistics
The twelve trainees who were trained by a didactic session
completed and submitted their cases for evaluation at the end
of the session. The three trainees who were instructed by telephone completed their cases within 2 weeks of the telephone
conference. Before the start of the study, a list of potential
answers that might be given for each data field was developed for the data-entry technicians. This provided uniformity
in data-entry coding. Data from the 45 cases (15 trainees with
3 cases each) were entered. The three answer keys were entered.
Data entry by the technicians was verified by having each
data-entry technician reenter selected cases. Data were compiled in an Excel spreadsheet and written into the statistical software (SPSS Inc, Chicago, Illinois) database file. There were
no detected errors in translation. Responses were coded as
follows: 1, trainee failed to enter a response; 2, response was
entered when none was called for; 3, incorrect response was
entered; 4, correct response was entered. The output reports for
the set of variables in a selected variable combinations group
consisted of total counts and a detailed frequency table for
each variable on the SOS form. Descriptive statistics and crosstabulation (Pearson Chi-square for significance of differences
between groups) were used to evaluate variables derived from
the SOS form. The combination of these two tests provided an
estimate of variability in areas of the form that called for
scaling. In addition, performance scores were computed on a
percentage basis. These computed scores were then compared
by t statistics (2-tailed independent-samples t-test or pairedsamples t-test, as appropriate).
There were 139 variables in each case and 3 cases for each
trainee, for a total of 417 variables analyzed. Trainee scores
(Table) ranged from 83.5% to 95.0% correct. Random case
checks indicated that data entry (from trainee forms) was
accurate (0.5% error); however, a single-pass verification of
all cases was not undertaken.
The mean score for the 15 investigators was 90.3%. There
were no outliers in the group, nor were there any differences
among the performances of residents (90.28% 4.66%) or
attending physicians (90.31% 3.85%) regarding this activity
(P .99; 2-tailed significance). All investigators qualified for
certification by scoring at least 80%.
Intraexaminer reliability among the 15 trainees was
assessed by comparing the trainees’ final scores obtained from
the two identical cases. Scores for each case and for each trainee
ranged from 79.9% to 96.4% ( 4.98%). Of the 139 variables
tested, 132 showed no significant differences between the two
similar narratives. Of these 132 variables, 89 showed significance greater than .70, and 43 showed significance values
between .70 and .10. This means that 95% of the variables
tested were the same for the two similar narratives (P .10),
indicating good intraexaminer reliability.
Problem variables were defined as any variable receiving
less than 80% correct responses. Of the 139 variables derived
from the SOS form, 25 were below the 80% correct responses.
Of these, 11 were between 70% and 80%, 8 were between 60%
and 70%, 4 were between 50% and 60%, 1 was between 40%
and 50%, and 1 was between 30% and 40%. Errors of entering
an incorrect response or entering a response when none was
required occurred in less than 1% of responses (0.74% and
0.88%, respectively). All remaining errors resulted from failure
to enter a response when a response was required. Two variables, somatic dysfunction severity and whether OMT was
performed in a region, demonstrated an inordinately large
number of instances in which trainees failed to respond when
a response was needed. There were many errors (11.1% for
each variable) for the specific variables involving OMT to the
rib and abdominal regions.
Each variable also was examined separately using the
cross-tabulation statistic (2-sided) comparing errors for the
narratives. The significance values indicated that trainees had
difficulty completing several areas. These included the observation and evaluation for the cardiac system, determining
lymph node presence, determining severity of somatic dysfunction in the head, OMT done to the left lower extremity, and
response to OMT in the right and left lower extremities sections
of the SOS form. Essentially, all of these errors were errors in
failure to enter a response.
Discussion
Variables
For 114 variables, scores were well above the 80% correct data
entry level and were deemed acceptable trainee performance.
Sleszynski et al • Original Contribution
Downloaded From: http://jaoa.org/pdfaccess.ashx?url=/data/journals/jaoa/932021/ on 06/16/2017
JAOA • Vol 104 • No 2 • February 2004 • 79
ORIGINAL CONTRIBUTION
For these variables, there were no apparent systematic data
recording problems, ie, errors were randomly dispersed among
trainees. It was concluded, therefore, that all trainees were
competent in filling out the SOS form and that interexaminer
reliability was within acceptable limits.
There were no outliers, nor were there differences between
residents and attending physicians. Errors committed were
primarily of two types: (1) failure to enter a response when one
was required, and (2) entering an incorrect response. A failure
to enter a response when one was required occurred in areas
for which physicians were not in the habit of entering data, such
as rating severity of somatic dysfunction and documenting
whether OMT was performed in specific regions. An incorrect
response was believed to be due to two things. One was the
lack of clarity as to which region certain muscle dysfunctions
should be documented. For example, should diaphragmatic
findings be documented in the rib, abdominal, or both regions?
The other reason was that some narratives listed information
in a more straightforward way than another, particularly with
regard to sidedness. Correction of these errors will require a
more directed approach and training effort in these areas.
With respect to regions in the musculoskeletal table for
somatic dysfunction, the thoracic and extremities regions presented the greatest difficulty to trainees.
On the SOS form, the thoracic region is divided into three
sections: T1-T4, T5-T9, and T10-T12. The significance of dividing
this region into logical sympathetic viscerosomatic distribution
regions was explained and emphasized in the training session. This subdivision, however, deviates from the format
used in the other regions of the table and is likely the source
of the poorer trainee performance within the thoracic section.
Nevertheless, trainee performance in the thoracic portion of the
table was considered to be better than acceptable.
Difficulty with the form also was apparent in the extremities regions of the musculoskeletal table. Errors in this area
seemed to be unclear within the narratives themselves
regarding extremity sidedness and in filling in the wrong side
on the form. Again, however, trainee performance was considered to be better than acceptable.
Training
In both the thoracic and extremity areas, the error data indicate
that better instruction is needed. If examination of a region
was performed and findings were normal, those findings
should be written in the somatic dysfunction area, and if a
region was examined but not treated, then a “no” is required
in the OMT done section.
There was also confusion between the use of two abbreviations. The abbreviation for osteopathy in the cranial field, cranial osteopathy, and cranial treatment is “CR,” and the abbreviation for counterstrain treatment is “CS.” The difference between
these two abbreviations must be emphatically emphasized.
All of the aforementioned issues were listed on the objectives instruction sheet and were covered thoroughly in the
training session; nonetheless, data showed that even greater
energy in instruction must be devoted to these sections of the
form.
Training must emphasize the need to complete each element of the form. Poorly gathered data regarding examination
and poor documentation of OMT that was given is poor clinical practice and results in reduced appropriate third-party
reimbursements for examinations and treatments that were
done. A research study is only as good as the training that
accompanies it. There must be reliability within a single examiner (intraexaminer reliability, ie, repetitive assessment of the
same example) and among examiners (interreliability, ie, agreement among examiners on the findings for each example).
Future of Standardized Notes
By analysis, the SNF was shown to capture considerably
greater amounts of patient data than physicians’ progress
notes. Because the SNF and the SOS form have similar formats,
it is reasonable to assume that the SOS form will also capture
greater amounts of patient data than the recording methods
currently used. Thus, diligence in completing all appropriate
elements of the SOS form can be expected to fully document
the office visit and enhance appropriate third-party reimbursements. This is because all appropriate data will have
been captured by this standardized, readily tabulated, and
easily interpreted recording instrument. Moreover, unlike
handwritten recordings as found in a physician’s progress
notes, nothing essential will be missed if the SOS form is completed.
If osteopathic postdoctoral training institutions use this
training and certification process in conjunction with the SOS
and SNF forms, it will help them to easily and accurately track
resident encounters, the diagnoses they make, the procedures
they do, and their continuity of patient care. In the future, a
“vertically integrated seamless osteopathic curricula” could
incorporate these standardized record-keeping instruments
as the core of their evaluation and accreditation missions. This
would be accomplished by introducing use of the SOS form to
osteopathic medical students during their second year of medical school. Subsequently, the form can be incorporated into the
current recommendations for Research-in-OPTI6 programs.
The training and certification processes can finally be made
available to regional, state, and national osteopathic physicians at specially held training seminars or annual conventions.
A goal of the Louisa Burns Osteopathic Research Committee is to establish a central data repository that will permit
physicians certified in the use of the form to submit their daily
clinical data to a centralized database for use in national osteopathic outcomes research. The resultant large quantity of data,
nominally 50 million patient visits per year to osteopathic
physicians, would provide evidence supporting the value of
osteopathic principles and practice in all types of clinical
encounters.
80 • JAOA • Vol 104 • No 2 • February 2004
Downloaded From: http://jaoa.org/pdfaccess.ashx?url=/data/journals/jaoa/932021/ on 06/16/2017
Sleszynski et al • Original Contribution
ORIGINAL CONTRIBUTION
Conclusions
Acknowledgments
During the SOS form study, training and the testing process
evaluated both intraexaminer and interexaminer reliability
and allowed trainees to become certified in completing the
SOS form. This process ensures that each form submitted by
a certified trainee will be completed correctly, thereby providing accurate data for a tabulated database. (It should be
noted that this training process did not address the accuracy
of the physician’s ability to perform an examination. It only
addressed the physician’s ability to transcribe his or her findings accurately from one form or thought process onto the
SOS form.) As no one can be familiar with all of the nuances
of any given study, all thorough clinical outcomes research
studies should have a training and certification process for
participating investigators, whether they be novices or experienced. Training workshops provide reasonable insurance
that forms will be completed correctly. The training process also
instills confidence and enthusiasm in its participants.
Investigator instruction by indirect and direct didactic
contact on the use of the SOS form was found to be adequate
and successful. Acceptable intraexaminer and interexaminer
reliability was demonstrated. There were no differences in
trainee success rate when the level of clinical experience of
the trainees was compared, and this ranged from undergraduate fellows through attending physicians with up to 30 years’
experience. This training process has developed a pool of qualified and certified investigators for participation in osteopathic
research.
The authors thank the following colleagues for their participation in
this project: David Abend, DO, Scott Chaffin, DO, Bernadette G.
Kohn, DO, Robert Paul Lee, DO, Kathleen G. Meyer, DO, Anette
K.S. Mnabhi, DO, Kenneth E. Nelson, DO, Charles J. Smutny III,
DO, Lawrence W. Waite, DO, MPH, Brooks Blake, DO, N. Nelle
Cotton, DO, Adi Philpott, DO, Stuart F.Williams, DO, Tony
Stupski, and Judy Yang. The authors also thank Todd Michael
Larsen, Scott Stoll, DO, PhD, David Yens, PhD, Bethany Mondrawickas, and David George.
References
1. Sleszynski SL, Glonek T, Kuchera WA. Standardized medical record: a new
outpatient osteopathic SOAP note form: validation of a standardized office
form against physician’s progress notes. J Am Osteopath Assoc. 1999;99(10):516529.
2. Nelson KE, Glonek T. Computer/outcomes: Hardcopy SOAP note preliminary report. Family Physician. 1999;3(8):8-10.
3. American Medical Association and Health Care Financing Administration.
Documentation Guidelines for Evaluation and Management Services. May
1997. Available at: http://acep.org/2,308,0.html?ext=.pdf. Accessed January 26,
2004.
4. Ward RC, ed. Foundations for Osteopathic Medicine. Baltimore, Md:
Williams & Wilkins; 1997: 489-508.
5. UMLS Knowledge Sources. 12th ed. Bethesda, Md: US Department of
Health and Human Services, National Institutes of Health–National Library of
Medicine; 2001.
6. American Osteopathic Association. Administrative Handbook for the
Accreditation of Osteopathic Postdoctoral Training Institutions (OPTI). Chicago,
Ill: American Osteopathic Association; 2001.
Sleszynski et al • Original Contribution
Downloaded From: http://jaoa.org/pdfaccess.ashx?url=/data/journals/jaoa/932021/ on 06/16/2017
JAOA • Vol 104 • No 2 • February 2004 • 81