2013/12/12 2 Objectives for this presentation 講演で学んでいただきたいこと KEY FEATURES TESTING FOR CLINICAL DECISION MAKING 臨床推論のテスト：KEY FEATURES • Be able to explain what a key features question is. Joyce Pickering, MD, FRCPC, FACP IRCME Visiting Professor 2013/2014 Dec. 12, 2013 • Demonstrate an understanding of how their validity Key Features (KF)が何であるかを説明できる • Be able to state why they were developed. なぜ開発されたかを説明できる has been tested. 妥当性がどう検証されてきたかを示す • Be able to describe how to create a key features question. KF問題の作り方を説明できる 3 History 歴史 4 • G. Norman – studied patient management problems. Concluded that NormanはPMPを評価し以下の結論 • Trying to test テストしたいこと • clinical reasoning skills 臨床推論スキル • Problem solving skills 問題解決スキル • Clinical decision making 臨床決断 • Problem solving skills are specific to the case or problem encountered. 問題解決スキルは症例特異的 • Patient Management Problems PMP • Used in the 1980s in the Canadian national licencing exams 1980年代にカナダの医師国家試験で利用された • Used a special pen to make a result visible • E.g. if a CBC was highlighted with the pen, the results of the CBC showed up なぞると文字が浮かび上がる特殊なペンを使う • Poor correlation across cases (12 to 15 cases used in the Canadian licencing exam). 症例間での成績の相関は低かった • These skills are contingent on the effective manipulation of a few elements of the problem that are critical to its successful resolution – “key features”. 問題の2、3の要素を効果的に扱えるかどうかが、問題解決にとって決定的な意味を持つ（情報を網羅しても、2、3の要素が扱えなければ問題解決できない） • Therefore many more questions need to be asked (e.g 30 to 40) across many clinical areas in order to have a more reliable assessment of clinical problem solving. 臨床問題解決に関し、より信頼性のある評価をするには、多くの臨床領域にわたる問題が必要（例えば30~40問） 5 6 That is: For example: • You must ask about many different areas 多領域の問題が不可欠 • Doing well in clinical judgment in an obstetrics/gynecology case does NOT mean you will do well in clinical judgement in a pediatrics case. • A KEY FEATURE may not be having a complete 産婦人科症例での正答は、小児科症例での正答の予測に役立たない • However, the time to do the exam cannot be too long, or it becomes very expensive. 試験時間が長すぎると高コスト • Therefore we need to ask many short questions about many areas. よって多領域で多くの短問が必要 • These short questions need to focus on the most important elements of a case – the KEY FEATURES. differential diagnosis, but being able to think of the one most dangerous possibility. KFだけでは完全な鑑別診断に至らないこともあるが、最も危険な可能性を思いつくことはできる • A KEY FEATURE may be even if the candidate does not know the diagnosis, they know that the situation is probably dangerous and the patient must stay in the hospital KFは受験者が診断を知らなくても、状況が危険かもしれないので、院内に留まらせるという判断ができればよい短問は症例の最も重要な要素に焦点を当てる→KEY FEATURES 1 2013/12/12 7 • These specific critical decision points are the KEY 8 • www.mcc.ca FEATURES of the problem これらのクリティカルな決断ポイントが問題のKFである 10 例題A あなたは地域病院に赴任して間もない医師である。小売業の55歳男性が、胸骨後面の痛みを訴えて来院。痛みは1年前から反復し、労作で悪化し、中央、深部に生じる。安静時には痛みはなし。頻度や強度は悪化なし。他に複数の医師を受診したが、今回再度受診を希望。喫煙歴は10本/日を20年間。 You are a physician who has recently come to practice in a community hospital. A 55-year-old merchant, who has run his own business for a number of years, comes to you with retrosternal chest pain. This pain has been recurring in the past year, is deep and central, and usually precipitated by exertion. He has no pain at rest. Episodes have not become more frequent or more severe. He has seen other physicians but he wishes a reassessment at this time. The patient has a 20-year history of smoking 10 cigarettes a day 質問１）考えられる診断として最も可能性の高いものはどれか。次の17の選択肢から一つだけ選びなさい。 1. 慢性膵炎 Chronic pericarditis 2. うっ血性心不全 CHF 3. 胸部大動脈解離 Dissecting thoracic AA 4. 過換気症候 Hyperventilation syndrome 5. 縦隔炎 Mediastinitis 6. 心筋梗塞 MI 7. 慢性心筋炎 Myocarditis 8. 筋筋膜疼痛症候群 Myofascial syndrome 9. 神経衰弱症 Neurasthenia 10. 冠攣縮性狭心症 Prinzmetal’s angina 11. 逆流性食道炎 Reflux esophagitis 12. 安定狭心症 Stable angina 13. T1-T6神経痛 Neuralgia 14. T1-T6神経根炎 Radiculitis 15. 慢性甲状腺炎 chr. thyroiditis 16. Tietze症候群（反復性肋軟骨炎） 17. 不安定狭心症 unstable angina 11 質問２）身体診察では、緊張感があるが、呼吸苦はなし。チアノーゼ、顔面蒼白、黄疸も見られない。 BT 37℃ BP 140/95 mmHg P 78/分, 整 RR 15/分身長 170 cm 体重 98 kg 頸静脈胸骨角上3 cm 心音正常呼吸音正常腹部所見なし末梢脈拍触知浮腫なし眼底正常 12 この患者に対する診断では、どの検査や処置を選択するか。次の21の選択肢から適切なものをすべて選びなさい。 1. 冠動脈造影 coronary angiogram 2. room airの血液ガス ABG 3. バリウム嚥下による造影 barium swallow 4. 気管支鏡 bronchoscopy 5. 胸部X線 CXR 6. 心エコーechocardiogram 7. 血沈 ESR 8. 食道運動機能検査 eso.motility 9. 負荷ラジオアイソトープ血管造影 exercise radionucleotide angio 10. 運動負荷テスト EST 11. 空腹・随時血糖 fasting and spot glucose 12. 上部消化管内視鏡 gastroscopy 13. Hb、白血球数と分画 HGB.WBC 14. HIVスクリーニング 15. 血小板数 platelets 16. 安静時心電図 EKG 17. 血清コレステロール（総、HDL、TG） 18. 血清CKとCK-MB 19. 血清蛋白電気泳 SPEP 20. タリウム心筋シンチ stress thallium 21. 血清総蛋白とアルブミン protein and albumin 2 2013/12/12 質問３ 13 この患者に対し、問題が生じるリスクを下げるためにどのようなアドバイスをおこないますか。下記17の選択肢から6つまで選びなさい。 1. 降圧療法 BP reduction 2. 血糖低減 bl.sugar reduction 10. 理学療法 physiotherapy 11. 心理療法 psychotherapy 3. 活動レベル低減 12. 呼吸機能検査 PFTs decrease activity 13. 休養プログラム 4. 運動プログラム relax. program exercise program 14. 禁酒 stop alcohol 5. 食物繊維の多い食事 15. 禁煙 stop smoking high fiber diet 16. ストレス低減 6. 高蛋白の食事 stress reduction high protein diet 17. 減量 weight reduction 7. 催眠療法 hypnotherapy 8. 低コレステロール食 low chol. diet 9. 低蛋白、高炭水化物食 low prot. High carb diet 14 • 例題B 不安そうな若い母親が1歳3か月の娘を連れて救急外来受診。この幼児は母親がジメンヒドリナート（制吐剤）とリンゴジュースを与えてから、 2日間下痢をしています。嘔吐はなし。具合が悪そうで、母親の腕の中で泣いているが、流涙はなし。体力が落ち、気力の低下が見られる。 An anxious young mother brings her 15-month-old daughter to the Emergency Department. The child has had diarrhea for 2 days for which her mother gave her dimenhydrinate and apple juice. The child has not vomited. She appears ill and is crying with no tears in her mother's arms. She seems weak and lethargic. バイタルサイン：体温（腋下） 36.5℃ 脈拍 120/分呼吸 40回/分血圧 85/55 mmHg 15 16 質問1）この幼児のマネジメントはどうするか？（4つまで選択可）体重11kg。流涙なし。粘膜乾燥あり。Capillary refill time 正常。皮膚ツルゴール低下し、腹部皮膚がテント状。おむつは少量の緑色の水様便がある以外は乾いている。 She weighs 11 kg. She has no tearing. Mucous membranes are dry to the touch. The capillary refill time is normal. There is diminished skin turgor with some tenting of the abdominal skin. The diaper is dry except for a small amount of watery green stool. 腹部レントゲン Abd. Xray リンゴジュースを与え、飲み方を確認動脈血ガス ABG 血糖 blood glucose BUN 血算 CBC 気の抜けたジンジャーエールを与え、飲み方を確認 flat ginger ale and observation 8. ブドウ糖水を与え、飲み方を確認 glucose water and observation 9. 5%のD-グルコースの静注 IV D5W 10. 5%のD-グルコース入り生食を静注 IV D5Nsal. 1. 2. 3. 4. 5. 6. 7. 経鼻胃水分補給 ORS by N/G 経口絶食 NPO 経口補水 ORS 血清クレアチニン creat 血清電解質 lytes 血清浸透圧 osmolal. 便培養 stool culture 便虫卵・寄生虫検査 stool O and P 19. 検尿 U/A 20. 尿中電解質 urine lytes 21. 尿浸透圧 U osm. 11. 12. 13. 14. 15. 16. 17. 18. 17 質問２）今回、この赤ちゃんにはどのような薬を投与すべきか。（薬剤の名前を２つまで挙げる。投薬不要の場合は“なし”と記入。）１．２． • What medications should you prescribe for this girl at this time? 例題C 55歳男性が巻き爪から感染症を発症し来院した。あなたは担当し、適切に処置した後、家族歴から、この患者の父親が大腸がんのため60歳で死亡したことに気づいた。 A 55-year-old man presents with an infected ingrown toenail. You treat this condition appropriately, and then notice on the chart that his father died of bowel cancer at 60 years of age. 3 2013/12/12 19 質問１）この患者からさらに情報を引き出すのに最も重要な内容は以下のうちどれか。（注意：16の選択肢から4つ選ぶこと）母親の膀胱がん家族歴 mother with bladder cancer 10. 腺腫様ポリープの病歴 personal history of adenomatous polyps 3. 11. 過敏性腸症候群の病歴 frequent formed BMs 4. 12. ジアルジア症の病歴 Hx of giardiasis 5. 13. 過形成性ポリープの病歴 Hx of hyperplastic polys 6. 14. 炎症性腸疾患の病歴 Hx of IBD 15. 座り姿勢の多いライフスタイル 7. sedentary lifestyle 8. 16. 喫煙習慣 smoking What additional information would be most important to elicit from this patient? アルコール依存症 alcoholism 兄弟の大腸がん家族歴 brother with bowel cancer 魚のくん製をよく食べるかどうか diet high in smoked fish 父親ががんと診断された際の年齢 fathers age at diagnosis of cancer 高脂肪で繊維の少ない食生活 high fat low fiber diet 高タンパクで繊維の多い食生活 high protein high fiber diet 乳糖不耐症 lactose intolerance 下剤濫用 laxative abuse 1. 2. 9. 20 質問２）この患者に推奨されることは次のうちどれか。（注意：12の選択肢から１つ選ぶこと） 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. S状結腸鏡検査 60 cm sigmoidoscopy CEA検査全結腸内視鏡検査 colonoscopy 腹部CT abdominal CT 直腸診 digital rectal exam 注腸造影 double contrast barium enema 便潜血反応 fecal occult blood 遺伝子スクリーニング genetic screening 経静脈尿路造影 Intravenous pyelogram 腹部MRI abdominal MRI 腹部触診 abdominal palpation 腹部超音波検査 abdominal ultrasound Which one of the following is your recommendation for this patient? 21 Objectives for this presentation 講演で学んでいただきたいこと Have been used: • Medical Council of Canada Qualifying Exam • (national licencing exam) • Be able to explain what a key features question is. • College of Physicians and Surgeons of Pakistan • America College of Physicians (MKSAP recertification) • American College of Colon and Rectal Surgeons – • • • • 22 Key Features (KF)が何であるかを説明できる • Be able to state why they were developed. なぜ開発されたかを説明できる CARSEP recertification Royal Australian College General Practitioners Swiss National Examination Board McMaster clerkships German Medical Schools • Demonstrate an understanding of how their validity has been tested. 妥当性がどう検証されてきたかを示す • Be able to describe how to create a key features question. KF問題の作り方を説明できる 23 • Key feature questions were developed and tested on final year Canadian medical students ( 1987-91). KF問題は、カナダの医学生最終学年に対して開発され、テストに利用されてきた。 24 • Face validity: Examinees were asked to judge the importance of the clinical problems and the importance and difficulty level of the competencies testing by the questions. 表面妥当性：臨床問題の重要性、コンピテンシーのテストの重要性や難易度について受験者に判断してもらった • High levels of face validity 表面妥当性のレベルは高い • 96% of students said the questions were at the “correct level” although the average score on the exam was only 50%. 96％の学生は問題が適切なレベルだと答えたが、平均点は50% 4 2013/12/12 25 • Content validity: 内容妥当性 • MCC test committee members and 69 Cdn medical faculty who were active in clerkship teaching asked to judge retrospectively if they agreed with the key features – 国家試験委員会とクラークシップ教育に関与する69名の教員がKFの一致度を判定 26 Canadian exams: • Medical Council of Canada Qualifying Examination 1 (MCCQE1). • Typically taken at the end of year 4 of medical school (graduation) • 75% multiple choice questions (196 questions) • 171 key features – 121 unanimous agreement, 158 > 80% agreement. 171KFに対し、121は無記名で一致、158は8割以上の一致度 • 25% key features questions (Clinical Decision Making = CDM) • Same judges asked to generate their own key features for scenarios different from the first set, and these were compared to the test key features. 同じメンバーに、異なるシナリオに対するKFを各自作成してもらい、テストとの一致度を判定 (80 questions) 4年生が受験。75%はMCQ（196問）。25%がKF問題（CDM－80問） • Medical Council of Canada Qualifying Examination 2 (MCCQE2) • Typically taken after 18 or 24 months of residency • OSCE type examination (16 stations over two days) 研修開始後18、24カ月後受験。16ステーション、2日間のOSCE • 94% level of concordance 94％の一致 • High level of content validity 内容妥当性のレベルは高い 27 28 Predictive validity 予測妥当性 • High correlation (0.8) between MCQ and CDM (key • Study by Tamblyn et. al. JAMA, 2007 features) questions on the MCCQE1. MCQとCDMは相関が高い（0.8程度） • Physician scores on Canadian national exams as a predictor of complaints to medical regulatory authorities. Tamblynら（2007）が、医事規制当局へのクレームを目的変数としてカナダ医師国家試験スコアの関与をみた 29 30 • Study population: 対象 • Independent variable: 説明変数 • Scores of candidates on these exams. 国家試験点数 • Dependent variable: 目的変数 • Retained complaints about these physicians, after they began practice, to the regulatory body for the Province of Ontario (College of Physicians and Surgeons of Ontario) or to the regulatory body for the Province of Quebec (Collège des médecins du Québec) オンタリオ州、ケベック州の医療規制当局へのクレーム • Population of Ontario about 9 million • Population of Quebec about 7 million • All physicians who took the MCCQE2 between 1993 and 1996 and were licenced to practice in at least one of Ontario or Quebec = 3424 physicians. MCCQE2に合格した3424人の医師 • All complaints filed with the provincial regulatory agency from the date of licensure until 2005 were studied. 2005年までに免許取得した医師のクレーム • Total 1116 complaints (4.9 complaints/100 practice years), 696 retained complaints 1116件（4.9件/100延べ診療年）、696 件は医師に問題ありと判定。 • 17.1% of physicians had at least one retained complaint. 17.1%の医師は少なくとも1回は問題ありのクレームを受けた 5 2013/12/12 31 32 Canadian exams: 問題のあるクレーム無しという測定値 Measured by lack of complaints • Medical Council of Canada Qualifying Examination 1 (MCCQE1). • Typically taken at the end of year 4 of medical school (graduation) • 75% multiple choice questions (196 questions) • 25% key features questions (Clinical Decision Making = CDM) ?? Exam scores (80 questions) 4年生が受験。75%はMCQ（196問）。25%がKF問題（CDM－80問） • Medical Council of Canada Qualifying Examination 2 試験スコア Quality of a physician’s practice 医師の診療の質 (MCCQE2) • Typically taken after 18 or 24 months of residency • OSCE type examination (16 stations over two days) 研修開始後18、24カ月後受験。16ステーション、2日間のOSCE 33 34 Associations with the number of retained complaints score RR 95% CI P value MCCQE 1 overall 1.39 (1.14-1.70) .001 MCCQE1 MCQ 1.25 (1.03-1.50) .02 MCCQE1 CDM 1.51 (1.25-1.84) <.001 MCCQE2 overall 1.19 (1.00-1.42) .05 MCCQE2 Communication 1.38 (1.18-1.62) score <.001 MCCQE2 data acquisition 0.98 (0.83-1.16) score .85 MCCQE2 Problem solving score 1.02 (0.88-1.19) 0.78 MCCQE 2 Communication score adjusted for CDM score 1.32 (1.13-1.71) <.001 • Note: McMaster University study showed correlation of multiple mini interview scores (for admission) with Clinical Decision Making (CDM) scores. McMaster大の研究では、入学試験でのMMIスコアとCDM スコアも相関している Change in relative rate per 2 SD decrease in score 35 36 Further evidence • Article by Bordage, Meguerditchian and Tamblyn, Academic Medicine 2013 • Defined physician-related practice indicators (PRINDS) that, if not done, contribute to either causing or preventing suboptimal care, and to what extent the two components of the Medical Council of Canada exams measured these. 医師の診療の質指標（PRINDS）と医師国家試験スコア • PRINDS defined via • literature and Google search for practice errors, • defined needed behaviour/skill, and then • validated this with MCC tests committees and physician experts. 診療過誤、必要な行動やスキル、国家試験委員会や医師専門家による妥当性検証によって、PRINDSは定義された • Examples of PRINDS: • Prescribe influenza vaccination for high risk patients. ハイリスク患者へのインフルエンザワクチン接種 • Provide heart failure patients with written instructions on activity, diet and medication 心不全患者への活動、食事、薬物療法に関する指導内容記載 • Recognize an abdominal aortic aneurysm • Recognize an ectopic pregnancy 腹部動脈瘤、子宮外妊娠の同定 6 2013/12/12 37 38 Objectives for this presentation 講演で学んでいただきたいこと • Three parts of the exams looked at: • Be able to explain what a key features question is. • Part 1 Multiple choice questions Key Features (KF)が何であるかを説明できる • Part 1 Clinical Decision making (key features • Be able to state why they were developed. • Part 2 OSCE (entire exam) • Part 1 Clinical decision making (key features) had the highest proportion of marks related to PRINDs (30%) なぜ開発されたかを説明できる • Demonstrate an understanding of how their validity has been tested. 妥当性がどう検証されてきたかを示す • Be able to describe how to create a key features question. KF問題の作り方を説明できる CDMにはPRINDSと関連する問題が3割含まれていた • Part 1 MCQ – 20% • Part 2 Exam – 5% 39 40 Process: • Define the domain 領域を決定 • What are the all skills that the candidate is expected to have? 受験者に持ってもらいたいスキルは何か • The problems must be drawn from defined domain of problems for which the candidate is accountable 受験者が対応可能な問題が及ぶ領域に基づいて問題を作成する • Define the blueprint ブループリントを設定 • How much of each discipline should be on the exam? 各科の内容がどの程度試験に含まれているべきか • The process of problem selection must be guided by an exam blueprint 問題選択プロセスは試験ブループリントによって方向づける必要あり • Write the scenarios and questions. シナリオと問題の記載 • The exam should have about 40 of these scenarios. 試験には40問のシナリオが必要 • The questions should only be about the key features. 問題はKFに関する部分のみ • Avoid “cueing” whenever possible. 可能な限り“手がかり”を避ける 41 Developing cases and questions 症例と問題の作成 42 Developing cases and questions • Problem Selection and Context Definition • Problem Selection and Context Definition • • • • • 問題選定と状況の設定 Key Features definition KFの定義 Case scenario シナリオ Test questions and format テスト問題とフォーマットの設定 Scoring keys keyの点数 Pilot test the question 問題の試用問題選定と状況の設定 • Key Features definition • Case scenario • Test questions and format • Scoring keys • Pilot test the question. 7 2013/12/12 43 Problem Selection and Context Definition 問題選定と状況の設定 • Select clinical problem: 臨床問題の選定 • Must be drawn from a list of problems that examinees are responsible to know how to address • Third trimester vaginal bleeding • 受験者たちが対応法について知っているべき問題リストから選定 • 今回は妊娠後期不正出血 • Select age group: 年齢層の選定 • Pregnancy neonatal and infant; pediatrics; adolescent; adult; geriatrics • Pregnancy, neonatal and infant • 妊娠・新生児・幼児、小児、思春期、成人、老年 44 • Select clinical situation and site of care 診療の設定 • Undifferentiated complaint; a single typical problem; a multiple or multi-system problem; a life-threatening event; preventive care and health promotion. • A life threatening event • 分類不能の主訴、単一の典型的問題、多数・多系統の問題、生命に危険が及ぶ状態、予防医学や健康増進 • 今回は生命に危険が及ぶ状態 • 今回は妊娠・新生児・幼児 45 46 Medical Council of Canada Objectives Developing cases and questions • http://apps.mcc.ca/Objectives_Online/ • Problem Selection and Context Definition • Key Features definition KFの定義 • Case scenario • Test questions and format • Scoring keys • Pilot test the question. 47 Key Features Definition KFの定義 • Ask this question: この質問をしてみる • What are the critical steps in the resolution of this problem for patients in this age group? • この年齢層の患者における問題の解決にクリティカルな段階とは • Where’s the challenge for a graduating medical student? • 医学部卒業時点の学生がちょうど悩むのはどのあたりか 48 Key Features Definition • The development of good key features can be challenging. よいKFの開発は難しい • Needs to be done by clinicians 臨床家が設定する必要性 • Discussion/review with colleagues is highly recommended 同僚とのブラッシュアップが強く推奨される • Each key feature has three basic elements: • If the critical issue is interpreting the data, then give the clinical data • 各KFに３つの基本要素 • A set of conditions: “Given a pregnant woman experiencing third trimester bleeding with no abdominal pain” • A task: “consider placenta previa as a leading diagnosis” • Some qualifier: “leading diagnosis” • 状況「妊娠後期の腹痛のない出血」、課題「前置胎盤を主診断と考慮」、 KFのポイント「主診断」 and ask questions aimed at interpretation. 臨床データの解釈が課題なら、データを示して解釈を問う質問をする • If the critical issue is one of eliciting the data, then give a patient complaint and ask the examinee to identify the specific data that would be elicited from the patient. 情報収集が課題なら、患者の主訴を提示し、患者から収集すべき特定のデータを同定するような質問を受験者にする 8 2013/12/12 49 50 Key Features Definition Key Features Definition • In diagnosis 診断 • If the key feature is focusing on eliciting a probable diagnosis, then ask for a differential diagnosis. KFが可能性のある診断を挙げることであれば、鑑別診断を尋ねる • If the key feature is focusing on eliciting THE working diagnosis or leading diagnosis at this time then ask for one diagnosis. KFが主たる診断を引き出すことであれば、一つだけ診断を尋ねる • Key feature steps or actions need to be a specific as possible – KFのステップやアクションは可能な限り特定的に • E.g. prescribe morphine 例：モルヒネ処方 • E.g. order a complete blood count 例：全血球血算の指示 • Avoid: 避けるべき表現 • “manage appropriately” 適切にマネージする • “investigate appropriately” 適切に検査する 51 52 Key Features Definition Key Features Definition • E.g. Given a pregnant woman experiencing third trimester • Need to be developed locally, with clinicians familiar with vaginal bleeding with no abdominal pain, the candidate will: 妊娠後期に腹痛のない不正出血を経験している妊婦を診たとき、受験者は・・・ • 1. Consider placenta previa as a leading diagnosis 前置胎盤を主たる診断とする the medical student “clinical reality”. 各国や地域に応じた形で医学生の臨床の現状を踏まえて開発される必要あり • E.g. do you expect all your graduating medical students to be able to recognize the clinical presentation of a placenta previa? 前置胎盤の臨床を全ての医学生が認識していると期待できるか • Possible reference: Key Features for many clinical • 2. Avoid performing a pelvic examination presentations have already been defined by the College of Family Physicians of Canada for finishing family medicine residents. すでに多くのKFはカナダの家庭医向けに定義されており、下記にて情報開示されている • http://www.cfpc.ca and search for “key features” • Or google “College of Family Physicians of Canada key features” 内診の実施を回避する • 3. Avoid discharging home from the emergency department 救急外来から帰宅させない • 4. Order a pelvic ultrasound 骨盤内超音波を指示する 53 54 Developing cases and questions Writing the Case Scenario シナリオ記載 • Problem Selection and Context Definition • Usual elements: 通常の要素 • Patient’s age and gender, presenting site (office, emergency room etc.), patients presenting condition, followed by whatever clinical details are required. 患者の年齢性別、場（診療所・救急など）、患者の症状など必要に応じて様々な情報 • Think of real patients you have seen in practice. 実際に診療している患者を想起する • Avoid “medicalese” – e.g. say “coughed up blood” rather than “hemoptysis” • Key Features definition • Case scenario シナリオ • Test questions and format • Scoring keys • Pilot test the question. 医学用語を避ける：血痰ではなく、咳をして痰に血が混じったという表現 9 2013/12/12 55 56 • At 20:00h, the nurse in the emergency department asks you to see a 24 year old woman who has two children and is now 31 weeks pregnant, reporting of bright red bleeding coming from her vagina that began about two hours ago. The three sanitary napkins that she used were completely soaked. Her pregnancy has been normal, as were the two previous ones. She has not had any pain in her belly and she says that the fetus is moving as usual. 夜8時ERに妊娠31週の24歳女性が2時間前から鮮血の不正出血で来院。ナプキンを3回交換。妊娠は3回目でいずれも順調。腹痛なし。胎動正常。 • Her BP is 110/70 and her pulse is 92/min. Your exam of the abdomen reveals a uterine height of 31 cms with soft, non tender uterus. The fetus is in a breech position and has a heart rate of 150/min. No bleeding has occurred since she arrived 25 minutes ago. BP110/70, PR92. 子宮底長31cmで圧痛なく軟。胎児は骨盤位で 150bpm。25分前に来院した後は出血は見られていない。 Developing cases and questions • Problem Selection and Context Definition • Key Features definition • Case scenario • Test questions and format テスト問題とそのフォーマットの設定 • Scoring keys • Pilot test the question. 57 58 Test Questions and Format Test Questions and Format • Two formats recommended:2つの推奨フォーマット • Short answer “write in”. 短答式で「記入」 • Short menu responses リストを挙げる • Use a direct formulation: 直接的な明快な言い回しで • E.g. What is your leading diagnosis at this time? 例：この時点での主たる診断は何か • NOT: Your leading diagnosis is _____________. 悪い例：主たる診断は（）である • Present list in alphabetical order ABC順に並べる • May include correct but not key options – not scored • Questions must derive from the key features you have defined. 問題は定義したKFに基づかなければならない • Questions must focus on clinical actions or decisions 問題は臨床決断に焦点を当てたものにしなければならない • Not on the reasoning behind these actions 決断の背景にある考え方に焦点を当てるのではない間違いではないがキーでもない選択肢もあり得る（スコアには含めない。例：安定狭心症患者に食物繊維の多い食事を指導する） • To develop the menu, can pilot test a question as a write in question, and use the incorrect responses to make the list. リストを作るため短答記入問題で試用し、誤答を列挙することも • Only one action per option. 各選択肢は単一のアクションを入れる • Almost always, include “none” as the last option. 「いずれでもない」というオプションはほぼ常に入れておく 59 60 Test Questions and Format Test Questions and Format • In general, use write in questions for questions that test • Every question is followed by one of the following diagnoses, management and prescriptions. 一般的に記入式は診断、マネジメント、処方の問題に利用 • Use short menu questions for history and physical exam findings or investigations. リストは病歴、身体診察、検査に利用 • Use standard questions as much as possible. 出来る限り問題は標準化 statements: 全ての問題は以下のいずれかの形式をとる • List (select) only one １つだけ選べ • List (select) up to X X個まで選べ • List (select) as many as are appropriate 適切なものをいくつでも列挙せよ • E.g. What is your leading diagnosis at this point? 例：この時点での主診断は何ですか • What tests, if any, will you order next? 次に検査をするとすればどれを指示したいですか 10 2013/12/12 61 62 Question 2. What steps will you take in your immediate assessment and management of this patient? Select as many as are appropriate. 問題2：この患者の評価やマネジメントに必要なものをいくつでも選べ • Question 1: What is your leading diagnosis at this time? List only one, or write “normal” if you judge this situation to be within normal limits. 問題１：この時点での主診断は何か。解答は１つ。正常範囲内と考えたなら、「正常」と記載しなさい。 • 1.________________________________ 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. Artificial rupture of members Cervical swab for chlamydia Complete blood count CT scan of abdomen and pelvis Cross and match for 2 units of blood Discharge home with instructions to return if bleeding recurs. Fetal scalp monitor to assess status of fetus HIV serology INR Large bore IV line Pelvic ultrasound Pelvic examination Syphilis serology Ultrasound using vaginal probe Vaginal swab for Group B streptococcus 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 人工破膜子宮頸部クラミジア検査全血球血算腹部・骨盤CT 輸血2単位クロスマッチ再出血時再診指示で帰宅胎児頭皮電極モニター HIV検査 PT-INR 大口径静脈ライン骨盤エコー内診梅毒検査経膣エコー膣部B群溶連菌検査 63 64 Developing cases and questions Scoring key • Problem Selection and Context Definition • List of the correct responses, and a system of assigning • Key Features definition numerical scores to the keyed responses. 正答のリストと、点数を付けるためのシステム • All the correct answers appearing in the scoring key should be part of the key features. キーのスコアに表される正答は全てKFの一部であるべき • Scoring should also be discussed by a group of clinicians – e.g., how many not harmful tests are too many? 臨床家のグループでスコアは議論を要する。例：リスクのない検査をどのくらい多くやったら多すぎるのか • Case scenario • Test questions and format • Scoring keys keyの点数設定 • Pilot test the question. 65 Scoring key – Key feature 1: Given a pregnant women experiencing third trimester vaginal bleeding with no abdominal pain, the candidate will consider placenta previa as a leading diagnosis. KF1：妊娠後期に腹痛のない出血に対し主診断に前置胎盤を想定 66 Key feature 2. Given a pregnant woman with third trimester vaginal bleeding with no abdominal pain, the candidate will avoid performing a pelvic examination because it may cause fatal bleeding. KF2：出血を引き起こす可能性がある内診を避ける Score Criteria クライテリア Score Criteria クライテリア Synonyms 同義語 1.0 1.0 Placenta previa 前置胎盤 Marginal placenta, low placenta, low insertion 辺縁前置胎盤、低置胎盤 Did NOT select #13, pelvic examination 「13. 内診」を選ばない 0 0 Listing more than one response 2つ以上解答 -Artificial rupture of membranes -Cervical swab for chlamydia -Fetal scalp monitor to assess status of fetus -Pelvic examination - Ultrasound using vaginal probe OR - Selecting more than 6 items 人工破膜、頸部クラミジア検査、頭皮モニター、内診、経膣エコー、6項目以上を選んだ場合 11 2013/12/12 67 Key Feature 3: Given a pregnant woman with third trimester vaginal bleeding with no abdominal pain, the candidate will avoid discharging home from the emergency department. KF3：ERから帰宅させないようにする 68 Key Feature 4: Given a pregnant woman with third trimester vaginal bleeding with no abdominal pain the candidate will order a pelvic ultrasound to confirm the placenta location. KF4：胎盤の位置確認のために骨盤エコーを依頼 Score Criteria クライテリア Score Criteria クライテリア 1 Did not select “Discharge home with instructions to come back if bleeding starts again” 「再出血時再診指示で帰宅」を選ばない 1.0 Orders pelvic ultrasound 骨盤エコーを依頼 0 0 Discharge home with instructions to come back if bleeding starts again OR Selecting more than 6 items. 「再出血時再診指示で帰宅」、6項目以上を選択 Ultrasound using vaginal probe or selecting more than six items. 経膣エコー、あるいは6項目以上を選択 69 • Note: • 3 points for the one question – “What steps will you take next?” 「次のステップは」は、1問で3点 • Each point related to the key feature. 点数はそれぞれがKFと関連づけられる（問題2にKF2、KF3、KF4が含まれている） 70 Key feature Question number 問題番号 Points Diagnosis of placenta previa 前置胎盤の診断 1 1 Does not do a pelvic exam 内診をしない 2 1 Does not send the patient home 2 帰宅させない 1 Orders a pelvic ultrasound 骨盤エコーを依頼 1 2 点数 71 72 Resources for writing Key Feature Problems KF問題の情報源 Biggest challenges • http://mcc.ca/about/test-committee-resources/ • Time to write the questions: • Get together groups of experienced clinicians, train them in how to write and score such questions • Time to score the exam: • Write in questions are scored manually up until now by MCC. • Research in progress on computer scoring of the write in questions 12 2013/12/12 73 • Using it on a smaller scale: 小規模に試用した研究 • Study by Hatala and Norman • Internal Medicine Clerkship 内科クラークシップ • 8 week clerkship, therefore 6 exams 8週で試験6回 • Blueprint taken from internal medicine clerkship objectives 内科クラークシップ学習目標からブループリント • 5 faculty members assigned to develop and review key features for 14 to 18 clinical problems each over an 8 week period – total 82 problems 5人の教員が、8週間で毎回14~18の臨床問題を作成、合計82個の問題を開発し、レビューした 74 • Workshop of faculty and residents to write the questions 教員と研修医が作問のWSを受けた • They were given the problem, the age of the patient, the clinical situation and the defined key features 問題、患者の年齢、臨床状況、定義されたKFを与えられた • Generated 35 problems. 35問を作成 • Remainder (47) done by one faculty member 残りは1名が作成 • 15 problems per exam (2 hours) 2時間で15問 • Cronbach’s alpha 0.49 75 76 Conclusions: References: • An excellent way to test clinical decision making • Bordage G, Meguerditchian A, Tamblyn R, Practice Indicators 臨床推論をテストするために優れた方法 • Consider: 是非こんな形で考えてみましょう • • Start small - clerkship exam 小規模からスタート • Then graduation exam 次いで卒業試験に • • Eventually on the national exam? 最終的には国家試験？ • Predictive validity in the Japanese context should be studied. 是非日本での妥当性研究をすべき • We need to determine what it would add to the current test methods. 現状の評価方法にKF問題が何を加えるかを決める必要がある • • of Suboptimal Care and Avoidable Adverse Events: A content analysis of a National Qualifying Examination. Acad. Med 2013; 88: 1493-1498 Hatala R, Norman G, Adapting the Key Features Examination for a clinical clerkship. Medical Education 2002;36:160-165 Page G, Bordage G, The Medical Council of Canada’s Key Features Project: A More Valid Written Examination of Clinical Decision-making Skills. Acad. Med. 1995;70:104-110. Page, G, Bordage G, Allen T, Developing Key-feature Problems and Examinations to Assess Clinical Decision-making Skills.Acad.Med 1995;70:194-201 Bordage G, Brailovsky C, Carretier H, Page G, Content Validation of Key Features on a National Examination of Clinical Decision-making Skills. Acad.Med 1995;70:276-281. 77 References • Medical Council of Canada, Guidelines for the Development of Key Feature Problems and Test Cases. Available on-line at www.mccc.ca • Tamblyn R, Abrahamowic M, Dauphinee D, et. al. Physician Scores on a National Clinical Skills Examination as Predictors of Complaints to Medical Regulatory Authorities. JAMA 2007; 298(9):993-1001 13