A Verbal Protocol Analysis of Test-takers’ Processes of
Taking IELTS Academic Reading Test
雅思学术类阅读测试受试答题过程
有声思维研究
Xiangdong Gu
Qin Lv
Lu Zhao
Chongqing University
6th Nov. 2013
Our Teams’ Studies Using Verbal Protocol Analysis
 Gao. X & Gu. X (2008) An introspective study on test-taking process for banked cloze ,
Teaching English in China (4), 3-16.
 Zheng. Y & Gu. X (2009) A study on the construct validity of long dialogue multiple-choice
listening comprehension tests: retrospection approach, Teaching English in China (6), 15-26.
 Gu. X & He.Y (2011) An introspective study on testees’ process of taking English to Chinese
translation test in TEM 8, Foreign Languages and Translation (3), 67-73.
 Gu. X & Shi. C (2012) A retrospective study on test-takers’ cognitive and metacognitive
processes in taking a compound dictation test, Chinese Journal of Applied Linguistics 35 (4),
400-420.
 Gu. X & Wang. J (2012) An introspective study on test-takers’ processes of taking sentence
translation test in CET-4
 Gu. X & Shen. Y (2012) An introspective study on test-takers’ processes of taking vocabulary
test in reading section of HSK-6
 Gu, X, Lv. Q & Zhao. L (2013) A verbal protocol analysis of test-takers’ processes of taking
IELTS academic reading test
Examples of IELTS Studies
• Vol 4 The Development of IELTS: A study of the effect of
background knowledge on reading comprehension
--Caroline Clapham (1996)
• Vol 19 IELTS Collected Papers: Research in speaking
and writing assessment
--Edited by Taylor & Falvey (2007)
• Vol 23 Assessing Academic English: Testing English
proficiency, 1950–1989 – the IELTS solution
--Alan Davies (2008)
• Vol 24 Impact Theory and Practice: Studies of the IELTS
test and Progetto Lingue 2000
--Roger Hawkey (2006)
• Vol 25 IELTS Washback in Context: Preparation for
academic writing in higher education
--Anthony Green (2007)
• Vol 34 IELTS Collected Papers 2: Research in reading
and listening assessment
--Edited by Lynda Taylor and Cyril J Weir (2012)
Academic Reading
Weir. C, Hawkey. R, The relationship between the academic reading construct
Green. A, Ünaldi. A & as measured by IELTS and the reading experience of
Devi. S
students in their first year of study at a British university
Construct validity in the IELTS Academic Reading test:
Moore. T, Morton. J &
a comparison of reading requirement in IELTS test items
Price. S
and in university study
Weir. C, Hawkey. R, The cognitive processes underlying the academic
Green. A & Devi. S
reading construct as measured by IELTS
Green. A & Hawkey. R
An empirical investigation of the process of writing
Academic Reading test items for the International
English language
Definition of Verbal Protocols
Rich data sources containing individuals’ spoken thoughts that are associated
with working on a task. Subjects usually either think aloud as thoughts occur
to them while working on a particular task (concurrent introspection) or
verbalize their thoughts upon completion of the task (immediate
retrospection). The verbalizations are recorded and then coded according to
theory-driven and/or empirically driven categories. (Gale Encyclopedia of
Education: Research methods)
Source: http://www.answers.com/topic/research-methods-verbal-protocols
Validity and Reliability of Verbal Protocols
Validity
Information captured
within verbal reports
Subject 1
corresponds closely with
similar verbal protocols
of the same task
Information
actually needed
Subject 1
Reliability
Coder 1
Coder 2
coding scheme accurately
captures the behavior
Coder 1
Coder 2
Research Questions
 What types of strategies do test-takers employ when taking
IELTS academic reading test?
 Are there any similarities and differences in the employment of
strategies between high-level group and low-level group of testtakers?
 Are there any similarities and differences about test-takers’
employment of strategies among different task types?
Task Types
All task types of IELTS
Task types in selected materials
• Identifying information
(True/False/Not given)
• Matching information
• Sentence completion
• Diagram label completion
• Short-answer questions
• Identifying writer’s
views/claims
(Yes/No/Not given)
• Matching headings
• Matching features
• Matching sentence endings
• Multiple choice
• Summary/note/table/flow
chart completion
1 Summary/Note/Table/Flow-chart completion
2 Identifying information (True/False/Not given)
3 Matching headings
4 Summary/Note/Table/Flow-chart completion
5 Matching features
6 Matching headings
7 Multiple choice
8 Sentence completion
Instrument: Test 2 in Cambridge IELTS Test 8 (2011)
Topic
Passage 1
Passage 2
Passage 3
Sheet glass
manufacture the float
process
The Little Ice Age
The meaning and
power of smell
Section 1 Summary/Note/Table/ Matching headings
Chart-flow completion
Tasks
Matching headings
Section 2 Identifying
Summary/Note/Table/ Multiple choice
information
Chart-flow completion
Tasks
(True/False/Not given)
Section 3
Tasks
Item No.
13
Matching features
Sentence completion
13
14
Operational Procedures
Task identification and analysis
Selecting an appropriate procedure
decision on whether to use verbal protocol
analysis
Selecting and
Training subjects
motivated, skilled at verbalization, different
proficiency levels, pilot, rehearsal
Collecting verbal reports
and supplementary data
simultaneous introspection, immediate
retrospection; subject performance first,
thinking aloud second; researcher out of
subjects’ view; retrospective interview; first
language; clear tape or video recording
Transcribing verbal reports
word by word, time markers, double check
Developing an encoding scheme
Segmenting and encoding protocols
Calculating encoder reliability
units division, intra- or/and inter- reliability
Analyzing data
descriptive statistics, Chi-square Tests
Inter-rater Coding Agreement of this study is 0.78.
Background Information of the Participants
Participants Gender Age
Major
Education
background
IELTS
score
SL1
Male
24
Mechanics
PhD candidate
6.0
SL2
Male
26
Power engineering
MS candidate
5.5
SL3
Female
22
Measurement and
control
MS candidate
5.0
SH4
Male
25
Electrical engineer
PhD candidate
8.0
SH5
Female
23
Chinese linguistics
and literature
MA candidate
7.5
SH6
Female
24
Law
MA candidate
7.0
SL: Subject of Low-level; SH: Subject of High-level
Abbott’s Reading Strategy Classification (2010)
Bottom-up strategies
Definition
Scanning for explicit information
requested in the item
Identifying a synonym or a paraphrase
of the literal meaning of a word, phrase,
or sentence
Relating visual or verbal information to
accompanying visuals
Matching key vocabulary in the item to
key vocabulary in the text
Breaking words into smaller units to promote
comprehension
Scanning the text for specific details or explicit
stated information requested in the item
Identifying or formulating a synonym or a
paraphrase of the literal meaning of a word, phrase,
or sentence in the text to help answer the question
Matching visual information in the item to answer
the question
Matching visual information in the item to the words
in the text to answer the question
B6
Using knowledge of grammar or
punctuation
Using awareness of grammar, syntax, parts of
speech, or punctuation to help answer the question
B7
Using local context cues to interpret a
word or phrase
Using the words in a sentence that precede or follow
a specific word or phrase to understand a particular
word or phrase
B1
B2
B3
B4
B5
Breaking lexical items into parts
Abbott’s Reading Strategy Classification (2010)
Top-down strategies
Definition
T1
Skimming for gist or identifying the main Drawing on the major points of the passage to answer
idea, theme, or concept
the question; summarizing main concept
T2
Connecting or relating information
presented in different sentences or parts
of the text
Relating new information to previously stated
information to help answer the question; synthesizing
scattered information
T3
Drawing an inference based on
information presented in the text
Making an inference, drawing a conclusion, or
forming a hypothesis based on information not
explicitly stated in the text to answer the question
T4
Speculating beyond the text
Using background information to speculate beyond
the text
Recognizing discourse format
Using discourse format or text organization to answer
the question (eg., discriminates between fact and
opinion or cause and effect; or notes how the
information is presented)
T5
Coding Schema of Bottom-up Strategy Use
Bottom-up strategies
Examples
B1
Breaking lexical items into parts
/
B2
Scanning for explicit information requested
in the item
这有一段话，让选词，weather during …in the distant are 什
么，应该是 tree rings and ice cores, tree rings, ice cores,这部分
在paragraph 3里面找的到，Records … tree rings and ice
cores（第三段的原文）。(Participant 1)
B3
Identifying a synonym or a paraphrase of the A段 的 标 题， 我觉 得它大 多数 都在 讲 emo t io n… 有一 个
literal meaning of a word, phrase, or
feelings很像，应该算是这个emotion的近义词吧，所以选…
sentence
。(Participant 5)
B4
Relating visual or verbal information to
accompanying visuals
先看这个图…这有两个轮子，刚好第二段讲到这个问题，估
计就应该填roller，因为文中提到两个two hot rollers。
(Participant 6)
B5
Matching key vocabulary in the item to key
vocabulary in the text
重新再看一下这个文章，找这几个关键词，第一个是
d is c o v e r a ll t h e la n d s …那 就逐 行看 ，找关 键 词 吧。
(Participant 1)
B6
Using knowledge of grammar or punctuation
however, 它这个呢就是一个转折，前面是优点，后面肯定就
是讲的一个缺点…所以就可以直接做题了。(Participant 1)
B7
Using local context cues to interpret a word
or phrase
/
Coding Schema of Top-down Strategy Use
Top-down strategies
Examples
（读文章）Global temperatures …这一段主要讲的是人类
对温度 的这 样一 个影 响 ， 恩， 所以 就是 选这 个四 ，
human impact on the climate. (Participant 3)
T1
Skimming for gist or identifying
the main idea, theme, or concept
T2
Connecting or relating information
presented in different sentences or /
parts of the text
T3
Drawing an inference based on
information presented in the text
12题，the process… improved,有没有说提到improved 呢?
就 是 说 ， （ 在 文 中 找 相 应 信 息 ） they finally
succeeded…15 years,就是说之前的时候它要停，而且要
校正，需要花费四个月，所以这个时候呢我们推断一下
，应该是improved. (Participant 4)
T4
Speculating beyond the text
9到13是一个判断问题true or false的问题，按照雅思的一
个规则，那它就不会在第一部分出现。(Participant 3)
Recognizing discourse format
前面讲到了优点，再看，后边应该就是缺点了吧，（继
续看文章），glass was marked, 恩，第五个空应该就是填
marked. (Participant 6)
T5
Frequency of Participants’ Bottom-up Strategy Use
Subject
B1
B2
B3
B4
B5
B6
B7
SL1
0
18
4
1
10
0
0
SL2
0
20
1
1
7
0
0
SL3
0
24
3
0
6
1
0
SH4
0
18
5
1
10
1
0
SH5
0
16
2
1
10
4
0
SH6
0
17
5
1
7
1
0
Frequency of Participants’ Bottom-up Strategy Use
B2: Scanning for explicit information
requested in the item
Frequency of Participants’ Top-down Strategy Use
Subject
T1
T2
T3
T4
T5
SL1
10
0
1
0
0
SL2
5
0
5
0
0
SL3
9
0
4
0
2
SH4
15
0
11
2
3
SH5
10
0
12
0
5
SH6
20
0
5
1
2
Frequency of Participants’ Top-down Strategy Use
T1: Skimming for gist or identifying the
main idea, theme, or concept
T3: Drawing an inference based on
information presented in the text
T5: Recognizing discourse format
Frequency of All Participants’ Strategy Use
Chi-square tests: Sig. .001
Frequency of Participants’ Bottom-up Strategy Use
in Each Task Type
Task
Strategy
B1
B2
B3
B4
B5
B6
B7
0
18
4
5
14
3
0
0
18
0
0
1
1
0
0
9
4
0
5
2
0
0
16
1
0
12
0
0
0
11
2
0
12
0
0
Task 5
Summary/Note/Table/Flow-chart
completion
Identifying information
(True/False/Not Given)
Matching headings
Summary/Note/Table/Flow-chart
completion
Matching features
Task 6
Matching headings
0
10
6
0
4
0
0
Task 7
Multiple choice
0
14
1
0
0
0
0
Task 8
Sentence completion
0
17
2
0
2
1
0
Task 1
Task 2
Task 3
Task 4
Frequency of Participants’ Top-down Strategy Use
in Each Task Type
Task
Strategy
T1
T2
T3
T4
T5
5
0
8
0
8
9
0
10
1
0
16
0
9
0
0
1
0
1
0
1
Task 5
Summary/Note/Table/Flow-chart
completion
Identifying information
(True/False/Not Given)
Matching headings
Summary/Note/Table/Flow-chart
completion
Matching features
2
0
5
2
2
Task 6
Matching headings
24
0
2
0
1
Task 7
Multiple choice
11
0
3
0
0
Task 8
Sentence completion
1
0
0
0
0
Task 1
Task 2
Task 3
Task 4
Bottom-up strategies more frequently used in Tasks 1, 4, 5 & 8
50
45
40
35
30
25
20
15
10
5
0
Summary/Note/Table/Flow-chart completion
Matching
features
Task 1
Task 2
Task 3
Task 4
Bottom-up Strategies
Task 5
Sentence
completion
Task 6
Task 7
Top-down Strategies
Task 8
Top-down strategies more frequently used in Tasks 3 & 6
50
45
40
35
30
25
20
15
10
5
0
Matching headings
Task 1
Task 2
Task 3
Task 4
Bottom-up Strategies
Task 5
Task 6
Task 7
Top-down Strategies
Task 8
Bottom-up and top-down strategies almost equally used in Tasks 2 & 7
50
45
40
35
30
25
20
15
10
5
0
Identifying information
(True/False/Not Given)
Multiple
choice
Task 1
Task 2
Task 3
Task 4
Bottom-up Strategies
Task 5
Task 6
Task 7
Top-down Strategies
Task 8
Conclusions
Q1: What types of strategies do test-takers employ when taking
IELTS academic reading test?
Test-takers’ cognitive processes of taking IELTS academic
reading test are very complex. They employ both bottom-up
strategies and top-down strategies when taking the test.
However, some strategies (B2, B5, T1 & T3) are much more
frequently used while some strategies (B1, B7 & T2) are not
used at all.
Conclusions
Q2: Are there any similarities and differences in the
employment of strategies between high-level group and
low-level group of test-takers?
Both high-level and low-level groups of test-takers tend to use
bottom-up strategies more frequently and there is no
significant difference in the frequency of their bottom-up
strategy use. But high-level group use more top-down
strategies than low-level group with significant difference.
Conclusions
Q3: Are there any similarities and differences about test-takers’
employment of strategies among different task types?
In different tasks, test-takers’ strategy use is different. Bottomup strategies are more frequently used in three task types:
Summary/Note/Table/Flow-chart completion, Matching features,
Sentence completion; Top-down strategies are more frequently
used in Matching features; Bottom-up and Top-down strategies
are almost equally used in tasks of Identifying information
(True/False/Not Given) and Multiple choice.
Implications
 A wider range of strategies need to be covered.
 The proportion of the tasks which measure test-takers’ top-down
strategies needs to be increased as they can differentiate high-level
group from low–level group.
 More training and practice is needed for test-takers to improve
their top-down strategy use.
Significance
 Theoretically, provided evidence to confirm the rationality of the
construct framework.
 Practically, put forward suggestions for language testers on test
development and test evaluation, and provided implications for
language teachers and learners on how to improve strategy use.
 Methodologically, proved verbal protocol analysis as an effective
approach to examine strategy use employed by test-takers.
Limitations
 Some subjects may be unrepresentative due to convenience
sampling.
 The validity and reliability of the coding schemes need to be
improved.
 The theoretical framework used for the analysis is inadequate,
e.g. test-wiseness strategies are not included in it.
 The generalizability of the research findings remains to be
further testified by triangulation methodology.
References and Further Readings
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
Abbott, M. L. (2010). An introspective study of Arabic and Mandarin speaker reading comprehension strategies. TESL Canada
Journal, 28.
Anderson, N. C. (1991). Individual differences in strategy use in second language reading and testing. Modern Language Journal,
75(4): 460-472.
Alderson, J. C. (2000). Assessing Reading. Cambridge University Press, Cambridge.
Alderson, J. C., Clapham, C. & Wall, D. (2000). Language Test Construction and Evaluation. Beijing: Foreign Language Teaching
and Research Press.
Barkaoui, K. (2011). Think-aloud protocols in research on essay rating: An empirical study of their veridicality and reactivity.
Language Testing, 28(1), 51-75.
Carver, R. (1992). Reading Rate: theory, research and practical implications. Journal of Reading, 36(2), 84-95.
Cohen, A. (1984). On taking language tests: What the students report, Language Testing, 1, 70-81.
Cumming, A. (1990). Metalinguistic and ideational thinking in second language composing. Written Communication 7, 482-511.
Enright, M., Grabe, W., Koda, K., Mosenthal, P., Mulcany-Ernt, P. & Schedl, M. (2000). TOEFL 2000 reading framework: A
working paper. TOEFL Monograph Series 17 ETS, Princeton.
Ericsson, K. A. & Simon, H. (1993). Protocol Analysis: Verbal Reports as Data. Cambridge, Massachusetts: The MIT Press.
Faerch, C. & Kasper, G. (1987). Introspection in second language research. Clevedon, Philadelphia: Multilingual Matters.
Green, A. (1998). Verbal protocol analysis in language testing research: A handbook. Cambridge: Cambridge University Press.
Goodman, K. S. (1973). Psycholinguistic universals of the reading process, In E Smith (ed.), Psycholinguistics and Reading. New
York: Holt Rinehart and Winston.
Grabe, W. & Stoller, F. L. (2002). Teaching and Researching Reading. Harlow, England: Pearson Education.
Glaser, B. G. (1978). Theoretical sensitivity: Advances in the methodology of grounded theory. Mill Valley, CA: Sociology Press.
Glaser, B. G. (1992). Basics of grounded theory analysis: Emergence vs. forcing. Mill Valley, CA: Sociology Press.
Grellet, F. (1981). Developing Reading Skills. Cambridge: CUP.
Hamp-Lyons, L. (1990). Second Language writing: assessment issues. In Kroll, B. (Eds.), Second Language Writing: Research
Insights for the Classroom. Cambridge University Press, Cambridge, 69-87.
References and Further Readings
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
Hammadou, J. (1991). Interrelationships among prior knowledge, inference, and language proficiency in foreign language reading.
Modern Language Journal, 75, 27-38.
Hawkey, R. (2006). Impact theory and practice: Studies of the IELTS test and Progetto Lingue 2000. Cambridge University Press,
Cambridge.
Hill, C., & Parry, K. (1992). The test at the gate: Models of literacy in reading assessment. TESOL Quarterly, 26, 433–461.
Hudson, T. & Park, S. (2002). Validity issues for selected versus constructed response Internet-based language test. Paper
presented at AAAL, Arlington, Virginia.
IELTS (1996). The IELTS handbook. Cambridge: UCLES. The British Council, IDP Education Australia.
IELTS (2007). IELTS official practice materials. Cambridge University Press, Cambridge.
IELTS Handbook. (1999). Cambridge: University of Cambridge Local Examinations Syndicate.
IELTS Handbook. (2007). Cambridge: University of Cambridge Local Examinations Syndicate.
Irwin, J. W. (1986). Teaching Reading Comprehension Process. New Jersey: Prentice Hall Inc.
Khalifa, H. & Weir, C. (2009). Examining reading: Research and practice in assessing second language reading. Studies in
Language Testing 29. Cambridge: UCLES & Cambridge University Press.
Lumley, T. (2002). Assessment criteria in a large-scale writing test: What do they really mean to the raters? Language Testing, 19,
246-276.
McDonough, S. (1995), Strategy and Skill in Learning a Foreign Language, Arnold.
McGinnis, D. J. & Smith, D. E. (1982). Analyzing and Treating Reading Problems. New York: Macmillan Publishing Co., Inc.
McNamara, T. F. (1999). Computer-adaptive testing: A view from outside. In Chaloub-Deville M. (Eds.), Issues in computeradaptive testing of reading proficiency. Cambridge University Press, Cambridge, 136-149.
Moore, T. & Morton, J. (1999). Authenticity in the IELTS academic module writing test: a comparative study of Task 2 items and
university assignments in Tulloh, R. (Eds.), IELTS research reports 2. IELTS Australia, Canberra, 64-106.
Munby, J. L. (1978). Communicative Syllabus Design. Cambridge: CUP.
Nevo, N. (1989). Test-taking strategies on a multiple-choice test of reading comprehension. Language Testing, 6, 199-215.
Nunan, D. (1995). Teaching methodology: a textbook for teachers. Phoenix ELT.
References and Further Readings
37.
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
48.
49.
50.
51.
52.
Oxford, R. (1990). Language learning strategies. New York: Newbury House.
Orr, M. (2002). The FCE speaking test: Using rater reports to help interpret test scores. System, 30, 143-154.
Parry, K. (1996). Culture, literacy and L2 reading. TESOL Quarterly, 30, 665-692.
Phakiti, A. (2003). A closer look at the relationship of cognitive and metacognitive strategy use to EFL reading achievement test
performance. Language Testing,1, 26-56.
Purpura, J. (1997). An analysis of the relationships between test takers’ cognitive and metacognitive strategy use and second
language test performance. Language Learning, 47, 289–325.
Rennie, D. (1984). Clients’ tape-assisted recall of psychotherapy: A qualitative analysis. Paper presented at the annual meeting of
the Canadian Psychological Association, Ottawa.
Richards, J. C., P late, J. & Platte, H. (2000). Longman Dictionary of Language Teaching and Applied Linguistics. Beijing: Foreign
Language Teaching and Research Press.
Rumelhart, D. E. (1980). Schemata:The building blocks of cognition. In Spiro, R. J., Bruce, B. C. & Brewer, W. E. (Eds.),
Theoretical Issues in Reading Comprehension. Hillsdale, N. J. Lawrence Erlbaum.
Rupp, A. A., Ferne, T. & Choi, H. (2006). How assessing reading comprehension with multiple-choice questions shapes the
construct: a cognitive processing perspective. Language Testing, 4, 441-474.
Schueller, J. (2004). Gender and foreign language reading comprehension: The effects of strategy training. Southern Journal of
Linguistics, 27(1), 45–65.
Shi, C. Y. (2010). A Retrospective Study on Test-takers’ Processes of Taking Compound Dictation Test. Master of Arts dissertation,
Chongqing University.
Taylor, G. (2009). A student’s writing guide, Cambridge University Press, Cambridge.
Urquhart, A. & Weir, C. J. (1998). Reading in a second language: process, product and practice. Longman, London.
Vaughan, C. (1991). Holistic assessment: what goes on in the rater's mind? In Hamp-Lyons, L. (Eds.), Assessing Second Language
Writing in Academic Contexts, 111-25. Norwood, NJ: Ablex.
Wang, H. Z. (2009). Introspection in L2 speaking performance assessment research. 2009 (1):12-22/127 CELEA Journal
(Bimonthly).
Weir, C. J. (1983). Identifying the language needs of overseas students in tertiary education in the United Kingdom, PhD thesis,
Institute of Education, University of London.
References and Further Readings
53.
54.
55.
56.
57.
58.
59.
60.
61.
62.
63.
64.
65.
66.
67.
68.
69.
Weir, C. J., Yang, H. & Jin, Y. (2000). An empirical investigation of the componentiality of L2 reading in English for Academic
Purposes, Studies in Language Testing, 12, UCLES Cambridge University Press, Cambridge.
Weir, C. J., Hawkey, R., Green, A., Devi, S. & Unaldi, A. (2009). The relationship between the academic reading construct as
measured by IELTS and the reading experiences of students in the first year of their courses at a British university, IELTS
Research Project Report.
Weir, C.J., Hawkey, R., Green, A. & Devi, S. (2009). The cognitive processes underlying the academic reading construct as
measured by IELTS, in Thompson P. (Eds.), Research Reports Volume 9, British Council/IDP Australia, London.
Widdowson, H. G. (1978). Teaching Language as Communication. Oxford: OUP.
Wu, Y. A. (1998). What do tests of listening comprehension test?-A retrospection study of EFL test-takers performing a multiplechoice task. Language Testing 15 (1), 21-44.
Xiang dong, Gu & Xiao ying, Gao. 2008. An introspective study on test-taking process for banked Cloze. CELEA Journal, 31(4),
3-16.
Yamashita, J. (2003). Processes of taking a gap-filling test: Comparison of skilled and less skilled EFL readers. Language Testing,
20, 267-293.
Yang, H. & Weir, C. (1998). Validation study of the national college English test. Shanghai: Shanghai Foreign Language Education
Press.
Zhang, Y. (2006). Validation Study of Translation Items in National Entrance Test of English for MA/MS Candidates. Master of
Arts dissertation, Wuhan University of Technology.
陈晓扣, 李绍山. 2006. TEM-4 完型填空测试结构 效度研究——答题过程分析法 [J]. 现代外语(1): 71-77.
金艳,吴江. 1998.以“内省法”检验CET阅读理解测试的效度[J].外语界(2).
郭纯洁（2007）．《有声思维法》．北京：外语教学与研究出版社．
刘建达，杨满珍（2001）．成段改错试题考了什么？．《现代外语》．第2期：170-180．
苗菊（2005）．有声思维——翻译内在过程探索[J]，《外语与外语教学》．第6期：43-46．
文军（2005）．翻译课程模式研究——以发展翻译能力为中心的方法．北京：中国文史出版社．
文军，孙三军（2006）．论使用出声思维研究翻译过程．《外语学刊》．第3期：93-97．
郑宇静，辜向东（2009）．长对话多项选择听力测试题结构效度的追述法研究(英文).《中国英语教学》. 第6期：15-26．
Acknowledgements
 Organizers
 My MA team
 My family
Thank you very much
for your attention!
Xiangdong Gu, [email protected], 13983732737
Qin Lv, [email protected], 18696567794
Lu Zhao, [email protected], 15826189568