International comparison of electronic reading literacy:
Uncovering Macao’s 15-year-olds’ unexpected strengths and weaknesses
using country differential item functioning
Pou-seong Sit
Faculty of Education, University of Macau, Macao, CHINA
[email protected]
Kwok-cheung Cheung
Faculty of Education, University of Macau, Macao, CHINA
[email protected]
Abstract
Electronic reading assessment (ERA) is a new form of international assessment in the
PISA 2009 Reading Literacy Study. Using students’ online reading responses in the field
trial, the present study examined Macao 15-year-old students’ unexpected strengths and
weaknesses using country differential item functioning. Specifically, characteristics of
ERA items identified as having country by item interactions are analyzed to throw lights
on the item format (multiple choice versus open constructed response) and the reading
aspects (i.e. access and retrieve, integrate and interpret, reflect and evaluate) that the
students are good at and/or weak in. Using four of the seven released items after the
field trial by the PISA Consortium, it is possible to examine item contents so as to have
a more concrete idea of Macao students’ online Chinese reading capabilities and
limitations.
Keywords: reading literacy, electronic assessment, differential item functioning, PISA
A.
Introduction
Programme for International Student Assessment (PISA) is a three-yearly sample survey
of reading, mathematical and scientific literacy in Organization for Economic
Co-operation and Development (OECD) and partner countries. The major domain of the
PISA 2009 Main Survey is reading literacy, and electronic reading assessment (ERA) is
envisaged as an innovation aiming at finding out to what extent 15-year-olds are able to
read on-line (OECD, 1999 & 2007). In ERA, the reading tasks generally necessitate
students to identify important questions, locate information which match the access
paths of the reading tasks, analyze the usefulness of the information retrieved, integrate
information retrieved from multiple electronic texts to answer questions, and then
communicate replies through electronic means (Cheung & Sit, 2008). Admittedly, in
Macao, students are not used to this innovative kind of online reading assessment,
which is emerging in the digital age of the new century (See Katz, 2007; Rush, Eakle &
Berger, 2007; Coiro, Knobel, Lanshear & Leu, 2008; and Coiro & Dobler, 2007 for the
emerging trends and methodologies). Language teachers seldom use electronic means to
assess students’ level of reading comprehension, and therefore not documented as
common assessment practices in official curriculum documents (Shan et al., 2007).
B.
Purpose of Study
As in any international comparison of student performance, reading tasks need to be
field trialed to ascertain that the test items behave in approximately the same way across
countries. The extent to which students from different countries with the same ability
are observed to respond differently is of prime concern in the present field trial.
Psychometrically speaking, the occurrence of country differential item functioning (DIF)
is an indication that respondents with the same ability from different countries have
different probabilities of responding correctly to an item (see Penfield & Lam, 2000 for
the assessment of differential item functioning in performance assessment; see also
Camilli & Shepard, 1994 for the methods for identifying biased test items).
The purpose of this paper is to analyze items identified as having item by country
interactions (i.e. country differential item functioning) in the PISA 2009 Field Trial
(FT09) so as to uncover Macao’s 15-year-olds’ unexpected strengths and weaknesses
when reading on-line. Compared with other OECD and partner countries, there are
many reasons why Macao experiences DIF on an item. The reasons range from relative
familiarity students have with the electronic reading texts to translation nuances or even
translation errors (see Haladyna & Downing, 2004 for the issue of construct-irrelevant
variance in high-stake testing). In particular for the Chinese students in Macao, they
may be frustrated with the slow or unfamiliar Chinese text input methods available to
them. All of these factors can lead to country differential response behavior. This may
be less so for multiple choice items but more so for items requiring constructed
responses.
Because most of the field-trialed items are selected for use again in the PISA 2009 Main
Survey (MS09), it is not possible to address curriculum and pedagogical issues faced by
language teachers and students in Macao in concrete terms without disclosing contents
of the items. In spite of this limitation, it is the intention of this paper to elucidate
Macao 15-year-olds’ unexpected strengths and weaknesses by reference to the few items
and the associated coding guides released by the PISA Consortium. In PISA MS09,
students generally used their everyday language of instruction in responding to the test
items, which have been translated from the common international English version of the
assessment items. The data for the present study were drawn from the 15-year-olds
studying in the Chinese schools field-trialed in April, 2008.
C.
Methodology in the calculation of country differential item functioning
In the present study, calculation of facility of constructed responses with partial credit
scoring proceeds as follows: (1) where the maximum score is 1 the facility is simply the
percentage of students who scored 1; (2) where the maximum score is greater than 1, the
facility is the sum of the percentage in each score group times its corresponding score,
divided by the item’s maximum score. Because of the small sample size of the field trial
data (i.e. N = ~100 for each reading item in the Macao sample) preventing the use of
item response scaling procedures, the DIF index reported is calculated for each item by:
(1) computing facility (or per cent correct) for each item, by country; (2) for each
country, subtracting the average facility for all items from each individual item, to
obtain national relative item facilities; (3) computing the international relative item
facility for each item (i.e. the arithmetic mean of values obtained in step 2); and
subtracting the international relative item facility from the national relative item facility.
(see Clauser & Mazor, 1998 for statistical procedures to identify differentially
functioning test items)
D.
Characteristics of ERA items with country by item interactions identified in
Macao PISA 2009 Field Trial
As shown in Table 1, there are altogether 25 ERA items with country by item
interactions identified in Macao PISA FT09. Because of confidentiality of the contents
of the ERA items. Only items released after FT09 will have their item name listed in
this article (i.e. PHISHING and LET’S SPEAK). Item facility ranges from 0% to 81%
with E010Q02 the easiest and E010Q07T the most difficult. Based on the international
data from Austria, Belgium, Canada, Colombia, Germany, Denmark, France, Hong
Kong-China, Hungary, Iceland, Ireland, Japan, Korea, Macao-China, Norway, Poland,
Spain, and Sweden, country by item interaction is calculated using the methodology
described in the preceding section. Twelve items are found to be associated with
positive country by item interactions, indicating that Macao students with the same
ability as the mean of the OECD countries/economies have greater probabilities of
responding correctly to these items. On the other hand, thirteen items are found to be
associated with negative country by item interactions, indicating that Macao students
with the same ability as the mean of the OECD countries/economies have lesser
probabilities of responding correctly to these items. In FT09, items are of two main
types, either multiple choice or constructed response format. The reading aspects which
items load on are: (1) access and retrieve; (2) integrate and interpret; and (3) reflect and
evaluate. It is with respect to these two item characteristics (i.e. item type and reading
aspect) that Macao’s 15-year-olds’ unexpected strengths and weaknesses are analyzed in
the ensuing sections.
Table 1: Characteristics of ERA items with country by item interactions identified in
Macao PISA 2009 Field Trial
Item Number
E002Q02
E002Q04
E005Q04
E006Q05
E007Q02
E007Q03
E007Q06
E008Q02T
E008Q03T
E008Q04T
E010Q02 (PHISHING) ***
E010Q04 (PHISHING) ***
E010Q07T
E013Q01
E013Q03
E014Q05
E014Q07
E014Q10
E017Q01
E017Q03
E017Q06
E021Q06
E022Q02
E022Q08 (LET’S SPEAK)***
E022Q09 (LET’S SPEAK)***
Item
Facility
(%)
53
42
13
4
79
80
41
79
41
54
81
75
0
45
72
74
64
26
37
7
24
24
34
14
15
Country by Item
Item Type
Interaction* (+/–) (MC/ CR)**
~35% (+)
15-20% (–)
20-25% (–)
10-15% (–)
10-15% (+)
~20% (+)
25-30% (+)
15-20% (+)
15-20% (+)
25-30% (+)
15-20% (+)
15-20% (+)
~25% (–)
10-15% (–)
10-15% (+)
20-25% (+)
10-15% (+)
10-15% (–)
10-15% (–)
~20% (–)
25-30% (–)
10-15% (–)
20-25% (–)
10-15% (–)
10-15% (–)
C_CR
O_CR
O_CR
O-CR
MC
MC
MC
MC
MC
MC
MC
MC
C_ MC
MC
MC
MC
MC
MC
MC
MC
O_CR
MC
MC
MC
O_CR
Reading Aspect
Assess and retrieve
Integrate and interpret
Reflect and evaluate
Reflect and evaluate
Assess and retrieve
Integrate and interpret
Integrate and interpret
Integrate and interpret
Integrate and interpret
Reflect and evaluate
Assess and retrieve
Assess and retrieve
Integrate and interpret
Integrate and interpret
Integrate and interpret
Assess and retrieve
Integrate and interpret
Integrate and interpret
Integrate and interpret
Integrate and interpret
Reflect and evaluate
Integrate and interpret
Integrate and interpret
Integrate and interpret
Reflect and evaluate
* Based on international data from Austria, Belgium, Canada, Colombia, Germany, Denmark, France,
Hong Kong-China, Hungary, Iceland, Ireland, Japan, Korea, Macao-China, Norway, Poland, Spain,
and Sweden.
** MC refers to the common multiple choice questions and C_MC to complex multiple choice questions;
whereas O_CR to open constructed response and C_CR to closed constructed response questions
*** Because of confidentiality of the contents of the ERA items. Only items released after FT09 will have
their item names listed in this table.
In Table 2, the twenty five ERA items with identified country by item interactions in
FT09 for Macao are cross-classified according to: (1) item type; (2) reading aspect; (3)
facility; and (4) DIF direction. Out of the 25 items, only 4 are included in the MS09
such that E014Q07 (i.e. a positive DIF item with medium difficulty) is in favor of
Macao’s 15-year-olds whereas E013Q01, E017Q01 and E006Q05 (i.e. the three
negative DIF items of medium to high difficulty) are not. Clearly, reasons for DIF,
whether positive or negative, should be identified to guarantee their valid use in the
MS09.
Table 2: ERA items of varied facility levels with identified country by item interactions
cross-classified with item type and reading aspect
ERA item with identified country by item interactions*
Item Characteristics
Easy
Positive DIF
Medium
Positive DIF
Negative DIF
Difficult
Negative DIF
Multiple Choice
1. Access and retrieve
2. Integrate and interpret
3. Reflect and evaluate
E007Q02
E010Q02
(PHISHING)
E010Q04
(PHISHING)
E014Q05
E007Q03
E008Q02T
E013Q03
E007Q06
E008Q03T
E014Q07#
E013Q01#
E017Q01#
E022Q02
E010Q07T
E014Q10
E017Q03
E021Q06
E022Q08
(LET’S
SPEAK)
E008Q04T
Constructed Response
1. Access and retrieve
2. Integrate and interpret
3. Reflect and evaluate
E002Q02
E002Q04
E005Q04
E006Q05#
E017Q06
E022Q09
(LET’S
SPEAK)
* Easy items for Macao’s 15-year-olds have item facility greater than 70%; whereas difficult items
smaller than 30%
# Items identified as having country by item interactions in Macao but nevertheless included in PISA
2009 Main Survey
The first finding stemming from Table 2 is that amongst the twelve ERA items with
identified positive item-country interactions in Macao’s FT09, ten revealed that Macao’s
15-year-olds outperform their international counterparts of the same ability mainly on
multiple choice items pertaining to “access and retrieve” and “integrate and interpret”.
These items are particularly of unexpected advantage to Macao’s 15-year-old students.
The second finding stemming from Table 2 is that amongst the thirteen ERA items with
identified negative item-country interactions in Macao’s FT09, all revealed that Macao’s
15-year-olds are inferior to the international counterparts of the same ability mainly on
items pertaining to “integrate and interpret” and “reflect and evaluate”, and these items
may be multiple choice and constructed response type. These items are particularly of
unexpected disadvantage to Macao’s 15-year-old students.
E.
Unexpected strengths as revealed in ERA items with positive country by item
interactions in Macao PISA 2009 Field Trial
Table 3 summarizes the unexpected strengths as revealed in ERA items with positive
country by item interactions in Macao’s FT09. Without disclosing the contents of
individual test items not yet released after the field trial, it can be summarized that the
“access and retrieve” tasks shown in Table 3 generally require students from a variety of
data sources to retrieve specific information (e.g. a directly and explicitly stated
definition, a synonymous match of data presented in various layout formats, a URL
linked for specified webpage, an identification of relevant search results by recognizing
explicitly stated terms that are not immediately visible).
On the other hand, the “integrate and interpret” tasks require students to form a broad
understanding of texts read (e.g. recognizing a summary of main ideas in a short text
containing complex or unfamiliar information, identifying relevant search results by
recognizing the main theme of a short description). There are tasks which require
students to develop an interpretation (e.g. synthesizing information in two texts so as to
identify a point of similarity, identifying a difference in evaluation between two
websites, identifying the general trend in a bar graph). Last, the “integrate and interpret”
tasks as exemplified by E008Q04T require students to evaluate the contents of search
results in terms of relevance and credibility/trustworthiness of information.
Table 3: Unexpected strengths as revealed in ERA items with positive country by item
interactions in Macao PISA 2009 Field Trial
Item Number
Item Type
Question Intent
Access and retrieve
E007Q02
E010Q02 (PHISHING)
E010Q04 (PHISHING)
E014Q05
E002Q02
MC
MC
MC
MC
CR
Retrieve information
Retrieve information
Retrieve information
Retrieve information
Retrieve information
MC
MC
MC
MC
MC
MC
Form a broad understanding
Develop an interpretation
Develop an interpretation
Develop an interpretation
Develop an interpretation
Develop an interpretation
MC
Reflect on and evaluate the content of text
Integrate and interpret
E007Q03
E007Q06
E008Q02T
E008Q03T
E013Q03
E014Q07#
Reflect and evaluate
E008Q04T
# Items identified as having positive country by item interactions in Macao but nevertheless included in
PISA 2009 Main Survey
It is possible to disclose the contents of released items after the field trial (i.e. E010Q02
E010Q04). Using the two released items of PHISHING for illustration, Macao’s
15-year-old students are good at locating information by clicking hyperlinks, whether
on the same webpage or different webpage. It appears that phishing is a topic of
sufficient familiarity so that students feel competent and happy to do (see Appendix 1
and 2 for the test items and their question intent, access paths and scoring guide).
Lastly, there is one item (i.e. E014Q07) identified as having positive country by item
interactions in Macao but nevertheless included in MS09. This item is evaluated as still
suitable for use in MS09. Without disclosing the content of this item, it can be observed
that this is a multiple choice item requiring students to identify a general trend in a bar
chart. Macao students generally are used to this kind of prediction tasks in their
mathematics lessons.
F.
Unexpected weaknesses as revealed in ERA items with negative country by
item interactions in Macao PISA 2009 Field Trial
Table 4 summarizes the unexpected weaknesses as revealed in ERA items with negative
country by item interactions in Macao’s FT09. Without disclosing the contents of
individual test items not yet released after the field trial, it can be summarized that the
“integrate and interpret” tasks shown in Table 4 generally require students to form a
broad understanding by recognizing the functions of a website. There are tasks which
require students to develop an interpretation such as combining information across an
email exchange to identify a common criterion, recognizing which one of a set of
suggestions is made in a text, applying criteria in one text to evaluate a different text,
and comparing websites and synthesizing a common message. In addition, students are
asked to identify the topics common to a series of short texts, a simple explanation for a
problem in a complex technical description, which of a series of advertisements can
supply a particular service, and the motivation expressed in a short e-mail text.
On the other hand, the “integrate and interpret” tasks require students to evaluate web
page in terms of credibility and trustworthiness of information, and drawing on prior
knowledge and information on a web page indicate reasoning behind a requirement in a
job advertisement. Evidences are sought to support a hypothesis with data from a
character’s personal profile combined with prior knowledge about a familiar context; as
well as an opinion about the authoritativeness of a text by combining prior knowledge
with information from the text.
Table 4: Unexpected weaknesses as revealed in ERA items with negative country by
item interactions in Macao PISA 2009 Field Trial
Item Number
Item Type
Question Intent
Integrate and interpret
E010Q07T
E013Q01#
E014Q10
E017Q01#
E017Q03
E021Q06
E022Q02
E022Q08 (LET’S SPEAK)
E002Q04
MC
MC
MC
MC
MC
MC
MC
MC
CR
Develop an interpretation
Develop an interpretation
Develop an interpretation
Form a broad understanding
Develop an interpretation
Develop an interpretation
Develop an interpretation
Develop an interpretation
Develop an interpretation
CR
CR
CR
CR
Reflect on and evaluate content of text
Reflect on and evaluate content or form of text
Reflect on and evaluate the content of text
Reflect on and evaluate the content of text
Reflect and evaluate
E005Q04
E006Q05#
E017Q06
E022Q09 (LET’S SPEAK)
# Items identified as having negative country by item interactions in Macao but nevertheless included in
MS09
It is possible to disclose the contents of released items after the field trial (i.e. E022Q08
and E022Q09). Using the two released items of LET’S SPEAK for illustration, Macao’s
15-year-old students are not good at developing an interpretation, and are not used to
engage in reflection and evaluation of the texts read. E022Q08 requires the student to
recognize which one of a set of suggestions is made in a text, whereas E022Q09 asks
student to support an opinion about the authoritativeness of a text by combining prior
knowledge with information from the text (see Appendix 3 and 4 for the test items and
their question intent, access paths and scoring guide).
Lastly, there are three items (i.e. E013Q01, E017Q01, E006Q05) identified as having
negative country by item interactions in Macao but nevertheless included in MS09.
These items are evaluated as still suitable for use in ERA assessment, on conditions that
translation nuances or translation errors are rectified. Without disclosing the contents of
the three items, it can be observed that there are wording problems in the two questions
(i.e. E017Q01, E006Q05) preventing students to have a correct and comprehensive
understanding for integrative thinking, reflection and evaluation. On the other hand,
E013Q01 is an item requiring students to combine information across an email
exchange so as to identify a common criterion for decision making. Content analyses
revealed that Macao students may not good at this kind of problem-based logical
thinking. Interestingly, this phenomenon is found also appearing in another
Chinese-speaking economy. As this item is kept unchanged for the MS09, Macau-PISA
Centre will watch out if this finding of unexpected weakness remains the same as that in
the field trial.
G.
Limitations of the present study
There are altogether two limitations. First, unless released after the field trial, it is
forbidden to disclose contents of the assessment items. Discussion of findings can only
be based on those characteristics pertaining to the general assessment framework, such
as item type, reading aspects and question intent. Second, unless authorized by other
countries, it is forbidden to reveal assessment results of other countries. In the light of
these two limitations, it is a pity that the items revealed in the present study to have
positive and negative country by item interactions cannot be compared with those of
other countries/economies with educational contexts similar to Macao, e.g. Hong Kong.
For a large number of items with identified country by item interactions for Macao, it is
not possible to disclose what we have discovered because these items are either yet to
be released for public viewing or remained to be kept for use in MS09.
H.
Conclusion
Using the twenty five items identified as having country by item interactions in the field
trial of the ERA assessment in the PISA reading literacy survey, it is possible to uncover
unexpected strengths and weaknesses of Macao’s 15-year-olds in Macao. The present
study capitalizes on the dodge items so as to uncover unexpected strengths and
weaknesses. Because all these dodge items will have been excluded so as to achieve fair
and valid international comparison, the present research design is not feasible in the
main survey again.
Based on this special research methodology, the present study discovered that Macao
15-year-olds outperform their international counterparts of the same ability mainly on
multiple choice items pertaining to “access and retrieve”, but are inferior to their
counterparts mainly on open-constructed items pertaining to “reflect and evaluate”. This
finding reveals that students are more used to answering multiple choice items by
clicking answer boxes than typing in constructed responses using Chinese input
methods. They are less inclined to secure full credits for constructed responses
demanding reflection and evaluation of materials read.
It is hypothesized that Macao’s
everyday testing and examination practices emphasizing correct understanding of
reading materials that are seldom questionable or debatable result in this kind of
weakness. It is envisaged that the scenario will be changed in the positive direction
when items with country by item interactions are eliminated in the main survey and that
students become more familiar with ERA assessment format.
Last, student performance on “integrate and interpret” can be good or bad, and items are
mainly of the multiple choice item format. Analyses of item contents with reference to
Macao schooling contexts are needed in order to understand strengths and weaknesses
on this reading aspect.
References
Camilli, G., & Shepard, L. (1994). Methods for identifying biased test items (Vol. 4).
London: Sage.
Cheung, K.C. & Sit, P.S. (2008). Electronic reading assessment: The PISA approach for
the international comparison of reading comprehension. Journal of Educational
Research and Development, 4(4), 19-40.
Clauser, B.E., & Mazor, K.M. (1998). Using statistical procedures to identify
differentially functioning test items. Educational Measurement: Issues and
Practice, 17(1), 31-44.
Coiro, J. & Dobler, E. (2007). Exploring the online reading comprehension strategies
used by sixth-grade skilled readers to search for and locate information on the
Internet, Reading Research Quarterly, 42, 2, 214-257.
Coiro, J., Knobel, M., Lankshear, C. & Leu, D. (2008). The handbook of research on
new literacies. UK: Routledge.
Haladyna, T.M., & Downing, S.M. (2004). Construct-irrelevant variance in high-stakes
testing, Educational Measurement: Issues and Practice, 23(1), 17-27.
Katz, I.R. (2007). Testing information literacy in digital environments: ETS’s iSkills
assessment, Information Technology and Libraries, 26, 3, 3-12.
OECD (1999). Measuring student knowledge and skills: A new framework for
assessment. Paris: Author.
OECD (2007). PISA 2009 reading framework – 23rd meeting of the PISA Governing
Board, Oslo, 12-14 March, 2007. Paris: Author.
Penfield, R.D., & Lam, T.C.M. (2000). Assessing differential item functioning in
performance assessment, Educational Measurement: Issues and Practice, 19(3),
5-15.
Rush, L.S., Eakle, A.J., & Berger, A. (2007) (Eds.). Secondary school literacy: What
research reveals for classroom practice. Illinois, US: National Council of Teachers
of English.
Shan, W. J., Ieong, S. L., Cheung, K. C., Zhang, C. L., Tian, Y., Wang. P. C., Vong,
S.K., Tse, K. S., Leung, S. O., & Sze, T. M. (2007). An Investigatory Study of the
Non-High Education Curriculum and its Direction for Reform in Macao: Final
Report. Macao: DSEJ.
Appendix1: Access structure pertinent to answering E010Q02 (PHISHING)
PHISHING: Task 1
E010Q02
You are at the home page of the Online Phishing Resource Site. According to the
information on this page, which one of the following is a feature of a phishing e-mail?
A
B
C
D
It asks for personal information.
It contains unwanted advertising.
It offers a genuine service.
It comes from a well-known company.
Question intent:
Access and retrieve: Retrieve information
Locate an important component of an explicitly stated definition
Access paths:
Based on the information shown on Online Phishing Resource Site to retrieve the
correct answer to the multiple choice question. (n.b. Blue dotted arrow shows
where the answer is located; there is no need to click to another webpage)
Scoring guide:
Full Credit
Code 1: A. It asks for personal information.
No Credit
Code 0: Other responses.
Code 9: Missing.
Appendix 2: Access structure pertinent to answering E010Q01 (PHISHING)
PHISHING: Task 2
E010Q01
How many phishing e-mails are sent around the world in an average month?
A
B
C
D
1,200.
over 6 billion.
about 25,000.
55,000.
Question intent:
Access and retrieve: Retrieve information
Identify the reference of a number in a list
Access paths:
Based on the information shown on Online Phishing Resource Site to retrieve
the correct answer to the multiple choice question. (Blue dotted arrow shows
where the answer is located; there is no need to click to another webpage)
Scoring guide:
Full Credit
Code 1: B. over 6 billion.
No Credit
Code 0: Other responses.
Code 9: Missing.
Appendix 3: Access structure pertinent to answering E022Q08 (LET’S SPEAK)
LET’S SPEAK: Task 3
E022Q08
Find the article by Doctor Nauckunaite. Which one of the following suggestions does
Doctor Nauckunaite make?
A
B
C
D
E
A casual and relaxed attitude is most effective when you give a speech.
If you think about your audience, you will worry less about yourself.
If you can hide the fact that you are afraid, you will feel less afraid.
It is safest to memorise your whole speech before you start.
It is best to look at different sections of the audience in turn during your speech.
Question intent:
Integrate and interpret: Develop an interpretation
Recognize which one of a set of suggestions is made in a text
Access paths:
Click the hyperlink in Education Network Forums (blue dotted arrow shows
where the hyperlink is located) to go to required webpage (shown by the red
small dotted arrow). Based on the information displayed to integrate a set of
suggestions made by Doctor Nauckunaite so as to answer the multiple choice
question.
Scoring guide:
Full Credit
C. If you can hide the fact that you are afraid, you will feel less afraid.
No Credit
Code 0: Other responses.
Code 9: Missing.
Appendix 4: Access structure pertinent to answering E022Q09 (LET’S SPEAK)
4
1
2
LET’S SPEAK: Task 4
3
E022Q09
Look at Mischa's post for March 10. Click on “Write a Reply” and write a reply to
Mischa. In your reply, answer her question about which writer, in your opinion,
knows the most about this issue. Give a reason for your answer. [Note: use the Back
button to refer to the Forum page.]
Click “Post Reply” to add your reply to the forum.
Question intent:
Reflect and evaluate: Reflect on and evaluate the content of a text
Support an opinion about the authoritativeness of a text by combining prior
knowledge with information from the text
Access paths:
Click on “Write a Reply” button either at the top or at the bottom on the
Educaton Network Forum webpage (i.e. step 1) and write by typing a reply to
Mischa (i.e. step 2). Then click on “Post Reply” to post the answer (i.e. step 3).
The answer can be edited by clicking “Edit Reply” button at the top of the
webpage (i.e step 4). (n.b. Blue dotted arrow shows where to start doing the
tasks, and red dotted arrows in sequence indicate the planned access structure
of the linked webpage)
Scoring guide:
Full Credit
Code 1:
Identifies Doctor Nauckunaite and/or Psychologist O.L. (explicitly or implicitly)
AND refers to their professional status. May express scepticism about their
professional status.
• The two professionals are the ones knowing the most, but only Dr. N gives advice on
how to work with the problem.
• Psychologist O.L. or Dr.Nauckunaite because they are both trained in the area.
• Doctor Nauckunaite. This is the only one that has the support of a university behind it.
• Dr Nauckunaite, because she’s from a university.
• A university professor has the most practical experience in talking in public.
• Mark has looked into it and found an article written by a person who knows what to do.
This man is obviously a professional on the matter, so I think you should follow his
advice.
• I’d take most notice of the one who wrote the book, because she has published a book on
this subject.
• Psychologist O.L. sounds authoritative, but of course you can’t really know that she is a
psychologist.
• You should follow Psychologist O.L's advice, as not only he is an experienced
psychologist but answers all the questions concerning public speaking accurately, and is
very believable.
• Psychologist O.L. because he’s a trained psychologist.
• The person that probably knows most about this is the Doctor. He has had most
experience (or at least more than Julie or Tobias) and I think he is therefore more
trustworthy.
Identifies any of the four writers named by Mischa (Julie, Tobias, Psych OL or Dr.
Nauckunaite) AND gives a reason that is consistent with the text, related to the
cogency, practicality or logic of the text.
• Psychologist O.L. because what he says makes sense in terms of the way you see small
children and teenagers behave.
• Tobias because he’s actually done it.
• Doctor Nauckunaite because she has set out her ideas in a practical way.
• I think Tobias has the greatest idea of what he's talking about. He gives you concrete
ways in which to improve your public speaking, and if you follow what he says I'm sure
you'll do fine! :)
• Probably go with Mark's link, it has the most useful hint about how to overcome fear of
public speaking. [“Mark’s link” implies Dr N.]
• I think that Tobias is right. It does help to rehearse and know your topic well. I also
agree with Julie to an extent, because some people are more outgoing than others. But
with preparation and a good attitude you can make a good speech. Avoiding it altogether
is not a solution!!
• Julie’s ideas describe the way people differ, so she is the one I’d believe.
No Credit
Code 0:
Names any of the writers without explanation.
• Psychologist O.L.
• Doctor Nauckunaite.
Gives insufficient or vague answer.
• Tobias because I agree with him.
• Doctor Nauckunaite because she is the best.
• Tobias because his ideas make sense.
Shows inaccurate comprehension of the material or gives an implausible or
irrelevant answer.
• Psychologist O.L. because he’s my favourite.
• Tobias because he tells you how to avoid public speaking. [inaccurate comprehension]
• I’d go for Mark. [not one of the four writers named by Mischa]
Code 9: Missing.