Adv Physiol Educ 41: 130–136, 2017;
doi:10.1152/advan.00174.2016.
HOW WE TEACH
Generalizable Education Research
Stop Think: a simple approach to encourage the self-assessment of learning
X Richard Guy, Bruce Byrne, and Marian Dobos
School of Health and Biomedical Sciences, RMIT University, Bundoora, Victoria, Australia
Submitted 28 October 2016; accepted in final form 23 December 2016
self-assessment; difficulty assessment; concept map; self-regulated
learning
that students are expected to “learn” but are
not taught how to learn. For example, students surveyed about
their learning strategies and problem solving were generally
unable to identify having received formal instruction in these
areas (27) and were often unaware or unable to identify what
strategies they used and whether or not those strategies that
they could identify were effective (31). Students asked to
identify their learning strategies commonly report rereading
material or highlighted comments rather than using other,
potentially more useful, approaches such as self-testing (21,
24). From these findings, one can infer that students often use
learning approaches they “picked up along the way” but are
unclear what these approaches are and whether or not they
work. This suggests that students may be lacking in their
ability to self-assess their learning (cognitive) approaches, and
thus this aspect of metacognition (17, 35) should be taken into
IT IS OFTEN THE CASE
Address for reprint requests and other correspondence: R. Guy, School of
Health and Biomedical Sciences, RMIT University, PO Box 71, Bundoora,
VIC 3083, Australia (e-mail: [email protected]).
130
account when designing learning activities. Cognition, metacognition, and motivation interact in a way that is referred to as
self-regulation (7, 46), and in the present study, we attempted
to encourage the self-assessment aspect of self-regulated learning (12, 14) using a “stop think” approach, requiring students
to rate their feeling of difficulty (FOD) before [FOD(pre)] and
after [FOD(post)] concept map exercises. The FOD provides a
rapid, abstract, metacognitive measure of perceived difficulty
(16) and can thus be regarded as a self-assessment tool. The
“stop think” approach used was based on the task evaluation
and reflection instrument developed by Belski and Belski (4);
however, it focused on individual appraisal of task difficulty
without reference to peer self-assessments (a feature of the
original instrument). The concept maps were chosen as an
active learning task because they provide an effective graphic
representation of information and also encourage students to
make associations between different facts and concepts (3, 20),
thus constituting a “deep” learning situation. Deep learning is
said to be characterized by intrinsic motivation, interaction
with and interest in the learning material (9), and an intention
to understand content (2). Since self-regulation is related to
deep learning (23, 29, 44), we used a “learning approach”
questionnaire to encourage students to think about their individual strategic and motivational deep learning and also to
investigate links between the concept map intervention and
deep learning per se.
Data for the present study were derived from a first-year,
first-semester nursing cohort studying human anatomy and
physiology. Students were allocated to two groups based on
their academic ability for the specific subject (overall mark).
This was done to allow an assessment of the effectiveness of
the interventions with respect to academic ability.
METHODS
Subject characteristics. The cohort consisted of first-year, firstsemester nursing students taking a one-semester Anatomy and Physiology subject. Although 128 students agreed to participate in the
study, only 85 students (42.5% of the total enrollment) attempted the
concept maps and provided the data for analysis. There were 75
female students and 10 male students with an average age of 20.5 yr
(SD: 4.25 yr).
Ethics. All participating students were provided with a plain language statement and signed an informed consent form, which included
the right to withdraw at any time. The project was approved by the
relevant RMIT Human Ethics Committee and assessed as low risk.
Experimental procedures. The study commenced during week 6 of
a 12-wk semester, and the late start allowed students to adjust (to
some extent) to starting their university studies and ensured that they
had covered relevant material before undertaking the concept map
exercises. Figure 1 shows the timetable for the various assessments
used in this study. The assessments were the same for the low-
1043-4046/17 Copyright © 2017 The American Physiological Society
Downloaded from http://advan.physiology.org/ by 10.220.32.246 on June 16, 2017
Guy R, Byrne B, Dobos M. Stop Think: a simple approach to
encourage the self-assessment of learning. Adv Physiol Educ 41:
130 –136, 2017; doi:10.1152/advan.00174.2016.—A simple “stop
think” approach was developed to encourage the self-assessment of
learning. A key element was the requirement for students to rate their
feeling of difficulty before [FOD(pre)] and after [FOD(post)] completing each of three authentic anatomy and physiology concept map
exercises. The cohort was divided into low- (group L) and highperforming (group H) groups (based on final subject marks). Both
FOD(pre) (group L) and FOD(post) (groups L and H) were significantly
negatively correlated with score for some maps. A comparison of
FOD(pre) and FOD(post) showed that students changed their mind
about difficulty in 58 –70% of the completed maps. Students who
changed their estimation were asked to provide explanatory comments, and an increase in difficulty was related to problems with map
link generation. For students who found the maps easier, 40% of
comments indicated that map generation prompted recall of information from memory. Both difficulty estimations and comments supported the contention that students were self-assessing their interaction with the concept maps. Group H was significantly older than
group L, had significantly higher levels of deep strategic and deep
motivational learning, and had significantly higher marks in two of
three concept map exercises. Notwithstanding these differences, the
results from the “stop think” approach were similar between groups,
indicating that it may be appropriate for students of varying academic
ability. It is suggested that “stop think” may be a useful approach to
encourage student self-assessment, an important step in assisting
self-regulated learning development.
ENCOURAGING SELF-ASSESSMENT
131
Fig. 1. Flow diagram outlining both the assessments carried out and the timetable of
these assessments during the semester. All
assessments were carried out during class
time. SPQ, Biggs study process questionnaire. FOD(pre) and FOD(post), feelings of
difficulty before and after the exercise, respectively.
cular, urinary, and nervous systems). The final concept map exercise
(map C) was based on someone picking up a heavy animal, which
damaged their back and also bit them (integrating skeletal, muscular,
cardiovascular, immune, integumentary, and nervous systems).
FOD measurement. The FOD measurement is related to the task
evaluation and reflection instrument for student self-assessment
(TERISSA) developed by Belski and Belski (4). TERISSA guides
students through two estimates of task complexity, with the first
estimate undertaken just before solving the task and the second
estimate immediately after completing the task. In the Belski and
Belski study, task complexity was judged on a five-point scale (where
1 ⫽ very simple, 2 ⫽ simple, 3 ⫽ so-so, 4 ⫽ difficult, and 5 ⫽ very
difficult), and students were asked to give reasons why their estimation was not one level less difficult than indicated. They were also
asked to reflect on any discrepancy between the two estimations and
to decide on any actions that would assist them when they next
encountered a similar task. The TERISSA approach was used in a
tutorial setting where feedback on student difficulty ratings were used
as part of the discussion.
In the present study, students were initially asked to rate the
perceived difficulty of the exercise [FOD)(pre)] on a scale of 1 to 10
(where 1 ⫽ easy and 10 ⫽ difficult) after reading the concept map
scenario but before generating the concept map. After completing the
map (10 –15 min), students rated the difficulty of the exercise again]
FOD(post)] on a scale of 1 to 10 (where 1 ⫽ easy and 10 ⫽ difficult).
The 10-point scale, rather than the 5-point scale, was used after
consideration of the advantages of the subjective mental effort questionnaire used in computer science usability studies (38). The FOD
rating has been used with reference to the work of Efklides et al. (16).
Students were not asked to comment on their difficulty rating (cf.
TERISSA approach); however, in keeping with TERISSA, they were
asked to provide a written comment if there was a difference between the
FOD(pre) and FOD(post) ratings. Written comments were initially surveyed to determine response categories, and the data were then coded
appropriately. The process was then refined to generate final categories,
and two academics independently completed the coding as indicated in
the RESULTS.
Both the current approach and TERISSA approach, in addition to
encouraging student self-assessment, also generate data that are relevant to an understanding of the monitoring component of the deep
aspects of self-regulated learning.
Student feedback questionnaire. Students were asked to complete a
set of 11 survey questions at the end of the study (semester week 11)
(Fig. 3). The first five questions related to the concept map intervention and the final six questions were related to the rSPQ and the
project in general. Students were told, before survey completion, that
questions 9 and 10 referred to the rSPQ. Survey questions were scored
using a four-point Likert scale (where 1 ⫽ strongly disagree, 2 ⫽ disagree, 3 ⫽ agree, and 4 ⫽ strongly agree). The four-point Likert scale
was used as we had very limited time to conduct surveys during class
time and it was felt that this would make it easier for students to
complete. However, the choice of four items is still within the
optimum range for Likert scales (30). An analysis of group homogeneity for questions 1–11 revealed the following values for Cronbach’s
␣: group L ⫽ 0.726 and group H ⫽ 0.684. Cronbach’s ␣ provides a
Advances in Physiology Education • doi:10.1152/advan.00174.2016 • http://advan.physiology.org
Downloaded from http://advan.physiology.org/ by 10.220.32.246 on June 16, 2017
performing group (group L) and the high-performing group (group H).
The tools used were the revised Biggs study process questionnaire
(rSPQ), which was used to measure student learning approaches (6) in
weeks 6 and 11. In weeks 7–9, students completed a concept map in
class (10 –15 min), preceded by FOD(pre) and followed by FOD(post).
If the FOD(pre) and FOD(post) values were different, for a given map,
students were asked to generate a free text response indicating why
they had changed their assessment. Students then completed both a
second rSPQ and a student feedback questionnaire in week 11 during
class time.
Learning approach measurement. The rSPQ generates two major
scores, one score for the deep learning approach (based on 10
questions) and one score for the surface learning approach (based on
10 questions). Although the Bigg’s rSPQ generates information about
both learning approaches, only the deep learning estimate was considered in the current work because the surface learning scale has not
been shown to be as statistically reliable as the former scale (41). The
deep learning scale was subdivided into two subscales dealing with
strategy and motivation, respectively, of five questions each. Since
each question has a five-point scale, each score can range between 5
and 25. The measurements of deep strategy are referred to as ds 1 (first
survey) and ds 2 (second survey). The measurements of deep motive
are referred to as dm 1 (first survey) and dm 2 (second survey) (see
Experimental procedures and Fig. 1). Students were not provided with
their scores from the rSPQ surveys.
Concept map intervention. The concept map intervention did not
form part of the summative assessment for the subject and was limited
in scope. The maps were marked using a modified version of the
scoring scheme proposed by Novak (34), e.g., each correct link scored
1 point and 5 points were given for each hierarchical level. The
answers were compared with a template that took into account
expected knowledge/understanding at that particular point in the
subject, and (as specified in the study) students were not provided with
their mark for the concept maps. However, the marks were used when
making comparisons with other aspects of the study. As the students
were in the first semester of their first year of study, the basic idea of
a concept map was explained in class before the first map. Prompts
were also provided for each map in the form of a precompleted section
(that demonstrated links and hierarchies for one relevant system) and
a list of body systems previously studied. The potential variability of
concept map expertise among the students was not considered to be an
issue, as our main point was to encourage them to think about the
exercise, irrespective of whether they found the task difficult or not.
Three simple authentic scenarios were provided (scenarios A, B, and C),
and students asked to generate a concept map for each. Three maps
were used to provide an opportunity for students to become familiar
with the concept mapping approach. The scenarios were related to
material that had previously been covered (in lectures and laboratories). One concept map was provided during week 7 and the following
two concept maps during weeks 8 and 9. The first concept map
exercise (map A) was based on someone drinking and spilling soup
that was too hot (integrating the integumentary, digestive, respiratory,
and nervous systems). The second concept map exercise (map B) was
based on someone who had eaten asparagus and was upset to notice
that their urine had changed color (integrating digestive, cardiovas-
132
ENCOURAGING SELF-ASSESSMENT
measure of internal consistency (reliability) of the survey question set
and the values are considered to be in the acceptable range.
Group division by academic performance. The cohort was divided
into groups L and H using the end-of-semester final subject mark. The
median of the mark distribution (75%) was used to divide the data set
into two halves. Students in group L achieved a score of up to 74%
and students in group H achieved a score of 75% and above.
Statistical analysis. Kendall’s correlation coefficients and paired/
independent t-tests were used for various elements of the data set as
appropriate. All statistical analyses were carried out using SPSS
version 24. All significant correlations were above the level of 0.3
considered as a minimum acceptable level.
A t-test was used for statistical analyses since the data were
normally distributed, and groups had similar variances. Since only
two groups were compared at a time and there were only two
independent variables, it was unnecessary to use ANOVA. Kendall’s
correlation was used as it was appropriate for both continuous or
ordinal data and it was thus suitable for use with Likert scales (ordinal
variables) as well as with other experimental data.
RESULTS
Characteristics of groups L and H. Median subject scores
for groups L and H were 66.0% (n ⫽ 39) and 84.5% (n ⫽ 46),
respectively. Groups L and H were compared with respect to
the number of participants, age, and deep learning approach,
and the results are shown in Table 1. Group H was significantly
older than group L and had a significantly higher level of deep
strategy and deep motive both at the beginning and end of the
study (Table 1).
Concept maps. Figure 1 shows how the assessments were
carried out as well as the timetable of these assessments. A
completed concept map is shown in Fig. 2. Group H scored
significantly higher than group L for maps B and C (55.1% vs.
32.5%, P ⬍ 0.001, and 44.6% vs. 34.9%, P ⬍ 0.05). However,
no significant between-group difference was found for map A
(group H: 44% vs. group L: 36%, not significant). Group H
also showed a positive correlation between the map B score
and the deep strategic learning approach (ds 1: 0.34, P ⬍ 0.01).
Difficulty index. A total of 221 concept maps (sum of maps
A–C) were generated by groups L and H (group L: 100 and
group H: 121). FOD(pre) and FOD(post) estimations were included in 93.0% of group L maps and 98.4% of group H maps.
A comparison of FOD(post) with FOD(pre) revealed students who
assessed the map as more difficult or easier than initially estimated
or showed no change in their rating. The FOD comparison data
are shown in Table 2 for each map and for each group of students.
Advances in Physiology Education • doi:10.1152/advan.00174.2016 • http://advan.physiology.org
Downloaded from http://advan.physiology.org/ by 10.220.32.246 on June 16, 2017
Fig. 2. Example of completed concept map
C. The student entries have been replaced by
typed text for clarity. The question (not
shown above but see text) also included a
body outline indicating damage to lumbar
and forearm regions. A list of body systems
already covered in the subject was also provided to act as a “prompt.” Boxes indicated
in bold were provided to the student as an
example of the map completion process. The
student was asked to estimate the difficulty
of the exercise before and after map completion. If there was a difference between the
“pre” and “post” assessments, the student
was asked to write their reason(s) for the
change. In this case, the student’s response
was as follows: “The cues listed on the side
made it easier to reflect on pre-existing and
previously learnt knowledge.”
ENCOURAGING SELF-ASSESSMENT
133
Approximately 58 – 67% of group L maps and 63–70% of
group H maps showed FOD(pre)-FOD(post) rating changes.
Overall, 92 written comments were made by students who
changed their FOD rating. For those students who rated the
FOD as less difficult after the map intervention, 43 comments
were recorded. For those students who rated the FOD as more
difficult after the map intervention, 49 comments were recorded. As no differences were found between the nature of the
comments from group L or group H students, the data were
combined.
For those students who found the maps easier than anticipated, the majority of comments were related to “Prompting
during map generation” (39.5%), e.g., “When I start to think
and write the concept map leads me to elaborate more information.” “Prompting due to available information” was also
well represented (14.0%), e.g., “The ones listed on the side
(note: list of body systems) made it easier to reflect on
pre-existing and previously learnt knowledge.” Students also
appreciated the “graphic representation” offered by the maps
(14.0%), e.g., “Physically seeing ideas mapped out is easier
Table 1. Characteristics of group L and group H students
Group L
Group H
Number of Students
Age, yr
rSPQ 1 (Deep Strategy)
rSPQ 2 (Deep Strategy)
rSPQ 1 (Deep Motive)
rSPQ 2 (Deep Motive)
39
46
19.4 (SD 3.4)
22.4 (SD 5.8)*
13.2 (SD 3.2)
15.3 (SD 2.9)*
13.5 (SD 2.6)
15.2 (SD 2.9)*
12.8 (SD 2.7)
15.2 (SD 3.1)†
13.6 (SD 2.5)
15.6 (SD 2.7)*
Values are means (SD). The learning approach survey [revised Biggs study process questionnaire (rSPQ)] was undertaken at the beginning (rSPQ 1) and end
(rSPQ 2) of the study. Group L, low-performing students; group H, high-performing students. Significant differences are between data in a given column.
*P ⬍ 0.01; †P ⬍ 0.001.
Advances in Physiology Education • doi:10.1152/advan.00174.2016 • http://advan.physiology.org
Downloaded from http://advan.physiology.org/ by 10.220.32.246 on June 16, 2017
Fig. 3. Responses of the low-performing group
(shaded bars) and high-performing group (solid
bars) to survey questions. The Likert scale extends from strongly disagree (1) to strongly
agree (4). rSPQ, revised SPQ. One SD is
shown. Significant differences are shown between the low-performing group and high-performing group for relevant questions. *P ⬍
0.05.
134
ENCOURAGING SELF-ASSESSMENT
Table 2. Proportion of concept maps A–C where students
showed no change in difficulty assessment, an increase in
perceived difficulty, or an increase in perceived easiness
proach had changed as a result of the concept map intervention
(question 5).
DISCUSSION
No Change
Group L, %
Map A
Map B
Map C
Group H, %
Map A
Map B
Map C
More Difficult
Easier
33.3
41.9
37.1
37.1
45.2
37.1
29.6
12.9
25.7
31.6
36.8
30.0
34.2
44.8
37.5
34.2
18.4
32.5
Advances in Physiology Education • doi:10.1152/advan.00174.2016 • http://advan.physiology.org
Downloaded from http://advan.physiology.org/ by 10.220.32.246 on June 16, 2017
than mentally visualizing a concept map.” Finally, the “authentic situation” stimulated positive comments (9.3%), e.g., “Relating myself into the situation and comparing how I would
have been affected helped me work out the systems affected.”
With respect to those students who found the maps more
difficult than anticipated, the comments fell into two categories: those related to “Difficulty in thinking or uncertainty
about how to approach concept maps” (55.1%) and “Difficulty
in remembering appropriate detail, knowing which systems to
include and in constructing concept maps” (44.9%).
In addition, links were found between the map scores and
FOD(pre) or FOD(post) scores. The group L map B score was
negatively correlated with map B FOD(pre) (⫺0.33, P ⬍ 0.05),
and thus higher map scores were correlated with student
“premap completion” ratings of “easier” or “less difficult.” In
addition, significant correlations were found between map
score and FOD(post) for both groups, with the map C score
negatively correlated with the map C FOD(post) (group L:
⫺0.33, P ⬍ 0.05, and group H: ⫺0.35, P ⬍ 0.01). Thus, in
these cases, higher map scores were correlated with student
“postmap completion” ratings of “easier” or “less difficult.”
Analysis of the student feedback questionnaire. The results
of the end-of-semester survey were based on group L and
group H student responses (group L students: n ⫽ 31–33,
range: 79.5– 84.6% of the group and group H students: n ⫽
37–39, range: 80.4 – 84.8% of the group). The survey questions, together with average responses for each group, are
shown in Fig. 3. Responses from groups L and H were
generally similar; however, group L was more positive for
questions 3, 6, and 8. With respect to the concept maps, overall
responses were positive regarding their utility and the feedback
they provided on what was known or not known (questions
1–3). Group L students were significantly more positive compared with group H with respect to the proposition that the
concept maps assisted with identification of problem areas
(question 3, 3.3 vs. 3.0, P ⬍ 0.05). In addition, concept maps
were also found to improve “thinking” abilities (question 4).
Group L gave a significantly more positive response (compared
with group H) to question 6 concerning how the project
encouraged them to think about how they approached learning
(2.8 vs. 2.5, P ⬍ 0.05) and also gave a positive response to
question 8 regarding the development of learning strategies.
Completion of the rSPQ surveys encouraged students to think
about how they preferred to learn (question 9), and group H
students were positive with respect to the project encouraging
their deep learning approach (question 11). From a negative
point of view, students did not agree that their learning ap-
The main aim of the present study was to encourage students
to self-assess their own learning, e.g., to improve the level of
metacognitive awareness of what they were doing (12, 18, 31,
40). The role of metacognition, in common with many learned
procedures, is diminished as cognitive approaches become
more established and “automated” (7, 10); thus, it is not
surprising that students may not always be aware of their
learning approaches nor how effective these learning approaches may be. The benefits of improved self-assessment are
clear since students who display appropriate metacognitive
approaches, e.g., establishing goals, planning, monitoring, reacting to, and reflecting on learning errors, achieve higher
levels of understanding and become more autonomous learners
(1, 37, 39, 40). The inclusion of the “stop think” approach at the
time of map completion provided an opportunity for immediate
self-assessment (4) as students were requested to stop and spend
time thinking about what they were about to do or had (just) done.
An important part of the “stop think” approach was the use of the
FOD, which provides a generalized assessment of perceived
problem difficulty. In the present study, students only had limited
time to make FOD assessments [FOD(pre) and FOD(post)]; however, FOD is typically generated where the situation does not
permit a full analysis of the problem (25). It would be reasonable
to suggest that FOD(pre) provides a measure of student assessment
of the learning task where knowledge, strategic approaches, and
task characteristics are taken into account. Although students were
not asked to provide text feedback on this phase of the exercise,
the significant negative correlation of the group L map B score
with map B FOD(pre) does lend some support for the notion that
FOD(pre) was a meaningful self-assessment measure. In addition,
FOD(pre) does provide a point of comparison for FOD(post) and
thus permits students to determine whether or not their estimation
of difficulty has changed during completion of the exercise. In
contrast to FOD(pre), FOD(post) is an assessment made after task
completion, anecdotally related to the immediate response of
students after an exam where a potentially complex scenario is
summarized as easy, difficult etc. The request for students to “stop
and think” at this point is a useful approach since reflection on
performance is an essential component of metacognitive understanding (12, 13), and FOD(post) was significantly negatively
correlated, for both groups, with the map C score. Similarly, in a
study by Moni and Moni (32), a strong positive correlation was
found between the strength of favorable perception of a concept
mapping task and task grade. The relationship between lower
levels of perceived difficulty and higher map scores, for both
FOD(pre) and FOD(post), suggests that students were attempting to
assess difficulty and were not simply entering random numbers
into the difficulty index.
Further evidence that the FOD estimations are a valid
approach to encouraging self-assessment comes from the fact
that a significant proportion of students changed their FOD
rating [FOD(pre-post)] and many provided free text comments on
why they changed their assessment. Although concept maps
have been widely used (33), there are relatively few studies
where students have been asked to give feedback on their
experience with this type of exercise (8, 32, 43). For those
ENCOURAGING SELF-ASSESSMENT
and deep strategic learning approaches and the need to improve
their learning; however, no deep learning improvement was found
for this group when pre- and poststudy rSPQ values were compared. It is possible that our concept map intervention was not
extensive enough to bring about such changes, and this suggestion
is supported by a previous study (1) where a relationship was
found between concept map completion and an increase in deep
learning. The idea that academically weaker students may benefit
more (in relative terms) from a concept map intervention is
supported by a study (36) where students in the lowest quartile
(based on prior grades and a measure of mental ability) who took
part in such an intervention did better than the equivalent control
group in a problem-solving exam.
Although the present study was focused on the “stop think”
approach, it is interesting to note that both groups of students
were very positive about the fact that rSPQ survey completion
made them think about how they preferred to learn (i.e.,
self-assessment) even though they did not receive the results
from these surveys. On the other hand, both groups were
neutral about changing their learning approach as a result of the
rSPQ completion.
Limitations. Because metacognition is an abstraction of
“real-world” actions, it is important to consider the accuracy of
student reflection on their learning. For example, inaccurate
metacognitive assessment may lead to situations where students become overconfident with respect to their learning
ability (5, 19, 24, 26 –28), and such effects must be considered
in any study encouraging student metacognition.
Although the concept maps did provide an active learning
situation where students were potentially able to improve existing
skills or develop new skills, we did not objectively assess the
range of cognitive approaches used by the students (11) nor
establish whether the study had a positive impact on their cognitive skills (15, 22, 45). Indeed, the end-of-study survey results
indicated a reluctance to change learning approach as a result of
the concept map exercises. However, encouraging monitoring of
self-learning is only the first step (15, 45), and further studies need
to be conducted to demonstrate tangible improvements in selfregulated learning after such an intervention.
Conclusions. The present study indicates that it is possible to
stimulate students to self-assess their learning activity through
the application of a relatively simple approach, the first step in
improving the effectiveness of their interaction with learning
materials. In particular, the “stop think” approach (4, 13),
where students assessed their FOD before and after the active
learning exercises, using a single question, proved to be particularly informative. Although it might be argued that a
numeric evaluation of difficulty by a student may not, in itself,
be metacognitive in nature, the correlations found between the
FOD and map scores suggests that students were “thinking”
about the level of difficulty. In addition, the text responses
related to changes in FOD(pre-post) clearly showed that metacognitive monitoring was taking place during the learning task.
It is important to note that the “stop think” approach appeared,
in general, to be successful for students of low and high ability,
and it will be interesting to see to what extent this method of
encouraging the self-assessment of learning may prove useful.
The simplicity of the approach may permit its incorporation
into a variety of different learning contexts, including those
that are computer based.
Advances in Physiology Education • doi:10.1152/advan.00174.2016 • http://advan.physiology.org
Downloaded from http://advan.physiology.org/ by 10.220.32.246 on June 16, 2017
students who changed their FOD estimation, the text responses
of groups L and H were similar within each cohort. For those
maps that were assessed as easier after completion, 40% of the
comments were similar to the following statement: “When I
start to think and write, the concept map leads me to elaborate
more information.” Thus, it appeared that “prompting” was
taking place where generation of concept maps triggered further relevant information retrieval from memory. It is interesting that prompting due to self-generated content was referred
to much more than the assistance provided by lists of relevant
body systems or notes (14% of comments). Prompting is a
powerful approach to information retrieval (42), and it appears
that, in the context of map completion within a limited timeframe, already available knowledge may be more important
than reference material. The authentic nature of the concept
maps also proved useful, with comments similar to the following statement: “Relating myself into the situation and comparing how I would have been affected helped me work out the
systems affected.” For those maps assessed as more difficult
after completion, the comments fell into two groups, with
~50% related to confusion in map generation and ~50% related
to lack of knowledge or ability to make the links. Given that
students did not receive any significant training in map construction and were not told in advance the map topics this is not
surprising. The large amount of content in anatomy and physiology and lack of experience in active learning approaches
may have also contributed to these comments. The main value
of the free text comments was that they clearly demonstrate
that many students were self-assessing their performance in the
concept map exercises.
Group H (based on final subject mark) differed from group L in
that they were significantly older, had significantly higher levels
of perceived deep learning (strategy and motive) at both the
beginning and end of the study, and had significantly higher marks
in two of the concept map exercises. As higher levels of deep
learning are often related to increased self-regulated learning and
positive academic outcomes, it seems reasonable to suggest that
this may be the case in the present study (23). Notwithstanding the
differences, the data from the “stop think” exercise were very
similar for the two groups, and thus we conclude that this approach may be suitable for academically weaker students as well
as those with stronger outcomes.
The end-of-study student feedback survey revealed both similarities and differences between groups L and H. For both groups,
the feedback supported the notion that concept maps act as
deep-level active learning tasks as students agreed that the maps
helped their thinking abilities, allowed them to identify deficiencies in their learning, and helped them understand how anatomy
and physiology facts can relate to a real-life situation. Other
studies have also found that concept maps were useful as an active
learning exercise that provided immediate visual feedback on
knowledge and assisted linking of concepts (8, 43); however,
perceived usefulness may depend on context, as in one study (32),
students were neutral about the advantages of maps as a learning
tool. In the present study, group L students were significantly
more positive than those of group H with respect to the usefulness
of the concept maps in identifying learning deficiencies and also
with respect to the project encouraging them to think about how
they approach learning. They also agreed that the project had
encouraged them to develop their own learning approaches. These
responses may be related to their significantly lower deep motive
135
136
ENCOURAGING SELF-ASSESSMENT
GRANTS
This work was supported by a STeLR grant from the College of Science,
Engineering and Health, RMIT University.
REFERENCES
Advances in Physiology Education • doi:10.1152/advan.00174.2016 • http://advan.physiology.org
Downloaded from http://advan.physiology.org/ by 10.220.32.246 on June 16, 2017
1. August-Brady MM. The effect of a metacognitive intervention on approach to and self-regulation of learning in baccalaureate nursing students.
J Nurs Educ 44: 297–304, 2005.
2. Baeten M, Dochy F, Struyven K. Students’ approaches to learning and
assessment preferences in a portfolio-based learning environment. Instr
Sci 36: 359 –374, 2008. doi:10.1007/s11251-008-9060-y.
3. Bar M, Aminoff E, Mason M, Fenske M. The units of thought.
Hippocampus 17: 420 – 428, 2007. doi:10.1002/hipo.20287.
4. Belski R, Belski I. Cultivating student skills in self-regulated learning
through evaluation of task complexity. Teach Higher Educ 19: 459 – 469,
2014. doi:10.1080/13562517.2014.880685.
5. Benjamin AS, Bjork RA, Schwartz BL. The mismeasure of memory:
when retrieval fluency is misleading as a metamnemonic index. J Exp
Psychol Gen 127: 55– 68, 1998. doi:10.1037/0096-3445.127.1.55.
6. Biggs J, Kember D, Leung DYP. The revised two-factor study process
questionnaire: R-SPQ-2F. Br J Educ Psychol 71: 133–149, 2001. doi:
10.1348/000709901158433.
7. Butler DL, Winne PH. Feedback and self-regulated learning: a theoretical analysis. Rev Educ Res 65: 245–281, 1995. doi:10.3102/
00346543065003245.
8. Carr-Lopez SM, Galal SM, Vyas D, Patel RA, Gnesa EH. The utility
of concept maps to facilitate higher-level learning in a large classroom
setting. Am J Pharm Educ 78: 170, 2014. doi:10.5688/ajpe789170.
9. Chamorro-Premuzic T, Furnham A, Lewis M. Personality and approaches to learning predict preference for different teaching methods.
Learn Individ Differ 17: 241–250, 2007. doi:10.1016/j.lindif.2006.12.001.
10. Corno L. The metacognitive control components of self-regulated learning. Contemp Educ Psychol 11: 333–346, 1986. doi:10.1016/0361476X(86)90029-9.
11. Corno L, Mandinach EB. The role of cognitive engagement in classroom
learning and motivation. Educ Psychol 18: 88 –108, 2009. doi:10.1080/
00461528309529266.
12. Cowan J. On Becoming an Innovative University. Maidenhead, UK: The
Society for Research into Higher Education and Open University Press,
1998.
13. Davis EA. Prompting middle school science students for productive
reflection: generic and directed prompts. J Learn Sci 12: 91–142, 2003.
doi:10.1207/S15327809JLS1201_4.
14. Dochy F, Segers M, Sluijsmans D. The use of self-, peer and coassessment in higher education: a review. Stud Higher Educ 24: 331–350,
1999. doi:10.1080/03075079912331379935.
15. Durkin K, Main A. Discipline-based study skills support for first-year
undergraduate students. Active Learn Higher Educ 3: 24 –39, 2002. doi:
10.1177/1469787402003001003.
16. Efklides A, Samara A, Petropoulou M. Feeling of difficulty: an aspect
of monitoring that influences control. Eur J Psychol Educ 14: 461– 476,
1999. doi:10.1007/BF03172973.
17. Flavell JH. Metacognition and cognitive monitoring: a new area of
cognitive-developmental inquiry. Am Psychol 34: 906 –911, 1979. doi:
10.1037/0003-066X.34.10.906.
18. Fleming SM, Dolan RJ, Frith CD. Metacognition: computation, biology
and function. Philos Trans R Soc Lond B Biol Sci 367: 1280 –1286, 2012.
doi:10.1098/rstb.2012.0021.
19. Hacker DJ, Bol L, Bahbahani K. Explaining calibration accuracy in
classroom contexts: the effects of incentives, reflection, and explanatory
style. Metacogn Learn 3: 101–121, 2008. doi:10.1007/s11409-0089021-5.
20. Halford GS, Wilson WH, Phillips S. Relational knowledge: the foundation of higher cognition. Trends Cogn Sci 14: 497–505, 2010. doi:10.1016/
j.tics.2010.08.005.
21. Hartlep KL, Forsyth GA. The effect of self-reference on learning
and retention. Teach Psychol 27: 269 –271, 2000. doi:10.1207/
S15328023TOP2704_05.
22. Hattie J, Biggs JB, Purdie N. Effects of learning skills interventions on
student learning: a meta-analysis. Rev Educ Res 66: 99 –136, 1996.
doi:10.3102/00346543066002099.
23. Heikkilä A, Lonka K. Studying in higher education: students’ approaches
to learning, self᎑regulation, and cognitive strategies. Stud Higher Educ 31:
99 –117, 2006. doi:10.1080/03075070500392433.
24. Karpicke JD, Butler AC, Roediger HL III. Metacognitive strategies in
student learning: do students practise retrieval when they study on their
own? Memory 17: 471– 479, 2009. doi:10.1080/09658210802647009.
25. Koriat A. The feeling of knowing: some metatheoretical implications for
consciousness and control. Conscious Cogn 9: 149 –171, 2000. doi:10.
1006/ccog.2000.0433.
26. Koriat A, Bjork RA. Illusions of competence during study can be
remedied by manipulations that enhance learners’ sensitivity to retrieval
conditions at test. Mem Cognit 34: 959 –972, 2006. doi:10.3758/
BF03193244.
27. Kornell N, Bjork RA. The promise and perils of self-regulated study.
Psychon Bull Rev 14: 219 –224, 2007. doi:10.3758/BF03194055.
28. Kruger J, Dunning D. Unskilled and unaware of it: how difficulties in
recognizing one’s own incompetence lead to inflated self-assessments. J
Pers Soc Psychol 77: 1121–1134, 1999. doi:10.1037/0022-3514.77.6.
1121.
29. Lonka K, Lindblom-Ylänne S. Epistemologies, conceptions of learning,
and study practices in medicine and psychology. Higher Educ 31: 5–24,
1996. doi:10.1007/BF00129105.
30. Lozano LM, García-Cueto E, Muñiz J. Effect of the number of response
categories on the reliability and validity of rating scales. Methodology 4:
73–79, 2008. doi:10.1027/1614-2241.4.2.73.
31. McCabe J. Metacognitive awareness of learning strategies in undergraduates. Mem Cognit 39: 462– 476, 2011. doi:10.3758/s13421-010-0035-2.
32. Moni RW, Moni KB. Student perceptions and use of an assessment rubric
for a group concept map in physiology. Adv Physiol Educ 32: 47–54,
2008. doi:10.1152/advan.00030.2007.
33. Nesbit JC, Adesope OO. Learning with concept and knowledge maps:
a meta-analysis. Rev Educ Res 76: 413– 448, 2006. doi:10.3102/
00346543076003413.
34. Novak JD. Concept maps and Vee diagrams: two metacognitive tools to
facilitate meaningful learning. Instr Sci 19: 29 –52, 1990. doi:10.1007/
BF00377984.
35. Pintrich PR, de Groot EV. Motivational and self-regulated learning
components of classroom academic performance. J Educ Psychol 82:
33– 40, 1990. doi:10.1037/0022-0663.82.1.33.
36. Rendas AB, Fonseca M, Pinto PR. Toward meaningful learning in
undergraduate medical education using concept maps in a PBL pathophysiology course. Adv Physiol Educ 30: 23–29, 2006. doi:10.1152/advan.00036.2005.
37. Ross ME, Green SB, Salisbury-Glennon JD, Tollefson N. College
students’ study strategies as a function of testing: an investigation into
metacognitive self-regulation. Innov Higher Educ 30: 361–375, 2006.
doi:10.1007/s10755-005-9004-2.
38. Sauro J, Dumas JS. Comparison of Three One-Question, Post-Task
Usability Questionnaires. Proceedings of the SIGCHI Conference on
Human Factors in Computing Systems, Boston, MA: April 04 – 09, 2009.
39. Schraw G, Moshman D. Metacognitive theories. Educ Psychol Rev 7:
351–371, 1995. doi:10.1007/BF02212307.
40. Schraw G, Crippen KJ, Hartley K. Promoting self-regulation in science
education: metacognition as part of a broader perspective on learning. Res
Sci Educ 36: 111–139, 2006. doi:10.1007/s11165-005-3917-8.
41. Snelgrove S, Slater J. Approaches to learning: psychometric testing of a
study process questionnaire. J Adv Nurs 43: 496 –505, 2003. doi:10.1046/
j.1365-2648.2003.02747.x.
42. Taminiau EMC, Kester L, Corbalan G, Alessi SM, Moxnes E, Gijselaers WH, Kirschner PA, Van Merriënboer JJG. Why advice on task
selection may hamper learning in on-demand education. Comput Human
Behav 29: 145–154, 2013. doi:10.1016/j.chb.2012.07.028.
43. Vadlapatla R, Kaur S, Zhao Y. Evaluation of student perceptions of
concept mapping activity in a didactic pharmaceutics course. Curr Pharm
Teach Learn 6: 543–549, 2014. doi:10.1016/j.cptl.2014.04.014.
44. Vermetten YJ, Lodewijks HG, Vermunt JD. The role of personality
traits and goal orientations in strategy use. Contemp Educ Psychol 26:
149 –170, 2001. doi:10.1006/ceps.1999.1042.
45. Zeegers P, Martin L. A learning-to-learn program in a first-year chemistry class. Higher Educ Res Dev 20: 35–52, 2001. doi:10.1080/
07924360120043630.
46. Zimmerman BJ. Self-efficacy: an essential motive to learn. Contemp
Educ Psychol 25: 82–91, 2000. doi:10.1006/ceps.1999.1016.