The Experience of Fresher Students in
Mathematics Diagnostic Testing
A survey by the UMTC/LTSN sponsored Action Research Group on Diagnostic Testing and
Student Support
Björn Haßler, Richard Atkinson, Mike Barry, Douglas Quinney
ABSTRACT
This paper presents the results of a survey of university fresher students into the experience of
mathematics diagnostic testing. A large majority of them believe that such tests are well timed
and executed and give both students and university staff alike a true position of academic
readiness, but many students also feel that they were unprepared for their test. The survey also
considered the relative familiarity of students with particular mathematical topics. Complex
numbers, and discrete mathematics appear to be largely unencountered, as does the concept of
proof, and many are aware of mathematical software and its facility to plot graphs.
1 Introduction
During autumn 2002, a total of 13 English and Scottish universities were visited in a survey of
diagnostic testing in mathematics. A wide range of procedures appears to be used. Some of the
case studies gathered during this survey have recently been reported elsewhere (Diagnostic
Testing for Mathematics, LTSN MathsTEAM, http://www.ltsn.ac.uk/mathsteam) and further
details regarding individual tests can be found in publications written by the departments visited
(references to be given).
In the survey of diagnostic testing several hundred students were observed and 98 of these
students, selected at random, were asked to fill out a questionnaire immediately after completing
their test. Sections One to Four of the questionnaire (see Appendix) cover administrative matters
such as the student’s profile, preparation for the diagnostic test, its timetabling, perceived
difficulty, execution, and any special features associated with taking a computerised test. This is
followed in Section Five with a detailed set of questions about the student’s familiarity with
mathematical topics. Section Six asked students to highlight the three topics that they found
most difficult and unpleasant, and Section Seven asked them to put in rank order favoured
strategies for follow up assistance in mathematics.
Although this paper relates to students’ experience of their diagnostic test, we will also address
the wide variety of testing practices used. When tests are taken in a classroom setting they tend
to last between one and two hours. However, some tests are taken by students in their own time.
Sometimes a test involves as few as 10 short questions whereas others can be much more
comprehensive. About 40% of the tests were computerised. Whilst the diagnostic test is always
meant to give staff information about student skills, the way it operates and is interpreted may
not always be the same. Usually the result of the test has no bearing on grades or student
learning, but sometimes it is not used as a learning tool and students may have to resit a series of
similar tests until the material covered by it is thoroughly mastered (‘test as drill’).
The evaluation of the student questionnaire is presented as follows. Section 2 of this report
covers Sections One to Four of the student questionnaire, Section 3 covers questions from
Section Five (including students' priorities), and Sections 4 and 5 the remainder.
2 The students background and perception of the diagnostic test
2.1 The Student's Profile.
Almost all students surveyed were embarking on a degree programme in which mathematics is
either their main subject, or a major shared part. The numbers were as follows.
Student Numbers
33
27
13
5
The main subject
A major shared part (near 50%)
Significant but key part (e.g. 30%)
Small
Most students had an A or AS level in mathematics, however their grades covered a wide range.
The entry qualifications in mathematics were as follows
Student Numbers
65
2
1
5
14
11
A or AS Level
BTEC
GNVQ
Non- UK Qualification
Other UK Qualification
No Answer
The vast majority of students are satisfied residential matters such as the quality of their
lodgings, the distance from their study places, shops, recreational facilities etc. They are also
satisfied that the daytime working environment is sufficient for them to work, as well as with the
level of access to computer facilities both in the study areas and in accommodation.
2.2 Pre-Joining Information in Mathematics.
Virtually all students believe that being given some information about how to prepare themselves
for their studies is a good idea (e.g. by being given a textbook list). This contrasts with students
perceptions as to whether they had received such information. Only 63% were given any idea
about how to prepare themselves, and the remaining 37% claimed to have been given no
information. From the student point of view pre-joining information is highly desirable, but
apparently only just over half of them receive this.
2.3 Information about Diagnostic Test and the Perception of its Usefulness.
Of the students who replied, just over 70% were given warning about the test, whilst about 60%
were told exactly what to expect.
It is often argued that the diagnostic test provides both staff and individual students alike with an
immediate picture of where the student stands with regard to his or her mathematical ability and
preparedness. When students were asked whether they accepted this, about 80% thought that the
test gave some useful information to both students and staff but much of the remainder thought
that the test did not do this with a small fraction (3%) believing that the test is useful to students
alone. Those unimpressed said that the test was not representative of their real ability and that
they made many mistakes that could have been avoided with just a little bit more preparation.
Typical replies were:
 Forgotten a lot of maths over the summer vacation. Not give enough preparation to
look over work.
 Have not practised math over the summer break.
 Haven't done maths for a long time i.e., rusty.
 I have passed my maths in Jun 2000 (two years before) and have found it difficult to
remember simple things!
 Incredibly out of practice, staff possibly not aware. Not done maths for approximately 3
months, and feel I have forgotten quite a lot. A quick briefing over past work could
help.
 Some students may prepare, some gap years students have had time away form subject.
 Students have just come back from holiday, done no revision, so a test doesn't really
indicate a student's ability.
A number of these answers suggest that students were under the impression that the test was
designed to measure their 'real ability', opposed to their 'present knowledge', and perhaps they
also felt that they were measured up in some way. It would be useful if institutions said quite
clearly that their aim in testing is to find out what the students know here and now, so that
teachers can know also. It may be true that there are a number of topics that students would be
able to do if they were just briefly reminded in advance. If a large number of students cannot do
a certain topic, substantial extra teaching time may be allocated to it in proportion to others.
There were also some comments unrelated to students' preparation and revision. Some students
thought the test was too basic, or did not contain enough questions. Here are some replies.
 Only 10 questions - not enough to accurately give a picture of ability etc.
 The test was just basic maths.
Other comments were:
 If I had a pad, paper & calculator I could have doubled my score.
 To an extent the test does help both the staff and student; however I feel that the test was set
too early on in the course as first year students are still only adjusting to the system of
universities.
2.4 Timetabling and Purpose
More than 70% thought that having the diagnostic test in the first week or so at university was
the best timing. Some students appreciated the fact that the test needs to be early so staff know in
due time, and it gives students a good chance to see what will be expected of them later on.
Among those who thought this was not the best timing, i.e. Freshers’ week, thought that a later
timing would allow students time to settle in. One student, for whom English is a second
language, thought there was much to adapt to already, so a later test would have been
appreciated. Another student felt that a lot of tasks in their diagnostic test depended on retrieval
of formulae, and that without remembering them one simply became stuck, so the test may not
be a true reflection of mathematical ability, but memory too.
The precise purpose of the test must be made clear to students before they come to the university.
They need to know that the test is there to help them and their teaching staff. It should for
instance produce a learning profile to guide the scheduling of material in lectures, or provide
special support. It isn’t and cannot be a measure of true mathematical aptitude, rather it offers a
reading of the state of present mathematical knowledge.
2.5 Preparedness for diagnostic test.
Many students believed that they were unprepared for their respective diagnostic tests and gave
their reasons as follows:
Reason
lack of revision
hard questions
new/unknown material
summer break
longer break
illness, accident, other
Applies (Out of 97)
64
14
13
45
29
12
Seemingly the lack of revision and summer/longer breaks is given by the majority as a reason
for being unprepared. Difficulty with questions (hard questions, new/unknown material) is
generally given less frequently as a reason for being unprepared.
2.6 Execution of Test.
93% of students thought that they were given enough time to answer the test. In 60% of cases,
certain materials, like calculators, were required to do their particular test, but usually these were
provided.
2.7 Perceived Difficulty of Test.
Percentage of questions
None (0%)
Some (10%-30%)
Half (40%-60%)
Most (70%-90%)
All (100%)
About right
1
8
27
31
12
Easy
6
35
4
10
3
Hard New
3
16
18
2
5
1
6
2
1
0
Very few questions were considered to be on new material, while some questions were judged as
hard. Most test questions were thought to range from easy to the right level.
2.8 Impact of computerised test.
41% took a computerised test and 73% of them thought that the computer interface did not cause
them distraction or unnecessary stress, but a significant 23% thought that it did.
3 Knowledge of Specific Topics and Subtopics
In Section Five of the questionnaire, students were given a list of thirteen mathematical topics
with several subtopics related to each, and were asked whether they were well practised,
acquainted, or hadn’t met the particular subtopic. In order to give an idea of the total number of
questions answered at particular universities, the following table presents the number of answers
obtained, as well as the number of questions not answered. The number of questions not
answered is listed under Not Applicable (N/A), together with the percentage in the last column.
The high percentage of questions not answered in this section for QMUL appears related to the
fact that the diagnostic test ran from 5pm to 7pm. More tentatively, the high rate for APU could
be explained that the whole test atmosphere was very informal, so it was hard to monitor how
students filled in the questionnaire. This was also the case elsewhere where the atmosphere was
similarly informal.
University
Apu
Brtl
Brunel
Cardf
Cov
Keele
Mmu
Ncut
Qmul
Stra
Su
Umist
York
OVERALL
Answered
347
865
616
435
1042
607
783
517
188
598
517
545
435
7495
N/A
175
92
80
0
2
89
0
5
334
11
5
151
87
1031
N/A % Students
33.5
9.6
11.4
0.0
0.1
12.7
0.0
0.9
63.9
1.8
0.9
21.6
16.6
12.0
98
3.1 Evaluation by Topic and Subtopic
A detailed discussion about how students felt about individual mathematical topics and subtopics
is given in the Appendix but here we look at the overall picture. For every subtopic students were
asked whether they had practised (i.e. knew the subtopic quite well and would have felt
competent to attempt an examination question), were acquainted (i.e. would have seen the
subtopic demonstrated by example, but little more ) or ‘no’ to indicate that the subtopic had not
been seen and was thus new to them. Roughly speaking `practice' was ticked by most students
for the majority of the subtopics under arithmetic, algebra, differential calculus, integral calculus,
classical geometry, sequence & series, trigonometry, and statistics & probability. For functions
and logarithmic/exponential `acquaint' and `no' were ticked more frequently. For discrete maths,
and proof, students tended to tick `practice', `acquaint', and `no' with roughly similar frequencies
for many of the subtopics. For complex numbers 'no' predominates. The full picture is below.
OVERALL
Trigonometry
Stats & Probability
Sequence, series, binomial
Proof
Logarithmic / exponential functions
no %
acqu %
Prac %
Integral calculus
Functions
Discrete Maths
Differential calculus
Complex numbers
Classical geometry
Arithmetic
Algebra
0
10
20
30
40
50
60
70
80
3.2 Comparison between Universities
The answers of students from all universities were all taken together, and then evaluated. We
shall now look at how the answers varied between different universities. The distribution is
broadly similar, except for APU and NCUT, which do worse than average. YORK and SU do
particularly well. We considered arithmetic and differential calculus as two topic areas to show
how the different universities typically compare. There is little difference with arithmetic that is
not surprisingly generally well practised, c.f.:
Arithmetic
york
umist
su
stra
qmw
ncut
no %
acqu %
Prac %
mmu
keele
cov
cardf
brunel
brtl
apu
0
10
20
30
40
50
60
70
80
90
100
But for differential calculus there is a more marked variation. In some universities, all students
perceive themselves as well practiced in all sub-topics, and there are quite a few universities
where most students are either practiced or acquainted. However, at two universities (NCUT,
APU) there are a large number of students who are not acquainted with a number of subtopics.
Differential Calculus
york
umist
su
stra
qmw
no %
acqu %
ncut
mmu
Prac %
keele
cov
cardf
brunel
brtl
apu
0
50
100
150
4 Priorities
Section Six was only answered/reached by about half of the students who filled in the
questionnaire. They were asked to pick out those three subtopics from the above list that they
found hardest. The most frequently listed were growth/decay models (mentioned 14 times) and
solving log/exp equations (mentioned 7 times).
5 Follow-up Assistance in Mathematics
Students were given five choices to be given in rank order:
 1 to 1 tuition - paid for GBP 15.00 per hour
 use of `walk-in' centre - with access to help materials, & very occasional advice
 support classes on diagnostic topics- possibly held at 1 p.m. or 5 p.m. weekly
 written tests on diagnostic topics- marked by staff, every 3 weeks
 WEB-based computer tests - point/click responses & feedback, always available
and asked to rate those in order of preference.
The most popular was support classes, while the least popular is one to one tuition priced at GBP
15.00. It is clear that such expensive tuition is probably unaffordable for most students but it is
not immediately clear why support classes would be most favoured. Among the other choices,
walk-in centres are slightly more popular than web-based support and written tests.
6 Conclusions
6.1: Most students believe that the diagnostic test serves a purpose and agree that the timing of
the test right at the beginning of their studies is suitable. Students want to be given information
about how to prepare themselves for their degree, but less than two-thirds claimed that they had
received information on how to get ready for it with a large number feeling that they could be
better prepared for the test.
6.2: A number of students raised questions about the precise form of the diagnostic test. Some
replies suggest that some of them thought the test was designed to measure their real
mathematical ability opposed to their present preparedness. Institutions must be quite clear about
their intentions, and should say quite explicitly that their aim is, e.g. to find out what the students
know here and now, so that students and their teachers can be informed in detail.
6.3: There is a deeper issue behind the information. We need to know that the diagnostic test
really measures fundamental shortcomings in knowledge, as opposed a false message indicating
a poor understanding when lack of revision might be at fault. This raises the question as to how
seriously does one need to take the outcome of the diagnostic test. We suggest that diagnostic
tests are evaluated in as much detail as possible, so one can advise upon remedial measures.
6.4: In the cases where students claim to feel least confident, e.g. complex numbers, discrete
mathematics etc, it would be useful to see if the test results actually bear this out this unease.
6.5: There are variations between different universities in what students know. The knowledge of
differential calculus may be a case in point, though this could be related to different
backgrounds.
6.6: With regard to student support, the most popular choice was for support classes but these
can be notoriously difficult to timetable. Some students also like walk-in centres but these are a
more expensive though more convenient option.
7 Recommendations
7.1: At the present time diagnostic testing appears to be a good idea for mathematics departments
in UK universities, and we recommend that departments conduct diagnostic tests.
7.2: If you are going to give a diagnostic test tell students and warn them what to expect, provide
a mock diagnostic test, and ask them to refresh their knowledge providing exercises as necessary.
7.3: Timetable the diagnostic test early and include the time of their test in the pre-joining
information sent out prior to Freshers Week.
7.4: Tell students before they come to the university exactly what the test is for. It is there to help
them and their teaching staff. It should produce a learning profile to guide the scheduling of
material in lectures, or to give support. It isn’t and cannot be a measure of true mathematical
aptitude, rather it is a reading of present mathematical knowledge.
7.5: We suggest that diagnostic tests are evaluated in as much detail as possible, so one can
advise upon remedial measures.
References
1.
Diagnostic Testing for Mathematics, LTSN MathsTEAM, http://www.ltsn.ac.uk/mathsteam
2.
A questionnaire to evaluate students experience of diagnostic tests, WEB ADDRESS
3.
Additional tables for Detailed Subtopic Analysis, WEB ADDRESS
Appendix
Appendix A: Abbreviations for Universities
Abbreviation
Apu
Brtl
Brunel
Cardf
Cov
Keele
Mmu
Ncut
Qmul
Stra
Su
Umist
York
University
Anglia Polytechnic University
Bristol University
Brunel University
Cardiff University
Coventry University
Keele University
Manchester Metropolitan University
University of Newcastle upon Tyne
Queen Mary University of London
Strathclyde University
Sussex University
University of Manchester, Institute for Science and Tech.
York University
Appendix B: Detailed Subtopic Analysis
We shall now consider the subject areas individually
3.1.1. Algebra.
Generally subtopics in algebra appear largely practised. The marginal exceptions are solving
modulus inequalities, three simultaneous linear equations, and the remainder theorem, but the
most of students still consider themselves appropriately practised.
3.1.2 Arithmetic.
Students generally felt they had good practice in most subtopics in arithmetic. The only major
exception is in estimating errors, where ‘acquaint’ claims relatively equal numbers. Logarithm
bases and modulus also have a high non-answer level of 35%-40%, indicating a well-recognised
insufficiency.
Arithmetic
estimating errors
logarithm base
modulus
no %
acqu %
Prac %
scientific notation
decimals
fractions
HCF LCM
powers primes
0
10
20
30
40
50
60
70
80
90
100
3.1.3 Classical Geometry.
Subtopics in classical geometry are generally well known. These include Pythagoras' theorem,
angle sums in polygons, and arc sectors. Reflection, translation, and rotation, as well as threedimensional shapes offer similar percentages of students answering acquainted and practised, but
most are acquainted with the three-dimensional classical solids. The percentage of students who
answered ‘not known’ seems less than 20%, and usually around 10%.
3.1.4 Complex Numbers.
For this there are often similar numbers of practice/acquaint/no, with the `no' category being the
largest category. Higher level subtopics such as solving zn=a, polar representation, and Argand
Diagram have generally not yet been met.
Complex numbers
solve zsupnsup a a is real
no %
polar representation
acqu %
vector sum and difference
Prac %
Argand diagram conjugate
definition modulusargument
0
10
20
30
40
50
60
70
80
3.1.5 Differential Calculus.
Consistent with algebra, arithmetic, and classical geometry, an overwhelming majority of
students state that they have practice with all the named subtopics. The percentage of students
claiming to be fully practised is always around 70% or more, while the percentage of students
answering unknown is low, around 10% or less. Differentiating powers of x not surprisingly is
best known followed by extrema and turning points. 20% or more students, still a minority, go
no further than acquaintance with respect to rate of change, standard derivatives, and
differentiating products and ratios, and using the chain rule.
Differetial Calculus
second derivatives
finding maximumminimum
turning point on graph
no %
tangent to graph
acqu %
chain rule
Prac %
sum product product ratio rules
know standard derivatives
differentiate powers of x
rate of change of a function
0
10
20
30
40
50
60
70
80
90
3.1.6. Discrete Maths.
The three subtopics in discrete mathematics, basic concepts of a set, laws of set algebra, Venn
diagrams) have not been met by many students. Relatively equal numbers of them feel that they
have practice, acquaintance, or no knowledge of them.
3.1.7. Functions.
The concept of limit is given equally for practice and acquaint (both c. 40%), with about half of
that for no knowledge (20%). Few students are well practised in using software to plot functions,
and the majority of students have never seen such software.
3.1.8 Integral Calculus.
Similar to differential calculus, subtopics in integral calculus are well practised. This includes the
knowledge of standard integrals, definite integrals, and area under a curve. For the trapezium and
Simpson's rule for integration, as well as for the solid of revolution there are higher number of
students answering unknown or acquainted (around 20%-30% for both respectively).
3.1.9 Logarithmic / Exponential functions.
For growth/decay models, there are equal numbers of practice and acquainted (just under 40%
each), with more than half the number answering not acquainted (more than 20%). For the other
subtopics (solving log/exp equations, graphs of exp and log, power laws), practice is in the
majority (around 50% or more). Note from above that this confidence does not extend to
logarithmic bases.
3.1.10 Proof.
There are two subtopics in this section: Pythagoras theorem, and the meaning of axiom / theorem
/ corollary. Again, there are about equal numbers of students who feel that they have practice,
acquaintance, and no acquaintance of following the proof Pythagoras' theorem. While some
students have practice in, and acquaintance with, the meaning of axiom, theorem, and corollary,
the majority (just under 60%) state that they have no knowledge of this topic.
3.1.11 Trigonometry.
Similar to other well-practised areas above (like algebra), most subtopics in this section are
practised (mostly 60% or more). In compound double angle formulae, sin^2+cos^2=1, and the
CAST rule, more students state acquainted, and more than 30% state unknown for the CAST
rule. For “related to motion”, more than 40% answered unknown.
3.1.12 Sequence, Series, Binomial
The three subtopics are binomial expansion, geometric progression and arithmetic progression,
are all practised by the majority of students (around 65%), and a further 20% claim to be
acquainted.
3.1.13 Statistics & Probability
Practice predominates over all the main subtopics with marginally higher levels of acquaintance
(20/30%) with probability rather than basic data analysis.
Stats & Prob
conditional probability independence
nonexclusive exclusive events
no %
probability of an event
acqu %
Prac %
define outcome event probability
mean median mode for data set
histogram bar chart stem leaf diagrams
0
10
20
30
40
50
60
70
80
90