METHODOLOGY FOR CONDUCTING FIRST TESTING
with students participating in LEARN IT project
This project has been funded with the support from the European Commission (project no: 2014-1-PL01-KA200-003353). This
publication reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made
of the information contained therein
Strona 1
The document describes methodology, according to which partners will conduct first series of tests witch
students, gather data and prepare reports.
To measure effectiveness of tools and solutions developed under the LEARN IT project, it is essential to
discover and define certain factors, which will be starting point in the process of measuring the
effectiveness of the developed tools and solutions. These are: level of mathematical skills of students at the
beginning of the project , level of motivation of each student, as well as cognitive abilities of each student.
On basis of this, project team will be able to measure progress in mathematical skills of each student, and
see in what extent it is thanks to influence of the tools developed under the project.
The group of students participating in the student will fill in series of test, examining the level of their
mathematical skills, cognitive abilities, and motivation. The tests battery will be administered twice - at the
beginning of the implementation of the project, and at the end of it.
After two series of test (conducted at the beginning and at the end of the project), researchers from WSEI
will analyze results, and prepare document summarizing results of these tests and containing conclusions
about effectiveness of new method of learning. Report will be published on the project website, as well as
on the websites of the partners.
Mathematical tests contain material taking into account the skills, that may be developed in Khan Academy
and the Key Competences defined by the European Commission, as well as the knowledge that students
ought to possess after completing the high-school graduate examination.
The research model consists repeated measurements. The repeated measurements with VTS and RPM use
leads to learning of test material. Latin square designs bring the remedy. A Latin square design is a blocking
design with two orthogonal blocking variables. These designs are used to simultaneously control (or
eliminate) two sources of nuisance variability, in our case – learning of test material. This is the reason of
adding post-tested group to the model of study.
This project has been funded with the support from the European Commission (project no: 2014-1-PL01-KA200-003353). This
publication reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made
of the information contained therein
Strona 2
I.
Objectives of testing:
 To measure of level of mathematical skills of each student participating in the project activities.
 To check students’ previous experience of learning mathematics, present motivation and cognitive
abilities levels.
 To discover, what factors influence the process of learning of each student.
II.
Testing sessions:
Mathematical tests:
1. There are 90 minutes to complete the test.
2. The test should be solved using blue ink pen, blue ballpoint and with pencil if necessary. Black ink
or black ballpoint are not allowed.
3. Tests ought to be signed with the full name of the student.
4. We suggest that the whole group of students should be divided into subgroups no greater that 18
people. That means that for 54 students engaged in the project each partner will need 3 groups in 3
separated classrooms with 3 supervisors.
5. Each subgroup should write the test at the same time so that there is no possibility of information
exchange among subgroups.
6. Students are not allowed to talk with one another.
7. Students are not allowed to cheat.
8. Calculators, smart-phones, books and mathematical tables are not allowed.
Psychological tests:
1. Each psychological testing session should take about 2 up to 2 and a half hour. The testing session
consists of three stages done in the following sequence:
 Information about testing. Students should be informed about confidence of results,
importance of reliable and open answers, possibility of refusing testing and canceling
participation in research. Researchers should address all students’ questions, and after
This project has been funded with the support from the European Commission (project no: 2014-1-PL01-KA200-003353). This
publication reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made
of the information contained therein
Strona 3
obtaining students’ explicit consent they should give instruction for the first test. This stage will
take about 10-20 minutes.
 Two paper-pencil tests will be administered as first, through about 45 minutes up to one hour;
 Three computer-based tests will be administered during the last stage of the process and it will
take about one hour, depending on students properties.
2. There should be a 10 minutes of break offered after the first stage of testing.
3. It is advised, that psychological tests should be administered by certified and trained
psychologists. Detailed rules of giving tests instructions and answering testee’s questions are the
subject of psychological training and can’t be presented in this methodology.
4. All sessions should be held under mostly identical circumstances. Test should be administered in
rooms with enough light level, ambient noise level and minimal distractions, not during typical
lunch time.
5. The paper-pencil tests (AMI and RPM) can be performed independently of the computer-based
test, during separate meetings. It is not mandatory to do all psychological test at the same day, but
they should be administered during the period of time no longer than one week. Paper-pencil tests
could be administered in groups not bigger than 20 students.
III.
Set of tools used for tests
Mathematical tests:
1. For each student there should be prepared one test set consisting:
 the test printout in national language.
 A4 pages with grid. They should be encoded, numbered or stapled so that they did not mix.
2. Test contains two parts. Part I contains 10 questions, each for 5 points. For each question write
“YES” or “NO” in indicated space. No calculations or justifications are needed. Part II contains 5
problems, each for 10 points.
Psychological tests:
This project has been funded with the support from the European Commission (project no: 2014-1-PL01-KA200-003353). This
publication reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made
of the information contained therein
Strona 4
The only tool used during psychological testing will be the personal computer – the base for Vienna
Tests System. It is a standard PC with mouse, keyboard and flat screen. Students will fill a set of three
tests in computers. The tests will be used to measure cognitive abilities of students. It is a part of the
licensed Vienna Test System – international system of psychological tests.
IV.
Recruitment of students for testing
Each partner should recruit 72 students of first year, who have mathematics in their study program.
The best idea is to recruit students of the same discipline and faculty, in the same age and at the same
stage of studying in each Partners’ country, for example students of informatics. Gender proportion in
all groups and sub-groups should be balanced (50:50% of K:M) and must be equal in each Partners’
countries.
In the first series of testing 54 students will fill in the tests. 36 of them will participate in sessions in
Learning Lab, and 18 will not.
V.
Data collection instruments
Mathematical tests
Students will do the paper version of tests, and after they are done, papers will be collected by the
examiners. After finishing the test all sheets ought to be collected and scanned, the put on the projects
share-point site. The originals will be taken by representative of project leader when he visits partners.
Each partner makes an assessment of the tests and sends it results to the leader.
Psychological tests
The set of tests includes paper-pencil tests and computer-based tests, which will be used to measure
cognitive abilities and motivation of students. Chosen tests administered in following sequence are:
 AMI (LMI) - the Achievement Motivation Inventory. The inventory measures achievement
motivation considering its various components such as preference for difficult tasks, confidence in
success, goal setting, competitiveness, persistence and others, thus enabling a profile diagnosis.
 SPM - Raven’s Standard Progressive Matrices. Aimed at nonverbal assessment of general
intelligence in people with average capacity on the basis of deductive thinking.
 NVLT, Form S2 – Non Verbal Learning Test. It is aimed at assessment of non-verbal learning.
This project has been funded with the support from the European Commission (project no: 2014-1-PL01-KA200-003353). This
publication reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made
of the information contained therein
Strona 5
 DAUF, Form S2 - test aimed at assessment of long-term selective attention and concentration and
of general performance and commitment.
 AHA, the second subtest “Coding symbols” measures respondent’s aspiration level and frustration
tolerance.
Notices:
Students should be informed that the result of the test will not have any influence on their final grades in
mathematical subject and that it only intends to check their level of mathematical skills. They should be
also informed that the results will be kept confidential.
This project has been funded with the support from the European Commission (project no: 2014-1-PL01-KA200-003353). This
publication reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made
of the information contained therein
Strona 6