weSPOT
IST (FP7/2007-2013) under grant agreement N° 318499
Deliverable Title: D5.1 Diagnostic Tool
Deliverable Authors: Elisabetta Parodi (ELS), Guido Cernuto (ELS), Alice Carminati (ELS),
Fotini Haimala (FORTH), Marcus Specht (OUNL)
Deliverable Workpackage: WP5
Deliverable Leader: ELS
Date: 30.09.2013
weSPOT Project - IST (FP7/2007-2013) under grant agreement N° 318499
1
weSPOT Project – D5.1 Diagnostic instrument
2
Table of Contents
1. Introduction .............................................................................................................................................................................. 3 2. Online survey tools................................................................................................................................................................. 3 2.1 QuestionPro ........................................................................................................................................................................... 3 2.2 Freeonlinesurveys................................................................................................................................................................ 4 2.3 SurveyGizmo ......................................................................................................................................................................... 4 2.4 SoGoSurvey ............................................................................................................................................................................ 4 2.5 FluidSurveys........................................................................................................................................................................... 4 2.6 SurveyMonkey ...................................................................................................................................................................... 5 2.7 Zoomerang ............................................................................................................................................................................ 5 2.8 PollDaddy ............................................................................................................................................................................... 5 2.9 Google Forms ........................................................................................................................................................................ 5 2.10 Lower Cost Integrated Solutions ................................................................................................................................. 6 2.10 More Advanced Survey Packages ............................................................................................................................... 6 3. 1st weSPOT Diagnostic Tool: set-up of questionnaire ................................................................................................ 8 3.1 Requirements .................................................................................................................................................................... 8 3.2 Set-up................................................................................................................................................................................... 8 4 Conclusions and Outlook ................................................................................................................................................... 10 Annex 1 Test items for online questionnaire...................................................................................................................... 11 weSPOT Project - IST (FP7/2007-2013) under grant agreement N° 318499
2
weSPOT Project – D5.1 Diagnostic instrument
3
1. Introduction
The weSPOT Diagnostic Tool uses the diagnostic indicators specified in WP2 to measure the level of
inquiry competence of students. The tool is meant to enable students to understand how their
competences in this area evolve over time and what deficiencies they need to address in order to
improve their inquiry abilities.
In the setup for different forms of assessment of the student skills and the implementation of a diagnostic
instrument different tools have been discussed. In Deliverable 2.3.1 the main skills and competences
have been defined and the indicators for their measurement have been identified.
This deliverable focus on the technical implementation of the first component of the diagnostic
instrument, extensions and the integration with the measures of the FCA tool and the Learning Analytics
framework will be released in a later version. With the extensions the tool provides implementation and
validation of theoretical assumptions about performance indicators and mastering of competences by
elaborating mathematical formulas where data about the user's experience are taken as input, resulting
into an assessment of his performance as output.
In order to get a set of scientifically sound equations, pedagogical partners need the results of the
empirical studies first. So it was agreed internally to the consortium to have first version of the Diagnostic
Tool as a set of questionnaires to be filled after running the first pilots. Then the outcomes of the
questionnaires will be elaborated to provide the formulas that in turn will be elaborated by a software
component.
In this document we present a short overview of tools that we analysed as software platform to provide
the questionnaire. We also present a summary table together with requirements for the table and, of
course, our choice and link to the online questionnaire. The questions for the online questionnaire and
the relations to the different phases of the IBL model based on the developments of work package 2 are
listed in Annex 1.
2. Online survey tools
We decided to use existing online survey tools as many are already available. We carried on an analysis
about most used, popular and recommended tools that we summarize below. We considered mainly the
features and limitations provided by the free accounts.
2.1 QuestionPro
QuestionPro1 allows to quickly create online surveys and polls with an intuitive web based software,
available both on desktops and mobiles. The free version offers reports online only and it is not possible
to download users’ answers locally.
QuestionPro packages range from limited free versions to more advanced $15-$99/month options. The
free package lets you re-use questions from one survey to the next, or pull questions from a standard
survey template library. The more advanced packages offer unlimited surveys, questions and responses
with skip logic, piping, randomization and even more complex survey logic, as well as multimedia and
multilingual support. QuestionPro also offers an API to exchange survey data with outside applications,
including a module for Salesforce.com integration.
1
http://www.questionpro.com/
weSPOT Project - IST (FP7/2007-2013) under grant agreement N° 318499
3
weSPOT Project – D5.1 Diagnostic instrument
4
2.2 Freeonlinesurveys
Also FreeOnlineSurveys2 allows to easily create questionnaires with an intuitive and user-friendly
interface. Thus, despite of the name, it does not offer reports for free.
2.3 SurveyGizmo
SurveyGizmo3 provides an SGLite Account with limited capabilities and no monthly cost.
Included in a SGLite Account:
• Unlimited Surveys
• 50 Responses per month, per project
• A Single Summary Report for each project
• 100s of Question Type Variations
• Access to Support
• Generate Test Data
Even if the following are not included in a SGLite Account:
• Unlimited Responses per Month
• Create Custom Scripting
• Send Email Invitations
• Blog and Website Embeds
• Create Tally & Pass/Fail Quizzes
• Advanced Skip Logic & Branching Features
2.4 SoGoSurvey
SoGoSurvey4 is a leader in online survey software. What sets it apart is ease of use, comprehensive
features, powerful reporting, and responsive 24/7 support.
Easy and even enjoyable to use, provides step-by-step survey wizards and a responsive customer support
24/7. Its features include nine powerful, real-time reports that can be easily shared via social media and
export reports to MS Office programs.
The free account provides only 10 surveys, 50 questions and 100 responses per month.
2.5 FluidSurveys
FluidSurveys5 allows building an unlimited number of surveys & forms, starting from scratch or making
use of expert designed survey templates. A simple drag & drop editor gives access to over 25 question
types, skipping, branding and more.
The basic, free account provides:
2
3
•
Unlimited Surveys
•
20 Questions Per Survey
http://freeonlinesurveys.com/
http://www.surveygizmo.com/sglite/
4
http://www.sogosurvey.com/static/packages.aspx?blnpackages=False
5
http://fluidsurveys.com/features/
weSPOT Project - IST (FP7/2007-2013) under grant agreement N° 318499
4
weSPOT Project – D5.1 Diagnostic instrument
•
150 Responses per Survey
•
Ready Made Templates
•
Basic Reports
5
2.6 SurveyMonkey
SurveyMonkey6 offers a popular online hosted survey tool that works well for basic surveys. The free
version might be useful for very small and informal surveys, but allows very little customization of the
look of the survey, no downloads of reports or data, and can only collect 10 questions and 100 responses
per survey.
2.7 Zoomerang
Zoomerang7 is similar to SurveyMonkey in many respects, but offers in general a somewhat more
powerful package for somewhat more money. Like SurveyMonkey, there’s a very limited free package;
the more useful Pro package is offered to nonprofits for $149/year for unlimited surveys, questions and
respondents, and a Premium package ($449/year for nonprofits) includes mobile surveys, comparison
reports, multi-user survey sharing and collaboration, and statistical analysis. The survey building tools are
not quite as intuitive as SurveyMonkey’s, and it can be more difficult to learn. However, Zoomerang offers
more extensive reporting, with a flexible cross-tabulation report tool that lets survey administrators see
the data relationships across any set of questions.
2.8 PollDaddy
8
PollDaddy offers surveys and polls that can be easily embedded into external websites and applications.
The free package offers a maximum of 10 questions per survey and 100 responses per month, plus basic
reporting. More advanced versions cost between $200/year and $900/year. Survey features are more
limited than some of the other options in this category, with no skip logic or piping, but survey
administrators have a lot of flexibility over the look of the survey, by selecting from pre-designed
templates, or fully customizing the template by editing stylesheet code. Surveys can be delivered in popup windows, and results can be tracked via RSS feed.
2.9 Google Forms
Google Forms is a useful tool to help planning events, sending a survey, giving students a quiz, or
collecting other information in an easy, streamlined way. Google Forms provides a flexible form and
survey development interface with built-in reporting. Google Forms is the start of an incredibly versatile
data collection framework. Survey data is far from being difficult, nor costly, to store.
A Google form can be connected to a Google spreadsheet. If a spreadsheet is linked to the form,
responses will automatically be sent to the spreadsheet. Otherwise, users can view them on the
“Summary of Responses” page accessible from the Responses menu.
6
www.surveymonkey.com
7
www.zoomerang.com
www.polldaddy.com
8
weSPOT Project - IST (FP7/2007-2013) under grant agreement N° 318499
5
weSPOT Project – D5.1 Diagnostic instrument
6
2.10 Lower Cost Integrated Solutions
There are a number of inexpensive solutions that bundle in additional features outside of surveys and
polls. However, the survey functionality within these integrated tools tends be fairly basic, and they’re
unlikely to meet the needs for advanced survey logic or analysis features.
•
Constant Contact
Although known primarily as an email marketing tool, Constant Contact9 offers “Listen-Up,” a
hosted survey tool with some interesting benefits. There’s no free option, but fee-based packages
are competitive with other lower-cost options, ranging from about $10/month to $150/month,
depending on the number of respondents who will be answering surveys. Constant Contact
offers more than 40 predesigned templates with some ability to customize. Surveys can include
skip logic, and can be scheduled in advance. The tool also offers a variety of emailing and email
management services, including contact importing, list segmentation and basic contact
management. Reports are quite basic, but data can be exported for analysis in another tool.
•
FormSite
FormSite10 offers a tool for building website forms—everything from simple “contact us” forms to
evaluation forms. Although the focus is forms for feedback and test-taking, FormSite offers a
basic set of survey features, and may be useful to those looking to collect a lot of different types
of information via web-forms. Features include multiple page surveys, question randomization,
basic skip logic and piping. Surveys can be customized to match your website by someone
familiar with HTML. They offer a variety of prices, including a limited free account, and packages
ranging from $10/month to $100/month.
•
Moodle
Moodle11 is a powerful open source course management software package primarily targeted at
schools and universities. It integrates website content management and online course
management with survey and test-taking solutions. The survey tools are geared toward those
gathering feedback from students to assess teaching methods, and several verified survey
instruments are provided for this purpose. Moodle may be appropriate for nonprofits running
training programs or schools seeking an all-in-one website, course and survey management
solution for their programs. The package is free to download, but will require substantial
technical knowledge to install, configure, customize and support.
2.10 More Advanced Survey Packages
For larger-scale research, marketing and feedback analysis projects, a more powerful survey package
could provide welcome functionality. These tools offer significantly more advanced question formats,
survey logic and data analysis. The more complex functionality makes them more difficult to use without
training, especially for those without prior survey design expertise.
•
Qualtrics
Qualtrics12 provides advanced survey logic and analysis targeted at research surveys, with a focus
9
www.constantcontact.com
www.formsite.com
11
www.moodle.org
12
www.qualtrics.com
10
weSPOT Project - IST (FP7/2007-2013) under grant agreement N° 318499
6
weSPOT Project – D5.1 Diagnostic instrument
7
on academic institutions. The Qualtrics Research Suite provides data analysis support for crosstabulation, conjoint analysis, subgroup analysis, time series analysis, and more.
A free account includes:
o
o
o
o
Unlimited surveys (1 active at a time)
250 completed responses
100 outgoing emails
Qualtrics University online support
•
LimeSurvey
LimeSurvey13 is a powerful, free and open source survey package appropriate for nonprofits
looking for advanced survey logic and analysis features and who have substantial technology
support. Its range of features includes full customization of survey look and feel, support for 40
different languages, piping, skip logic, a library of available survey questions and blast emailing.
The tool has a large support community and is under active development. This is an open source
package that can be downloaded for free, installed on your own web server, and customized to
your needs by a developer with knowledge of PHP/ MySQL. Although the tool itself is free, be
prepared to bear the costs for your own web hosting, and the time it takes to properly install,
configure, customize and support this product on your own.
•
Key Survey
Key Survey14 is the most robust of the advanced survey tools covered in this article. Prices range
from $1,950 to $5,950/year for single-user subscriptions. It offers a full set of features, including
several unusually advanced ones such as LDAP integration to allow single-sign-on models for
large organizations, role based permissions, support for “teacher/student” surveys, multimedia
questions and much more. Key Survey also offers an API to exchange survey data with outside
applications, with a module for Salesforce.com integration.
QuestionPro
Main advantages
Intuitive interface
FreeOnlineSurveys
Intuitive interface
SurveyGizmo
• Unlimited Surveys
• 50 Responses per month, per
project
• A Single Summary Report for each
project
• 100s of Question Type Variations
• Access to Support
• Generate Test Data
Ease of use, comprehensive features,
powerful reporting, and responsive
24/7 support
Main drawback
The free version offers reporting online
only, no possibility to download data
The free version offers reporting online
only, no possibility to download data
No:
• Unlimited Responses per Month
• Create Custom Scripting
• Send Email Invitations
• Blog and Website Embeds
• Create Tally & Pass/Fail Quizzes
• Advanced Skip Logic & Branching
Features
The free account provides only 10
surveys, 50 questions and 100 responses
per month.
Unlimited number of surveys &
forms, starting from scratch or
The basic, free account provides:
• Unlimited Surveys
SoGoSurvey
FluidSurveys
13
14
www.limesurvey.org
www.keysurvey.com
weSPOT Project - IST (FP7/2007-2013) under grant agreement N° 318499
7
weSPOT Project – D5.1 Diagnostic instrument
SurveyMonkey
Zoomerang
PollDaddy
Google Forms
making use of expert designed
survey templates. A simple drag &
drop editor gives access to over 25
question types, skipping, branding
and more.
Online hosted survey tool that works
well for basic surveys. The free
version might be useful for very
small and informal surveys.
Similar to SurveyMonkey in many
respects
Surveys and polls can be easily
embedded into external websites
and applications.
It provides a flexible form and survey
development interface with built-in
reporting. Survey data is far from being
difficult, nor costly, to store.
8
• 2only 0 Questions Per Survey
• 150 Responses per Survey
• Ready Made Templates
• Basic Reports
Very little customization of the look of
the survey, no downloads of reports or
data, and can only collect 10 questions
and 100 responses per survey.
Very limited free package
The free package offers a maximum of
10 questions per survey and 100
responses per month, plus basic
reporting. More advanced versions cost
between $200/year and $900/year.
Survey features are more limited than
some of the other options in this
category, with no skip logic or piping,
but survey administrators have a lot of
flexibility over the look of the survey.
Essential GUI.
Table 1: Summary of tools for questionnaire, their main advantages and drawbacks
3. 1st weSPOT Diagnostic Tool: set-up of questionnaire
3.1
Requirements
The approximate total number of students for all test beds is 730 students, this is also the maximum
estimated number of students accessing the questionnaire.
The number of users accessing the questionnaire at the same time depends on the formation of groups
of students which will be done after the schools open (after September 2013), as all pilot partners have
stated. Before September groups of students cannot be formed.
According to the evaluation framework the number of students in each group should be minimum 15
and maximum 60. On average, each group should be one classroom so this is max 30 students for
schools. As such an approximate number of 30 students may use the questionnaire in the same time in
each test-bed (for schools). For Universities where classrooms might be composed of more than 30
students, I believe that a max of 50 students could be in each group.
There is a possibility that 2 groups in each test-bed work on the questionnaire the same time.
It is rather unlikely that 2 or more groups from different testbeds work on the questionnaire in the same
time.
3.2
Set-up
In the end we decided to set up a Google form. This appeared the best solution in terms of:
• affordance (no costs),
weSPOT Project - IST (FP7/2007-2013) under grant agreement N° 318499
8
weSPOT Project – D5.1 Diagnostic instrument
9
supported number of users and questions/answers (no limits/restrictions/constraints, while
other softwares require to count total/concurrent users and/or number of answers per
month/questionnaire),
• availability of basic reports (tables with diagrams and statistics are available, while other tools
sometimes don’t provide reports or have limitations over them).
In addition to these features, a basic administrator’s interface is also provided for the creation of the
questionnaire. Even if quite skinny, it provides in an intuitive way many basic facilities for the creation of
the questions and a few graphical options.
The 1sst set up resulted in the questionnaire at:
•
http://tinyurl.com/qe7st6a
Figure 1and Figure 2 below present respectively a screenshot of a questionnaire page and a sample of
available reporting information and display.
Figure 1: Online questionnaire for Diagnostic Tool
weSPOT Project - IST (FP7/2007-2013) under grant agreement N° 318499
9
weSPOT Project – D5.1 Diagnostic instrument
10
Figure 2: Sample reporting
4 Conclusions and Outlook
As agreed within the consortium, we set up an online questionnaire as 1st version of the Diagnostic Tool.
We will collect feedback about the questionnaire content and supporting tool in the next period, such
feedback will go respectively to WP2 for the theoretical part and to WP5 for deciding if keeping the
Google Form or moving to another software platform and/or how to change the questionnaire to
enhance usability, if needed. After validation of the first questionnaire in English also localization is
planned.
Once enough questionnaire answers are collected to progress with the theoretical definitions and
diagnostic indicators are specified and described, we will be able to move to the development of further
software prototypes of the Diagnostic Tool.
weSPOT Project - IST (FP7/2007-2013) under grant agreement N° 318499
10
weSPOT Project – D5.1 Diagnostic instrument
11
Annex 1 Test items for online questionnaire
IBL Diagnostic Questionnaire-Draft
Instructions: Please examine all options before responding; select the single best answer.
Thank you.
0.
Context
Scientific ideas are based on reason and evidence, and not on guesses, hunches, or insights. This
statement is
____ because ____.
a) true – scientists only deal with facts and avoid bias due to guesses, hunches, and insights.
b) true – scientists always follow the “Scientific Method” and it does not allow for guesses,
hunches, or insights.
c) false – scientists only deal with theories that are subject to error.
d) false – reason, evidence, and creativity are all important for the creation of scientific ideas.
1. Question/hypothesis
A scientific hypothesis is best defined as a:
a) well tested explanation supported by lots of evidence from experiments
b) proposed explanation that incorporates available evidence
c) statement of the relationship between two variables
d) prediction of what will happen
1.1 Embedding
Scientists should ______________ the work of other scientists, especially if that work will serve as
the basis for other scientists’ future efforts.
a) review and check
b) read
c) do nothing
1.2 Existing knowledge
Scientists with good existing knowledge of an area can identify more accurately what needs to
be tested.
a)True
b)False
1.3 Mental representation
Scientists who have a clear mental image/representation of their hypothesis can proceed with
their inquiry because________________
a) they are more confident
b) they have a better idea of what needs to be done
c)they could guess the expected outcomes
d)they could draw their approach
1.4 Language/definitions
weSPOT Project - IST (FP7/2007-2013) under grant agreement N° 318499
11
weSPOT Project – D5.1 Diagnostic instrument
12
Do scientists need to consider the differences in language/definitions in the field before they
start their inquiry?
a)Yes, because differences in definitions can indicate different approaches
b)Yes, because different definitions indicate problems
c)No, because all definitions are the same
d)No, because different definitions just reveal different language understanding
1.5 Field of research
Scientists usually do a standalone research that does not need to define the field/domain that their
research will take place?
a)True, because research fields are unrelated
b)True, because they know the domain very well
c)False, because the definition of the field will help them to place their research into the current work
d)False, because the definition of the field will help them conduct better research
1.6 Ethics
The research hypothesis for a study of a drug on hypertension requires the administration of the drug
at different doses to 50 laboratory mice, with chemical and behavioural tests to determine toxic
effects. Do scientists involved in the study need to think about ethics at this early stage?
a)Yes, because it involves animals
b)Yes, because it involves laboratory experiments
b)No, since it does not involve human participants
b)No because it is too early
1.7 Empirical meaning
Empirical research is based on ______________________
a) personal and social beliefs
b) observed and measured phenomena
c)theories
d)scientific guesses
1.8 Reflection
Reflection is a process that begins with looking back on a situation, evaluating it, learning from it and
then using the new knowledge to help you in future similar situations. Reflection at this stage can
help scientists’ to______________________
a)have better results
b)identify mistakes
c)make better analysis of their data
2. Operationalisation (realisation of idea with the aim to measure)
Scientists think of scientific knowledge as:
a) beliefs supported by repeatable, observable evidence
b) unchanging eternal truths
c) assumptions about the world
d) self-evident statements about the world
2.1 Indicators
When running experiments usually scientists do need to_____________activities.
a)only observe
weSPOT Project - IST (FP7/2007-2013) under grant agreement N° 318499
12
weSPOT Project – D5.1 Diagnostic instrument
13
b) measure
c)describe
2.2 Predictions à Which of the following is the alternate hypothesis:
a) Ho: μ = 5
b) Ha: μ ≠ 5
c) Ha ≠ Ho
2.3 Resources
A scientist needs to have a clear idea about the recourses (e.g. library, textbooks, technology, tools
etc.) he/she needs to perform his/her inquiry because that will help him to ____________
a)find the information he needs quicker and more efficiently
b)feel better and reassured about his inquiry
c)calculate how many resources he will need
2.4 Methodology (of data collection and processing)
A scientist needs to run an experiment to find out the right dosage for a new drug against fever. The
method he will use to run his experiment is _______________
a)Qualitative method
b)Quantitative method
c)Mixed research methods
2.5 Ethics (Ethical issues)
The research hypothesis for a study of a drug on hypertension requires the administration of the
drug at different doses to 50 laboratory mice, with chemical and behavioural tests to determine toxic
effects. Do scientists involved in the study need to think about ethics at the operationalisation stage?
a)Yes, because ethics are important throughout the process
b)Yes, because it involves laboratory experiments
b)No, since they only concerned with the methodology and measurement
b)No because they considered at an earlier stage
2.6 Reflection
Reflection is a process that begins with looking back on a situation, evaluating it, learning from it and
then using the new knowledge to help you in future similar situations. Reflection at this stage can
help scientists’ to______________________
a)have better results
b)identify mistakes
c)make better analysis of their data
3.
Data collection
Scientists think of scientific knowledge as:
a) beliefs supported by repeatable, observable evidence
b) unchanging eternal truths
c) assumptions about the world
d) self-evident statements about the world
3.1 Information foraging
weSPOT Project - IST (FP7/2007-2013) under grant agreement N° 318499
13
weSPOT Project – D5.1 Diagnostic instrument
14
When good scientists using a book, article, report, or Web site for your research, it is important to
judge how reliable the source is.
a)True
b)False
3.2 Systematic observation
When scientists do systematic observation they ______________________
a) evaluate activities and performance
b)only watch activities and performance
c) measure any activities or performance
3.3 Experimentation
Scientific knowledge is usually not based only upon experiments. An experiment usually tests
____________
a) facts
b) a hypothesis
c)personal beliefs
3.4 Tools
Scientists use a wide variety of tools to help them learn about the world around us. What tool will
help a scientist to see microorganisms?
a)lens
b)microscope
c)telescope
3.5 Simulation
Scientists cannot run experiments on computers.
a)True
b)False
3.6 Data storage
3.7 Data security
3.8 Documentation
For all three (Data storage, Data security, Documentation)
Scientists working at public institutions share certain values such as careful record keeping,
making accurate measurements, and reporting procedures, data and results accurately. This
statement is ____ because scientists ____.
a) false – hide their procedures and data from other scientists
b) false – don’t have a common set of values
c) false – don’t follow the “Scientific Method”
d) true – generally want others to be able to repeat their observations and experiments
3.9 Classification
After collecting data scientists usually classify their data, which involves_______________
a) organise their data into groups
b)storing their data safely
c)encrypting their data
3.10
Reflection
weSPOT Project - IST (FP7/2007-2013) under grant agreement N° 318499
14
weSPOT Project – D5.1 Diagnostic instrument
15
Reflection is a process that begins with looking back on a situation, evaluating it, learning from it
and then using the new knowledge to help you in future similar situations.
a)True
b)False
4. Data Analysis (processing)
4.1 Quantitative analysis (Statistical methods/analysis)
A scientist wishes to conduct an experiment to determine the relationship between distance, speed,
and time using one or more motorized toy cars. Which would be the best way to proceed?
a) Observe the motion of several motorized cars going different speeds and make a graph – plotting
distances and speeds on one axis and times on the other.
b) Use one car with a fixed speed. Measure distances at various times. Make a graph of distance
versus time.
Do this again with cars of different fixed speeds.
c) Take the speed of all cars and average them, and divide this average by time.
d) Use the formula distance = speed * time
4.2 Qualitative analysis
A scientist wishes to determine how students perceive their teachers. Which would be the best way
to proceed?
a) Observe students
b) Record grades
c) Conduct interviews
d) Measure temperature
4.3 Tools
Scientists need to be certain about the tools they will use to collect their data.
a)Always
b)Never
c)Sometimes
4.4 Visualisation
Scientists are using graphs to present their data because they want __________________
a) to simplify data values and promote understanding
b) to improve the appearance of their documents
c) to show to the fellow scientists that their work is professional
d) to save time
4.5 Noise reduction (of data)
Noisy data is meaningless data. Therefore scientists by removing corrupted or uncompleted data can
_____________________
a) save time in their analysis
b)improve their results
c)simplify data analysis
d) enhance data analysis
4.6 Reflection
Scientists do not usually reconsider their actions at this stage since they have done it earlier in the
process.
a)True
b)False
weSPOT Project - IST (FP7/2007-2013) under grant agreement N° 318499
15
weSPOT Project – D5.1 Diagnostic instrument
16
5. Interpretation/discussion
5.1 Embedding (Embedding into existing theories/results/domain knowledge (classification))
When scientists judge evidence, they make reference to what they already know about
_______________
a) the world around them
b) their personal beliefs about the world
c)popular beliefs about the world
5.2 Confirmation/falsification (of the initial question/hypothesis)
After making observations or conducting an experiment, a good scientist will report all evidence,
including valid evidence that contradicts the expected outcome.
a) True
b) False
5.3 Significance (statistical)
When researchers say a result is "highly significant" they mean it is _______________.
a) always true
b) very probably true
c)never true
d)sometimes true
5.4 Relevance (of the results)
Well established scientific conclusions will generally remain unchanged with the passage of time, but
are subject to change in the light of new evidence. This statement is ____, because ____.
a)true – science is composed of theories that have a high probability of being wrong.
b) true – scientific conclusions might change when new contradictory evidence is found.
c) false – once scientists make scientific conclusions, these conclusions can and will never change in
the future because laws of the universe are always and everywhere the same.
d) false – science is the search for truth, and truth never changes.
5.5 Threshold
Threshold is statistical concept and it is expresses as ____________.
a)a value
b)text
c)a hypothesis
d)a theory
5.6 Exhaustion
Jean has a hypothesis about how a certain chemical reaction occurs. Based on her hypothesis, she
predicts that mixing chemical A with chemical B will produce chemical C. She mixes A with B and the
result is mostly chemical C, but chemical D is also produced in a small amount. Which of the following
conclusions may Jean properly draw about her hypothesis?
a) her hypothesis has been proven correct
b) her hypothesis appears to be supported, but needs to be refined
c) her hypothesis is most likely completely wrong
d) she cannot draw any conclusion from the results
weSPOT Project - IST (FP7/2007-2013) under grant agreement N° 318499
16
weSPOT Project – D5.1 Diagnostic instrument
17
5.7 Reflection
When scientists interpret their results do not usually reflect upon them.
a)True, since they have done it before
b)True, because it is too late to change anything
c)False, because they can still identify and note errors and problems
d)False, because they can identify the errors and exclude them
6 Communication
6.1 Strategy
Effective communication requires effective strategy and a coherent plan of action. Therefore a
scientist should _____________________
a)put in place a strategy to communicate his/hers work in the most effective way
b)use the most convenient way to communicate his/hers work
c)not bother, because the most important thing is his research and not its communication
d)choose the most scientific way to communicate his results
6.2 Audience
When presenting scientific work, scientists should consider their audience and adjust their
presentations accordingly.
a)True
b)False
6.3 Tools
A good scientist would not consider many different tools to present his/her work because that will
make his message accessible to wider audience and his main audience is his fellow scientists.
a)True
b)False
6.4 Dissemination (Events/Presentation/Publication)
After making observations or conducting an experiment, a good scientist should communicate the
results using__________________.
a) many different events
b) one main event
c)only private events
d)only public events
6.5 Discussion
After making observations or conducting an experiment, a good scientist will report, discuss and
evaluate all evidence, ______________________________________.
a) understating valid evidence that contradicts the expected outcome
b) excluding valid evidence that contradicts the expected outcome
c) including valid evidence that contradicts the expected outcome
6.6 Feedback (Receiving and reacting)
After disseminating the results, a good scientist should be open to positive and negative feedback.
a)Always
b)Never
weSPOT Project - IST (FP7/2007-2013) under grant agreement N° 318499
17
weSPOT Project – D5.1 Diagnostic instrument
18
c)Sometimes
6.7 Writing up
After finishing the analysis of the results, a good scientist usually produces __________.
a) a video
b) a summary of his results
c) a detailed report
d) a power point presentation
6.8 Reflection
At the communication stage of their research, scientists usually do_______________________.
a) not reflect because they cannot change anything
b) not reflect because they are sure about their work
c) reflect because they learn from it and then use the new knowledge to help them in future
similar situations
d) reflect because they are uncertain about their work
Answers
0. d
1. b
1.1 a
1.2 a
1.3 b
1.4 a
1.5 c
1.6 a
1.7 b
1.8 b
2. a
2.1 b
2.2 b
2.3 a
2.4 b
2.5 a
2.6 b
3.
a
3.1
3.2
3.3
3.4
3.5
3.6
3.7
3.8
a
c
b
b
b
d
d
d
weSPOT Project - IST (FP7/2007-2013) under grant agreement N° 318499
18
weSPOT Project – D5.1 Diagnostic instrument
19
3.9 b
3.10 a
4. Data Analysis (processing)
4.1 b
4.2 c
4.3 a
4.4 a
4.5 d
4.6 b
5. Interpretation/discussion
5.1 a
5.2 a
5.3 b
5.4 b
5.5 a
5.6 b
5.7 c
6. Communication
6.1 b
6.2 a
6.3 b
6.4 a
6.5 c
6.6 a
6.7 c
6.8 c
weSPOT Project - IST (FP7/2007-2013) under grant agreement N° 318499
19