Download PeLe user manual for teachers

LLP KA3‐ICT Prroject 2011‐13 511485‐LLP‐
5
‐1‐2010‐1‐NO
O‐KA3‐KA3M
MP
Do
one--IT
Devellop of open systems
s
servvices for smaartphones that facilitates new evaluaation methodds, and
enhannces use of im
mmediate feeedback on ev
valuation ressults obtained
d in tests as a creative leaarning
tool.
WP
P 8: De
eveloping eva
aluation
n syste
em for 4
Op
perative
e Syste
em platforms
s for
Sm
martpho
ones
D8
8.1: Do
one-IT User
U
M
Manuals
s – Tea
acher Client
C
Author and editor:
Co-Authors:
C
Version:
Date:
S
Start
monthh
End monthh
Pack
kage leaderr
L
Language of
o the reportt
T.
T M. Thorsseth
R.P.
R Pein, M.
M Uran, S. Keseling, B.
B Voaidass, R.
Støckert,
S
B.. Zorc, M. Jovanović,
J
A. Koveš
Final
F
31.03.2013
1
27
In
nstitut za Varilstvo
V
(IIzV), Ljubljjana, Sloveenia
English
E
This project has been fu
unded with su
upport from the
t
European Commission.. This publication
mmission cann
not be held responsible for any
reflects the views only off the authors,, and the Com
e of the inform
mation contain
ned therein.
use, which may be made
Done-IT
LLP-Project Nr. 511485-LLP-1-2010-1-NO-KA3-KA3MP
This document may not be copied, reproduced, or modified in whole or in part for any purpose without written permission from the Done‐IT consortium. In addition to such written permission to copy, reproduce, or modify this document in whole or part, an acknowledgement of the authors of the document and all applicable portions of the copyright notice must be clearly referenced. All rights reserved. This document may change without notice, but consortium members should be informed, and number of version, stage and date should be given. Project consortium 





Sør‐Trøndelag University College, Faculty of Technology, Trondheim, Norway Centrum for Flexible Learning, Söderhamn, Sweden Petru Maior University of Targu‐Mures, Tirgu Mures, Romania Magyar Hegesztéstechnikai és Anyagvizsgálati Egyesülés (MHtE), Budapest,Hungaria Institut za Varilstvo (IzV), Ljubljana, Slovenia HiST Contract Research, Trondheim, Norway 2 Done-IT
LLP-Project Nr. 511485-LLP-1-2010-1-NO-KA3-KA3MP
PeLe User Manual
Teacher Client
3 Done-IT
L
LLP-Project
Nr.
Nr 511485-LLP-1-2010-1-NO--KA3-KA3MP
Peerr Learning A
Assessmentt System forr mobile devvices PeeLe Pe
eLe
e Use
U er Ma
Manu
ual
Tea
ache
er Cliient
4
4 Done-IT
LLP-Project Nr. 511485-LLP-1-2010-1-NO-KA3-KA3MP
Table of Contents
Done-IT ................................................................................................................ 1 Overview .............................................................................................................. 7 Requirements ...................................................................................................... 8 Access to WiFi Network .................................................................................................. 8 Running on Windows ..................................................................................................... 8 Running on Mac and Linux ............................................................................................. 9 Starting up and log in ......................................................................................... 9 Getting access .............................................................................................................. 12 General information .......................................................................................... 13 Create an assessment in Pele ......................................................................... 14 Create a new assessment ............................................................................................. 14 Assessment metadata .................................................................................................. 15 Assessment size ............................................................................................................ 16 Rules ............................................................................................................................. 16 Editing the assessment ................................................................................................. 17 Save assessment setup to file ...................................................................................... 20 Uploading an assessment to the server ......................................................... 20 Monitoring of assessments in Pele ................................................................. 22 Ending the assessment .................................................................................... 25 In class ............................................................................................................... 26 Running a Post Assessment Activity (PoAA) ................................................................. 27 Choice of method guide ............................................................................................... 27 Selecting questions to bring in Class ............................................................................ 27 In class and display of results ......................................................................... 28 The percentage chart ................................................................................................... 29 The histogram ............................................................................................................... 30 Sending back the question as SRS ................................................................................ 31 Working with the play controller 31 Adjusting timer time. 32 5 Done-IT
LLP-Project Nr. 511485-LLP-1-2010-1-NO-KA3-KA3MP
Ad‐hoc SRS .................................................................................................................... 33 Results ............................................................................................................... 34 Appendix A Simulating an Assessment ......................................................... 35 Appendix B Marks in questions ...................................................................... 38 6 Done-IT
L
LLP-Project
Nr.
Nr 511485-LLP-1-2010-1-NO--KA3-KA3MP
Ove
erview
w
Pele is a weeb based Peerr Learning Assessment
A
S
System
for mobile
m
devices
(Pele), desiggned to makee the assessm
ments and areena for learniing.
A
sssessments aree normally given
g
perioddically to deteermine at a particular po
oint in time
what
w students know and don’t
d
know. A
Assessmentss are used to provide the teacher and
th
he students with
w feedbackk. Quite ofteen the results from a test, is given 1-2 weeks after
th
he performan
nce, and the information back to the student mayy be everythiing from ok
or not, app
proved or no
ot, a grade orr a solution. These
T
kinds of assessment are normaally referred
to as summ
mative assesssments. In contrast, forrmative asseessments arre assessmen
nts done as a
part of the instructionall process. While
W
summattive assessmeents are assocciated with standardized
s
e
the formative
f
tests are desiggned to givee students feedback
f
andd guide the
tests and exams,
students to
owards the learning
l
goaals. Although
h formative assessmentt strategies appear
a
in a
variety of formats, th
here are some distinct ways to distinguish
d
t
them
from summative
ment is that students
s
are involved
i
as assessors
a
of
assessmentts. One criterrion for a forrmal assessm
their own learning and as resourcess to other stuudents. Pele allows
a
runnin
ng both sum
mmative and
formative tests.
t
Figure 1 Todays challenge, large
l
groups off students in a huge
h
hall, takingg their final exaam. Are the reesults from the
exam the onlyy feedback theyy get during the course?
In Pele youu don’t havee to enter th
he questionss into the syystem. It is assumed thaat you have
created an assessmentt on paper, where therre are multip
ple alternativ
ives for each question.
Handing ouut a paper with
w the task makes this p
possible with
hout any lim
mitations of the
t question
by the systtem. You can
n then desiggn the multip
ple choice teest as you want,
w
using yo
our favorite
editor and include eqquations, graaphics or syymbols at your
y
own will.
w You disstribute the
assessmentt as a task on
o paper andd the studen
nts work witth the probleems as usuaal. The only
i that your students
s
subm
mit the resultts digitally. This
T allows yo
ou to start working
w
with
difference is
providing feedback
f
on the
t assessmeent immediattely after the assessment.
7
7 Done-IT
LLP-Project Nr. 511485-LLP-1-2010-1-NO-KA3-KA3MP
Pele has been designed to allow students to respond to tests electronically and hence allow the
teacher to see what happens during the test and prepare for a Post Assessment Activity (PAA).
When the students have given their response, the teacher can start to explain and use the
students’ results immediately. If a lot of students have struggled or there are many
misunderstandings, the system has a build in Student Response System (SRS) that can be
applied to give a second chance, create a group discussion or engage the students in a new
response after giving some hints. The system has been designed as a tool to allow working with
the problems addressed in the assessment, immediately after the when the problem is still fresh
in mind. How the system is used is dependent upon the teacher and what he/she wants achieve.
Requirements
Access to WiFi Network
In order to be able to run Pele in the classroom you need a good coverage of a stable WiFi
Network in the room. Students can log in using most kinds of web enabled devices like, iPod,
iPad, iPhone, Android phones or computers. The only requirement is that the web browser is a
webkit1 based web browser. You find a short list of browsers in Table 1 below.
Table 1 The most common webkit enabled browser that we have tested. A more extensive list can be found
online.2
Computer
PC
Browser
Google Chrome
Safari
Status


Mac
Safari

iOS
Android
Safari
Dolphin


Running on Windows
The system had been developed and tested mainly on windows. However, the air application
can run on other operating systems that support the Adobe Air runtime.
We have intentionally kept the implemented application at a compatibility level that coincides
with the last supported version for Linux. The support for Adobe Air on Linux has been
dropped by Adobe in favor of mobile platforms, but they kept the support for Mac OS.
1
http://en.wikipedia.org/wiki/WebKit
2
http://en.wikipedia.org/wiki/List_of_web_browsers#WebKit-based
8 Done-IT
L
LLP-Project
Nr.
Nr 511485-LLP-1-2010-1-NO--KA3-KA3MP
Running on Mac and Linu
ux
The system
m needs furth
her testing to optimally ruun on Mac OS
O and Linuxx. Even if thee runtime at
the base iss smoothingg out the diffferences between the platforms
p
there are still issues with
certain featuures.
In some in
nstances we have encouuntered pro
oblems with the way traansparent w
windows are
treated. Mo
oreover theree are differen
nces between
n the operatiing systems in
i the way th
hey treat the
always on top
t windowss.
Starrting
g up and
a
log in
i
When Pele has been insstalled you sstart by doub
ble clicking th
he Pele icon.. Then the ap
pplication starts and d
displays the login panel. By default th
he login paneel will show no connectio
on information
n. After the first log in, tthe information entered last time and
d that it has been used successfullyy will be prelloaded in thee login form.. There are ttwo way to eenter the con
nnection information
n. First way is the manuaal entry of the server details, suitablee when you h
have been provided w
with these at registration.. Second app
proach is to u
use usernam
me@institutio
on_id in the usernam
me box. The available insstitutions will be shown in an autoco
omplete list tthat will allow the user to avoid any typos. TThe application will autom
matically seeek and fetch the connection
n information
n for the giveen institution
n id using thee designated
d zero configuration directory. Here you haave to give yo
ou usernamee and password and presss ok. Figure 2 Thee login Panel, where
w
the user has to providee username and
d password in order
o
to accesss the server.
You can ch
hose to “Rem
member password” and the system will
w rememb
ber till the neext login on
the same co
omputer. Th
he username and
a server co
onfiguration will automattically be rem
membered.
9
9 Done-IT
L
LLP-Project
Nr.
Nr 511485-LLP-1-2010-1-NO--KA3-KA3MP
If there forr any reason is an error, either that you
y usernam
me or passwo
ord is wrong,, the system
will notify you with a sound andd message in
n the login panel. Check your useername and
a
password again.
Figure 3 Userrname or the associated
a
passsword is not acccepted.
If you havee to change the
t server, click on “Sho
ow details” and
a the login
n panel will expand
e
with
the addition
nal server setttings, and yo
ou can changge the server and the path
h to the serveer.
If you inserrt an invalid web
w address to the serveer, the system
m will notify you
y that the server can’t
be reached..
1
10 Done-IT
L
LLP-Project
Nr.
Nr 511485-LLP-1-2010-1-NO--KA3-KA3MP
If the serviice address iss wrong, butt the server responds
r
witth a messagee that the serrvice path is
not accessib
ble. You theen have to ch
hange the setttings to corrrect settings.
1
11 Done-IT
L
LLP-Project
Nr.
Nr 511485-LLP-1-2010-1-NO--KA3-KA3MP
Gettin
ng access
The users can
c register using
u
either th
he mobile cllient or the portal
p
page. These
T
are avaailable at the
desired servver (e.g. http://aft-test.hiist.no/Hist/)). After regisstration an addministrator must assign
the teacher role to the n
newly registerred user in order
o
to give him the teaccher rights.
Figure 4 Reg
gistration form from the mobiile app – use th
he link to the app
a
1
12 Done-IT
L
LLP-Project
Nr.
Nr 511485-LLP-1-2010-1-NO--KA3-KA3MP
Figure 5 Reg
gistration form from the portaal - use the link
k to the registrration
Gen
neral info
orma
ation
When you have been granted
g
acceess to the syystem, you will
w be providded with thee main Pele
T is a men
nu bar that displays thee normal lineear flow in th
he way you will use the
interface. This
system. The black butto
ons will chan
nge and a po
ointer will po
oint at the lo
ocation wheree you are in
the workflo
ow. The flow
w is as follow
ws: When youu have loggedd in, you are in
i the Login area.
Figure 6 show
ws the menu bar with session
n code placeho
older and comm
munication staatus area.
Then you move right to the assesssment wherre you can create
c
a new
w assessmentt. When the
assessmentt is created, you start and move to monitoriing of the assessment. When the
assessmenti
tis finished, it
i has to be ended, and when
w
you arre ready youu go “in Classs” with the
1
13 Done-IT
L
LLP-Project
Nr.
Nr 511485-LLP-1-2010-1-NO--KA3-KA3MP
results and start workin
ng through th
he results witth the studen
nts. Then youu have an Addministrator
n Exit button
n.
area and an
Above the menu butto
ons, you findd two light arreas, used to
o display the session codee to the left
y have uploaded you assessment
a
to
o the server,
and commuunication staatus to the rigght. When you
you get a seession code iin return from
m the server that will be displayed.
d
Below the menu bar you
y have a work
w
area an
nd down in the right corner you fin
nd a control
button. Thiis control buutton will alw
ways stay on top
t if you mo
ove to anoth
her program. If you have
your questiions on a pow
werPoint forr later work in
i the classro
oom, you can
n hide Pele and
a move to
Figure 7 shows the menu system in the Pele client. Th
he tasks are dissplayed in a lin
near manner ass you will work
k through an
assessmentt.
PowerPoin
nt, but the co
ontrol button
n will remain
n on top of you
y program
ms. Pressing itt, will reveal
Pele again.
p
asseessments thaat is stored on
n the server.
On the logiin interface, a list of you previous
Crea
ate an
a as
sses
ssme
ent in
n Pelle
When we create
c
an asseessment in Pele
P we have to setup a document,
d
wh
hich resemblle the actual
assessmentt that you give
g out on paper. We have to feeed the system
m with the number of
questions, alternatives
a
a what alteernative is correct.
and
Create
e a new
w asse
essmen
nt
When you want to creaate a new asssessment, move to Assesssment and cliick Create. Th
hen the first
ment” wizard will
w appear.
step of the “create assessm
1
14 Done-IT
L
LLP-Project
Nr.
Nr 511485-LLP-1-2010-1-NO--KA3-KA3MP
Asses
ssmentt metadata
Figure 8 show
ws step one of the create asseessment wizard
d where you en
nter metadata.
The first sttep is to inseert metadata about the assessment.  The namee of the asseessment, the
Course nam
me , a notte area  where
w
you ass a teacher can
c write wh
hat you wantt, and a last
message th
hat will be displayed
d
on the studentts when theyy start the assessment. Then you
move to the next step.
1
15 Done-IT
L
LLP-Project
Nr.
Nr 511485-LLP-1-2010-1-NO--KA3-KA3MP
Asses
ssmentt size
In the second step you give an apprroximate setti
ting of the assessment sizze.
Figure 9 show
ws step two of the create asseessment wizard
d, where you define
d
the size of
o the assessm
ment.
First you give
g the num
mber of quesstions , byy pulling thee slider to th
he correct po
osition. The
maximum number
n
of qquestions aree 50. Then you
y select thee average nuumber of alteernatives by
pulling the second slideer . You caan later add questions
q
and change thee number off alternatives
on each question individdually. Then
n you select next
n step.
Rules
The system
m will need some rules to
o calculate th
he score from
m the assesssment. In thee third step,
shown in Figure
F
10, yo
ou set the geeneral rules for how thee score are given
g
on gen
neral for the
assessmentt. These ruless can again be
b changed on
o each indivvidual questio
on separatelyy later. Allow
bleeding  is
i an option you can usee if you allow
w more than one correct alternative and
a you use
punishmen
nt (negative score)
s
for wrrong answer. If you don’’t use negativve score, thiss bleeding is
not relevan
nt. In most no
ormal cases you
y can leavve this unselected. Unlimitted choices  aallow you to
allow the sttudents to seelect more th
han one alterrnative on eaach question.. You can alsso leave this
unchecked.. Correct choicee  sets the general scorre for a correect answer. The
T default value
v
is one,
but you can
n change thiss if you wantt to give morre for a correect answer. Wrong
W
choice  gives you
the default score by seelecting a wrrong alternattive. No choicce  allows you to selecct the score
given if no response is given.
g
If youu now press next,
n
we will start editing the created assessment.
a
1
16 Done-IT
L
LLP-Project
Nr.
Nr 511485-LLP-1-2010-1-NO--KA3-KA3MP
Figure 10 sho
ows step tree of create assessm
ment wizard, where
w
you define the general rules.
The final part
p is to givee the weight to the srs quuestions . When
W
the asssessment is done and if
you want to
o you can givve additionall points to th
he responses given by SR
RS. If you don
n’t want the
srs part to count,
c
you will
w set SRS weight
w
to 0.0. If you want to give them
m somethingg you might
give a weigght of 0.5 on
n the srs resuults. The sco
ore from thee srs section is added to the default
assessmentt section and given in the final results..
Editin
ng the assess
a
sment
When you have finisheed the create assessment wizard, you get a created
d assessmentt that needs
work. You have to speecify the corrrect number of alternativves, the correct alternativves and fine
hown in the figure
f
below. The main area
a consists
tune you sccore settings. The overall picture is sh
of two tabss where yo
ou can see yo
our questionss or can cheeck or changge the metadaata. There is
a rules areaa where th
he default rulles that you selected are shown and where
w
you can
c still alter
the values. The question
ns are aliggned vertical in a table wh
here you can
n change each
h individual
Y can app
pend a questtion at the en
nd . The maximum and
a minimum
m score are
question. You
calculated and
a displayed  in the overall scorre area. If yo
ou want to, you
y can also
o simulate a
scenario wiith these setttings and seee how the score distrib
bution looks like if everyyone is just
guessing.. More abouut simulatingg the assessm
ment and thee Monkey caage you will find
f later in
Appendix A.
A
1
17 Done-IT
L
LLP-Project
Nr.
Nr 511485-LLP-1-2010-1-NO--KA3-KA3MP
Figure 11 sho
ows the main areas
a
of the editt assessment in
nterface
In each queestions you have
h several options
o
that you can use to change th
he setup of th
he question.
Figure 12 sho
ows details of how
h you can ed
dit each individ
dual question.
 shows the
t question number. Th
he correct alternative can
n be selected
d , and you can allow
more than one question
n to be correect. If you neeed to add orr remove an alternative 
, just click
+ or – as you
y want to. You can override
o
the standard rules  and addjust the sco
ore for each
alternative as you want to. Min andd max gives you
y the posssible minimuum and maxiimum score
for this queestion .  displays thee number off choices the students aree allowed to select. You
can delete this
t question
n , by presssing delete. Iff no options are selected you are giveen a warning
.
1
18 Done-IT
L
LLP-Project
Nr.
Nr 511485-LLP-1-2010-1-NO--KA3-KA3MP
Now we caan start speciifying each question
q
and finalize the assessment
a
setup.
s
The quuestions are
arranged seequentially downwards
d
a we can select the rigght alternativve by just ch
and
hecking the
correct alteernative. If questions
q
onee has one co
orrect alternaative c, you check
c
alternaative c. You
can notice how the sco
ore adds up and
a change as
a you select correct alterrnatives. Theen we move
w have altern
native b and d.
d
on to questtion 2 and seelect alternativve d, on 3 we
Figure 13 sho
ow the edit asssessment interface after seleecting correct alternatives on
n tree question
ns. Note how
question 3 haas two correct alternatives.
a
If we unch
heck use stan
ndard rules yo
ou can see th
he score setttings. On quuestion 3, if for
f a reason
alternative d is more co
orrect that b, you can givve the weigh
ht here to 1.55 and the tottal score on
this questio
on can now add
a up to 2.55.
Figure 14 sho
ow details on question
q
tree when
w
we overrule the standard
d rules with new
w score.
If you use negative
n
scorre for wrongg answer this might end up
u to a negatiive score in ttotal, if both
selected altternatives aree wrong. If you want th
he score to be
b negative from
f
this quuestion, you
have to cheeck bleed.
Figure 15 sho
ow details wheere we have seelected negativve scores on wrong
w
alternatiives, and wherre bleeding is
allowed. Notice how the miinimum score on this questio
on can now be negative.
1
19 Done-IT
L
LLP-Project
Nr.
Nr 511485-LLP-1-2010-1-NO--KA3-KA3MP
By default, you studentts will not bee able to selecct more than
n the correctt number of alternatives.
nts to be ablle to select as many as po
ossible, up to
o all, you can
n do that by
If you want your studen
he Unlimited choices.
c
This will
w give the students thee possibility to
t select them
m all. Think
checking th
carefully beefore you allo
ow this one and
a considerr using the neegative score.
Save assess
a
sment setup to file
You can save
s
the assessment by pressing saave button in
i the main navigation bar, below
Assessment, so you can
n reload thee assessment document later.
l
Notice that the savve button is
disabled if you
y have no assessment document
d
seetup in memory.
Figure 16 sho
ows the menu with
w the save button
b
under Assessment.
A
When yourr assessmentt has at least one correctt alternative for each queestion, the fiinish button
lights up an
nd you can run
r the assesssment by pressing finish
h. When youu are finished, press the
finish butto
on and the ap
pplication wiill move startt uploading th
he assessmen
nt to the servver.
Uplo
oadin
ng an assess
smen
nt to
o the
serv
ver
When yourr assessmentt document is
i set up andd the system has no indiccation about an obvious
fault in the setup, you are
ar allowed to
o upload the assessment to
t the serverr.
Figure 17 sho
ows the start up
pload assessmeent document to server butto
on.
If you don
n’t have any setup readyy this will no
ot be enabled. If you forget
fo
to set the correct
alternative to
t a question
n this will app
pear on start
2
20 Done-IT
L
LLP-Project
Nr.
Nr 511485-LLP-1-2010-1-NO--KA3-KA3MP
Figure 18 shows the starrt interface if th
he assessment is not complettely set up.
If you try to
o start witho
out any prepaared assessm
ment, you willl get the messsage as show
wn in Figure
19 below.
Figure 19 sho
ows the start paage when no document
d
is preepared.
You then have
h to optio
on to  creatte a new asseessment or  load an asssessment from
m file.
2
21 Done-IT
L
LLP-Project
Nr.
Nr 511485-LLP-1-2010-1-NO--KA3-KA3MP
Mon
nitoriing of
o as
ssess
smen
nts in
Pele
e
When you have uploadded the assesssment to th
he server, yo
ou will be givven an assessment code
that is disp
played on top
p of your scrreen. Then
n the applicaation move to
t the monittoring stage.
Monitoringg is done on individual caandidates andd on individuual questionss in two diffe
ferent views.
The monito
oring matrix shows the sttudent orientted list of thee results send
dt.
Figure 20 sho
ows the monito
or interface witth the retrieved
d session code..
Now you can
c monitor your
y studentts as they loggin and respo
ond to the assessment. Th
he first page
of the mon
nitor interfacee contains a student
s
orien
nted view witth their delivvery. If youu press Show
results summ
mary , you will
w get an overview
o
of the submitteed responsess oriented to
owards each
question.
As studentts log in theey will appeaar in this listt with small dots for eaach question that is not
responded to. As they respond, th
he dots will grow
g
to a sm
mall black sqquare. You can
c turn on
t
squaress by pressingg Show status , and answ
wers that aree correct willl be colored
colors on these
green and wrong
w
answeers will have a red color. By
B default th
he responses are indicatedd as neutral,
since you should
s
not sh
how this to the
t students.. If you keep
p the interfacce neural, yo
ou can show
the login to
o the studentts and give th
hem a feelin
ng that techno
ology is worrking and theey are in the
correct placce, but this iss recommend
ded only in a first demon
nstration. Latter the studen
nts trust the
technologyy, they will no
ot need it. Yo
ou also get a count of ho
ow many stud
dents are logg
gged inn and
a percentagge bar indicatting how man
ny of your sttudents havee submitted th
heir answerss.
Figure 21 sho
ows the studen
nt list with threee students in. Only student 1 has respondeed to the first six
s questions.
2
22 Done-IT
L
LLP-Project
Nr.
Nr 511485-LLP-1-2010-1-NO--KA3-KA3MP
You will be
b able to seee if their resp
ponses are co
orrect or wro
ong by clicking “Show sttatus”. Now
each respon
nse will be colored red or
o green. Wh
hen the resultts are shown
n you can swiitch back to
Neutral view
w of the mo
onitoring mattrix.
If you chosse “Show ressults summarry” the view will change from studen
nt oriented to
o a question
oriented lisst as shown in
i Figure 222 below. Heere you find the question
ns oriented vertically
v

with the nuumber of corrrect, unansw
wered and wrrong results .
 The percentage bar  shows the
percentage correct in grreen, the perrcentage wro
ong in red an
nd the percen
ntage still unaanswered in
n
ofm
marked quesstions is show
wn in a sepaarate column
n  and as a percentage
gray. The number
bar . Youu find more information
i
about markss and how to
o use them in
n Appendix B.
B
Figure 22 sho
ows the questio
on oriented mo
onitoring interfface.
When you click on a question
q
youu immediatelyy get a histo
ogram of thee results for the
t selected
question. This
T
histogrram and th
he monitoriing data will
w update as
a the resuults appear.
2
23 Done-IT
L
LLP-Project
Nr.
Nr 511485-LLP-1-2010-1-NO--KA3-KA3MP
Figure 23 shows
s
the hiistogram  for the seleccted question
n with the taable showingg the option
, the num
mber of studdents who haave selected the
t option ,
 the percen
nt of studentts who have
selected thee option , and
a a flag tellling if the op
ption was sett to be the co
orrect answerr .
Figure 23 sho
ows the monito
ored question with
w the histog
gram for one qu
uestion
When the students havve filled inn all their resuults they havve to submit their respon
nses. To do
h to go an
nd deliberateely submit. Figure
F
24 sho
ows the interrface where the
t number
that, they have
Figure 24 shows the stud
dent oriented list
l when all stu
udents have su
ubmitted their assessment.
a
of studentss that have submitted
s
is displayed. On
O top  yo
ou see the number
n
of sttudents that
have logged in and th
he number of
o students that
t
have suubmitted. Th
he blue bar shows the
2
24 Done-IT
L
LLP-Project
Nr.
Nr 511485-LLP-1-2010-1-NO--KA3-KA3MP
percentage of students that have submitted. You
Y can also see on each
h student, whether
w
they
 By clickiing on the S column you can have tho
ose submitteed sorted on
have submiitted or not .
top or at th
he bottom. When
W
all youu students have
h
submitteed their resp
ponses, it’s tim
me to close
the default section assesssment, and start preparin
ng for the po
ost assessmen
nt activity.
Ending the
t
a
asses
ssme
ent
When all sttudents havee submitted you
y can now
w close the assessment.
a
T do that, move
To
m
to the
end monito
oring stage byy pressing th
he End butto
on as shown in
i Figure 25 below.
Figure 25 sho
ows the menu bar
b and the loccation of the en
nd button.
Then the en
nd button wiill appear witth a warning.
Figure 26 sho
ows the end bu
utton with the warning.
w
Pressing ST
TOP will cllose the asssessment andd no studen
nt will be able
a
to respond to the
assessmentt. A new inteerface with th
he results foccusing on th
he questions will appear as
a shown in
Figure 27 below.
b
Here it fits nice to give the students
s
a sh
hort break so
s you can cconsider the
results.
2
25 Done-IT
L
LLP-Project
Nr.
Nr 511485-LLP-1-2010-1-NO--KA3-KA3MP
Figure 227 shows the en
nd interface aftter the assessm
ment is closed.
Now it’s tim
me to considder the respo
onses that yo
ou students have
h provided and start p
planning the
Post Assesssment Activiity. You havve to select th
hose questions you wantt to bring intto the PAA.
When you have only a few question
ns you can bring
b
them all
a by selectin
ng Select all .
 You can
also select manually  each individdual or you can invert th
he selection . If you want
w to, you
can reopen
n the assessm
ment again an
nd go back to
o monitoringg here. Wheen you selectt one row in
the respon
nses  a hisstogram sho
owing the acctual responsses will appeear . In th
he example
shown Figuure 27 you haave to balancce the time you
y have got with how many
m question
ns you want
to go throuugh. Questio
on one appeears to be un
nderstood an
nd a quick positive
p
can be enough.
Question 2 is answeredd wrong by one
o student, but
b question
n 3 has 50% errors and iss marked by
80% of thee students. You
Y should sp
pend time on
n the last thrree questions. But since the number
What you sh
of question
ns were few,, you could bring
b
them all into the classroom.
c
hould do is
dependent upon whatt you want to achieve with the PAA.
P
As a teacher youu know the
ms the students might haave.
assessmentt and the posssible problem
In class
When yourr results are ready you caan start displlaying resultss and work with
w their deeliveries, it’s
time to gatther the stuudents again. If you wan
nt your studdents to learrn somethingg from this
assessmentt, now is the time
t
to put in the effort.
2
26 Done-IT
LLP-Project Nr. 511485-LLP-1-2010-1-NO-KA3-KA3MP
Running a Post Assessment Activity (PoAA)
When your students have delivered the results on the assessment and you as a teacher have the
overview over their performance, it’s time to start working with their results. You know what
problems they have solved and what problems they haven’t solved. Now you have a window
of time to provide feedback and you can go over the results in a variety of ways, dependent on
the information you have. Below you find a short guide giving you suggestions for how you can
perform a PoAA based on the results you find. The method you chose, depend on your test,
what you are testing and how you have designed your alternatives.
Choice of method guide
If we assume that we have only one correct alternative on all questions, we can start with the
simplest case where all students have given the correct answer. Here you should give a short
positive feedback.
If the results are scattered all over the place, this means that the class does not have a clue and
probably are guessing. You have to consider if the question is bad formulated or anything else
might be wrong of if they do not have any clue.
When you have your results, and you want the do a PoAA with the class you can use the
response system build into Pele.
If the class is divided in two, this might be a good start to a group discussion where they argue
against each other why one is correct over the other.
But it is important that you always give a final verification of what results are correct and why,
and what results are wrong and why. Your explanation is highly valued.
Selecting questions to bring in Class
In the End page, you can select questions that you want to display in class. You can click and
select one at a time  or select them all . If you have only a few questions, I would
recommend that you select all. You can also invert the selection . Notice as you make the
selections, how new buttons appear up on the menu line , where you select each question
from later.
27 Done-IT
L
LLP-Project
Nr.
Nr 511485-LLP-1-2010-1-NO--KA3-KA3MP
Figure 28 Sellecting questio
ons for an In Class
C
sequencee, manually  or selecting alll . The figu
ure also shows
the selected
question
ns appearing in
n the top .
In class and
d disp
play of re
esultts
When you move over tto the in claass page, youu application turns transp
parent . Now you can
display the assessment  as you display
d
resultts. Now wheen you selectt the first quuestion  a
ontrol panel appears . From here you can seleect the form of chart or activity that
question co
you find suuitable for eacch
queestion.
Figure 29 sho
ows the In Claass page with a word documeent containing
g the questionss below . Wh
hen you select
question 1 in
n the menu  the question control panel appears  wh
here you can choose
c
what to
o do with this
queestions.
2
28 Done-IT
L
LLP-Project
Nr.
Nr 511485-LLP-1-2010-1-NO--KA3-KA3MP
Figure 30 sho
ows the questio
on control pan
nel.
The questio
on control panel
p
displayss the main question
q
num
mber . From
m this questiion you can
navigate to
o the previo
ous or next question . For the seelected quesstion you caan display a
percentage chart , a histogram
h
 as a way to
o display thee results or chose
c
not to display any
results yet. You can alsso send out the
t question again after focusing
f
on something
s
th
hat has been
qu
or fo
or any other rreason.
misundersttood in the question
The percenttage ch
hart
Displaying the results as
a a percenttage chart caan be given to
t give some feedback but
b without
ny pattern off what the sttudents have chosen. Thiis chart only shows whatt percentage
revealing an
chart has giiven a correcct response or
 a wrongg response
. The chart also displayss how many
students haave chosen to
t mark this question in a percentagge bar . Th
his bar can be
b used also
from the sttudents to rem
mind the teaacher that theey have markked this question.
2
29 Done-IT
L
LLP-Project
Nr.
Nr 511485-LLP-1-2010-1-NO--KA3-KA3MP
Figure 31 sho
ows the percen
ntage chart with
h percent correect, percent wrong
w
 and marks
m
on thiis question.
The histogra
am
When you select to diisplay the hiistogram of what the sttudents havee chosen, no
o correct or
wrong markkings are disp
played.
Figure 32 sho
ows results from
m a question without
w
any ind
dication of whaat is wrong or correct.
c
3
30 Done-IT
L
LLP-Project
Nr.
Nr 511485-LLP-1-2010-1-NO--KA3-KA3MP
The histoggram can be scaled or moved
m
as youu wish on th
he screen. You
Y move th
he dialog by
dragging th
he panel title  and scale using the scaale handle in
n the lower riight corner .

Using the histogram
h
yo
ou can markk questions as
a correct byy clicking on
n the bar or in the area
where the bar
b should be.
b If no one has selectedd alternative a as in the ab
bove result, you
y can still
select to mark
m this as correct
c
by cliicking in thee area where the bar sho
ould be. Youu can always
remove thee correct marrk by clickingg on the bar aagain.
Figure 33 sho
ows the histogrram where alteernative a) and d) has been marked
m
as correect by clicking in the figure.
Sending bac
ck the questiion as SRS
What happ
pens if you giive students a hint, or if tthey are allow
wed to discuuss the probllem as small
groups or in
i a class disscussion? Yo
ou can send the question
n out as an student resp
ponse single
question aggain. The sysstem has thee question co
onfigured an
nd you only have
h
to presss Send SRS
and the students will haave the altern
natives ready to vote again
n.
If you presss Send SRS the
t play conttroller appearrs on your sccreen
Figure 34 sho
ows the SRS pllay controller.
The SRS syystem has beeen preconfiggured to requuest as manyy alternativess as the questtion you are
working wiith. The onlyy thing you haave to considder is the am
mount of timee given.
Workin
ng with the pla
ay contrroller
The play co
ontroller has many option
ns and we wiill use some time
t
to explaain them. Firrst of all you
can notice how the labeel  displayys how manyy options are requested. Then you haave the play
3
31 Done-IT
L
LLP-Project
Nr.
Nr 511485-LLP-1-2010-1-NO--KA3-KA3MP
button  that
t will sendd out the question and ch
hange to a pause
p
button
n when the vote
v is open.
Pressing paause will pause the timer but your stuudents will bee able to sub
bmit results. If
I you press
Stop  thee vote is clossed and the students
s
will no longer seee the vote paage.
The sound  button alllows you to turn sound from timer on
o or off. Th
he timer buttton  turns
he voting lastt until you cllose it with the
t stop buttton and not
timer on orr off, allows you to let th
the timer. The
T next buutton  lets you display results autom
matically wh
hen the votin
ng is closed.
Automatic results are byy default disaabled. Then
n finally at thee end you haave the chancce to display
r
are submitted
s
byy the studen
nts. In the
results live, showingg the histogrram as the results
n panel  you see the timer tim
me, how man
ny of the registered
r
sttudent have
information
submitted and
a a bar inddicating the percentage
p
off students th
hat have voted. The two numbers
n
a:b
correspond
d to a, the nuumber of studdents that haas logged in to
t you sessio
on and b, thee number of
students that given theirr response.
Adjusting time
er time.
If you rightt click the tim
mer icon ( on a digital blackboard click and ho
old) you get an
a extended
part of the play contro
oller where you are allow
wed to changge the timer time. You then
t
simply
drag the slidder  and cllose the winddow .
Figure 35 sho
ows the timer adjustment
a
pan
nel that appearrs on right-clicck on timer opttion 
When you press play, the
t vote pagge will appeaar on the stuudent’s devicce and the students can
respond to your questio
on.
3
32 Done-IT
L
LLP-Project
Nr.
Nr 511485-LLP-1-2010-1-NO--KA3-KA3MP
Figure 36 sho
ows the play co
ontroller after having
h
pressed
d run. 10 studen
nts are conneccted to the sesssion and none
of them have responded to the question yet.
y
The play co
ontroller gets less and yo
our options are
a removedd. If you use the timer, th
he vote will
close after the
t countdow
wn and the student
s
pagee will close. Iff you want to
o pause the countdown,
c
press pausee and the tim
mer will stop.. The studen
nts can still giive their resp
ponses. By pressing
p
run,
the countdo
own will con
ntinue.
Figure 37 sho
ows the resultss from assessm
ment question 1--5 and one SRS
S result.
Results fro
om the SRS round will appear up together
t
with
h the other results. SRSS results are
tagged with
h a SRS num
mber and thee assessment question nuumber in ligh
ht gray. You can get the
results thro
ough clickingg on the resullts button.
Ad-ho
oc SRS
If you wantt to ask a queestion, with a different seetup from thee assessmentt, you can run
n an ad-hoc
SRS. This means that you have to
t set up th
he number of
o alternativees. Press  to get the
m alternatiives press “m
more”  to get more altternatives or
alternativess panel . Iff you want more
press “less”” if you waant less altern
natives again.
Figure 38 sho
ows the play co
ontroller when running an ad
d-hoc SRS.  display
d
the num
mber of alternaatives that you
can select, an
nd  allows yo
ou to have one selection or many selections on the student side.
In an ad-hoc SRS you also have to set the number of altern
natives that th
he student caan select. By
default the student can select only one,
o but by pressing
p
the cardinality button
b
 yo
ou can select
to have un
nlimited num
mber of seelected itemss. This allow
ws you to have “selecct only one
alternative”” or “select up
u to all alterrnatives”. When
W
you run
n the ad-hocc SRS , resultts appear as
the other reesults , but without
w
the reeference to th
he assessmen
nt question.
3
33 Done-IT
L
LLP-Project
Nr.
Nr 511485-LLP-1-2010-1-NO--KA3-KA3MP
Figure 39sho
ows the assessm
ment results frrom question 1-5,
1 one SRS reesult based on question 1 (SR
RS 1) and one
ad-hoc SRS ittem (SRS2).
Results
When the assessment with the Po
ost-Assessmeent activity is
i finished, it’s
i time to look at the
results. Youu can look at
a the resultss from each individual user
u under Admin.
A
Byy default no
results will show up herre. This is do
one deliberateely so that yo
ou as a teacher can avoid by pressing
this by a miistake and sh
how the resullts to all studdents. You haave select Sh
how Results Table
T
.
You will ho
owever get a warning, to
o avoid any mistake,
m
wheere you have to confirm that you do
this action deliberately.
d
Figure 40sho
ows the warning
g dialog when selecting to lo
ook at the indivvidual results.
If you wan
nt the studen
nts to look at
a their indiviidual score, you have to press “Show
w results to
students” button.
b
Th
his will allow
w the studentts to see on their own deevice the sco
ore they got
on the asseessment andd from the SRS
S section. You can fro
om this inteerface exportt data as an
excel file.
3
34 Done-IT
L
LLP-Project
Nr.
Nr 511485-LLP-1-2010-1-NO--KA3-KA3MP
App
pendiix A Simu
ulating an
a
Asse
essm
mentt
When you design a mulltiple choice assessment, you want to
o have a feelin
ng with how
w easy is it to
do a pure guess
g
and pass your assesssment. Let’’s look at thee possibility to
t simulate th
he outcome
from an assessment and what you can learn fro
om doing it. If I have an
n assessmentt with seven
questions set
s up as bellow. I have one
o correct alternative on
o each quesstion, exceptt question 3
where I havve rules that deviate from
m the standarrd setting. I have
h one question that bleeds
b
to the
total score.
Figure 41 sho
ows the edit assessment interrface with a deffined assessmeent and where you find the siimulate
button.
You can siimulate a situation wheere a given number
n
of students respond rando
omly to this
assessmentt. Press the sim
imulate link an
nd the monkkey cage conttrol appears.
3
35 Done-IT
L
LLP-Project
Nr.
Nr 511485-LLP-1-2010-1-NO--KA3-KA3MP
Figure 42 sh
hows the mon
nkey Cage co
ontrol where yyou select thee number of randomly filliing inn your
assessment.
Each questtion the resp
ponse is randdomly selectted, as manyy as I am allo
owed to seleect. Assume
that we releease 5000 monkeys
m
to th
his assessmen
nt. Then wee press Run Simulation.
S
Yo
ou will now
get a simulaated outcome space of yo
ou assessmen
nt. ©
Figure 43 shows the simullation results with
w
the sliderr where you caan “measure” the probabilityy of getting a
score in the selected
s
range.
3
36 Done-IT
LLP-Project Nr. 511485-LLP-1-2010-1-NO-KA3-KA3MP
When you use the slider on top and move the lower limit up to 1.9 and the upper limit to max,
you get the probability to get a score of 2 or more, by pure guessing. Remember that our
students are smarter that monkeys and they will not do a pure guessing and hence this estimate
is low. If you have one obvious wrong alternative, they can do a qualified guess and reach a
quite good score.
37 Done-IT
L
LLP-Project
Nr.
Nr 511485-LLP-1-2010-1-NO--KA3-KA3MP
App
pendiix B Mark
ks in
n que
estio
ons
The studen
nt page has aan option wh
here the studdent can marrk the questiion. This can
n be used in
many differrent ways. The
T most ob
bvious way might
m
be a diigital remindder that “thiss question I
have to takke a second lo
ook at if I haave time”. Th
he figure belo
ow shows the student intterface, after
the studentt has markedd question thrree.
Figure 44 shows the student interface w
when question three has been
n marked. Thee question buttton on top is
tagged with a flag and thee number of marks
m
that has been placed in
n the assessment is shown iin the bottom
menu.
Those queestions that are marked will appear in you mo
onitoring. Th
he question monitoring
interface haas a column dedicated
d
to the total am
mount of marrks and a bar showing thee percentage
of students that has marrked the queestion.
Figure 45 sho
ows the questio
on monitoring interface with
h question 3 maarked by two sttudents.
3
38 Done-IT
LLP-Project Nr. 511485-LLP-1-2010-1-NO-KA3-KA3MP
If you as a teacher want to, you can use these flags as indicators. You might give your students
the instruction:

“Mark those questions that you find difficult, and I will place additional focus to
questions that are marked.”

“Mark those questions that you find simple”
This allows you students to influence the post assessment activity. You as a teacher can also
chose to ignore the marks.
Notice that marks does not show in the monitoring matrix. If a student want you to pay
attention to a question, he or she should be able to do that without exposing themselves to the
teacher. Marking is made anonymously.
39