Download FTEP-question-writin.. - University of Colorado Boulder

The Gentle Art of Questioning
WRITING GREAT CLICKER
QUESTIONS
Dr. Stephanie V. Chasteen
Physics Department
&
Science Education Initiative
Univ. of Colorado at Boulder
http://colorado.edu/sei
Web and blog: http://sciencegeekgirl.com
Email: [email protected]
This presentation is copyrighted under the Creative Commons License
Attribution Non-Commercial Share-Alike
That means: Please watch it, share it, and use it in your presentations.
Just give us credit, don’t make money from it, and use the same kind of
license on the works that you create from it.
More information about Creative Commons licenses here:
http://creativecommons.org/licenses/
Credit should be given to: Stephanie Chasteen and the Science Education
Initiative at the University of Colorado,
http://colorado.edu/sei
Who are you?
"  Show of hands
A.  Natural sciences
B.  Social sciences
C.  Humanities
D.  Arts
E.  Languages
F.  Other
Have you used response systems (clickers) in
your teaching?
"  Take a clicker & turn it on
"  If the green light flashes, your
vote has been counted
A.  Not at all, and I haven’t seen them used
B.  Not at all, but I’ve observed their use somewhat
C.  I’ve used them a little
D.  I’ve used them a lot
E.  I could be (should be?) giving this workshop
How familiar are you with Mazur’s “Peer
Instruction”
"  Colored cards
A.  Fairly familiar, and I like it
B.  Fairly familiar, but I’m not sure that I like it
C.  I’ve heard of it but only have a vague idea what it is
D.  Not familiar at all
E.  Not sure
Introducing Me
6
Science Education Initiative
http://colorado.edu/SEI
Applying scientific principles to improve science
education – What are students learning, and which
instructional approaches improve learning?
Physics Education Research Group
http://PER.colorado.edu
One of largest PER groups in nation, studying technology,
attitudes, classroom practice, & institutional change.
Blogger
http://blog.sciencegeekgirl.com
Why question?
7
Credit: Rosie Piller
whiteboard
Agenda
8
1.  When and how we can ask questions
2.  About clickers as a way to ask questions,
3.  Writing good questions. Example questions,
writing our own.
4.  Action plan
Learning goals: Participants will be able to….
A.  Explain several benefits of questioning and of using clickers to question
B.  Defend the use of best practices in questioning to overcome common
challenges
C.  Formulate and revise clicker questions to target student learning goals
Exercise #1: Question goal brainstorm
9
—  What goals might clickers be used to achieve? Or,
put another way, what might you use clicker
questions to accomplish in your class?
—  Brainstorm in your handout, then we will discuss as
a group.
2 minutes
whiteboard
Some of my question goals
10
BEFORE
Setting up instruction
Motivate
Discover
Predict outcome
Provoke thinking
Assess prior knowledge
AFTER
Assessing
learning
Relate to big picture
Demonstrate success
Review or recap
Exit poll
DURING
Developing
knowledge
Check knowledge
Application
Analysis
Evaluation
Synthesis
Exercise skill
Elicit misconception
Credit: Rosie Piller and Ian Beatty.
Clickers are a tool for questioning
11
Don’t confuse the pedagogy with the technology!
But not a magic bullet!
The perfect question doesn’t solve all problems!
Why use clickers to target the class?
An outline of Peer Instruction.
12
A quick outline of peer instruction
13
Anatomy of Peer Instruction
14
Ask Question
(Maybe vote)
…Lecture…
Class Discussion
Peer Discussion
Vote
* See also: Peer Instruction, A User’s Manual. E. Mazur.
What is special about clicker questions?
15
—  Similar in terms of goals
—  Multiple choice
—  Anonymous (to peers)
—  Every student has a voice – the
loud ones and the shy ones
—  Forced wait time
—  You can withhold the answer
until everyone has had time to
think (choose when to show the
histogram)
What does this tool help
us to do?
Which of these could be clicker questions?
16
BEFORE
Setting up instruction
Motivate
Discover
Predict outcome
Provoke thinking
Assess prior knowledge
AFTER
Assessing
learning
Relate to big picture
Demonstrate success
Review or recap
Exit poll
DURING
Developing
knowledge
Check knowledge
Application
Analysis
Evaluation
Synthesis
Exercise skill
Elicit misconception
Credit: Rosie Piller and Ian Beatty.
Let’s try it
17
Which superpower would you
rather have? The ability to…
A.  Change the mass of things
B.  Change the charge of things
C.  Change the magnetization of things
D.  Change the boiling point of things
What is the goal of this question?
When might you use it?
17
Question: Ian Beatty, UNC Greensboro
Image: Thibault fr on Wikimedia
Example question: Ethics
If you were a judge, how would you assess the “responsibility” of
the U.S. Government, for what happened in the world between
1933 and 1945?
A.  Not responsible
B.  Minimally responsible
C.  Responsible
D.  Very responsible
What is the goal of this question?
When might you use it?
Origin unknown
18
Example question: Math
Your sister in law calls to say that she’s having twins. Which of
the following is the most likely? (Assume she’s having
fraternal, not identical, twins)
A. 
B. 
C. 
D. 
Twin boys
Twin girls
One girl and one boy
All are equally likely
What is the goal of this question?
When might you use it?
19
Derek Bruff, Vanderbilt
Example question: Literature
If Homer wrote the Iliad today, Stanley Fish and Harold
Bloom would argue, respectively, whether the work
should be categorized as:
A.  Existential vs. Romantic
B.  Postmodern vs Classical
C.  Modern vs Romantic
D.  Postcolonial vs Modern
E.  Preliterate vs Postliterate
Origin unknown
What is the goal of this question?
When might you use it?
The Technology Enhanced Learning and
Research center at Ohio State
Questions?
21
Ask Question
(Maybe vote)
…Lecture…
Class Discussion
Peer Discussion
Vote
* See also: Peer Instruction, A User’s Manual. E. Mazur.
Exercise #2: Try writing a question
—  Choose one of the question goals (slide #3 on page 2
in handouts)
—  Write a draft question that aims to achieve this goal.
3 minutes
Question-writing tips
—  Move away from simple quizzes
—  Use questions that prompt discussion
—  Use questions that emphasize reasoning or process
—  Use clear wording
—  Use tempting distracters
—  Use questions at a mixture of cognitive depth
—  Ask challenging questions – don’t just test
memorized facts
See handout
Effective multiple-choice questions have believable
“distracters.”
1)  Talking with other instructors that have
taught the course in the past.
2)  Talking with your students one-on-one
before class, after class, during office
hours.
3)  Using student responses to openended questions that you include in
HW and exams.
4)  Asking your students to come up with
answers that will be used as the
choices.
5)  Use researched and documented
student misconceptions.
D. Duncan, Univ. of Colorado
24 An example question
What causes the seasons?
A. 
B. 
C. 
D. 
E. 
The change in the earth’s distance from the sun
during the year
Bad question.
The tilt of the earths axis
Students can
answer by
Changes in the sun’s brightness
memorizing a
word (“tilt”)
Changes in clouds
None of the above
Can we make a better question on the SAME topic?
Yes…
Better seasons example
What would happen to the seasons if the earth’s
orbit around the sun was made a perfect circle
(but nothing else changed) ?
There would be no seasons
B.  The seasons would remain pretty much as they
are today
C.  Winter to spring would differ much less than
now
D.  Winter to spring would differ much more than
now
Much better question. Requires reasoning!
A. 
Exercise #3: Revise your question
—  Use what we’ve just talked about, and the “tips” in
your handouts, to revise your question
—  If you wish, swap with your neighbor and discuss.
5 minutes
Use questions at a variety of cognitive depth
28
Do the questions you use intellectually
challenge your students or simply assess
their factual knowledge?
Higher order
---------------Lower order
handout
Exercise #4: Rate and swap
—  Use the Bloom’s Taxonomy worksheet to rate the
Bloom’s level of your question
—  Swap your question with a neighbor. Do you agree
on the Bloom’s level of your question?
—  Use the verbs on the detailed Bloom’s handout to
“Bloomify up” the level of your question.
5 minutes
What was the Bloom’s level of your question?
A.  Remembering
B.  Understanding
C.  Applying
D.  Analyzing
E.  Evaluating
Share out
—  What did you learn in this process?
—  What worked well, what was challenging?
—  How might you go about writing questions in your
class?
Gallery Walk
—  Look at the “example questions” trios on the wall.
—  What do these three questions have in common?
Jot any ideas down on the sheet.
—  Might you use this type of question? When? For
what purpose?
10 minutes
Aihofanz2010 on Wikimedia
But…
The perfect question doesn’t solve all problems!
I think the toughest thing about using clickers
and peer instruction in class will be:
A.  Writing good questions
B.  Getting students to really think about them
C.  Getting students to discuss the questions
D.  Getting students to share their ideas in the class
discussion
E.  It takes too long / I have a lot of content to cover
Or something else? Write it on your handout!
What are some *solutions* to some of these challenges?
Some “best practices” and solutions
We’ll do these quickly
These are repeated in handouts
—  Underlying philosophy to most of these “best
practices” are:
“It is important for students to discuss their
ideas, to feel safe doing so, and for the
teacher to be aware of those ideas”
1. Ask Question
36
What are some challenges/ things to consider
when posing a clicker question?
• Ask several times during lecture
• Ask challenging, meaningful questions
• Don’t post until ready
• Give time to read (read silently)
• Don’t read question out loud
36
Handout/worksheet / whiteboard
2. Peer Discussion
37
•  Students learn more deeply by
teaching each other
•  Makes them articulate answer
•  Lets you see inside their heads
Why is peer discussion
important?
What are challenges /
how can you help make it work?
• Make it clear why you’re doing this
•  Circulate and ask questions / model
• Use questions they want to discuss
• Allow enough time (2-5 mins)
• Focus on reasoning in wrap-up
Student buy-in is key!
3. Wrap-Up Discussion
39
Challenges?
What might you
do to facilitate an
effective wrap-up
discussion?
• Establish culture of respect
• Consider whether to show the
histogram immediately
•  Ask multiple students to defend their
answers
•  Why are wrong answers wrong and
why right answer is right
Giving the answer stops student thinking!
40
Action Plan
41
—  Take a few minutes to write down your action plan to
implement ideas you heard about in the workshop
U. Colorado clicker resources…
42
Videos of effective use of clickers
http://STEMvideos.colorado.edu
Clicker resource page
http://STEMclickers.colorado.edu
•  Instructor’s Guide
•  Question banks
•  Workshops
•  Literature / Articles
2-5 mins long
References & Resources
Web and blog: http://sciencegeekgirl.com
Email: [email protected]
Thanks!
—  Clicker Resource Page from the Science Education Initiative: http://
STEMclickers.colorado.edu. Has clicker question banks (in the sciences), an instructors’ guide, and videos of
classroom use. Useful books (such as Eric Mazur’s Peer Instruction are cited there.
—  Workshop handouts will be uploaded to the above website, or email me.
— 
— 
— 
Many materials in this workshop (particularly the questioning cycle and the participant exercises) were
adapted from Rosie Piller, Making Students Think: The Art of Questioning. Short papers published in:
Computer Training & Support Conference, 1995; ISPI International Conferences, 1991 and 1996; ASTD
National Conference on Technical & Skills Training, 1990. Related workshop description at http://
www.educationexperts.net/nstworkshop.html
Other materials (particularly sample clicker questions and goals of clicker questions) adapted from Ian
Beatty’s Technology Enhanced Formative Assessment (TEFA) program. http://ianbeatty.com/crs
My upcoming travel: Chicago, West Virginia, Pittsburgh. Need a speaker?
—  Upcoming free webinar for K12. See flyer.
Learning Goals
—  Biology: Recognize the components of a cell and
describe why each is necessary for the function of a cell
—  Physics: Identify the different ways that light can
interact with an object (i.e., transmitted, absorbed,
reflected).
—  Chemistry: Explain trends in boiling points in terms of
intermolecular interactions
—  Earth science: Understand the formation of the three
major types of rocks (igneous, sedimentary, and
metamorphic) and the processes by which they form,
relating them by the rock cycle.
—  Math: Solve a system of linear equations in two
variables using algebra or graphing.
What Do I do if…?
45
What can you do if you ask questions and..
—  There is no response
—  The same people keep raising their hands
—  The answers are called out before everyone has a
chance to think
We’ll discuss in
—  The answers take too long
Workshop #2.
For now: Many of
—  Someone gives a wrong answer
these challenges are
addressed by clickers
—  Only some students are prepared
—  ?