Question banks and question pools
From ETS
Deliverable One:
ANGELshort: What Instructors Need to know about Question Banks published on 9/26. A supporting video tutorial was also released. Question Bank Video Question Banks/Question Pools
Lead
- Suzanne Weinstein
- Jeff Swain
- Carol McQuiggan
What are Question Banks/ Question Pools
The only place ANGEL seems to mention "pools" is in adding a quiz question. That is where there is a choice for "Pool Item" which "Selects one or more random items from a question pool based on specific criteria." Everywhere else I checked, ANGEL refers to "banks." In ANGEL Help, the documentation to import a publisher's question bank is located at Courses -> Manage Tab -> PSU Data Management -> Import Wizard -> article #1464 "Import a WebCT Question Bank." The documentation to create your own question bank is located at Courses -> Lessons Tab -> Quizzes -> Import Quiz Questions -> article #1215 includes "Create a Question Bank Using a Word Template" and "Create a Question Bank Manually."
Pedagogical Starting Point:
- Why would you want to use them?
- To create unique exams for each student, reducing the ability of students to share answers with each other, particularly if tests are given over a period of time longer than a single day.
- To offer multiple opportunities for practice and ensuring that each quiz a student takes has different questions.
- To offer students chances to re-take quizzes with different questions to obtain mastery of the content.
Examples: Who is using them?
Who is for our purposes to provided examples and non-examples that are generalizable to our audience.
- We can ask faculty whom we know are using this to provide us with a brief description.
- Pool example: Shirley Clark, PSH-Environmental Engineering, uses ANGEL quizzes as mastery homework in her ENVE 417 Hydraulic Desgin course. The questions either come from professional review manuals of solved problems or textbook homework. Purpose: The multiple choice questions resemble the questions on the professional licensure exam. She doesn't use the timing feature, but tries to have large enough pools of questions so when they take the quiz the 2nd or 3rd time they are not getting the same questions. This is the motivation for spending time solving the questions on the first quiz correctly.
- Test bank example: Martha Strickland, PSH-Education, uses Prentice Hall's text's test bank for her EPSY 014 undergrad course. This is the student's first education course. The test bank is set up in case study format requiring an experience in applying theory to practice. She believes that every assessment experience should be a learning experience and these case studies fit well within that philosophy. "This critical thought and reflective approach to working with informational texts is crucial to effectively beginning their coursework in the area of preservice teacher preparation."
- Test bank example: Refik Culpan, PSH-Business, uses 6 major quizzes with 15 questions each in 2 of his undergraduate courses. He combines publisher's test bank questions with his own questions.
- (Not sure what to do with this information - Would this be considered 3rd party software?)- I have a PSH instructor who uses a publisher's website as his test bank. He creates a class list on the publisher's site, has his students register, and then they take the quizzes right on the website. He didn't realize that test banks could be integrated in ANGEL. (He knows now, and I'm meeting with him tomorrow.)
Import Question Bank from Word/RTF/Text File
See Article 1215 in ANGEL Knowledgebase - includes how-to and templates to use.
Publishers
- Prentice Hall
- Thompson South-West
How do they work? An instructor enters, copies or otherwise imports many more questions than are needed for a single quiz or test into the test bank. The quizzing software creates unique assessments by choosing either randomly from the questions in the bank or randomly within content groupings.
Significance: Why are they a valuable tool/resource?
- Over time, when a large enough test bank has been built, instructors don't have to create new tests each semester.
(May roll into the pedagogical starting point)
- May be valuable in one way for mastery learning and another way for norm-referenced.
What are the limitations of this tool/resource?
- limitations will be determined based on how the quiz is intended to be used.
- Time is a big limitation; it can take a great deal of time to develop a good bank of questions.
- Faculty review of questions in a question bank may also take a considerable investments of time, e,g, Chem expects to take 12-18 mos to review question banks from a publisher
- If the test bank is to be used for tests that are associated with a relatively high percentage of the course grade, then instructors need to ensure that all questions available for a single concept are of similar difficulty and discrimination. This ensures that each student receives a test that is equal in difficulty to tests received by other students. But it requires instructors to measure and keep track of the statistics of each item.
- It can be very difficult to create questions of equal difficulty and discrimination. However, it is easier in a math course (where the same question with different values can be included in the test bank) than it is in a humanities course, for example.
Where can I find more information about this tool/resource?
http://www.schreyerinstitute.psu.edu/Tools/
Troubleshooting
This section will be different from resources. Resources are more informational whereas troubleshooting willbe about getting immediate assistance. It will list the contact information to go to when encountering specific issues. e.g. if the problem is with ANGEL go to your support person, contact the help desk.
Related Publications
These are topics that are related to the main topic. May be published as an addendum ANGELshort, on the Community Hub, or as a video tutorial.
Additional topics for question banks include:
- Importing questions from word
- Using a Word template
Deliverable(s)
- ANGEL Short publication 9/26/08
