July 7
From ETS
Agenda
- Review research methodologies
- Qualitative vs. quantitative
Minutes for meeting with Vicki Could do the survey first, and then come up with your focus group questions out of that. This is a way to get individual responses.
Survey will reveal patterns and trends. When you are reading the responses---this can feed into the questions for the focus group.
Ideally, it’s a two part thing. You look at trends, patterns, and then discuss further in focus group.
Is there a certain sized group that we should shoot for when conducting focus groups? Most studies say that it is somewhere around eight---plus or minus two.
If you have five people or fewer, you don’t have critical mass. Try to make it 8-10 people. Sign up the groups for 8-10.
Compensate the groups in some manner—pizza is one way. Paid cash to students. Drawing for an ipod.
When putting together the focus groups—there a methods for selecting and Vicki can consult with us once we get to this point.
Vicki typically audio tapes her groups but does not video them. This has to be in the IRB approval, consent form, etc…
Will read through the entire transcript, and then will go back and manually begin pulling out patterns by occurrences of certain words, etc…
Ideally, it would be wonderful to have a written transcript of the entire thing, and then run it through software like inVivo to see trends / frequencies, etc… Then you can just do the interpretation of the focus group data.
Can always do a two part process-do naturalspeech recognition with Dragon Speak.
If you are trying to say, “this intervention was effective, you must do a pre-test, post-test. The post-test will show marked changes.
If you can calculate the change in behavior from beginning to end, that is the best use for it.
The pretest and post test have to be instructionally equivalent. You really should give the same test at the beginning and the end.
Our hypotheses would be that those students who took advantage of library resources
Tie the grading rubric into the survey responses. Post-grading the assignment, the instructor can provide the grades.
Consider tying the data that we get from the survey into the grading information. As soon as all of the data has been collated, any identifying student numbers or names will be stripped from the data set. (put this into the IRB)
Vicki will share IRB.
We will pin down what we are trying to learn here as far as our hypotheses.
Measure of prior learning as opposed to outcomes. The pretest and posttest were not identifcal tools.
Used the pretest part to measure prior learning and the post test was not so much a test of knowledge.
Could ask students to demonstrate skills and ask them to actually demonstrate outcomes that they may have learned thoughout the class.
Could always do reliability (sort of like an item analysis)
Just because something is significant, it doesn’t mean that it is actually significant.
Start out with self-reporting on pre-test and post-test
Consider phrasing questions where you parse out the definitions for the respondents, i.e.,
How well do you know how to find the information that you need? Instead of saying , expert or proficient, you say, “I hope others with their computer problems’ “I know about as much as others,’ or “I
Use a group of five students to help us create / test survey
Pilot the survey with Emily and Anne’s class. Then tweak the survey as needed. This will allow for comparisons across classes.
We can do focus groups
Write this up in two phases—self reported and focus groups. Once we have analyzed the results from the first part, then we will add this to the next stage of this study.
Math 21: First time: Survey at the end; Second time—survey at beginning. Phase one: Make it simple. Even as you are doing this and looking results, new questions will come to you.
Math study: Vicki is asking questions about how much time they are contacting math instructors during the week;
Chris has a bank of validated questions that he used for a blog survey.
Vicki will share her questions from the math study.
