|
|
|
|
|
|
|
|
CAT series (4) --
Pre-/post-tests; but first, a plug for an important workshop this week
This month, as the semester ends, we're going to talk about how to use pre- and post-testing as a classroom assessment technique (CAT). I know, I know, final exams week is a little too late to be making out a pre-test. But there's always next semester. And I can recommend a nice webpage, from the University of Wisconsin system, for your holiday reading. I'll even include a link below, sort of like the sixpence in an old-fashioned Christmas plum pudding.
But, first, this message: Benedictine University's Eileen Kolich will conduct an assessment workshop at 5:30 p.m. Thursday, Dec. 9, in SCI's Becker Library. This will be an important meeting, and it will give us a context for the things we do in the name of assessment -- including the classroom assessment techniques we've been discussing this year in Nuts & Bolts.
I'll admit it, I've been working with assessment nearly five years now, and a lot of the time I've felt like I was playing blind man's bluff. But Assessment Committee members and I have spoken with Dr. Kolich a couple of times now about her plans for the workshop, and I 'm getting a stronger sense of how we can do assessment at SCI so it fits into the larger scheme of what we for students across our curriculum.
In large part, Thursday's workshop will be a planning session. We'll be brainstorming common student learning objectives for all SCI graduates. In other words, we'll be asking ourselves, as Dr. Kolich said last week during a conference call, "What will students who complete the core curriculum be able to know and do?" We'll also look for specific observable behaviors -- perhaps things like defending a written thesis, analyzing statistical data or putting together a juried art exhibit -- that our students master as a result of studying with us. Then, as time permits, we'll ask ourselves, "How does the curriculum nurture and develop those objectives?" We especially need to hear from our adjunct teachers, who may be the only voice we hear in some of the pre-professional disciplines. As we gear up for next year's North Central Association re-accreditation process, we'll develop specific goals, objectives and outcomes for Gen Ed and our pre-professional programs. These, in turn, will tie into goals, objectives and outcomes for specific courses. And all taken together, they will provide us with a foundation we've been lacking up till now for our assessment program.
Some of this is entirely new, at least to me, but a lot of it we've done before. After taking part in retreats and talking with other teachers for 10 years now, I think we have a good sense of SCI's mission as a two-year liberal arts college. But it's essential that we define, or redefine, our goals and objectives now as we prepare for next year's NCA site visit.
As I've said before, I didn't go to education school, and I felt like I was in the earliest -- and shakiest -- stages of on-the-job training when I started drafting assessment plans in 2000. Nor could I honestly see how classroom assessment answered any larger purpose than confirming my in-class hunches about why my students looked bewildered at times. But I see more and more now how it all fits together.
Pre- and post testing. One of the buzzwords in education these days is "value-added assessment." In plain English, it means looking for evidence our students know more when they finish a course than they did before they took it. One way to do that is to give a test at the beginning of school and give the same test again at the end. Hence the name pre- and post-testing.
We've talked a lot about pre- and post-testing at the classroom level over the years at SCI, but I haven't been able to find much information about it on the Internet. Keyword searches on "pre," "post," "test," "classroom assessment techniques" and "CATs" got me a lot of hits on institutional assessment, not to mention pre- and post-operative procedures for small animal veterinarians, but nothing about using pre- and post-tests in the classroom. Nor is the subject mentioned in Thomas A. Angelo and K. Patricia Cross, Classroom Assessment Techniques (1993), the standard reference on CATs. I did, however, find a brief discussion in an outcomes assessment manual put out by the Office of the Provost for the University of Wisconsin.
"Pre-test/post test assessment is a method used by academic units where locally developed tests and examinations are administered at the beginning and at the end of courses or academic programs," the UW Provost's Office says. "These test results enable faculty to monitor student progression and learning throughout prescribed periods of time. The results are often useful for determining where skills and knowledge deficiencies exist and most frequently develop. Academic departments at other research institutions currently using this form of assessment to measure student learning include communications, economics, geography, linguistics, theatre, and dance."
So there you have it. I've never had much luck with pre- and post-tests myself, and I suspect they work better for institutional assessment and the evaluation of programs at Research I universities like Wisconsin than they do in my classroom. But there are excellent teachers who swear by them.
And their experimental design is elegant in its simplicity: (1) Give a test; (2) give the same test again; and (3) compare the scores. Here's how I'd do it if I were using the technique, by way of example, in my introductory mass communications class. I would start with the goals, objectives and outcomes, which include: "To articulate the complexity of practices in the industries that make up the media, as well as the trend toward concentration of ownership." Like many freshman-level courses, my masscom survey is partly an introduction to the jargon of the field. So I would choose several key terms and ask students to define them. I might ask them the definition of "media conglomerate," for example, since it relates to the quoted objective. Or I might ask them to identify Rupert Murdoch or News Corp. (his conglomerate). For other questions, I would consult other goals, objectives and outcomes. Whatever questions I asked, I'd ask them twice -- once in August as the semester begins and once in December as it ends. If the students did better on the post-test, I could chalk it up to value added by my teaching. If they didn't, I would have something to work on in my next cycle of continuous improvement!
And, now, the sixpence in the pudding. So ... that's pre- and post-testing in a holiday nutshell. And now, you're asking, what about that sixpence in the plum pudding? Here it is. The UW assessment manual is available on line, and you can get to the section on "Assessment Instruments and Methods Available to Assess Student Learning in the Major" I quoted from by going to http://www.wisc.edu/provost/assess/manual/manual2.html. It's one of the best brief overviews of program assessment I've seen. And even though it's written for a Research I university, it can be easily adapted to assessment strategies for a two-year college like SCI that doesn't have departments and majors. I especially recommend it to anyone who will be working with NCA's re-accreditation process in the coming year. In fact, it would be a good thing for all of us to read before Thursday's workshop.
See you there. And Happy Holidays!
-- Pete Ellertsen, chair, Assessment Committee
Fourth in a series on classroom assessment techniques.
Nuts & Bolts is an electronic newsletter published by SCI's Assessment Committee. Members are: Bob Blankenberger, history and philosophy; Brian Ferguson, chemistry; Amy Lakin, English; Matt Mogle, fine arts; Rick Rossetto, biology; Steve Stowers, math; Barb Tanzyus, math; and Pete Ellertsen (chair), English and mass communications. Kevin Broeckling, dean of students; and Jeff Mueller, dean of the college, serve ex officio. The newsletter is available on line at http://www.sci.edu/assessment/newsarchive.html
If you have information, comments or feedback, please contact any committee member or Nuts & Bolts editor Pete Ellertsen, in Becker L-16A on the SCI campus, 525-1420 ext. 519 or by e-mail at <ellertsen@sci.edu>.
