CATs in nutshell --
Workshops for SCI/BenU faculty on classroom assessment techniques
Welcome back, everybody. How was your summer? OK, good, now we've got that out of the way, let's think about the end of the semester. Specifically, let's think about the Classroom Assessment Report Forms you'll receive from your friendly local neighborhood Assessment Committee a scant 14 or 15 weeks from now. And, if you think about it, the time to start thinking about them is now. For planning purposes, if nothing else.
Here's the nutshell: Since we'll be asking you to document what CATs you used, now's as good a time as any to survey the field and start deciding which ones to use. To lend a hand, we are scheduling an informal workshop on Classroom Assessment Techniques during the first week of classes at the Resource Center in SCI/Benedictine's Becker Library. In order to encourage attendance by traditional daytime program teachers and night school teachers alike, the same workshop will be offered at three times:
Attendance is voluntary. The workshops will be convened by SCI’s faculty assessment committee chair Pete Ellertsen (who also writes and edits Nuts & Bolts, and who is going stop referring to himself in the third person right now)! The workshops will be very informal, and I'll be available to answer questions on: (1) how the Common Student Learning Objectives (CSLOs) on your syllabuses were derived from SCI's mission statement; (2) how the Course Based SLOs (CBSLOs)derived from them and listed below them on the syllabus can be related to daily lessons and assignments; and (3) how instructors can choose classroom assessment techniques appropriate to the SCI and/or Benedictine University mission statement and the SLOs in the courses they teach.
I like to pitch the workshops to new adjunct instructors who are encountering more acronyms and buzzwords than they'd expected in college teaching. I remember too well how confusing all the academic gobbledygook was when I made the transition from newspapering to higher education, but you'll get through it, too. At last year's workshops we had a nice mixture of adjunct instructors and experienced faculty, and they evolved into round-table discussions of what works and what doesn't work in the classroom, and how to measure and document it ... so we can learn from it how we can improve our performance as instructors, and so we can show it to "outside stakeholders" when re-accreditation time rolls around in eight years. (Did you notice, by the way, the countdown is beginning?) I know I learned things I hadn't known before at last year's workshops. I think we all did, and that's one measure of a good workshop.
At SCI and Benedictine, we survey instructors on the classroom assessment techniques they use. That's what those Classroom Assessment Report forms you'll be receiving are about. It's part of a longitudinal study members of the Assessment Committee are undertaking, so we have a list of specific techniques we're tracking. Eight, to be exact. But there are others. And we have a space on the forms where you can check "Others" and briefly explain. A perfect fit! I'd encourage you to search the literature and find your own ways of doing assessment.
Southern Illinois University at Edwardsville has an excellent website on classroom assessment. It's by far the best I've found for a quick overview. Or for a very accurate refresher when I need one. When I'm not sure what to do next, sometimes I'll just go to SUI-Edwardsville website and shop around for an assessment technique that fits my needs at the moment.
More important than specific techniques, though, is the underlying attitude behind assessment, and that's something the SIU-Edwardsville website sets out. I especially like the "Seven Principles for Good Practice in Undergraduate Education" on the main page page and the list of "What Students Want in a Teaching Professor" linked to it.
Why do we do assessment, after all? Well, for one thing, we're required to by outside stakeholders like government and accrediting bodies. But that's not the real payoff for us as educators. "First and historically, assessment is what we faculty members can do in order to demonstrate to ourselves that we actually do what we say we do," says SIUE's Douglas Eder. "It is our source of in-process feedback." By assessing what our students actually learn, Eder explains, we can analyze "the curriculum (or an assignment, class, or course) into component parts." By doing it as we go along, we can see where we're going with a class and change course as needed.
One-minute essay. Here's how it works: At the end of class, you take a minute or so. Hence the name. Ask the students two questions: (1) What was the most important thing you learned during this class? (2) What important question remains unanswered? Let the students write their answers, take up their papers, analyze them and report back to the class what you learned from the exercise. That last step, reporting back is important. It completes the loop, and over time it helps the students realize learning is a two-way street and they're responsible for what and how they learn. According to assessment gurus Thomas Angelo and Patricia Cross, "no other Classroom Assessment Technique has been used more often or by more college teachers" (Classroom Assessment Techniques: A Handbook for College Teachers [San Francisco: Jossey-Bass, 1993] 148). It's not hard to see why.
Pre- and post testing. A lot of instructors at Springfield College/Benedictine swear by this technique, especially in basic courses that don’t call for a lot of analysis, synthesis and evaluation of alternative strategies. The experimental design is elegant in its simplicity: (1) Give a test; (2) give the same test again; and (3) compare the scores. Whatever questions I asked, I'd ask them twice -- once in August as the semester begins and once in December as it ends. If the students did better on the post-test, I could chalk it up to value added by my teaching. If they didn't, I'd have something to think about I planned the next semester.
Muddiest Point. Students are asked at the end of each class to write and submit the most difficult or confusing topic of the lecture. It's explained further on the SUI-Edwardsville website, which notes, "Actually, asking students to identify that which is least understood is an interesting and potentially powerful integrative exercise because it requires students, first, to rate their own understanding across several topics and, second, to ponder, if even momentarily, why one particular topic should be selected as least understood." One word of warning: When I've tried it, some of my students found the word "muddiest" to be, well, uh, muddy. I substitute "most confusing."
Three Question Survey. Students are asked what they have learned, what they are uncertain about, and one question they would like answered. Used by some instructors at SCI, it's basically a variation on the minute essay and "muddiest point" exercises that provides a little more direction for the next day's lesson. Directed Paraphrasing. Students are asked to briefly paraphrase important idea(s) from the day's class. If you want an example of how it works, try it yourself: The SIUE website has a sample form that asks instructors to define classroom assessment in terms "that would make sense to your faculty colleagues regardless of their academic disciplines." Muddy enough?
Misconception/Preconception Check. Students are asked to describe any misconceptions or preconceptions that they had which were brought into question by the day's class. It's supposed to be especially good for social sciences and other fields that take up controversial issues. Here's a detailed explanation of how to go about it that was the "CAT of the Semester" at Westminster College. I hope that doesn't sound too much like "flavor of the month!" Anyway, it looks like a valuable way of sorting out controversial issues -- if done carefully -- and the Westminster College approach looks like a careful approach to subject matter that can be tricky.
RSQ2C (Recall, Summarize, Question, Comment, Connect). A complex exercise in which students describe an issue from the day's class and try to connect it to a broader understanding. It's explained, also quoting Angelo and Cross, on the SIU-Edwardsville website: "RSQC2 is a powerful and multifaceted assessment tool that takes time to administer and evaluate. ... Student feedback through RSQC2 may be crucial for aiding a professor to identify where students really need help the most but get it the least -- on relationships betwen ideas."
Assignment Assessments. Student input is requested to evaluate assignments. A more elaborate form is used at Curtin University of Technology in Australia. "Students self assess, attach their assessment to each assignment they submit, and are able to compare their marking with the instructor's assessment," says Peter Radloff of the Behavioural Health Science program in the School of Nursing. "Two drafts of the assignment are also involved and are submitted with the assignment. Use of this procedure is valued by students who appreciate its structured support of their assignment preparation, planning, writing and revising."
Other. Don't feel like you have to use the specific CATs we're tracking. A lot of faculty check "other." I do myself. The technique I've found most useful in my classes, for example, is something called a self-reflective essay. So when I fill out my survey forms at the end of the semester, I check "other." And that gives the committee useful statistics, too.
Committee members needed
We lost key members of the Assessment Committee over the summer, and we need volunteers to take their places. The coming year is especially crucial as we phase in key components of the continuous improvement planning cycle we committed to when SCI was reaccredited in 2005. We especially need people with training, experience or an interest in statistical analysis, strategic planning and/or program evaluation. But we also need rank-and-file instructors to ensure we don't get too specialized, and everyone's needs and interests are reflected in our assessment plan. Please contact me for more information at pellertsen@sci.eduu.
Editor's note. The discussion of individual CATs is condensed from Nuts & Bolts articles in November 2004, December 2004 and January 2005. Fuller discussion and documentation of techniques mentioned here is available in those issues of the the newsletter.
Squirrel image by Aaron Logan, LIGHTmatter Galleries, licensed under Creative Commonsons Attribution 1.0 License
-- Pete Ellertsen, chair, assessment committee
Nuts & Bolts is an electronic newsletter published by SCI's Assessment Committee. If you have information, comments or feedback, please contact any committee member or Nuts & Bolts editor Pete Ellertsen, in 211 Beata Hall on the SCI/Benedictine campus, 525-1420 ext. 519 or by e-mail at <pellertsen@sci.edu>.