|
|
|
|
|
|
|
|
Open letter to students --
We now have "linkage reports" back from ACT Inc. comparing our students' scores on the Collegiate Assessment of Academic Proficiency (CAAP) reading test to others in Illinois and at two-year private colleges nationwide, and the data suggest students at SCI are doing pretty well compared to their peers.
But more importantly, the CAAP data give us a tool for analyzing how well our students develop their language skills across the curriculum.
Don't forget Assessment Day Wednesday, March 30. Please encourage your students to take part in the test, which will begin at 11 a.m. in Dawson 224-226, and release them from 11 o'clock classes. Please join us for a Free Food Day afterward, too. It's our way of saying thanks.
The linkage reports involve benchmarking of CAAP reading test scores earned by SCI sophomores who took the test in the spring of 2003 and 2004. The linkage report data consist of comparisons of: (1) our sophomores' CAAP scores in 2003 and 2004; (2) Illinois high school students' ACT scores; and (3) nationwide CAAP scores for students in our private two-year reference group.
Because CAAP is an ACT product and ACT Inc. does standardized testing nationwide, the reports compare SCI students' scores to ACT reading test scores the students earned on standardized ACT tests they took in high school. Also part of the linkage reports is a comparison of SCI sophomores to students nationwide in our "reference group" of private two-year colleges. Thus, the reports give us a rough measure of what our students learn. Results are as follows:
It's far too early to draw conclusions about academic programs and curriculum from these initial data. For one thing, we're dealing with very small numbers here. But I think the scores are encouraging. If nothing else, ACT's methodology and the way the scores are broken down suggest that over time we are going to be able to get useful data from what is essentially a voluntary testing program.
And for that, I think our students deserve a vote of thanks.
"Just in the last month," I said in a letter to sophomores eligible to take this year's CAAP reading test that's going in the mail this week, "we got back statistics from ACT Inc. showing that 94 percent of SCI 'made expected progress on CAAP.' Since only 79 percent of students in our reference group of private two-year colleges made expected progress, that suggests we're doing something right. But we also have detailed information we can analyze now in order to improve what we do with reading and writing across the curriculum."
So far, frankly, the data don't tell us much. But they will. The most important part of the process will come later, when we analyze the information in terms of specific skills and competencies.
Here's how ACT explains the uses of CAAP scores: "Institutions concerned with program evaluation use CAAP to provide evidence that general education objectives are being met, to document change in students' performance levels from one educational point to another, and to compare performance gains across general education instructional programs within an institution."
What objectives do we measure with CAAP? Well, so far, we're measuring reading -- or referring -- and reasoning skills. Referring skills allow "the student to derive meaning from text by identifying and interpreting specific information that is explicitly stated." Test questions measure the ability to "recognize main ideas of paragraphs and passages, to identify important factual information, and to identify relationships among different components of textual information." Reasoning skills allow "students to determine implicit meanings and to go beyond the information that is explicitly presented." Test questions measure "students' ability to determine meaning from context, to infer main ideas and relationships, to generalize and apply information beyond the immediate context, to draw appropriate conclusions, and to make appropriate comparisons." In time we will be able to relate these skills to the specific behaviors in our Common Student Learning Outcomes statement. And the data can help us design a language across the curriculum program that's more than just a pedagogical buzzword. In the meantime, we're starting to get useable data coming in. That's progress in itself.
The letter to students is my effort to explain why we do standardized testing, and what's in it for them. What's in it, really, for all of us. In next-to-final draft form, it reads as follows:
Here's a brief reminder of our assessment day on Wednesday, March 30, and a few words about what you can expect. Like other schools and colleges everywhere, SCI is required to use standardized tests and other indicators as multiple measures of what you learn in your classes. So on March 30 we will give you the CAAP Reading Test, which we will compare with other data to measure what you have learned since you started college. These comparisons will not only help us comply with government mandates and keep our accreditation; they also give us information we need to know in order to improve the quality of the classes you take at SCI.
The test will given in Dawson Hall at 11 a.m., and again at 5 p.m. for New Horizons students and others who couldn't make it earlier in the day. It is mandatory. If it conflicts with a class, your instructor will allow you to make up any assigned work you miss. Please report to Dawson 224 at one of the two times, and we'll assign you to a room and let you know other last-minute details. The test will last 40 minutes.
The CAAP Reading Test is published by ACT Inc. It consists of 36 questions designed to measure reading and reasoning skills. While it is mandatory for sophomores who expect to graduate this year, it is not a "high stakes" test. In other words, you do not have to make a certain score on it in order to graduate. But it helps us measure how well we're doing at SCI. Basically, we do that by comparing the overall, average scores of SCI sophomores to ACT tests taken by high school students; the difference gives us an indicator of "value added" by your college work, which we can use to evaluate how well our programs at SCI are succeeding in helping you refine this basic skill. Just in the last month, we got back statistics from ACT Inc. showing that 94 percent of SCI "made expected progress on CAAP." Since only 79 percent of students in our reference group of private two-year colleges made expected progress, that suggests we're doing something right. But we also have detailed information we can analyze now in order to improve what we do with reading and writing across the curriculum. Next year we'll add a math test, so we can start analyzing the way we teach mathematical skills as well.
We wouldn't be able to do any of this without your cooperation. So we'll follow the 11 o'clock test with a Free Food Day on Gingko Square, as a small way of saying thanks. If you have questions, please don't hesitate to get in touch with me in my office, Room L-16, Becker Library, or by e-mail at <pellertsen@sci.edu>.
So reads the letter. For the faculty, staff and others in the SCI community reading it in this newsletter, I would only add one thing: You all please join us for the Free Food Day, too. Assessment is for all of us.
-- Pete Ellertsen, chair, Assessment Committee
References: The CAAP reading module is described on the ACT Inc. website at http://www.act.org/caap/tests/reading.html. And the uses of CAAP scores are explained at http://www.act.org/caap/uses/index.html. Links on the ribbon at the lefthand side of each of these webpages will take you to other valuable information about the CAAP tests.
Nuts & Bolts is an electronic newsletter published by SCI's Assessment Committee. Members are: Bob Blankenberger, history and philosophy; Brian Ferguson, chemistry; Amy Lakin, English; Matt Mogle, fine arts; Rick Rossetto, biology; Steve Stowers, math; Barb Tanzyus, math; and Pete Ellertsen (chair), English and mass communications. Kevin Broeckling, dean of students; and Jeff Mueller, dean of the college, serve ex officio. The newsletter is available on line at http://www.sci.edu/assessment/newsarchive.html
If you have information, comments or feedback, please contact any committee member or Nuts & Bolts editor Pete Ellertsen, in Becker L-16A on the SCI campus, 525-1420 ext. 519 or by e-mail at <pellertsen@sci.edu>.
