|
|
|
|
|
|
|
|
CAAP scores look good, storm clouds on horizon
Test scores are back from the Collegiate Assessment of Academic Proficiency (CAAP) tests we administered at the end of March, And, like others, they suggest our students are about at national average. So even though I'm not a statistician, I think we can crisply and unequivocally say we're more-or-less kinda in the ballpark. That'll have to do until a subcommittee of the Assessment Committee can take a closer look at the CAAP test results over the summer, to see what we've got and start making some preliminary decisions on what it means and what we want to do about it.
On the CAAP reading test, our students who took it (n = 87) scored an average of 59.0; the nationwide reference group of second-year students in private two-year colleges scored an average of 60.4. That's slightly less than the national average. But our students in 2005 averaged 61.2 on the reading test, compared to 60.4 nationwide, and in 2004 our students' score was 59.9 compared to 60.3 nationwide. So given the inevitable fluctuation on something like test scores, I think they're pretty consistent. Our first math scores may suggest the same trend. Our students averaged 56.5 as compared to 56.1 nationally. But we won't know for a couple of years if the trend holds.
In addition, this week we got a linkage report that gives us at least a rough benchmark for measuring how much our students learned about reading and math in their college years. ACT Inc., the vendor, explains: "This report contains an analysis of performance for students who tested with the ACT Assessment on entry to college and CAAP after general education work has been completed. ... Because the content specifications of some pairs of ACT Assessment and CAAP tests are similar, it is possible to track student performance for your cohort." In other words, we get comparative data on the students who took both the ACT state tests and the CAAP with us.
The linkage results:
What does this tell us? Since the results only came in this week, we've only had time to give them a cursory look. But here's what I take them to mean at first glance. Like the CAAP reading test scores, the linkage reports would appear to suggest our students stack up near the national average in terms of improvement. At least, the distribution of students making less than expected, expected and more than expected gains seems to fall into a "bell curve" pattern that isn't too far off the national linkages. That's one of the things the subcommittee will look into, however, and I expect to rely on the expertise of other members of the Assessment Committee in analyzing the results.
Over the summer, a subcommittee will take an exploratory look at all the CAAP test results and make some preliminary decisions on how they can be utilized as a planning tool for continuous improvement of instruction. We now have several years of reading test results back. That was our first step, getting a testing program back in place and collecting data again. Now we're ready to move on to the next step, figuring out what to do with the data we're collecting. That takes time, and it takes the expertise of more than one person. So we have empaneled the subcommittee. Serving on it will be Academic Affairs dean John Cicero, Languages and Literature chair Amy Lakin, and math instructor Barb Tanzyus, who is also a doctoral candidate in the education school at Illinois State University. I'll chair the subcommittee.
Anyone else who wants to spend part of the summer frolicking in the groves of academe is welcome to join us. Contact me by email at <pellertsen@sci.edu> if you're interested.
No blue-ribbon commission left behind?
It's beyond ironic -- just as we're starting to get a handle on our standardized testing at SCI, along comes the federal government with a proposal to politicize the issue nationwide. The U.S. Education Department's blue-ribbon Commission on the Future of Higher Education met this month in Indianapolis, and apparently backed away a little -- but not very far -- from its earlier proposals to implement nationwide standardized testing analogous to the No Child Left Behind tests at K-12 level and to replace regional accrediting bodies like the North Central Association with a omnibus federal agency. In one of the few articles to appear in the mainstream media on the commission's activities, Jody Cohen of The Chicago Tribune reported:
During the meeting, commission members heard views on higher education's shortcomings, from problems providing financial aid for low-income families to an inability to provide proof that students are learning. And the accreditation process used to measure the quality of a college is too secretive and expensive, some said.
It's unclear what solutions the commission ultimately will recommend. For the first time last week, the group ranked its priorities, putting at the top of the list increased college access and success for low-income, minority and adult students; a commitment to lifelong learning and need-based financial aid; the doubling of the number of scientists and engineers; and more accountability and transparency from institutions.
A word of translation is in order: "accountablity" is the political buzzword for standardized testing, for assessment in general. So it's still on the list, although maybe not at the top. In an article "Coalescing Around Concepts" published in the online magazine Inside Higher Ed, staff writer Doug Lederman noted the same set of priorities. And he noted the relief with which they were greeted by educators:
The mainstream nature of the recommendations — which emerged from a New Age/corporate-style discussion known as “nominal group technique,” in which the commissioners took turns suggesting “values” and then “goals” they thought were important, and then picked their top three choices individually, to create a group ranking — very much pleased those college officials who have been most troubled by the explosive nature of some of the ideas emanating from the panel’s chairman, Charles Miller, and position papers he has released.
"After the parade of the disaffected, the commission has begun to move back toward the middle, where American public policy will work its will," said David L. Warren, president of the National Association of Independent Colleges and Universities, who has tracked the commission’s work closely and expressed great concern about what he calls proposals to "demolish accreditation," "dismantle the federal financial aid system," and create a "nationwide system of student outcomes."
But, quoting Miller of the blue-ribbon commission, Lederman added:
... in an interview after the day's deliberations, which also included panel discussions about college costs and accreditation, Miller suggested that Warren and other college officials might be engaging in a bit of wishful thinking if they believe the commission isn't still headed toward issuing a "strong report with strong language," and turning the general "goals" agreed to Thursday into a set of tough recommendations that hold institutions much more accountable for their performance. Miller said he was confident that the commission was nearing consensus on proposals on "reforming [accreditation] and changing it for the better," "more and better assessment of student learning," and "making much more consumer information available" on how colleges "perform for students and the public." He also said the financial aid goals endorsed by the commission can be achieved not by "tweaking the current system," but only by "starting over."
It all bears watching. With the exception of college papers and major metro dailies like The Trib, the commission hasn't had much media attention. About the only paper, at least on the World Wide Web, to follow it in sustained way has been The Harvard Crimson. Best sources for keeping up with it are The Chronicle of Higher Education and Inside Higher Ed at http://www.insidehighered.com/. Unless I miss my guess, the political issues it raises aren't going to go away. And neither, I suspect, is its proposal for standardized testing patterned after No Child Left Behind.
Let's fill in the bubbles. Commission chair Miller and U.S. Education Secretary Margaret Spellings are both longtime associates of President Bush, and both were prime movers in NCLB and the Texas standardized testing program NCLB was based on. The NCLB act was the bright, shining moment of bipartisan legislative achievement in Bush's first term, and he sorely needs to find a winning issue for his second. The commission is due to report in August, and the congressional elections are in November. Should we make our heavy, black marks with our No. 2 pencils on A, B, C or all of above?
-- Pete Ellertsen, chair, Assessment Committee
Nuts & Bolts is an electronic newsletter published by SCI's Assessment Committee. Members are: Bob Blankenberger, humanities and social science; Amy Lakin, languages and literature; Steve Stowers, math; Barb Tanzyus, math; Ray Bruzan, chemistry; Brian Carrigan, science; Sr. Anna Izydorczyk, student; and Pete Ellertsen (chair), communications and humanities. Kevin Broeckling, dean of students; and John Cicero, academic affairs dean, serve ex officio.
If you have information, comments or feedback, please contact any committee member or Nuts & Bolts editor Pete Ellertsen, in Becker L-16A on the SCI campus, 525-1420 ext. 519 or by e-mail at <pellertsen@sci.edu>.
