Our standardized test scores holding steady
Scores from the standardized Collegiate Assessment of Academic Proficiency (CAAP) tests we gave Springfield College sophomores in March are back, and they suggest our students are still within a percentage point or two of the national average on the baseline General Education competencies we measure. That's about where our students have been testing since we began using the ACT Inc. product in 2003, and I take that as a good sign.
When we gave the CAAP math and reading modules to sophomores eligible for graduation at the end of March, 61 took it. That's down considerably from last year, when 87 took the test, and closer to 2003 when 66 students took the test and 2004 when 69 took it. This year's participation represents about half of the 120 students who were eligible, which isn't a bad response rate for what is essentially a survey. But it's a warning not to get complacent about our policy of voluntary compliance with standardized testing. So during the summer members of the Assessment Committee will be discussing, mostly by email, why the numbers are down and what we should do about it.
In the meantime, the scores aren't half bad. On the reading module, our students scored in the 59.6 percentile. That's compared to 60.5 percent on ACT Inc.'s nationwide reference group of second-year students in two-year colleges. On the math test, our students scored 56.2 percent compared to 56.1 on the national reference group. So we're in the ballpark compared to the national norm. As assessment committee chair, very frankly I find that reassuring.
Perhaps even more reassuring, our scores this year are in the same ballpark as last year's. In math, this year's sophomores scored within 0.3 points of last year's score of 56.5 (compared to 56.1 nationwide). We only started testing math last year, so we don't yet have enough data to come to any conclusions. I'd be worried if the two year's scores weren't in line with each other's. But so far -- cross your fingers! -- they seem to be consistent. We've been giving the reading test for five years now, and those scores are consistent, too. Year by year, they are:
CAAP Reading Scores, 2003-2007
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
59.6 |
60.5 |
(- 0.9) |
What do these numbers mean? I think it's important to sort out what they tell us -- and what they don't tell us, at least not yet. When we started looking into the CAAP test in 2003, the test vendor cautioned us that the results from any one year wouldn't mean very much, largely because our test-taking population was too small: "Generally, the larger the sample size, the more useful the data. ACT recommends a minimum of 100 students tested for each module used."
With that caveat in mind, I try to resist the temptation to read very much into our year-to-year fluctuations. (And for the same reason, I'm not ready to assume yet that our math scores are on target. I want to see another year's data before I get too comfortable with them.) But by lumping together all the reading scores from 2003 to the present, I'm think it's safe to say we have a data pool of more than 350 students who have taken the CAAP module and they have tended to test out within a percentage point or two of the national average.
As tentative as it is, that's important information. If nothing else, I take it to mean our Gen Ed program isn't way out of step.
It's important to remember, too, standardized testing is only one component of our assessment plan at Springfield College. We use it primarily to compare our efforts -- and our students' outcomes, or measurements of what they learn -- with other two-year colleges nationwide on two baseline skills that we consider foundational to our Gen Ed program. The CAAP test results take on meaning only in combination with the program assessment and classroom assessment that fill out the plan, together with the attention we pay to the statewide learning standards set out in the Illinois Articulation Initiative.
Radio host Garrison Keeler has a running shtick about the fictional Midwestern town of Lake Wobegon where "all the children are above average." So a cynic might say our CAAP scores tell us our students are a little above average (except when they're a little below average), but in fact they can tell us more than that when we take them into account as we work up our individual classroom assessment pieces and evaluate the results.
Assessment committee news
Outgoing academic affairs dean John Cicero will be missed as he returns to the Chicago area to take a post with Roosevelt University. John's unfailing good humor and low-key management style have served the Assessment Committee well as we steered our way through the final stages of a North Central Association accreditation process and charted our course for the next 10 years. We wish him all the best at Roosevelt.SCI/Benedictine Resource Center Director Joanna Beth Tweedy will join the Assessment Committee as the fall semester begins in August. She was nominated to the committee by the chair, and her nomination was approved by an email poll of the committee subject to ratification at our first meeting fall semester. Joanna's best practices initiatives will mesh well with the committee's classroom assessment and professional development initiatives.
States mark turf on testing?
The State Higher Education Executive Officers, an advocacy
group for statewide governing and coordinating board officials, has come out
against U.S. Education Department efforts to force changes in higher education,
including high-stakes standardized testing, through the accreditation process.
But that doesn't mean the state officials don't want the changes. They just
don't want the feds to push them. In other words, the SHEEO statement sounds
like the states are marking their territory. Reports governmental affairs
reporter Doug Lederman of the online newsletter Inside Higher Ed:
“The line we’re drawing here is that the federal government can’t and shouldn’t try to accomplish all of these ends through the accreditation process,” Paul E. Lingenfelter, president of the SHEEO group, said in an interview about his group’s new statement. “It’s asking too much of accreditors, and the blunt instruments of federal regulation are not the right tools for dealing with these issues.”
“Student learning is taken for granted at too many institutions, including some of the very best ones, because they just have a great deal of confidence in what will naturally happen given the quality of their faculties,” said Lingenfelter. “But I think they can do better, and benefit from a little more discipline and rigor.”
The letter ... encourages the federal government to respect the roles of institutions, the accreditors and state governments (for public institutions). “The federal government should not seek to materially shape or constrain the work of institutions and the states in delivering instruction, setting learning objectives or degree requirements,” the SHEEOs write.
But that doesn't mean the state boards aren't interested in forcing
change themselves. Lederman adds:
... while colleges and accreditors may embrace the SHEEOs’ view that the government should not change federal rules to dictate changes in measuring student learning or transfer of credit standards, they are unlikely to welcome the state officers’ overall argument that it is time — as the Education Department argues philosophically — for colleges and accreditors to set “more explicit minimum standards for the knowledge and skills required for different degrees,” and for the academic community, “through accrediting associations,” to hold degree granting institutions accountable “for rigorous academic standards resulting in demonstrable student achievement.”
Standardized testing isn't specifically mentioned in the SHEEO statement. But it is widely
seen, especially by non-educators, as the most cost-effective way of enforcing
minimum standards set by government.
Nuts & Bolts is an electronic newsletter published by SCI's Assessment Committee. Members are: Bob Blankenberger, Brian Carrigan, Dave Holland, Barb Tanzyus and Pete Ellertsen (chair). Kevin Broeckling, dean of students; and John Cicero, academic affairs dean, serve ex officio. If you have information, comments or feedback, please contact any committee member or Nuts & Bolts editor Pete Ellertsen, in 211 Beata Hall on the SCI/Benedictine campus, 525-1420 ext. 519 or by e-mail at <pellertsen@sci.edu>.