When a series of modifications to Springfield College in Illinois' current Assessment Plan was approved by the Higher Learning Commission of the North Central Association of Schools and Colleges in August 2001, NCA associate director John Taylor noted a lack of progress, up to that time, in implementation of earlier plans for the assessment of student learning outcomes and noted, "[a]ssessment at SCI remains a work in progress." Two years later, outcomes assessment remains a work in progress. But a good deal of progress can be reported during the 2001-02 and 2002-03 academic years, and SCI has a better sense of what needs to be done in order to build on that progress. It can accurately be said that SCI now is actively building a "culture of assessment," i.e. an organizational culture characterized by awareness of the value of assessment and ongoing implementation of measures designed to: (1) document the learning that takes place when students come to Springfield College for their education; (2) communicate the evidence of student learning to those who make decisions for the College; and (3) complete feedback loops so the documentation of student learning can be used by faculty, staff and administration in a wide variety of planning for continuous improvement of the quality of education offered to SCI students. As we take steps toward a merger with Benedictine University and inclusion in the Academic Quality Improvement Program for purposes of strategic planning and accreditation, we have exciting opportunities to make further improvements in assessment and generate learning outcomes data for use by decision-makers in all sectors of the institution. Since this year's report of the Assessment Committee comes at a time of important transition, its scope has been expanded somewhat to serve as progress report on efforts since 2001 to jumpstart the assessment program and to provide background on SCI's assessment program since its inception more than 10 years ago.

In his staff analysis of SCI's assessment plan on Aug. 28, 2001, Dr. Taylor noted that SCI was then in a time of transition. "The College attributes its lack of great progress in an institutionalized assessment commitment to administrative changes in personnel and to changes in the direction and foci of the assessment program. However, assessment of student learning is now an administrative priority." Taylor also noted: "The College reports that a core group of faculty are committed to develop a College-wide assessment process to (1) enhance and improve student learning; (2) assess learning outcomes; and (3) provide assessment data for the College planning and budgeting processes. The College has made least progress in this last area. However, to fulfill #1 and #2 above, the College is working to build 'a culture of assessment' to help faculty take ownership of these assessment process and to recognize its value to their programs." Since that time, SCI's Assessment Committee has made a priority of the creation of an organizational culture that values assessment and stands ready to use the data in decision-making processes across the board. In furtherance of the assessment plan as amended, the Committee has undertaken initiatives in the following areas:

1. General Education Outcomes Testing. After a hiatus in General Education testing after 1998, a revamped program was designed in 2001 and phased in during 2002 and 2003 using the Collegiate Assessment of Academic Proficiency and other ACT Inc. instruments. Resumption of the program began in the 2001-02 academic year with conversion from locally written placement testing to use of the COMPASS tests developed by ACT Inc. During the same time frame, as a trial run for the administrative activities involved with a successful standardized testing program, the Assessment Committee and the Student Affairs office coordinated the administration of the ACT College Student Needs Assessment Survey during Fall 2001 and Spring 2002 freshmen orientation; results of the survey were made available to the Dean of Student Affairs and to faculty by means of reports at subsequent faculty meetings, and they were written up in a newsletter initiated by the Committee. The first CAAP test was selected in 2002; the reading module was chosen, because the Assessment Committee determined that it would give immediately useful data across the curriculum better than other components of the CAAP test. Results of the first test, given on April 2, 2003, suggest that SCI sophomores scored slightly higher than the national cumulative average for all sophomores in two-year colleges taking the reading test; it should be stressed, however, that data from the first test are only preliminary since the 66 students who took the test do not comprise a large enough statistical population for valid conclusions to be drawn from the results. It is envisioned, however, that over time the use of standardized, nationally normed tests will yield data allowing us to: (a) measure how well the learning outcomes of SCI students measure up against those of their peers in other colleges and universities; and (b) establish linkages between CAAP test results, entering freshman scores on the COMPASS placement tests and other ACT data that will facilitate assessment of SCI's General Education program.
2. Philosophy of General Education and Core Curriculum. In this area, SCI has been affected by administrative transition and policies of outside stakeholders. In the late 1990s, SCI's mission statement was replaced by a statement drafted by the College's then-president. Some faculty found the new statement too broadly stated for use in determining course goals and objectives; therefore, in the year 2000 an earlier statement of College goals and objectives was reinstated during the administration of the next president for use by faculty in assessment and published on the Assessment Committee's Web site until such a time as a new mission statement can be devised. Under this section of the College's assessment plan, significant efforts have been put into bringing SCI programs and course offerings over the years more closely in line with guidelines set out by the Illinois Articulation Initiative, the Illinois Board of Higher Education and other external stakeholders, including professional bodies that evaluate academic programs in their fields. In 2003, at the recommendation of the College's Academic Cabinet, more rigorous procedures for the proposal and approval of new course offerings and new programs of instruction have gone into effect, to be overseen by a new Curriculum Committee with primary responsibility for seeing to it that such initiatives reflect applicable statements of SCI's mission, goals and objectives as well as the requirements of outside stakeholders. Proposals submitted to the Curriculum Committee are required to state learning goals and objectives that flow from the missions statements, and to specifically address how learning outcomes will be assessed. A possible model for future program initiatives was the inclusion of a sophomore-level capstone course in a pre-Communication Arts program approved in 2003 and now being phased in with the addition of an introductory freshman-level course in Fall 2003 and other courses subsequently.
3. Assessment of Courses and Programs of Study. Individual course assessment is a key area of SCI's plan, and the one in which perhaps the most progress has been experienced to date, as instructors grapple with the collection and use of learning outcomes data other than grades. Assessment issues in the classroom have been addressed by a variety of measures, ranging from faculty development workshops and an on-line assessment newsletter called Nuts & Bolts, published on the Committee's Web site, to a new contractual requirement that faculty conduct assessment as a condition of employment. The Assessment Committee's Web site was opened in January 2001, to serve a faculty/staff development function and to complete one feedback loop designed to make assessment data available to interested members of the SCI community and outside stakeholders as well. Another feedback loop is in the process of completion this year. Nearly all full-time faculty and a majority of adjunct instructors in the 2002-03 academic year inventoried the classroom assessment techniques they use in questionnaires filled out each semester and returned to the Dean of the College and the College's joint faculty-administration Assessment Committee; these results are being tallied and will be published to the College's assessment Web site in the near future.

In addition, the Assessment Committee has taken on the added responsibility of collecting survey results relating to student needs and attitudes administered by the Student Affairs Office. A locally developed survey of student "views and opinions of [their] time here at Springfield College" was conducted for the Assessment Committee in the spring of 2002, and 75 out of a 128-student sample completed questionnaires. A similar survey was conducted in 2003 in connection with SCI's assessment day, administered to freshmen while the sophomores were taking the CAAP reading test referenced above. It is believed that ongoing survey results can be used in evaluating learning outcomes assessment data as the latter become available, and that the Assessment Committee can serve a purpose by undertaking the collection of these data in the absence of more formal institutional research.

Important areas remain to be addressed. While the collection of data is proceeding, time constraints on members of the Assessment Committee make difficult the timely analysis and promulgation of those data to the College community. While the assessment Web site has shown a great deal of promise as a tool for completing feedback loops and enhancing communication throughout the SCI community, its full range of potential is not yet being utilized. Much the same might be said of SCI's assessment activities as whole: They show promise, but their full potential is only slowly being realized.

Background, 1987-2001
Since Springfield College is in the midst of a particularly significant transition as plans proceed for an eventual merger with Benedictine University, the current General Education assessment report will briefly recap assessment efforts at SCI from the beginning before outlining specific assessment measures during the two-year period under review. Reference is also made to SCI's 1996 General Education Assessment Plan and updates submitted to the North Central Association of Schools and Colleges in November 1998 and June 2001, as well as NCA analyses of SCI documents and minutes of the Assessment Committee from 1994 to the present. It would not be overstating the case to say that over the years assessment has proceeded at SCI by fits and starts.

SCI's assessment program began in 1987 when an ad hoc "student outcomes" committee was empaneled at the suggestion of NCA. That body recommended that the College administer standardized tests on critical thinking (the Cornell Critial Thinking Test) and principled reasoning or informed moral judgment (James Rest's Defining Issues Test), which were considered appropriate to SCI's mission, goals and objectives as stated at that time. Some faculty development also was undertaken, especially in the area of setting course goals and objectives. In 1994 the ad hoc committee was reconfigured as a standing committee of the College, and an assessment plan was submitted to NCA in 1995 in connection with SCI's evaluation by NCA for reaccreditation in that year. It was during that time frame that the three "phases" still reflected in the current assessment plan were determined: Phase I consisted of the standardized testing, and was so designated because it was the first part of the assessment plan to be undertaken, beginning in 1987; Phase II of the assessment plan, relating to SCI's philosophy of education and core curriculum, began in 1993 with an evaluation of those issues in connection with the Illinois Articulation Initiative then proposed by the Illinois Board of Higher Education and other agencies. Phase III, dealing with individual and program assessment, was formalized in connection with the self-study conducted for SCI's 1995 reaccreditation visit by NCA evaluators; it was to consist of meetings with instructors in SCI's instructional divisions at which core competences would be discussed. The 1995 plan, however, was not accepted by NCA; the evaluation team that visited the campus in September of that year stated: "While there are some parts of the plan submitted which meet North Central expectations, in general, the team finds that . . . the plan as submitted is not comprehensive with respect to all institutional goals, does not obviously have institution-wide support, does not involve multiple measures of outcomes, has little provision for analysis and feedback of the results, possesses no locus of administrative responsibility for implementation and review of the plan and its components, and has no clear process for assessment of the plan's effectiveness" (32). The evaluators saw this as part of a larger concern: "Springfield College can do a much more effective job of collecting and analyzing data of all kinds on students, programs, etc., and then utilizing this data in helping to make the most appropriate decisions for the institution" (28). SCI's current assessment plan came into being in June 1996, in response to NCA's request for a revision of the proposal offered by SCI the previous year.

While important features of the 1996 plan have been modified over time, it remains the basis for SCI's outcomes assessment program. As drafted in 1996, the plan: (1) retained the standardized CCTT/DIT testing program then being phased in; (2) proposed a more rigorous process of tying the core competencies then being discussed by IAI to SCI's program/course goals and objectives; and (3) identified multiple assessment measures, including classroom assessment techniques, the use of juries and exhibitions in the fine arts, and review of programs such as those in nursing and aviation by outside professional bodies. The revised 1996 plan was accepted by NCA, and a good deal of effort went into securing assessment reports by the College's instructional divisions and individual instructors, especially in developmental course offerings, during the next year. A 1997 change in SCI's administration, however, brought a change in priorities. While reports were prepared on developmental English, and developmental writing course offerings were upgraded (from three to four credit hours and from one to two courses) in response to the assessment reports, other assessment initiatives were followed up with rather less vigor over the next few years. In 1998, the CCTT/DIT program was dropped as well due to what SCI reported to North Central in November 1998 as concerns about the validity of testing and a small number of students taking the tests. The College elected to "cease administering them and put more energy into developing and documenting the wider use of effective classroom and course-specific assessment techniques" (4). The next two years saw some faculty development activity, both on the division level and in collegewide faculty meetings, relating to the use of classroom assessment techniques (CATs) and planning for continuous improvement by instructors. As part of that emphasis, faculty were asked to indicate in their annual reports to the Dean of Academic Affairs what use they had made of CATs and what changes they had made in their teaching as a result of assessment. But meetings of the Assessment Committee became less frequent and appear to have been suspended altogether during the 1999-2000 school year. Other than the emphasis on CATs, the College's assessment program appeared to be more-or-less stalled until another change of administration took place in the summer of 2000.

During the 2000-01 school year, a new president took office shortly before an interim progress report was due to NCA. New members were named to the Assessment Committee, and a flurry of activity took place. Among those activities were setting as an Assessment Committee priority a proactive effort to establish an organizational "culture of assessment" at the College; opening an assessment Web site, maintained by the then-secretary (now chair) of the Assessment Committee, as part of the effort to establish such a culture; the initial planning for phased-in resumption of a standardized Gen Ed testing program; intensified emphasis on documenting the use of CATs by faculty; and the establishment of feedback loops to get learning outcomes assessment data in the hands of faculty, staff and administrators for decision-making purposes. Two faculty members were sent to the Assessment Institute that fall at Indiana University-Purdue University at Indianapolis, and a past chair of the Assessment Committee conducted a workshop in assessment and the use by instructors of assessment data in a "Plan-Do-Study-Act" cycle of planning for continuous improvement. In March 2001, Dr. Taylor of NCA visited SCI to speak of the accrediting agency's expressed concerns at the time of its site visit in 1995; the importance of assessment in addressing those concerns as part of the ongoing accreditation cycle; and the importance of changing SCI's organizational culture. "In order for an assessment program to be successful, it has to be faculty-driven," Taylor said. "Why? We're talking about student learning, and most of that student learning we perceive as taking place in the classroom. We've shifted the paradigm -- the focal point is now on the learner" ("We've Shifted"). CI's progress report on implementation of the 1996 assessment plan, completed in June 2001, involved substantial revision of the plan as written in 1996 although its main outlines were retained. Since 2001, therefore, activities of the Assessment Committee have consisted largely of implementation of the 1996 plan as revised in 2001. They will be detailed below.

Implementation, 2002-2003
Very early in its deliberations in the fall of 2000, the new Assessment Committee decided to make a priority of developing a "culture of assessment" at SCI. We were guided especially by two points raised by associate NCA director Cecilia Lopez in "The Commission's Assessment Initiative: A Progress Report" (1998):
· Teams and institutional administration and faculty are clear that a successful program to assess student learning must be faculty owned, developed, and implemented. Almost all have determined that [an] accountable individual, such as the chief academic officer, should have administrative oversight or directly administer the assessment program. (p. 4 of 12 in on-line printout)
· Many institutions have determined that in order for assessment to become an institutional priority, faculty efforts need to be recognized and rewarded, and senior leadership needs to publicly express strong support for those efforts. Other institutions have found that establishing feedback loops that regularly communicate the results of assessment activities are critical because feedback loops generate a great deal of interest and debate on student learning across departments and the campus. (p. 4 of 12)

In academic institutions that have fully embraced assessment at an ongoing level of implementation, according to Lopez, "[p]roposed changes based on the analysis and faculty discussion of the assessment results are linked to departmental and/or institutional planning and budgeting processes" (p. 11 of 12). SCI is clearly not at that level, but strides are being made.

While SCI's assessment program had been hampered by changes in administration during the 1990s, members of the Assessment Committee determined that several factors made it likely that SCI can grow a culture that recognizes the value of assessment and other means of collecting data to feed into a cycle of continuous improvement. Workshops and other faculty development initiatives as far back as 1995 had made instructors aware of a continuous quality improvement technique known as the "PDSA Cycle" (an acronym for Plan-Do-Study-Act, itself a formulation of the Shewart Cycle planning model promoted by Edwards Deming and other Total Quality Management experts), as well as a wide array of classroom assessment techniques. A December 2000 study by Peter Ellertsen, then secretary of the Committee, found that most instructors were already doing assessment, although new feedback loops needed to be established and existing means of feedback used more effectively for communication. Especially important to success of the Committee's initiatives has been the support from SCI's chief academic officer, Jeffrey Mueller, since he assumed those responsibilities in January 2001. Specific initiatives will be outlined below, in the order into which they fit into SCI's three-phase assessment plan:

General Education testing. When the revised 2001 plan was adopted by the Assessment Committee, it was clear that a nationally normed standardized testing component of the original plan needed to be resumed. It was also clear that it would have to be phased in over time in order to give the College a statistically valid measure of General Education learning outcomes. As it stands in the summer of 2003, the Gen Ed testing program is partially up and running, with ACT Inc. products now being used for: (1) placement testing, with entering students taking the COMPASS tests in math and language skills since the 2001-02 school year; and (2) the beginning of an outcomes testing program, with the reading component of the CAAP battery of tests administered to sophomores on April 2 of this year. Special attention was given to vigorous efforts to secure student cooperation in the CAAP test; even though taking the test is mandatory for graduation, the literature strongly suggests that other measures are needed in order to publicize assessment issues to students if response rates are to be high enough to yield useful data. Accordingly, a drawing for prizes including gift certificates to area restaurants and clothing stories was held on the day of the test - with students filling out their tickets at the testing site - and an issue of the assessment newsletter was given over to an open letter to students, explaining assessment and its benefits to SCI. Student response exceeded expectations.

We considered the first CAAP test a qualified success, and data on response rate and learning outcomes will be analyzed during the coming year when the Committee meets again. In all, 66 students took the test. Two preliminary patterns were noticed in the aggregate test scores. Taken as a group, SCI students scored slightly higher than the national cumulative percentage for all sophomores in two-year colleges, with correct answers of 61.7% for the SCI group as opposed to 60.9% nationwide. Possibly lending support to the idea that students can be motivated to take part in assessment is the fact that 63 of 66 students indicated that they "tried my best" or "gave moderate effort" to do well on the test. Issues to be addressed by the Assessment Committee during the coming academic year will include the matter of response rates; whether to phase in another component of the CAAP battery; and, if so, which test would be most appropriate for measuring student learning outcomes across the curriculum in a two-year Gen Ed program.

Philosophy and Core Curriculum (IAI). Over the years, the Illinois Articulation Initiative has become more important, as SCI has brought its course offerings into line with guidelines and requirements posted to the IAI's "iTransfer" Web site; the purpose of this exercise brings our course offerings under the scrutiny of multiple outside stakeholders, since the IAI is a quasi-governmental initiative involving not only Illinois regulatory boards but also representatives of two- and four-year colleges and universities statewide. It is, thus, an important ongoing measure of the quality and comparability of SCI's course offerings. Review of syllabi for IAI purposes also allows faculty to exercise greater rigor in setting forth their course goals and objectives, and assessing to determine whether students have in fact attained stated learning objectives. While the IAI's activities and policies are external to SCI, in the past year a good deal of faculty development effort has gone into helping our instructors, most of whom do not have formal training in pedagogy, define and measure learning objectives, goals and outcomes via two issues of the assessment newsletter, memos and informal means of communication.

Faculty and program assessment. Since most faculty reported as early as 2000 that they were using Classroom Assessment Techniques, the committee's attention has focused on the establishment of feedback loops. In the spring of 2002, a questionnaire was distributed at a faculty meeting and in on-campus mailboxes on which instructors were asked to check which CATs they use; of approximately 35 distributed, three completed questionnaires were returned. So in the fall of 2002, a somewhat simplified questionnaire was distributed again and Dean Mueller made strong efforts to ensure that both full-time and adjunct faculty returned the questionnaires. In the spring, the questionnaire was given out again, and completed instruments are being gathered; while a detailed analysis of the survey has not at this time been completed and exact statistics therefore are not yet available, it would appear that the response rate increased dramatically over the previous year and that most faculty were taking steps, at least informally, to get multiple measures of student learning in addition to grades and that they were using the results of these measures for planning purposes. As time permits, results of the faculty development survey will be posted in the assessment newsletter Nuts & Bolts. Program assessment was also addressed during the 2001-02 and 2002-03 academic years, as the Language and Literature Division discussed issues relating to what we expect students to learn and how we assess whether those learning outcomes are attained at division meetings. A planned replication of the Language/Literature discussions by other SCI divisions with less cohesive disciplinary boundaries has not, however, proven as feasible for the division dealing with humanities, fine arts and social sciences; or the division of mathematics and science. Members of the Assessment Committee have formed an ad hoc subcommittee to determine which of SCI's programs can be assessed by measures specific to a discipline, which are best assessed in connection with General Education and which might lend themselves to a variety of assessment measures.

Recommendations for 2003-04
It would appear that Gen Ed testing and faculty development are proceeding well, and that SCI is beginning to put in place some of the components of an overall learning outcomes assessment program. An informal review of the CAT questionnaires from the fall semester suggests that most faculty realize the importance of assessment and want to know more, but they are uncertain whether their practices conform to specific steps in such commonly encountered CATs as the "one minute essay" or background knowledge probes; this squares with results of a more elaborate survey in December 2000, and it suggests a need for ongoing faculty development. Similarly, the turnout for the April 2003 CAAP test was encouraging, but it is only one of the first steps in an ongoing renewal Gen Ed testing.

Where to go next with the General testing program is one of the most important items before the Committee in the coming year. It is not the only area that needs to be addressed, however. Time constraints on the faculty who serve on the Assessment Committee make difficult the rapid dissemination of information as it is collected, but as more and more data are compiled, we need to place greater emphasis on the establishment of feedback loops. Faculty and students need to know more about the results of the testing and surveys we ask them to take part in, so we can help increase buy-in into the program, and as learning data become available those data must be communicated effectively to the people who make decisions. A minor administrative matter that needs attention is the establishment of a filing system for the Assessment Committee to supplement the files of official minutes and testing information properly housed in the Academic Affairs Office. At present, Assessment Committee files, notes and supporting documentation are scattered among several faculty offices, a classroom and the circulation desk in Becker Library; in the spring of 2003, the former faculty reading room in Becker L-16 was set aside for use by the Assessment Committee, and plans are to centralize the files in that location.

A possible avenue for addressing the need for more effective communication of information compiled by the Committee may be presented by discussions attending SCI's contemplated application for inclusion in the Academic Quality Improvement Program. Since SCI is beginning the process of application to AQIP for accreditation purposes, members of the Committee will be engaged in that process anyway, since it relies heavily on the use of continuous improvement cycles similar to the PDSA cycle and other quality improvement measures addressed in assessment. In fact, assessment is specifically addressed at considerable length in the AQIP literature as a key component of planning for continuous academic improvement. Learning outcomes assessment all too often has been seen, at SCI and other colleges and universities nationwide, as an extrinsic requirement linked to accreditation rather than an integral part of teaching. As chair of the Assessment Committee, I have been involved in some of the AQIP planning, and it strikes me that the AQIP process can set up organizational structures and procedures that will facilitate the completion of feedback loops and the communication of learning outcomes data to decision-makers.

-- Peter Ellertsen, chair, Assessment Committee, August 20, 2003


References
Assessment Committee, Springfield College. Minutes. 1995-present.
__________. Nuts & Bolts [electronic newsletter]. 2001-present.
Crow, Steven D. Staff Analysis of Institutional Report, October 29, 1996. North Central Association/Commission on Institutions of Higher Education, Chicago.
Lopez, Cecelia. "The Commission's Assessment Initiative: A Progress Report." Presented at the 103rd Annual Meeting of the NCA/CIHE, March 1998. Formerly posted to Parkland College's Web site. A hard copy printout is in the possession of SCI's Assessment Committee chair.
"NCA Official: We've Shifted Pargdigm." Nuts & Bolts [SCI assessment newsletter], 7 March 2001.
Report of a Visit to Springfield College in Illinois, September 25-27, 1995, for the Commission on Institutions of Higher Education of the North Central Association of Colleges and Schools.
Springfield College in Illinois. Progress Report on Assessment in General Education, to North Central Association of Colleges and Schools. November 1998.
__________. Progress Report on Assessment in General Education, to North Central Association of Colleges and Schools. June 2001.
Taylor, John. Staff Analysis of Progress Report on Assessment in General Education, August 28, 2001. North Central Association/Commission on Institutions of Higher Education, Chicago.