Establishing a Culture of Assessment

Related Content:

College representatives typically start accreditation visits by talking to the visiting team about their institution’s culture of assessment. Too often, however, the speakers lack an understanding of what that truly means. To determine whether an assessment culture exists—that is, whether the predominating attitudes and behaviors that characterize the functioning of an institution support the assessment of student learning outcomes—one must look at the attitudes and behaviors of individuals within that institution.

Just claiming that a culture of assessment exists does not make it so. In fact, there are fifteen major elements contributing to the attitudes and behaviors of a true culture of assessment. Few institutions of higher education can assert an expert level for all fifteen items, but those who claim to have an assessment culture must recognize them, be expert at some, and be moving toward achieving the rest. Only when an institution is on the path to meeting these standards can its claim to have a culture of assessment be taken seriously.

The fifteen elements needed to achieve a culture of assessment are the following: clear general education goals, common use of assessment-related terms, faculty ownership of assessment programs, ongoing professional development, administrative encouragement of assessment, practical assessment plans, systematic assessment, the setting of student learning outcomes for all courses and programs, comprehensive program review, assessment of co-curricular activities, assessment of overall institutional effectiveness, informational forums about assessment, inclusion of assessment in plans and budgets, celebration of successes, and, finally, responsiveness to proposals for new endeavors related to assessment.

General Education Goals

General education goals are critical for assessment. These are the core competencies that all students, regardless of major, are expected to demonstrate. Although each institution must determine what those competencies should be, most colleges and universities stress oral and written communication, critical thinking, quantitative and scientific reasoning, and information literacy. In recent years, global competence has also earned a high level of attention as a general education goal. Because general education goals must be assessed on a regular, perhaps rotating, basis, the number of goals should be manageable.

One challenge is that many faculty view general education goals and the assessment of them as the responsibility of colleagues who teach general education courses, such as first-year composition, introduction to biology, or introductory mathematics. Thus, they see written communication as the duty of first-year composition teachers.

Assessment professionals challenge this viewpoint for several reasons. First, if graduates should meet general education goals, assessing written communication should not be limited to firstyear composition. If a goal is important, and all goals should be, then the skills involved in achieving it should be taught and reinforced throughout all four years of a bachelor’s program, regardless of major. Second, skills need to be developed over time and integrated throughout the curriculum to enhance fully a student’s abilities. Third, faculty must verify that each degree program has multiple opportunities for students to learn and practice all general education skills assessed. The basic tenet of never testing students on something they have not been taught holds especially true of general education goals.

Common Use of Assessment Terms

Too often, faculty discussions about assessment lead to frustration. The cause can be as simple as a lack of common language. To avert unnecessary assessment angst, it is imperative to work on a glossary of terms. In other words, everyone involved in assessment should come to the table to develop a list of assessment terms and working definitions of those terms. Once this tentative list has been compiled, it should be made available to the entire academic community for further input and, ultimately, collegewide adoption.

Faculty Ownership

If the faculty does not own it, it is not going to happen. At best, if faculty fail to assume responsibility for assessing student learning outcomes, a college will have a faculty of “defiant compliant.” As anyone who is employed by a college knows, the real energy for program implementation comes from faculty members. They need to take part in planning and developing an assessment program, because they will certainly be the implementation team.

The success of the program will depend on having a faculty-led team composed mostly of faculty from across disciplines who plan the program, develop tools for and implement it, and use the data obtained. Otherwise, a college merely has an assessment program in theory, not in practice.

Ongoing Professional Development

Faculty members are not born with an innate knowledge of how to assess student learning outcomes. Some will learn on their own or while attending conferences. To ensure widespread understanding, however, a college must offer an ongoing professional development program that begins by building understanding of assessment concepts and elevates faculty competence through a series of higher-level assessment workshops.

By establishing assessment workshops, the college demonstrates its commitment to assessment and raises expectations among faculty. The workshops lead to conversations about assessment, encourage faculty to use the language of assessment, and help them gain competence and confidence. It will become clear that assessment is accessible and important.

Sending teams of faculty to assessment conferences is another way to foster the culture of assessment. Even though doing so is more expensive than bringing speakers to campus for in-house workshops, off-campus conferences allow faculty to gain a broader perspective and establish a network of resources. In addition, the college delivers a clear message that assessment is valued, and faculty view the conference as a reward.

Through ongoing professional development, faculty will come to understand that assessment is an achievable and engaging part of their jobs.

Administrative Support and Understanding

If faculty members think that their administration views assessment as a fad that will go away, or even that it sees it only as the job of faculty, they will hesitate to engage in the assessment process. But assessment is everyone’s business, beginning with the college president. Too often, college presidents fail to see their role, which is to keep their colleges honest in their assessment efforts. An informal survey conducted among assessment coordinators identified several key responsibilities of presidents: they must review student satisfaction surveys, assess institutional effectiveness, and use assessment findings in the budgeting and planning process. Presidents can also demonstrate earnest support of assessment by attending workshops and becoming conversant in assessment, as well as by committing sufficient resources and incentives to the effort.

After more than twenty years, it is clear that assessment is not going away. It has outlasted a plethora of educational innovations. But perhaps most important, assessment is about accountability. Once the 2006 report of the Secretary of Education’s Commission on the Future of Higher Education was published, regional accreditation bodies stepped up the urgency of local assessment programs in an effort to ward off federally mandated programs that many worried would overlook local needs.

Practical, Sustainable Assessment Plan

When a college community develops an assessment plan, it must keep in mind the need to do so cost-effectively and realistically and to revisit the plan frequently. If colleges are assessing six core competencies, it is probably not practicable to assess all students on all six competencies every year. Doing so would be an onerous task.
 
Another way to consider the plan is through the individual effort of faculty members. Each faculty member needs to select a learning objective to assess each year. Results should inform future assessment in that course or department in terms of change and assessment focus. They should also support assessment efforts going on elsewhere on campus. Such course-embedded assessment is a practical approach. It allows assessment to be integrated into normal course implementation instead of added on as an extra task for overworked faculty.

Systematic Assessment

The assessment plan must provide for a methodical assessment process. In other words, assessment of student learning outcomes must be consistent and orderly over time. As a regional accreditor explained, assessment should not be “once and done.” For example, if a college is assessing critical thinking, it should do so regularly, using uniform tools and a steady schedule—which does not necessarily mean that testing should be done every year.

Student Learning Outcomes

It is difficult, if not impossible, to implement assessment without identifying outcomes for student learning and program success. A college must understand what students will be able to do by the end of each course. Faculty must therefore agree on basic learning outcomes for each course, and the outcomes should be stated in the course catalog—for example, “By the end of the course, students will be able to x, y, and z.” The outcomes must be measurable. “Understand” and “learn” are not measurable. To find verbs that will lend themselves to measurable outcomes, faculty should consult Bloom’s Taxonomy. It includes appropriate verbs such as “define,” “describe,” “illustrate,” “compare,” “appraise,” and “construct.”

Comprehensive Program Review

Program reviews can take many forms. Institutions assessing progress at the department level often use prescriptive formats to guide them in auditing their academic programs. Assessment of student learning outcomes is one factor in such a report. Some institutions have posted information about their processes on the Internet, and excellent examples are available, including that of North Carolina State University, available at www.ncsu.edu/uap/academic-standards/uapr/ Undergrad_ Prog_Review_6%20Final.pdf.

Of course, each institution needs to customize its review process, which is about improvement, growth, and accountability, not merely meeting the requirement of a regional accrediting body.

Assessment of Co-curricular Activities

Discussions of assessment all too often focus entirely on courses and academic programs, even though learning can and does take place outside the classroom. On many campuses, significant sums of money support co-curricular activities that may provide learning opportunities. For example, if a college offers a globalawareness series through a student activities program, the series should be included in the assessment plan. As part of the plan, it should also have its own set of desired outcomes and be systematically assessed for student learning.

Institutional Effectiveness

There are many areas to assess in addition to student learning outcomes. Considering all areas, and how well the college is meeting its mission and goals, is essentially assessing institutional effectiveness. This makes assessment everyone’s business.

Assessing campus climate is an important part of evaluating an institution’s effectiveness. Students’ perceptions and attitudes about their campuses and college communities have been shown to affect student learning. “Taking the temperature” of a campus will reveal how students view its commitment to their success. Areas to assess include opportunities for student-faculty interaction, academic support services, personal support, academic challenge, enrichment, and library services. In addition, the president’s office, administrative services, the board of trustees, workforce development, the foundation, public relations, and all components of student services should not be overlooked. Every area ultimately has an impact on students and their success.

Information Sharing

Sharing the results of assessment, good or bad, is an essential part of a successful assessment program. Faculty can learn from one another. Each department needs to see what the others are doing and how well their efforts are working. Such sharing provides opportunities for departments to engage in peer review, steer away from failed experiences, and replicate successes where appropriate. It also permits faculty to identify activities from other disciplines that they can combine with their own to produce richer results, and it highlights areas of the curriculum that can benefit from cross-disciplinary efforts.

Planning and Budgeting

Often, faculty are turned off by the budgeting process. It does not have to be so. All faculty can contribute to their department’s plan and provide data for informed budgeting. The operational plan should highlight areas that need to be improved and have specific objectives to guide the department in working toward improvement. Assessment results can demonstrate areas of need within a department, and faculty can evaluate how much money might be necessary to rectify a problem. Also, if faculty engagement in assessment has yielded positive results, faculty can think about rewards that could be put in their department’s budget to continue the good work.

Although some faculty may resist supporting budgeting and planning efforts as additional work that the department head is paid to do, others who are regularly involved in improving teaching and learning will gladly provide input that will move them closer to the level of success they seek. Likewise, for an administration committed to a culture of assessment, the planning and budgeting process can help it accomplish this goal. When an administration explains the budgeting process to the entire college community and invites interested parties to participate, it signals that it is going to close the loop in planning, assessment, and budgeting. Of course, administrators must follow through and make sure that more money flows into classrooms as a result.

Celebration of Success

Too often, assessment reports are turned in and the faculty members involved never hear another word about them. Celebrating successes demonstrates the importance of assessment. In fact, celebrating participation in assessment is of tremendous value. Acknowledgment and appreciation serve to reinforce a college’s assertion that assessment is important and that assessment reports are read and valued.

Among faculty, revealing failed assessment strategies suggests an openness to collegiality and trust of colleagues. It is also useful to discuss what went wrong and why; sometimes, minor adjustments will make all the difference.

New Initiatives

Perhaps the most compelling indicator of an assessment culture is what occurs when any new initiative or proposal is advanced. Automatically, the questions asked will be: What are the goals and objectives? How will we assess the effort? These questions will be followed by lively discussions at meetings around the college, including not only the president’s staff but also other interested parties. When such a response occurs, it will confirm that an institution possesses a culture of assessment.

Wendy F. Weiner is professor of education at Chatham University. Her e-mail address is [email protected].

Preparing for the Visiting Team

The announcement of an accrediting agency’s visit can send your blood pressure into the stratosphere if you are among the faculty members who must write the accreditation report and host the team. To keep your systolic and diastolic numbers in the normal range, keep in mind the following alliterative list of principles to guide you in preparing for the visit.

Procrastination is one of the major downfalls affecting college assessment programs and reports. Visiting teams look for timelines in reviewing when assessment activities took place; a college that began to address assessment issues only four months before a visit will most likely be viewed as “defiant compliant.”

Planning is demonstrated in various ways, including by the presence of an institutionalized assessment team, ongoing professional development, and a consistent vocabulary of assessment terms. Timelines for departmental and collegewide assessment activities and program review cycles also document long-term planning.

Practice of assessment activities shows a commitment to improving and monitoring student learning. Such practice can include faculty piloting of departmental assessment activities, the sharing of findings at least divisionally, and determination of next assessment steps based on outcomes.

Preparation for the visit can best be demonstrated by documentation of regular, frequent, and meaningful assessment committee meetings. The follow-up information flow is just as critical. Evidence should exist of regular presidential briefings as well as collegewide communication. Assessment newsletters, minutes of meetings, and summaries of constituent input are welcome evidence of such efforts.

Presentation of assessment efforts in oral and written form should be honest and realistic. Visitors expect a clear communication of a college’s activities and plans. If the report and the evidence are confusing, team members often wonder what is really going on. Is something being intentionally obscured, or is the college confused, too? One way to avoid giving a bad impression is to have a simple, clear template for the assessment plan and another one for documenting assessment progress. All faculty members should be able to explain the template and its individual components, complete with examples, to a visitor.

Perceptions of members of the college community about the assessment program, plans, and progress are an excellent reflection of assessment efforts. A team could ask faculty, staff, and administrators to rate a college’s assessment program using a scale of 1 to 10 (with 10 being assessment nirvana). More practiced and knowledgeable faculty members typically will not give a rating higher than 8.5, and they will have significant evidence to support that number. They might identify goals yet to be achieved and people still not on board with assessment. For institutions developing assessment plans, a score of 5 could be viewed as realistic and thus acceptable. It is always better to recognize what one does not know. Constituent perceptions could be solicited before a visit by a survey that includes a rubric for achieving each score and a comment section in which respondents can state what still needs to be done. Such a survey not only documents campus perceptions and understanding, but also demonstrates assessment in action.

Perfection is impossible. Honest, systematic efforts are best. Even mistakes can provide meaningful information to help inform next assessment steps. When things go wrong, discussing the causes and what to do next is a steppingstone to improvement. Claims of perfection will only lead to skepticism.

Productive use of assessment is critical. Manageable plans and activities that faculty know about and use demonstrate a constructive approach. And the ability of faculty members to articulate clearly the way assessment influences institutional planning and budgeting is viewed as significant evidence of closing the loop in the assessment cycle.

Presidents and provosts are critical to successful assessment. They must demonstrate their knowledge of and commitment to assessment. Institutional commitment can be confirmed by attendance among top administrators at professional development meetings and funding of assessment activities. If presidents and provosts encourage assessment with grants, travel funds, and incentives to present and share findings, a visiting team will recognize the institution’s seriousness about assessment.

Although these guidelines do not guarantee a successful visit, following them will put a college on the right track to a culture of assessment. Long-term, systematic, and collegewide assessment activity is the standard by which an institution will be judged.

Comments

Wendy Wiener writes that "Understand” and “learn” are not "measurable." Throw out such concepts if you wish a "proper culture of assessment," But understanding and learning are what it is all about so I guess I don't want a culture of assessment.

D. M.

In the July-August issue of  Academe, Wendy F. Weiner tells us, in the manner of a parent scolding a child, "After more than twenty years, it is clear that assessment is not going away."  I am hoping against hope this is misplaced finality.  I would like to think assessment can go away; I am certainly convinced that it should go away.
 
Later in the same paragraph, Weiner states, "Assessment is about accountability."  Is it?  Who assesses the assessment process?  Where is the proof that prior to the establishment of the assessment bureaucracy, universities were not accountable to their students?  What indication do we have that assessment has actually improved the quality of education today?  Doubtless the measures we devise seem to show "progress"--they *must* do so, after all, or we are called on to rectify our "failure."  But they are founded on a flawed premise--the idea that college-level "learning" can be cut apart and reduced to "learning outcomes."  Students must be able to *do* something, measurably, by the end of a course; it is not sufficient, for the purposes of assessment, that students merely "learn" or "understand."  But "learning" and "understanding" should be at the heart of college courses.  A stress on "learning outcomes" over and above learning itself imparts a mechanistic feel to the process that both teacher and student may begin to accept as the true point of a higher education.

I am not advising that faculty ignore the issues on which assessment is centered.  Accountability is important, and I suspect that the vast majority of professors and administrators bear it in mind at all times, and take steps to ensure it.  That we now must suffer this additional layer of bureaucracy, whose dictates (disingenuously embedded in a ploy that seemingly vests authority in ourselves) actually steer us away from a focus on learning proper, adds a great burden to overworked personnel and university budgets, and ultimately distorts the very principle of accountability it purports to serve.  My apologies to colleagues whose salaries and professional reputations are enhanced by their involvement in the current assessment establishment, but I suspect you are vastly outnumbered by those of us who dearly want assessment to go away, and may someday achieve sufficient solidarity to act on this desire.
 
C. B.

In order to celebrate the success of  the academic achievements, it is crucial that every department comes to a common goal of the definition of success.  It’s very important to understand that the success lies within our students.  And if one departments improvements, and growth and even their accountabilities have been met, but the other departments are still struggling, then we have still failed as an institution.  Success has to be met through each department, otherwise, failure has impacted our students for the rest of their duration of being a part of our institution.

T. C.