The following observations were approved for publication by the Association’s Committee on Teaching, Research, and Publication in May 2005 on the occasion of the thirtieth anniversary of the AAUP’s adoption of the Statement on Teaching Evaluation. Comments are welcome and should be addressed to the AAUP's Washington office.
The Statement on Teaching Evaluation remains sound policy, and its guidance “for arriving at fair judgments of a faculty member’s teaching” continues to be invaluable.1 The world of higher education has changed significantly, however, since the publication of the statement thirty years ago. The proportion of part-time and non-tenure-track faculty appointments has grown to more than 60 percent of all faculty appointments; student evaluations of teaching are increasingly relied upon in decisions about renewal, tenure, promotion, post-tenure review, and salary increases; new computer-based tools have been developed to administer, disseminate, and interpret student evaluations of teaching; and corporate forms of governance are threatening to dominate higher education.2 In light of these new and important changes to higher education, we thought it desirable to bring forth these comments.
The Statement on Teaching Evaluation was published in 1975 “as a guide to proper teaching evaluation methods and their appropriate uses in personnel decisions.” It recommended that colleges and universities and their academic departments have clear, written policies about expectations concerning teaching, and provide support for meeting those expectations. The statement emphasized that descriptions of a professor’s teaching and data about the teaching obtained from other sources must be accurate. It cautioned that “the full dimensions of teaching should not be slighted in the desire to arrive at usable data and systematic practices.” The statement further recommended that the faculty member being assessed have a meaningful role in the evaluation process, and it called for faculty members to have the primary, although not the exclusive, role in evaluating an individual faculty member’s performance as a teacher. Last, it urged that “factual data, student opinion, and colleague judgments should be central in the formal procedures for review, which should involve faculty discussion and vote.”
Since the statement’s publication, a growing body of scholarship, supplemented by extensive experience, has developed in the areas of effective teaching and learning strategies, the role of teaching centers in assisting faculty to enhance their teaching, and the evaluation of teaching by faculty peers and students.3 Both this scholarship and experience show that faculty members share pedagogical and evaluation materials with colleagues; that self-evaluation, assessment by teams of faculty members, and student evaluations provide a regular flow of data that facilitates continual improvement in teaching; and that technology has contributed new tools with which teachers may assess and improve their teaching and conduct student evaluations.4 In addition, the dissemination of evaluations of teaching can provide students with useful information with which to plan their course of study, and offers faculty peers and administrators a richer body of material on which to base judgments of professional merit.
Two key issues identified in the 1975 statement continue, however, to trouble the evaluation of teaching: how best to ensure that evaluations of teaching provide accurate information about the effectiveness of teaching and how to ensure that faculty have the primary responsibility for devising and implementing teaching evaluations. We will offer the following comments on these two issues, and will then turn to an issue of academic freedom.
The ever-expanding use of, and reliance on, teaching evaluations since 1975 have given rise to an abundance of data about classroom performance. Although survey instruments, data collection, and methods for disseminating information vary within and across institutions, several common areas of concern exist with regard to the type, quality, and accuracy of the data collected. One concern involves the practice of relying solely on numerically based student evaluations. Although survey forms that call for numerical evaluations have the advantage of offering a common instrument that can provide meaningful comparative data, in many places they have become the dominant, or even sole, component of the evaluation of teaching and tend to displace less standardized and more individualized forms. We encourage the inclusion of a section to encourage students to provide written comments relevant to a particular course and a particular instructor.
A second concern is that numerically based evaluations are sometimes misunderstood and therefore misinterpreted, resulting in erroneous conclusions about the absolute and relative merit of faculty members. The data employed are often ill-suited for the type of statistical analysis carried out, being neither continuous in nature nor useful for making the fine distinctions on which rewards are often based. For example, teachers with numerical ratings falling below a certain percentile obtained by the entire faculty, such as 50 or even 90, may be characterized as inadequate or “bad teachers.” Such a use of relative position as an absolute measure of merit overlooks the possibility that the majority of faculty in some departments or institutions will be ranked as “superior,” so that some or even all of those in the department or institution with evaluations below the mean may in fact be good teachers. In other departments, most faculty will be ranked as “poor,” and some or even all of those ranked above the mean may be poor teachers.
Another problem is the belief that judgments can be made about the relative merit of faculty members even though response rates of students can vary widely from one class to another. For example, if the response rate by students in one class is higher than 90 percent while in another class of the same size the response rate is 50 percent, no comparison of the sets of responses is statistically valid.5
Beyond the concerns about the interpretation of numerical data, a growing body of evidence suggests that student evaluations create pressures that work against educational rigor. Rather than exclusively measuring teaching effectiveness, evaluations tend also to measure the influence of personal style, gender, and other matters extraneous to the quality of teaching.6One possible way to moderate such influences is to include questions directed at student self-evaluation in evaluations of teaching (for example, did the student dedicate sufficient time to course work?).
Further evidence suggests that the use of student evaluations in faculty personnel decisions may produce incentives to weaken or dilute course content and contribute to grade inflation. This problem can be acute for tenure-track faculty and especially for part- and full-time faculty who serve with no expectations of reappointment and who are subject to nonreappointment at the discretion of the administration. Because these faculty members typically have no security of appointment, they may come under pressure to give higher grades than students deserve to improve their teaching evaluation scores.7 The pressure could be diminished in several ways: through reevaluations with established criteria for faculty members on renewable term appointments; through appointment and reappointment decisions based on criteria specific to the positions; and through opportunities for faculty on renewable term appointments, part or full time, to move into tenured or tenurable positions.8
The faculty should have primary responsibility for developing reliable methods for evaluating teaching, which should include a clear division between questions that are appropriately answered by students and those that are appropriately addressed by peers. The numerical scores that students give instructors on evaluations should be interpreted with valid statistical methods, and a comments section should be included in the instrument. Questions directed at student self-evaluation should also be included.
A separate set of concerns arises as a result of well-documented shifts from collegial forms of academic governance, with the emphasis on consultation and participation, to corporate, top-down managerial styles. The corporate model has been characterized by the tendency to judge the success of teaching and teachers in numerical terms, and by the redefinition of education as a commodity that universities are expected to sell and that students, as paying customers, are expected to consume.9 A potentially troubling effect of this approach to education is that it can create pressure on faculty members to please students rather than to educate them. In this context, teaching evaluations may be seen less as a guide for improving teaching and more as an important marketing tool, which online capabilities can extend beyond their traditional scope and purpose. Thus it is not unknown for links to teaching evaluations to be provided in online course catalogues, or for class-by-class, year-by-year evaluation scores to be posted on the Web and thus made available to all faculty and students. Since these measures do not require permission of the instructor, faculty oversight of the evaluation process can be compromised. We urge faculty members to take an active role in formulating and adopting policies that specify the proper use and dissemination of student evaluations of teaching, including the purpose and scope of their online availability.
The relationship between faculty members and those in positions of institutional authority, including department chairs, has also changed. In collective bargaining situations, consideration should be given to including contractual provisions to restrain the potential for administrators to extend their role in the evaluation process by requiring a single approach or evaluative method of the faculty within a department or school, or by mandating the inclusion of certain quantifiable features of evaluation, such as timetables for improvement and specified educational outcomes.
With regard to the role of administrators in the evaluation process, the Statement on Teaching Evaluation states, “Emphasis in evaluation should be upon obtaining first-hand evidence of teaching competence, which is most likely to be found among the faculty of a department or college and the students who receive instruction. Evaluation of teaching in which an administrator’s judgment is the sole or determining factor is contrary to policies set forth in the Statement on Government of Colleges and Universities.” We reaffirm the primacy of the faculty in the evaluation of teaching and encourage collective bargaining units to pay careful attention to provisions concerning teaching evaluation. In addition, we recommend that institutional evaluation policies preserve the essential role of scholarly peer evaluation of courses and teaching performance.
The increasingly important role that technology has assumed in the instruction of virtually all disciplines has raised an issue of academic freedom: the widespread expectation that instructors use particular forms of technology in their courses. Evaluation questionnaires often ask about the use of technology in the learning experience (for example, How many assignments used Web-based resources, or How often did the instructor use the smart workstation for lecture demonstrations?). The routine inclusion of such questions seems to suggest that faculty members are expected to use these technologies in their classrooms, even though they may have no pedagogical reason to do so. Instructors should be free to determine the extent to which they employ technology in their classrooms according to their professional assessment of its benefits and costs to instruction in a particular subject matter.
The authors of the 1975 statement advocated the evaluation of teaching for the “development of the teacher and the enhancement of instruction.” Instructors may accomplish these purposes in several ways. One is to make more extensive use of the teaching portfolio. The portfolio typically contains a statement of teaching philosophy, copies of syllabi, sample corrected work and other course materials, and summaries of student evaluations of the faculty member’s teaching. Teaching centers have also proved valuable. They provide faculty members with access to fellow teachers or mentors of recognized quality, to courses and workshops on pedagogical matters, to assistance in videotaping of classes, and to libraries with relevant literature. We encourage the use of classroom visitation by peers on a regular basis with advance notice of such visits. We recognize that the logistics of implementing peer evaluation, with at least two or three visits to each class for each evaluation, may be cumbersome. Recognition of the practical difficulties, however, does not relieve members of the profession of their obligation to ensure the quality of teaching, or their responsibility to evaluate excellence in teaching performance according to knowledge of the subject matter and of teaching methods that students may not possess. With respect to guidelines for classroom visitation, we reiterate those outlined in the 1975 statement: “There must be an understanding among the visitors and the visited upon such matters as who does the visiting, how many visits are made, what visitors look for, what feedback is given to the visited, and what other use is made of the information.”
In conclusion, institutions, departments, and faculty members should ensure that the evaluations of teaching promote and sustain excellence of teaching and education, that faculty be primarily responsible for devising systems of evaluation and monitoring their use, and that the development and implementation of teaching evaluation methods be consistent with principles of academic freedom and shared governance.10
Committee on Teaching, Research, and Publication
Carol Simpson Stern (Performance Studies), Northwestern University, chair
Linda L. Carroll (Italian), Tulane University
Kevin Mattson (History), Ohio University
Daniel P. Murphy (History), Hanover College
James T. O'Brien (Physics), Montgomery College
Paul Ortiz (Community Studies), University of California, Santa Cruz
Phillip H. Smith (Cell and Developmental Biology), State University of New York Health Science Center at Syracuse
Karen G. Thompson (English), Rutgers University
Robert A. Wheeler (History), Cleveland State University
Mary Gray (Mathematics), American University, consultant
Anita Levy, Association staff
1. Since the publication of the 1975 Statement on Teaching Evaluation, the Association has addressed the issue of college teaching in numerous policy statements. Published statements in Policy Documents and Reports, 9th ed. (Washington , D.C. : AAUP, 2001), include Mandated Assessment of Educational Outcomes (1991), 166–75; The Work of the Faculty: Expectations, Priorities, and Rewards (1993), 158–61; The Assignment of Course Grades and Student Appeals (1997), 113–14; Statement on Faculty Workload with Interpretive Comments (2000), 153–57. Published articles in Academe: Bulletin of the AAUP and its predecessor, the AAUP Bulletin, that address this issue include Wilbert J. McKeachie, “Student Ratings of Faculty: A Reprise,” 65 (September–October 1979): 384–97; Steven M. Cahn, “The Authority of the Teacher,” 68 (May–June 1982): 13–14; Kenneth E. Eble, “Teaching’s Highest Aims,” 70 (March–April 1984): 10–16; Bill Barol, “The Threat to College Teaching,” 70 (November–December 1984): 10–17; Rouben Cholakian, “The Value of Evaluating,” 80 (September–October 1994): 24–26; “Education Bytes: The Problems and Promises of Technology,” 85 (September–October 1999): 1–68; “How Are We Doing? Assessment, Accountability, Accreditation,” 86 (January–February 2000): 1–50; Douglas A. Davis, “Millennial Teaching,” 89 (January–February 2003): 11-22; Steven M. Cahn, “Taking Teaching Seriously,” 90 (January–February 2004): 32–33; Dan Bernstein and Randy Bass, “The Scholarship of Teaching and Learning,” 91 (July–August 2005): 37–43; and Sherry Lee Linkon, “How Can Assessment Work for Us?” 91 (July–August 2005): 28–32. Back to text
2. Association policy documents and reports that address these developments include Contingent Appointments and the Academic Profession (2003), and the following, all of which appear in Policy Documents and Reports; The Status of Part-Time Faculty (1980), 57–67; On Full-Time Non-Tenure-Track Appointments (1986), 69–76; The Status of Non-Tenure-Track Faculty (1993), 77–87; and Post-Tenure Review: An AAUP Response (1999), 50–56. Back to text
3. For useful sources on teaching evaluations, see Nancy Chism, Peer Review of Teaching: A Sourcebook (Bolton, Mass.: Anker, 1999); W. J. McKeachie and M. Kaplan, “Persistent Problems in Evaluating College Teaching,” AAHE Bulletin (February 1996): 5–8; Peter Seldin, “Current Practices—Good and Bad—Nationally,” in Changing Practices in Evaluating Teaching: A Practical Guide to Improved Faculty Performance and Promotion/Tenure Decisions, ed. Peter Seldin (Bolton, Mass.: Anker, 1999). Back to text
4. For an in-depth review of online evaluations, see D. Lynn Sorenson and Trav D. Johnson, eds., “Online Student Ratings of Instruction,” special issue, New Directions for Teaching and Learning, no. 96 (San Francisco: Jossey Bass, 2003), especially Nedra Hardy, “Online Ratings: Fact and Fiction,” 31–38, and Christina Ballantyne, “Online Evaluations of Teaching: An Examination of Current Practice and Consideration for the Future,” 103–13. Back to text
5. See Mary Gray and Barbara Bergmann, “Student Teaching Evaluations: Inaccurate, Demeaning, Misused,” Academe 89 (September–October 2003): 44–46. For reader letters responding to the article, see Academe 90 (January–February 2004): 3–6. Back to text
6. See Daniel Hamermesh and Amy M. Parker, “Beauty in the Classroom: Professors’ Pulchritude and Putative Pedagogical Productivity.” See also Faith E. Fich, “Are Student Evaluations of Teaching Fair?” Computing Research News 15 (2003): 2–10. Back to text
7. For a useful review of the research on the relationship between course content and student evaluations of teaching, among other issues, see Robert E. Haskell, “Academic Freedom, Tenure, and Student Evaluation of Faculty: Galloping Polls in the Twenty-first Century.” Education Policy Analysis Archives, v. 5, 1997. On the relationship between grade inflation and teaching evaluations, see Jane Buck , “The President’s Report: Successes, Setbacks, and Contingent Labor,” Academe 87 (September–October 2001): 18–21; Valen E. Johnson, Grade Inflation: A Crisis in College Education (New York: Springer-Verlag, 2003); Brent Staples, “Why Colleges Shower Their Students With A’s,” New York Times, 8 March 1998; and Barbara Wendell, “Can Untenured Faculty Members Stop Grade Inflation?” Chronicle of Higher Education, 13 December 2001. Back to text
8. See The Status of Part-Time Faculty and The Status of Non-Tenure-Track Faculty, especially the following passage from the latter document:
The performance of faculty members on renewable term appointments, full time and part time, should be regularly evaluated with established criteria appropriate to their positions. Failure to evaluate professional appointments diminishes the institution and the professional standing of the faculty. Evaluation of performance provides essential information for sound and fair institutional decisions regarding compensation, promotion, and tenure.
See also Contingent Appointments and the Academic Profession, especially the following passage:
Collegial support of academic freedom for the profession requires conscientious and thorough reviews of the work of all faculty members, including contingent faculty. Reviews should be conducted by faculty peers and should be structured to permit faculty members to demonstrate their competence and accomplishments in their respective fields. The records of reviews should validate faculty members’ effectiveness in their positions. Appointment, review, and reappointment processes should incorporate accepted practices of academic due process, and should give careful attention to the quality of education that the faculty member contributes to the institution. Back to text
9. For an informative discussion of the corporate restructuring of higher education, see Jennifer Washburn, University Inc.: The Corporate Corruption of Higher Education (New York : Basic Books, 2005). See also Joan Wallach Scott, “The Critical State of Shared Governance,” Academe 88 (July–August, 2002): 41–49, and “Report of the Association of Departments of English Ad Hoc Committee on College and University Governance,” Association of Departments of English Bulletin 129 (fall 2001. Back to text
10. In Policy Documents and Reports, see 1940 Statement of Principles on Academic Freedom and Tenure, 3–10, and Statement on Government of Colleges and Universities, 217–23. Back to text
AAUP statements not available on the AAUP's website are available by request from Nanette Crisologo.