A recent report from the highly respected Community College Research Center paints a skeptical picture of performance funding as practiced by states so far. The publication, which surveys current empirical literature on the subject, will likely become a focal point for discussion by Texas policy makers in the next several months. It should.
Daniel Luzer of Washington Monthly picked up on it and cut right to the chase. "Why Grade Inflation Might Happen" is the title.
It is difficult to examine the entire report ("The Impacts of State Performance Funding Systems on Higher Education Institutions: Research Literature Review and Policy Recommendations," by Kevin Dougherty and Vikash Reddy—December 2011) without taking away more questions than answers. Does performance funding actually work? What are the potential unintended consequences? Are students learning more with performance funding? Has it ever been implemented with valid, consistent results? Are there ways to game the system? These are just a few.
Most interesting from a faculty perspective are the abundant caveats in the report's narrative concerning potential grade inflation and weakening of standards. Several excerpts will be provided below, but first please read the brief abstract (emphasis added):
Over the past three decades policymakers have been seeking new ways to secure improved performance from higher education institutions. One popular approach has been performance funding, which involves use of a formula to tie funding to institutional performance on specified indicators. This report reviews findings from studies on performance funding programs in a multitude of states. The studies suggest that tying funding to outputs has immediate impacts on colleges in the form of changes in funding, greater awareness by institutions of state priorities and of their own institutional performance, and increased status competition among institutions. Because of these immediate impacts, performance funding produces intermediate institutional changes in the form of greater use of data in institutional planning and policymaking and in changes in academic and student service policies and practices that promise to improve student outcomes. However, claims that performance funding does indeed increase ultimate outcomes—in the form of improved rates of retention, completion of developmental education, and graduation—are not validated by solid data. In the face of this finding, this report identifies obstacles to the effective functioning of performance funding, as well as unintended impacts. The report closes by providing recommendations for overcoming the many obstacles to the effective functioning of performance funding and addressing the unintended impacts documented by the studies reviewed.
The report is complex and tentative enough to allow selective use of the data. For instance, those who support performance funding could argue that states just didn't devote enough funding or time for the policy to achieve its goal. As the narrative indicates, some states tried it and subsequently abandoned the effort. Also, the pervasive budget crisis of recent years has caused overall funding cuts, making it difficult to draw any conclusions independently. The report doesn't say performance funding can't work, and it addresses the obstacles constructively (see the last excerpt below, for instance). But the obstacles are numerous and formidable.
Under HB 9, passed by the Texas Legislature in 2011, "not more than ten percent" of overall formula funding would be devoted to "outcomes based" funding—a higher potential level than other states have attempted. Here's a previous post on HB 9. The bill in its final form failed to impose performance funding, but the statute does mandate an official proposal by the Coordinating Board for the 2013 Regular Session. TCCTA opposed the measure in testimony.
Faculty members virtually never express reservations about efforts to help students succeed. Summer bridge programs, enhanced advisement, and better diagnoses of student achievement are all fine with teachers, who often lead such efforts in fact. As we say often in our publications at TCCTA, no one wants students to succeed more than their teachers. The central problem arises when instructors believe that standards could be lowered or grading decisions affected by performance funding. Such teachers will find the following excerpts from the report instructive (full documentation and elaboration can be obtained in the document linked above). The emphasis is added in each passage.
On side effects:
In addition to these different elements of the theories of action underlying performance funding, we also need to consider its unintended impacts and frequent obstacles (Dougherty & Hong, 2006, pp. 69, 73). The unintended impacts constitute outcomes that are not intended by the enacting body, but which arise as side effects of funding institutions based on their performance. These can take such forms as the weakening of academic standards or the narrowing of institutional missions to those that are financially rewarded. The obstacles to the success of performance funding can include such things as performance indicators that do not adequately capture institutional performance, performance funding not keeping pace with improvements in institutional performance, and inequality in institutional capacity to diagnose performance problems and determine workable solutions.
On "gaming" the system:
Several studies on Tennessee, and one each on Florida, Missouri, Ohio, South Carolina, and Washington, document ways in which institutions try to game the performance funding program to secure high performance scores without actually improving their performance. This gaming takes two main forms: setting low institutional goals that can be easily attained and taking actions that produce apparently desirable performance but in ways that require minimal effort and are not in keeping with the spirit of performance funding.
On "Grade Inflation and Weakening of Standards:"
Three studies on Tennessee and one each on Florida and Washington discuss how performance funding leads to grade inflation and the weakening of academic standards. In Florida, several community colleges removed various obstacles to their students’ degree completion. Often these obstacles were simply unnecessary hindrances, but the obstacle clearing could also result in the elimination of difficult, but important, intellectual requirements (Dougherty & Hong, 2006, pp. 73–74). There is also some evidence that colleges are being pushed by accountability demands for higher retention and graduation rates to pressure faculty to avoid giving failing grades. The president of the American Association of University Professors chapter at a Florida community college noted: “There’s a lot of pressure to retain every single student no matter what it takes. ... We have to report every conference we’ve had, the outcome, if the student wasn’t retained, why, how many efforts were made” (as quoted in Dougherty & Hong, 2006, pp. 74–75). These faculty fears also crop up in the studies of Tennessee and Washington (Banta et al., 1996, p. 36; Freeman, 2000, p. 90; Jenkins et al., 2009, p. 39; Tanner, 2005, p. 84). In Washington, faculty and administrators at several community colleges raised the alarm that the Student Achievement Initiative might lead to pressure on instructors to lower their academic standards so that more students would pass courses and the colleges could gain more performance points (Jenkins et al., 2009, p. 39).
Also:
The pressure on colleges to resort to grade inflation and lower standards in order to retain and graduate students is not easily relieved. However, states and colleges can carefully monitor degree requirements and course grade distributions to see if they have changed substantially after the advent of performance funding. Moreover, they can conduct anonymous surveys of faculty to see if they report substantial pressure to weaken academic requirements in order to keep up rates of retention, course completion, and graduation.37 Finally, states can conduct learning assessments. While indicators for specific kinds of learning are important, consideration should also be given to general student learning (Dougherty et al., 2009).
On exclusion of students as a result of performance funding:
Two studies on Florida and one on Missouri note how performance funding can lead colleges to restrict the admission of less prepared students in order to boost their retention and graduation rates, an example of what has been called “creaming.” In Florida, a local community college official noted: There are people who may need to take a course in a program and we would not necessarily want to attract those people because you’re going to be working for performance-based funding ... in the health sciences, this is a major concern because it’s not who you start with, it’s who completes that matters. (As quoted in Dougherty & Hong, 2006, p. 75) In fact, another study found that a Florida community college had restricted enrollments to maintain program quality. Moreover, that college had eliminated its Center for Disabilities because its completion and job placement rates did not justify the high cost of running the program (Bell, 2005, p. 146).
Interesting:
At the same time, the research literature does not provide firm evidence that performance funding significantly increases rates of remedial completion, retention, and graduation. When these claims are made, they are not based on solid data that control for other possible causes of changes in student outcomes beyond performance funding. In fact, the few multivariate quantitative analyses of the impacts of performance funding on institutional retention and graduation rates uniformly fail to find statistically significant positive impacts.
Finally:
Conclusions About the Ultimate Impacts of Performance Funding
The absence of findings that performance funding does produce significant improvements in student outcomes should not lead us to dismiss it. The multivariate studies mentioned above are still too few in number to reach definitive conclusions. Moreover, they apply to the traditional form of performance funding (what has been dubbed performance funding 1.0), involving small bonuses to base state funding for higher education. They do not apply to new forms (performance funding 2.0) that embed performance indicators in the base state funding formula and involve much more money. Those new forms may have significant impacts, if only because they involve considerably greater funds. Finally, there are a host of obstacles to performance funding that, if removed, might greatly improve its effectiveness.
Comments