It Is Time to Move Research Expenditures to the Denominator in University Metrics

An unintended consequence of using “research expenditures” as a figure of merit for universities is to reduce the research output per dollar invested by discouraging the diffusion of superior, lower-cost, open-source scientific equipment.

Many universities have begun to adopt business management practices to improve productivity and have thus simplified the collection of metrics to gauge research output. One metric that has gained traction is “research expenditures.” It is easy to see why: research expenditures reflect the faculty’s cumulative ability to acquire competitive grants and other funds. As a metric, “research expenditures” enables us, albeit a bit crassly, to compare individual faculty members on what appears to be a level playing field. It also boils down the research efforts of an entire university to a single number to be used for ranking.

Unfortunately, using research expenditures as a proxy for academic output is simplistic thinking in the best cases and has become counterproductive to the scientific enterprise.

There are clear cases where the metric fails. I leave it to those in the humanities and arts to explain why their diminutive research expenditures do not reflect the value of their work to their fields or society. In addition, the modest expenses of the theoretician in, for example, physics may obscure much more valuable contributions. For example, I vividly remember a symposium by Penn State’s Vincent Crespi where he physically used a sheet of paper to derive carbon nanotube folding characteristics, which have numerous applications potentially worth billions. What a bargain! The theoreticians, however, must be backed up by experiments. Since experiments are where the large research expenditures occur, this is where we find the best argument for using expenditures as a metric. Yet even in the strongest case for this metric—experimental science—new methods of technological development call it into question.

One important phenomenon that has come out of experimental physics is an observer effect, where the act of being observed changes what is being observed. In this case, measuring and ranking research expenditures encourages driving them up and potentially reducing economic efficiency in research. In general, researchers are frugal with their hard-earned funds, but if one of the primary metrics of success is spending, acceptance of increasingly onerous overhead rates and discouragement of investments that stretch research funding are more likely. Overheads have climbed steadily, and ones of over 50 percent are now common, effectively increasing the research expenditures for a project without materially contributing to the research itself. This is unfortunate, but far worse is the perverse incentive of the metric slowing science in a second way, by discouraging the mass diffusion and development of open-source scientific hardware.

Scientific equipment, which is normally highly customized, low-volume specialized instrumentation, has historically been extremely expensive (read: large research expenditures and high rankings for those who use it). However, as the free and open-source process, which has so powerfully transformed the Internet, is applied to hardware, we are able to radically reduce the cost of experimental research in the sciences. This is obviously good news, but the current metric of research expenditures says just the opposite. Scientists now have an option to invest a small amount of time and money into open-source equipment rather than simply pulling out the credit card and shopping at traditional equipment vendors. As I documented in my book Open-Source Lab, with dozens of examples from around the world, a revolution is occurring as formerly highly specialized, high-cost scientific equipment are increasingly custom fabricated in-house using digital designs, at tenfold or one hundredfold cost reductions. This makes it possible to stretch research expenditures much further to accomplish the same or even superior research.

How is this possible?

The combination of open-source microcontrollers and low-cost open-source 3-D printers has made an enormous range of high-cost scientific equipment customizable at the lab level for pennies on the dollar. The Arduino is a powerful yet easy-to-learn microcontroller that can be used to run a burgeoning list of scientific apparatuses. In addition, an open-source 3-D printer such as the RepRap, which costs roughly five hundred dollars, can fabricate approximately 50 percent of its own parts and can be assembled in a weekend. As many scientists have found, it is less expensive to design or download and print research tools with a RepRap rather than to purchase them, and digital designs have begun to flourish in Internet repositories. As additional research groups use this model and ignore the “research expenditure disincentive,” we all benefit. As they begin to freely share the designs of their own laboratory hardware, not only can everyone in the greater scientific community enjoy the same economic savings on equipment but, following the open-source approach, the equipment will continue to evolve and improve at a much faster rate than in closed models. Any scientific field that adopts this model will immediately gain an innovation advantage not only for equipment but also in the ability to quickly replicate, verify, and build on one another’s experimental work. For example, chemists have already begun to share reactionware designs that enable rapid diffusion of new chemical synthesis methods. In this way the science and the applications enabled by it will also accelerate for the benefit of the greater society.

We can improve the effectiveness of research funding by moving “research expenditures” to the denominator, comparing universities, colleges, departments, and faculty by “citations per research dollar spent” or similar metrics. Then those who can frugally stretch research dollars the furthest while having the greatest impact will be encouraged rather than discouraged. In addition, this will create an incentive for universities to increase their efficiency by reducing parasitic overhead rates. Most important, granting agencies should solicit proposals for open-source scientific hardware development for the tools most relevant to the fields they oversee. In this way investments now will pay large dividends in the future as science benefits from increased efficiency.

Joshua Pearce ([email protected]) is an experimentalist with significant research expenditures and thus no conflict of interest with the policy recommendation made here. Professor Pearce is an associate professor in the Department of Materials Science & Engineering and the Department of Electrical & Computer Engineering at Michigan Technological University.