Insecure in the knowledge

How do you measure academics' scholarship? That's a question that worries Timothy Gerber, professor of music at Ohio State University. "Let's say a professor goes to Brussels to give a paper on a 15th-century composer. Wonderful! The professor now has an international reputation. How many were in the audience? Eight. No one asks about impact."

This may sound like bliss to UK academics who have the word "impact" ringing in their ears as the country gears up for the first research excellence framework. In the highly decentralised US system, individual institutions make the majority of the decisions about what kinds of research and scholarship to value. Defining and assessing scholarship is central to the debate over criteria for academic tenure and promotion - but some believe that the criteria used too often encourage the academy to be inward-looking.

Take, for example, research conducted by Kent State University sociologist Jerry M. Lewis for his 2007 book, Sports Fan Violence in North America. His work was widely quoted in popular media outlets such as Sports Illustrated, which has a circulation of over 3 million. Yet "that would count for nothing in promotion and tenure today", says Lewis. "Your research has to be in a refereed journal and it has to be in the right refereed journal."

The stakes are high. The criteria for academic tenure ultimately affect an institution's ability to win multimillion-dollar research grants, its reputation, teaching and public service missions and, of course, the careers of its academic faculty.

The tenured faculty member is not yet on the endangered species list in the US, but it is well known that landing a tenure-track job is tougher than it used to be. In 1975, more than half of all academics in higher education in the US were tenured or on the tenure track. By 2005, the number was down to less than a third. The latest (2009) statistics from the US Department of Education show a further drop to just over 30 per cent.

Moreover, these numbers do not tell the whole story; if graduate assistants are added, tenured and tenure-track faculty may account for as little as a quarter of instructional staff. Public universities with graduate programmes and elite private institutions have the highest ratios. At two-year community colleges, most courses are taught by contract or part-time academic faculty known as adjuncts.

"It's a continuing slide," says John Curtis, director of research and public policy for the American Association of University Professors, the trade union that represents both tenure track and non-tenure track faculty at higher education institutions with collective bargaining agreements.

"Institutions have been trying to keep their costs down by hiring part-time faculty who don't receive health or pension benefits and have no job security," says Curtis. "What seemed to be a temporary cost-saving measure has now become a major feature of the academic workforce."

The AAUP acknowledges that universities face financial problems, but says the decline in tenure-track hiring also reflects a change in budget priorities.

"We've seen spending shift away from instruction and research to support services and facilities - student housing, food service, new gymnasiums, the things that attract students to the campus," says Curtis.

In a recent survey of university leaders, seven out of 10 said they would prefer most academic faculty to work under long-term or annual contracts. Only half of presidents at four-year public institutions and just 30 per cent at private, four-year colleges wanted full-time tenured or tenure-track faculty to make up the majority of their academic staff.

"There's no question that professors like me are disappearing," says Frank Donoghue, a professor in the department of English at Ohio State and the author of The Last Professors: The Corporate University and the Fate of the Humanities (2008). "Any time a professor retires, you hire part-time labour." Poorly paid adjuncts with heavy teaching loads "don't have a reason to be loyal to the universities they work for and not much reason to be loyal to the students. Over time, without any policy change, you have the flexible labour force that presidents want."

"I refer to that as the F word," says Curtis, "because when administrators talk about flexibility what they mean is the flexibility to get rid of people when they're no longer convenient."

The survey results did not surprise some online commentators. "College presidents don't like tenure?" wrote one. "In other shocking news, corporate bosses prefer non-union labor to union labor, major polluters don't like environmental regulation, and criminals don't like cops."

The AAUP sees tenure as the best way to protect academic freedom and give faculty members a role in decision-making. However, the system has continued to come under attack from political leaders, lobby groups and even from some within the academy.

University of Chicago economist Steven D. Levitt, the co-author of the influential 2005 book Freakonomics: A Rogue Economist Explores the Hidden Side of Everything, set off a lively debate with his 2007 blog posting "Let's just get rid of tenure (including mine)", calling for an end to the practice in his discipline, economics. Wesleyan University historian Claire B. Potter, who blogs as the "Tenured Radical", argues that tenure destroys mobility in the job market, and "creates a kind of power that is responsible and accountable to no one".

Writing in the US press in January this year, former American University provost Milton Greenberg denounced tenure's "dirty little secret, its protection of doctrinal orthodoxy and curricular inflexibility to accommodate long-serving faculty". In a difficult economic climate, he argues, political leaders "are not sympathetic to claims to extraordinary privilege such as lifelong employment for tenured faculty". However, defenders such as AAUP president Cary Nelson argue that the system provides the "stable, dedicated workforce" that is essential to good teaching and research, shared governance and academic freedom.

At most institutions, academics are considered for tenure after a six-year probationary period. Several groups, including the American Council on Education and the Alfred P. Sloan Foundation, have called for a more flexible system, with a 10-year timeline and options for part-time work and personal and professional leave of absence.

"What's so magical about six years?" asks Cathy Trower, research director of Harvard University's Collaborative on Academic Careers in Higher Education.

"Scholars develop differently and have quality of life issues. If a woman could come up sooner for tenure she might still be able to have a child. I support a system where you can come up in five years if you're ready, or 10 years if that's what it takes."

Trower argues that the "six-years-up-or-out" system puts pressure on faculty to publish as much as they can, rather than focus on research that advances their field. Ohio State's Gerber agrees. "Does it really matter if your paper on Nathaniel Hawthorne is published in February 2012 or February 2013?" he asks. "The only place where it really matters is in the timetable for your tenure clock."

Jeffrey Bowman, professor of history at Kenyon College, thinks the debate over whether tenure is good or bad misses the point. "No single system of tenure is going to be right for all institutions," he says.

With tenure under a double threat on economic and moral grounds, institutions are forging new paths to academic tenure, but they are taking different routes. Differing concepts of tenure at three Ohio institutions - Kenyon, Kent State and Ohio State - illustrate what Bowman calls "one of the strengths of higher education in America - its polyculture".

Kenyon, a selective private liberal arts college in rural central Ohio, has about 1,600 students and a student-to-faculty ratio of 10:1. Almost all Kenyon faculty members are tenure track or tenured. Individual departments set tenure criteria, but "the primary driver is teaching excellence", says provost Nayef Samhat.

Kenyon academics bring their research into the classroom rather than engaging in what Samhat, a political scientist, calls "the paper chase within the academy that encourages the proliferation of journals and conferences and publication at all costs. The narrower the scholarship, the more difficult it is to integrate it with teaching - and that's our primary mission at Kenyon."

Bowman, who serves as chair of the faculty, says departmental tenure guidelines "talk about standard models of scholarship" but also accept applied research that serves the community and non-academic audiences.

Kenyon's tenure principles are in the spirit of the so-called Boyer model of scholarship - discovery, integration, application and pedagogy (see below).

At Kent State, a public university in north-eastern Ohio with almost ,000 students on its main campus and 15,000 more spread across seven regional campuses, Boyer's principles came to be embraced under the presidency of Carol Cartwright (1991-2006).

"Boyer legitimated the scholarship of teaching," says Lewis, an emeritus professor. "I ended up writing a book on teaching introductory sociology." Faculty members in Kent's professional schools (including journalism, architecture, education and nursing) found that the Boyer model rewarded applied research, public presentations and media work. By 2000, Boyer was the coin of the academic realm; a presentation to faculty senate was titled "When 'Boyer' became a verb: The process of changing policy, practice and culture".

But Luett Hanson, associate dean of the College of Communication and Information at Kent State, believes that the Boyer approach did not result in major changes in departmental tenure and promotion guidelines. "Boyer's treatise said faculty do many things that don't fall into the traditional definition of scholarship and they should be valued, but it didn't tell anybody how to do that," she says. "It wasn't a set of definitions, it didn't explain how integration differs from discovery. It was fuzzy."

And Albert Ingram, associate professor of instructional technology, says some Kent State departments misinterpreted Boyer and thought that faculty members had to show scholarship in all four areas. "I saw people who were very good at traditional empirical research obsessing over whether they had enough application scholarship. I thought that was pretty silly."

The definition of scholarship began to shift in 2006 with the arrival of a new president, Lester Lefton. His mandate was to raise Kent's profile from the "high research activity" tier to the "very high research activity" tier in the Carnegie Classification of Institutions of Higher Education. From mid-2008, administrators led by provost Robert Frank began working with the faculty senate to revise the university-level tenure and promotion policy.

"It's my fervent hope we don't use Boyer any longer," Frank told faculty in a 2009 presentation on preparing their dossiers. The model, he said, tends to "obfuscate and confuse" the process.

New university guidelines adopted in 2010 dropped the four Boyer categories in favour of a broader statement about scholarship with an emphasis on "external measurements of quality". Boyer, says Frank, "had broadened the criteria to such a degree that scholarly productivity was diluted in the decision-making process".

Each department at Kent State sets its own criteria for scholarship. Currently, faculty members have the option of applying for tenure under the 2010 university guidelines or the previous Boyer policy.

"Probably 90 per cent of what counted in Boyer still counts," says Frank. "It's just presented differently and there's a bit of a different emphasis around scholarly activity. I don't think anyone was really disadvantaged."

Although some academics endorsed the change, others felt that it created a double standard: they feared that unwritten rules at the college and university levels about what kinds of scholarship were valued might penalise those who had been following the Boyer criteria.

"It's very difficult for a mid-tenure-track faculty member to rejig," says Kara Robinson, Kent State chapter president of the AAUP. For the past four years, the AAUP's Reappointment, Tenure and Promotion counsellors - faculty members who help others with dossier preparation and appeals - have seen "steady business", she says.

While Kent State has moved away from the Boyer model in an effort to raise its research profile, the leading public research institution in the state, Ohio State University, seems to be going in the opposite direction.

Ohio State, with almost 57,000 students on its main campus in Columbus, has the third-largest enrolment in the US and an annual budget of more than $5 billion (£3.16 billion), larger than those of some state governments. With almost 3,000 tenured and tenure-track academic faculty, large scientific and medical research centres and 113 doctoral programmes, it is one of the 108 Carnegie "very high research activity" institutions in the US. It ranks ninth among public universities for research spending and second of all universities for industry-sponsored research.

That accomplishment is not sufficient for president E. Gordon Gee, named by Time magazine in 2009 as one of "The 10 best college presidents in the US". In his annual address to faculty members that year, he called for a change in promotion and tenure criteria in order to recognise "quality and impact".

"Some arbitrary volume of published papers, on some narrowly defined points of debate, is not necessarily more worthy than other activities," said Gee. Instead, Ohio State should value applied research that has an impact on people, he argued. "We can dare to say, 'No more' to quantity over quality. We can stop looking at the length of a vita and start measuring its heft."

Implementing the grand vision is challenging. Ohio State has more than 100 tenure-granting units, each with its own set of criteria. Donoghue says he welcomes a broader definition of scholarship, and hopes that digital publishing will be recognised. "But I still don't know what the president means by 'interdisciplinarity'," he says. "In the English department, we're baffled about how to interpret that."

On 16 February, the university senate adopted new definitions of teaching, research and service, the result of months of debate and wordsmithing by a rules committee. The definition of research was broadened to include "discovery, scholarly and creative work, applied research and the scholarship of pedagogy". Although it is not called the Boyer model, it includes three of the four Boyer categories.

In his address to Ohio State's senate, provost Joseph Alutto cited Gee's goal of moving the university "from excellence to eminence". He noted that it lagged behind other institutions in commercialising research, and he pledged an 8 to 10 per cent increase in the number of tenure-track faculty in a bid to help make it one of the top 10 public research universities in the country by 2020.

Alutto said several of the university's constituent colleges had already adopted more flexible criteria, particularly with regard to promotion to the rank of full professor. Gerber, who served on the rules committee, says the sciences and some social science disciplines will still give weight to the number of publications and citations in a promotion or tenure dossier, but that the "quantity measure" should not be the general rule.

"The whole way we conduct research, and produce and distribute it, isn't working," says Donoghue. "Everyone has to produce research as a credential for promotion and tenure, but no one is under any obligation to consume it. No one (except university libraries) buys academic monographs. If the general public knew this they would be all over university presidents to increase teaching loads."

At a time when universities are struggling to meet teaching needs, academics are too often rewarded for "esoteric research", says Harvard's Trower. "A lot of what is published is read by only very few and it has little impact. Who is judging the quality?"

Without government policies to assess and reward certain kinds of research, quality will continue to be judged by institutions - from department promotion and tenure committees to senior administrators, all with their different priorities. How they make these judgements, and what they value as scholarship, will determine the future of academic tenure in the US.

The Boyer Model: wide definitions of scholarship

In an influential 1990 book, Scholarship Reconsidered: Priorities of the Professoriate, Ernest Boyer, former chancellor of the State University of New York system and president of the Carnegie Foundation for the Advancement of Teaching, called for a revolution in what counts in higher education.

He proposed broadening the scope of traditional scholarship, dividing it into four categories:

• scholarship of discovery: original research or creative activity that advances knowledge

• scholarship of integration: bringing together knowledge across disciplines, or across topics within a discipline

• scholarship of application (also called scholarship of engagement): using research findings to address societal problems

• scholarship of teaching: systematic study and evaluation of teaching and learning processes.

Research policy in the US: funding and influence

In the decentralised US higher education sector, the ability of government agencies to push research in a particular direction is limited.

Although agencies such as the Department of Defense, the National Science Foundation and the National Institutes of Health provide major research funding, the federal government has little formal role in higher education beyond funding student loan and grant programmes.

At the state level, legislatures provide oversight of public universities and (in good budget times) fund research, usually to promote economic development. However, as state budget subsidies have shrunk, forcing universities to rely more heavily on income from student tuition fees and from investments, state governments have lost some leverage.

Universities with research ambitions look to two non-government bodies for recognition.

The Carnegie Foundation for the Advancement of Teaching classifies doctorate-granting institutions into three tiers - very high research activity, high research activity and doctoral/research.

The National Research Council rates the quality of about 5,000 doctoral programmes in almost 60 fields. Its latest rankings, finally released in 2010 after a lengthy dispute over criteria and methodology, drew criticism from some in the sector because they were based on data collected in 2006-07.