Practise what you preach

Like the proverbial physician unable or unwilling to heal himself, attending to the ills that are closest to home can often figure low in higher education's priorities.

Writing in 1963, Sir Eric Ashby, then master of Clare College, Cambridge, observed that "all over the country, these groups of scholars, who would not make a decision about the shape of a leaf or the derivation of a word ... without painstakingly assembling the evidence, make decisions about admission policy, size of universities, staff-student ratios, content of courses and similar issues based on dubious assumptions, scrappy data and mere hunch".

As a profession, higher education is not alone in avoiding self-analysis, but because its raison d'etre is, in large part, to help others forge an evidence-based path, such inadequacy would seem to be particularly incongruous.

Nevertheless, many argue that just such a blind spot exists, and that it is exacerbated by fundamental problems with the profile of higher education research, its focus and scope, and the unwillingness of governments and policymakers to pay it the attention it deserves.

Roger Brown, a former vice-chancellor and civil servant who is now a professor of higher education, has an insider's view of how the discipline is looked upon, both by the sector and by the Government.

For Brown, the whole enterprise is undermined by the low esteem in which schools of education are held by the academy, a disadvantage compounded by the status of higher education research within those schools.

"Within universities, higher education is not a highly regarded subject area," he says. "I can think of only a handful of vice-chancellors who have had education as their academic discipline.

"The research assessment exercise is partly responsible for its low status because it tends to denigrate the practice-based disciplines; but on top of that, if you take any typical university school of education, most of its effort goes to training and developing teachers, so higher education is a bit of a Cinderella within a Cinderella.

"There seems to be a genuine blind spot, and this is not confined to the UK, where people in academia simply do not practise what they preach. It may be true of all professions, but it is particularly obvious in higher education because that is our profession."

The sense that the study of higher education lacks credibility or prestige within the sector is shared by David Dill of the University of North Carolina at Chapel Hill in the US. He describes how interest in, and funding for, his work grew only when, mid-career, he switched schools, from education to public policy.

"I speak with personal experience on this," he says. "After I made that move, I received two major grants from the Ford Foundation in the US and started getting invited to lots of review panels in Europe.

"So the fact that a lot of research in higher education is being done by people who are located in schools of education probably does diminish the influence of that research."

Another American, Philip Altbach, director of the Centre for International Higher Education at Boston College, points to the closure of the School of Education at his alma mater, the University of Chicago, to make a similar point. "That the school was closed down shows you where the fields of education in general, and higher education in particular, rank in the hierarchy of the disciplines - typically at the bottom, maybe a little bit above nursing or library studies," he says.

So far, so gloomy, but Brown is quick to point out that such concerns are nothing new.

However, while worrying that the discipline is, to a certain extent, in the doldrums, Brown, who is now at Liverpool Hope University, is adamant that this was not always the case.

"In the early 1980s, the Leverhulme Foundation funded a huge programme of research into all different aspects of higher education in the UK. A lot of it is still relevant today, and there was a generation of academics who made their reputations through the project, people such as Gareth Williams, Oliver Fulton and Tessa Blackstone. There was a whole group of these people who went on to get their chairs on the basis of this research, but there has been no successor generation really."

Gary Rhoades, the outgoing head of the Centre for the Study of Higher Education at the University of Arizona and the next general secretary of the American Association of University Professors, agrees that the discipline has lost its way in the UK.

"In my view, you folks have a lost generation or two. There are giants in the field such as Maurice Kogan, Tony Becher, Gareth Williams, Oliver Fulton, Mary Henkel, to name a few, but they are all quite senior. And frankly, you don't have a large stock of younger scholars coming up.

"That's partly because you don't really have these departments of higher education training scholars and partly because, as in much of Europe, the structure of your academic career, the demographic pattern, and your relative lack of investment in secure positions in the academy has deprived you of intellectual capital in this realm."

Brown adds that the focus on training teachers and school-related research has left few schools of education with a "critical mass" of higher education experts.

"The only place that does have that critical mass is the Institute of Education (University of London), and even that isn't very large by international standards," he says. "The Centre for Higher Education Policy Studies at the University of Twente in the Netherlands, for example, has as many people working on different aspects of higher education research as I would reckon we have in the whole of the UK, so it's quite a serious weakness."

In assessing the state of the discipline, a clear distinction must be made between practice-based and policy-based research.

Dill, currently on a visiting professorship at the European University Institute in Florence, says that, if anything, there has been an "explosion" of research around the world over the past 15 years, tied to the "massification" of the higher education sector.

The problem, he says, is that very little of it is policy related. "There are many more journals of higher education in Europe, and a lot more literature is being published now than at any other point in my lifetime. This happened in the US several generations ago with the development of a mass system of higher education after the Second World War.

"Now that's being paralleled in the UK and elsewhere, so from an empirical point of view the amount of higher education research has expanded. One question, though, is how much of this research is policy related, and my criticism would be that much of it isn't. Much of it is very qualitative, focusing on internal matters of the university, and although there are policy areas intrinsic to that, there are also a lot of other policy areas that are being ignored."

While Brown calls for "fresh blood" to reinvigorate the discipline, Dill insists that many of the researchers he has encountered and worked with in Europe have been "bright and clever" and that the issue is not their calibre but the orientation of their research.

He says: "There's no question that if you do quantitative research, and perhaps economically oriented quantitative research, you are going to have more influence.

"That can be done within schools of education, but there aren't many of those people there. The majority are doing qualitative work, and you can see that if you go to any of the professional meetings: I search in vain for quantitative research, and in Europe that seems to be increasingly the case."

In the UK, last month's Society for Research into Higher Education (SRHE) conference offered something of a sampler of the body of research being produced.

A cursory sift through the papers presented suggests that, as Dill has found elsewhere, quantitative research is in the minority.

Ron Barnett, professor of higher education at the Institute of Education, is a former chair of the SRHE and remains a member of its governing council. He says that while it is important to acknowledge the problems, it is also vital to recognise the positive developments and the work of younger researchers in the field in recent decades.

"I am so aged that I can remember going to annual conferences of the SRHE in the 1970s when there were perhaps 60 people there and it was a very low-level, conversational meeting. Now the annual conference attracts 300 or more people, with papers from around the world.

"As an adjunct to that, for the past three to four years there has been a separate conference for research students that has attracted between 50 and 100 young researchers from around the world. So, over the past ten to 20 years, this has developed into a major field of research in its own right. That said, there is still some way to go."

Among his concerns are the "narrowness" of the field, the fact that quantitative studies are "few and far between" and the scarcity of high-level theoretical studies.

"It has often been remarked over the decades that higher education will study everything except itself, and the result is that the field of higher education studies is still relatively parochial.

"I would say, in theoretical terms, we are not drawing sufficiently on the major cognate fields, in philosophy and sociology, and I would want to throw in anthropology, economics and theology to that list, too. There are all manner of fields that should be contributing more; the field is narrow theoretically as well as quantitatively."

For Brown, a consequence of these limitations is that not enough of the research being done is relevant to policymakers, which stunts the field's profile.

"There isn't much interaction between researchers and policymakers, and the British Government has a longstanding resistance to incorporating academic thinking into its policymaking, so this is a demand problem as well as a supply problem," he says.

Barnett, while arguing that Brown underestimates the vitality of the field, agrees that there are issues here, although he suggests that the blame lies in several quarters.

"I've always tried to write in a way that reaches out to people, even though I am talking about quite abstract matters, and I'm not sure that (other) researchers in higher education studies do that sufficiently.

"That could be said about most social and philosophical fields of intellectual endeavour, but nevertheless, there are issues there and the SRHE is conscious of that, because it has a responsibility in effecting a bridge both with policymakers and the Government as well as with vice-chancellors and the like in the sector.

"As a senior member of the organisation, I wish we'd made more progress on that front than we have."

He continues: "A limitation of research into higher education is that it is unduly self-serving. A lot of it analyses and critiques current policies or the way the world is in higher education, but - and this is perhaps why policymakers aren't interested - it fails to offer ways forward, it insufficiently tries to look into the future and provide imaginative ideas for how we might do things better.

"That is a defect in contemporary research into higher education, and it has become much more marked in recent times as the field has become more professionalised ... I see the responsibility of the researcher or intellectual as trying to move things forward, but I am afraid a lot of my research colleagues don't have sympathy with that point of view; they don't see it as part of their intellectual responsibility."

A point made by several commentators is that there is more to higher education research in the UK than simply the output of schools of education.

While Barnett argues that tangential fields could offer more to higher education research, there is a degree of consensus that some of the most influential work of recent times has been conducted by economists and sociologists.

One high-profile example is Nicholas Barr, professor of public economics at the London School of Economics, whose research on student loan models has had a wide-reaching and international impact.

Barnett suggests that the very fact that the likes of Barr - and, many would argue, he himself - stand out as "jewels in the crown" of higher education research demonstrates how few and far between such heavyweights are. They are, he says, individuals "beavering away, doing their own thing, but lacking a critical mass of other scholars around them".

For his part, Barr acknowledges that his position as an economist allows him to tackle aspects of higher education research that are more policy friendly than others.

"Economists are highly trained in quantitative techniques, so where data exists economists can do good work on it," he says. "I'm thinking of things such as work by Richard Blundell (professor of political economy at University College London) on the impact of education on earnings. That is high-grade quantitative work.

"People in schools of education are typically less well trained quantitatively ... (and) I have seen some work coming from education departments that I do not rate because I think that it is analytically flawed.

"Having said that, a lot of the work that education departments do is such that, while you would ideally answer the questions with quantitative data, the things that you need to measure, you can't measure. So there might be a quality issue, but there is also another issue: not that the task is impossible, but that it is made very difficult by measurement problems."

Stating that higher education research in this country "could be much stronger than it is", he says that increased competition between universities could be a catalyst for advance.

"Let me tell you a caricature story," he says. "The world we are coming from was one in which people in the Treasury doled out money to their chums the vice-chancellors - it was a gentleman's club. That world wasn't competitive, so why should universities really concentrate on what they were doing?

"As we move into a more competitive world, universities become more concerned about their own quality. My suspicion is that it is only in the past two or three years, since the 2006 reforms (and the introduction of top-up tuition fees), that most universities have started to realise that they really are in a more competitive environment.

"As a result, there is a growing interest in this now, and if the (student tuition) fees cap were to rise a bit, strengthening competition, that would also strengthen the self-interested motives that institutions have, to look at what they are doing and how they can improve. Until relatively recently, this hasn't been much of a priority."

Across the Atlantic, however, Christopher Loss, assistant professor of public policy and higher education at Vanderbilt University, a private research-intensive university in Tennessee, believes government interest in higher education policy waned once the sector became a "mature industry".

He says: "The rise of student loans, both subsidised and unsubsidised, was particularly critical. Not only did it help increase access, but it really took the Government and its policymakers out of higher education policymaking.

"It seemed as though all the real questions and debates had been settled: loans became the fallback policy, and higher education policymaking increasingly involved simply making more money available to students and their families ... Some will disagree here, but I think the case of the (2006) Spellings report (on the future of higher education in the US) says a lot. It found little interest and stimulated little debate. In sum, the rise of America's credit culture and of K-12 (primary and secondary school education) policymaking combined to derail interest in higher education policy."

He continues: "Significantly, it wasn't just the Government that became disenchanted. Higher education was also a bit weary of its relationship, so when the money began to dry up for research, it shifted focus to the private sector to help make up the difference."

Loss believes that the impact of the recent economic turmoil on student funding models in America could reverse this hands-off approach, and that it could reinvigorate demand for research.

"My guess is that interest in higher education research and higher education policy might yet make a comeback, at least in Washington. Only time will tell," he says.

Other American commentators point to Barack Obama's election as US President as another potential trigger for change.

Dill says: "Obama has shown a lot of evidence to me of being someone who is going to be even-handed and thoughtful about looking at evidence, which is in stark contrast to the Bush Administration, when we had the Spellings Commission, which was very clearly stacked with people of particular perspectives."

Altbach agrees: "We will have to wait and see, but Obama was, after all, an academic, and has an analytical way of thinking. It is interesting to me that no one has really been speculating about who will be the next Secretary of Education, which tells you something about the level of thinking about education in the US at the moment.

"But I think there will be a sea change in the way that policy is thought about in the US in all areas, including in education, under Obama."

Back in the UK, Barr gives an alternative perspective on the question of whether policymakers are as interested in higher education research as they should be.

He says: "I actually wonder how important it is that the Government is interested in higher education research, as opposed to empowering students and employers by requiring universities to disgorge the relevant information (including the results of student surveys on teaching quality and graduate employment data).

"That would strengthen the competitive pressures on universities to pull their finger out and deliver. So in a way, I am asking: how important is it that the Government is interested in these things, as opposed to being highly interested in ensuring that quality assurance maximises the information that consumers get?"

AN INTRINSICALLY VALUABLE FIELD

The late Maurice Kogan's important contribution to higher education studies was to argue that the field is of value in its own right. It does not exist as a critical tool in the box marked "evidence-based policy".

Instead, the best higher education research seeks to understand a rich internal culture, connected in a variety of ways to the environment in which it lives and moves. The outcomes include intense and valuable conclusions about human behaviour, organisational effectiveness and the sources and application of knowledge.

For some of us, this is not enough. All around the world, the expectations of higher education are growing, and they can be contradictory. Universities and colleges are expected to serve governments, economies, societies, groups and individuals in a wide variety of ways. However, the engagement between higher education researchers, practitioners and policymakers can often seem like a dialogue of the deaf.

At their best, researchers can bring a strong historical sensibility, an understanding of the wider role of universities and colleges, and novel insights. At their worst, they can be defensive, apologetic, self-serving and repetitive.

At their best, the practitioners (teachers, researchers, managers and support staff) will attempt to create, validate and use evidence about their practice. At their worst, they will want simply to be left alone.

At their best, the politicians will bring a sense of urgency, as well as resources and democratic validation, to the higher education enterprise. At their worst, they, too, can be self-serving, but also impatient and simplistic.

Perhaps most significant in all this is the lack of corporate memory: when a policy fails to be assessed against the history of the last time it was tried. The myth of Cassandra springs to mind.

A recent report from the European Science Foundation spoke eloquently about the role of higher education research in "resolving conflicting social and economic expectations", as well as overcoming "public myths".

But it also referred to the "fatalism" with which researchers approach questions such as funding and government direction. Higher education researchers need to be more than Cassandra. But policymakers should listen to her more carefully.

Sir David Watson is co-director of the Centre for Higher Education Studies at the Institute of Education, University of London, and president of the Society for Research into Higher Education.

STILL IN SEARCH OF THE GOLD STANDARD

Does higher education research in the UK influence policy and practice? It certainly does, to judge from the evidence base called upon to support the Dearing review of higher education in the 1990s as well as the many subsequent White Papers on financial reform and the more recent debate on social mobility in higher education.

However, among policymakers and across academe, the rigour, as much as the relevance, of education research has been questioned. This debate about quality has often degenerated into arguments about methodology. Quantitative methods are arguably in the ascendancy as they give answers to policy questions in quantifiable terms.

Yet some quantitative studies tend to generate as many questions as they do answers because they cannot tell us why people do the things they do. On the other hand, qualitative research findings regularly get generalised well beyond their original context.

Surely no one would claim that higher education policy should be made on the basis of interviews with a handful of students, however well conducted. Good-quality higher education research needs to adopt a healthy mix of methodologies in a robust manner, and such studies are still a rarity.

Higher education research is also by its nature multidisciplinary, since analysis of education requires the analytical tools of sociologists, economists, historians and many more besides. It is crucial however, that multidisciplinary research is recognisable as good quality by the individual disciplines concerned.

We also have to acknowledge fundamental and often implicit differences in researchers' views as to what constitutes research. Advocacy as research has a long tradition in education.

Yet surely research that is polemic rather than robust, irrelevant rather than potentially applicable to policy or practice, will be of limited long-term value. Substantial progress in improving the quality of higher education research has undoubtedly been made, but this issue will continue to challenge us in years to come.

Anna Vignoles is professor of education, Institute of Education and Centre for the Economics of Education.