Report underlines difficulty of benchmarking scientific productivity

Brussels, 20 Sep 2002

A report published by the European Commission has highlighted the difficulty of measuring scientific and technological productivity.

The study follows up on a call from the Lisbon Council of March 2000 for a 'new open method of coordination' of RTD (research and technological development) policies. A series of benchmarking reports have now been finalised, which compare processes and performances, identify best practices and establish a continuous process of mutual learning.

'Science and technology cannot easily be defined in categories of simple input-output relations. They comprise a variety of actors, missions, objectives and organising principles, and can only compared in their 'systemic' context,' reads the report.

The paper recognises that 'science and technology are recognised as producing tangible and intangible, codified and tacit outputs.' Output can be measured to an extent by the number of scientific publications published and patents acquired, but the report writers acknowledge that this information provides only half of the story.

'Scientific publications and patents are only intermediate outputs of research. Their measurement is a partial proxy for the achievement of broader goals in the advancement of knowledge and achievement of social and economic progress. Their use should be conditioned by these limitations,' says the paper.

In spite of their limitations, this data does present some interesting facts. For example, in terms of scientific publishing and patenting, the statistics show no evidence of the EU lagging behind the USA and Japan. The report claims that instead, these figures point to an 'input gap'.

In absolute numbers, the EU represents the largest source of scientific publications, slightly ahead of the USA, and way ahead of Japan. In publications per inhabitant, the EU is ahead of Japan, but lags behind the USA. This gap almost halved however between 1995 and 1999. In terms of publications per money spent in university research, the EU comes ahead of the USA and Japan, and doubled its lead between 1995 and 1999. Looking at patents per money spent in business R&D (research and development), Europe lags behind the USA and Japan

Within the EU, two thirds of EU publications come from the UK, France and Germany. In publications per inhabitant, the Nordic countries lead, followed by the Netherlands and the UK.

Publications per money spent in university R&D show a wide divergence around the EU average. Top of the table are the UK, Finland and Denmark with 40 per cent above the mean, while Germany and Portugal are only 10 per cent above the mean.

The number of patents per business R&D expenditure also shows a substantial variation between EU Member States, with Sweden, the Netherlands and Germany as top scorers. The sectoral composition of industry strongly influences such rankings, with countries with high levels of activity in machinery, chemicals, communications equipment and electrical components are more likely to score highly in numbers of patents.

The report writers believe that such a benchmarking study allows countries to ask questions about the configuration of their research and innovation systems. 'Comparison may also be seen as a driver to improve performance,' states the report.

To see the full report, please consult the following web address: ftp://ftp.cordis.lu/pub/rtd2002/docs/bench_stp_0602.pdf

CORDIS RTD-NEWS/© European Communities, 2001