Benchmarking is not everything, says EU official

Brussels, 29 Oct 2004

In an increasingly globalised world, the EU should stop worrying so much about benchmarking and focus more of the framework conditions that shape localisation decisions if it wants to stay ahead of the game in research and innovation, according to a European Commission official.

Presenting his view on the 'Internationalisation of research and benchmarking of European performance' in Maastricht, the Netherlands on October, Keith Smith from the JRC's Institute of prospective technological studies in Seville, Spain, warned that many of the indicators used by the EU are not robust enough for international comparisons.

'International comparisons appear in all European Research Area (ERA) documents,' said Mr Smith. 'We often hear that the GERD [gross domestic expenditure on research and development] to GDP ratio is too low or that the patenting is too low. But too low in respect to what? Mainly the US and Japan. We must remember that in the EU, diversity, heterogeneity and variety are the norm, both in terms of companies and countries,' said Mr Smith.

The diversity of industry varies from country to country, as do patterns of innovation inputs, technological specialisation, operations of labour markets, institutional set-ups and infrastructures, explained Mr Smith.

This means that there are going to be many different possible performance indicators and that companies and countries will perform differently according to different indicators, and differently in terms of final outcome.

'The problem,' stated Mr Smith, 'is that performance indicators are not necessarily well-correlated with each other. Science and technology indicators may correlate poorly with different indicators of growth and welfare. Aggregate R&D intensity, for example, does not always correlate well with income levels and growth rates.'

Using the ICT productivity example to illustrate his point, Mr Smith explained that the US uses 'Hedonic price' indexes for calculating real output of computing equipment, while Europe uses the 'matched model' method. This means that European productivity in the computer sector has been underestimated relative to US and Japan because of statistical differences in price deflators. 'Precision of international productivity comparisons is 'severely limited', emphasised Mr Smith.

Turning to unemployment, Mr Smith said that the EU is always compared unfavourably to the US. 'But really, all it means is that the EU is better at calculating unemployment and that the US doesn't advertise the fact that it has two million men of working age in prison.'

'Thus,' said Mr Smith, 'we have enormous amounts of diversity and heterogeneity in the systems being compared. We therefore need a serious effort to improve the types of data collected, their coverage, and their collection methodologies and we need comparative analyses that look at systems as a whole and that are much more sensitive to diversity.'

To conclude, Mr Smith warned that inter-economy diversity means the EU must be very careful about numerical evaluation indicators. Furthermore, globalisation trends suggest that policies should focus more on the framework conditions (regulation, policies, infrastructure, human resources, market conditions) that shape companies' localisation decisions, rather than on specific performance targets.

CORDIS RTD-NEWS / © European Communities
Item source: http:///dbs.cordis.lu/cgi-bin/srchidadb?C ALLER=NHP_EN_NEWS&ACTION=D&SESSION=&RCN= EN_RCN_ID:22849