This week policy makers and educationalists across many parts of the developed world will become gripped by a phenomenon known in some circles as ‘PISA envy’.

On Tuesday, the OECD will publish its latest tables comparing performance in education systems across the world.

Once again we in Ireland and others elsewhere will wring our hands and look to countries like Finland, wondering how they are getting it so right and whether, or why and how, we are getting it so wrong. 

RTÉ's Emma O Kelly previews PISA, and its pitfalls.

The OECD’s Programme for International Student Assessment – known as PISA – is the World Cup of the education world.

Conducted every three years, it’s a massive exercise.

15-year-olds in around 70 countries and systems across the world are tested in reading, maths and science.

500,000 children take part, across a swathe of the globe that accounts for more than 80% of the world’s economy.

The outcomes are collated and a league table is drawn up ranking all systems from top to bottom.

The last tables were published in 2010. The latest will be published on Tuesday morning and are eagerly awaited.

The big question for Ireland is how will we fare this time round? 2010’s data showed dramatic declines in our ranking for both literacy and maths.

In literacy alone Ireland slid from 5th place in 2000 to 17th place, the biggest fall of any country. Will this picture be sustained? Or was 2010 some kind of strange aberration?

PISA is the brainchild of Andreas Schleicher, the OECD’s Deputy Director for Education. British Education Secretary Michael Gove has called Schleicher “the father of more revolutions than any German since Karl Marx”.

Because of the huge influence these tables now wield among policy makers, some regard Schleicher as the most powerful man worldwide in education today.

In 2000 for example Germany was so shocked by the mediocre performance of its 15-year-olds that it instituted sweeping changes.

It increased education spending and introduced national testing across schools.

Such was the level of public soul searching that a TV quiz show was even established, based on PISA questions.

But is it really possible to compare student performance in this way? And are there dangers associated with attempting to do so? A growing number of educationalists internationally are voicing concerns around both the methodology of PISA and the consequences the tables can have.

They say there are cultural, linguistic and structural factors that make it impossible to compare like with like.

How can the same set of questions work in the same way for a 15 year old child in a remote village in Peru for example and a 15 year old from Berlin, or Kerry, or Vietnam or Shanghai? 

Canadian statisticians were commissioned to examine Ireland’s sudden decline in PISA last time round.

Among their conclusions they pointed to factors associated with the design and reporting of PISA that may have contributed to Ireland’s low scores.

“Constructing a test that measures achievement accurately across many languages and cultures is a significant challenge”, they noted.  

Among other factors they pointed to greater numbers of students here who don’t have English as their first language.

They also said Irish schools’ success in preventing early school leaving meant conversely that there were weaker performing children in the system. 

And then there’s the danger associated with the outcomes. How wise is it to look to systems that do well in PISA with a view to emulating them?

The Finnish education system always does well in these tests but how much of its success has to do with its relatively homogeneous society, and that society’s cultural attitudes towards endeavour and education? And what about systems like South Korea and Shanghai? They too come out on top.

But their systems are based on rote learning and sometimes extremely long hours in school.

Not surprisingly there is no clamour to imitate these two top performers.
Andreas Schleicher says he can’t claim that PISA does a perfect job, but he staunchly defends the exercise. Economies compete in a globalised marketplace, he says, so education systems must measure themselves against each other too.

The OECD goes to huge lengths to try to ensure that methodology is fair to as many as possible and it's satisfied that it achieves this.

No one denies there is a vast amount of rich and valuable data contained in PISA.

While people point to flaws, very few would dismiss the outcomes outright. But many educationalists are uneasy with the blunt ranking tables that will dominate tomorrow’s headlines.

And as we pore over tomorrow’s figures and, possibly, lament our positioning on those tables, it’s worth remembering that 'PISA envy' will probably also be spreading across Japan, the UK, the US, Austria, Australia and many more countries too. 

It is also worth sparing a thought for Kyrgyzstan. Imagine being ranked last – yes, last - of all 65 countries surveyed?

When PISA 2010 was published that’s where Kyrgyzstan found itself.

Coincidentally, teachers in the impoverished Central Asian state were on strike at the time, protesting against average pay of just $30 to $40 a month.