An Australian university has been given the green light to publish student course evaluation data, despite concerns that the information can be used to “reverse engineer” staff league tables.
UNSW Sydney has successfully appealed a decision from Australia’s industrial relations umpire, the Fair Work Commission (FWC), banning it from giving students broad access to data compiled from quantitative course ratings.
The information includes students’ appraisals of course quality, resources, assessment and feedback on a six-point scale. UNSW says it does not intend to publish students’ qualitative commentary about the courses or their evaluations of teachers.
The university only plans to publish quantitative data on courses that involve more than one teacher – so that individual staff cannot be directly identified – and where at least 10 responses have been received, so that aberrant assessments cannot unduly influence the data.
But the National Tertiary Education Union (NTEU) said that the information allowed individual staff to be identified by academics in the same discipline or school, and – in cases where courses were taught by pairs of tenured and casual staff – by academics across the university.
“This is a public university, not a reality TV show,” the union’s New South Wales branch tweeted on the eve of a FWC hearing in August.
FWC commissioner Leigh Johns sided with the union. He dismissed the university’s argument that the data’s publication did not directly identify staff because “detective work and additional effort” was needed to match courses to teachers.
But an FWC appeals panel has overturned that decision, finding that the university’s plans comply with a strict reading of the enterprise agreement.
“We accept that in some cases a person can be identified without being named,” the 7 March judgment says. “A reference to [an] attribute could identify the person, even though the person’s name might not be used. But the proposed form of the data to be published simply does not do this. No staff are identified.”
UNSW believes that publishing the data will convince students that their feedback is being “considered and utilised”, encouraging more of them to fill out course evaluation surveys and improving the questionnaires’ response rates and accuracy. “Our students…take such care in providing all this feedback,” said deputy vice-chancellor Merlin Crossley.
“This gives us an opportunity to celebrate and showcase many courses that provide an outstanding student experience, and work with staff to develop courses that still need work.”
But NTEU state secretary Damien Cahill said that the “very disappointing” ruling potentially sent a signal to managers at other universities who “want to use these surveys as performance metrics”.
Dr Cahill said student evaluations were useful for teacher and course development when they were “framed and conducted” properly. “But there’s a lot of scholarly literature on…the inherent gender and racial biases [and] limitations based on the context. Whether a course was online, the time of day, the room in which it was held, the dynamics of the cohort of students – all of those situational factors play a role in determining the outcome of those evaluations.”
In the latest such study, published in the Journal of University Teaching & Learning Practice, a Victoria University analysis of more than 22,000 students’ teaching evaluations found no differences in the score ratings for teacher gender. But male students’ commentary about the teaching style of female academics became increasingly negative during Covid-related lockdowns.
