U Of M Refuses To Take Part In Maclean's Survey
Canadian universities are unhappy with ranking process
CHRISTINE LEONG VOLUNTEER
The University of Manitoba, along with ten other Canadian universities, has decided not to take part in the Maclean’s magazine annual university rankings this year. The letter that outlined the reasons for this decision was signed by the 11 university presidents and sent out Monday, August 14, 2006, to Maclean’s managing editor of special projects, Tony Keller.
In terms of Maclean’s categories, the presidents of nine medical-doctoral universities (Dalhousie, McMaster, Alberta, British Columbia, Calgary, Manitoba, Montreal, Ottawa, Toronto), one primarily undergraduate university (Lethbridge) and one comprehensive university (Simon Fraser), stated in the letter their reservations with the Maclean’s rankings’ “reliance on survey data with low response rates” and described the magazine’s overall methodology as “oversimplified and arbitrary.”
“Some people view that the quality of an institution is very much dependent on the quality of the students that come in,” said University of Manitoba president Emöke Szathmáry while commenting on the larger weight given to entrance grade (11 per cent) as opposed to other variables such as graduation rate (two per cent). “In Manitoba, there is only one medical-doctoral university. Our mission as an institution, therefore, is to educate broadly. That means that we admit people that we know perhaps are at the lower end of the scale, but nevertheless, we believe that they have the potential to succeed in a medicaldoctoral environment.”
For some time, universities have raised concerns regarding the methodology that is employed by the Maclean’s ranking system. Some issues that institutions have include the magazine’s use of inappropriate variables to measure quality in a university. The 11 university presidents contend that staff at Maclean’s has not addressed these issues to date. The coalition has therefore agreed not to respond to the magazine’s information-gathering questionnaire.
Dalhousie University president Tom Traves commented that, “When [the survey] lumps all these categories together into a single ranking, arbitrarily assignming more points to one category than another based on its own idiosyncratic judgement, it fundamentally misrepresents the character of every institution.
Tony Keller, who is responsible for putting together the annual university rankings, says that using an overall average is how universities grade their students and this method may not properly represent the student. He said Maclean’s is doing things differently.
“The universities are saying that you can’t combine a bunch of grades together into a grade point average. Right,” said Keller, “It’s the equivalent of a student submitting their transcript.”
“I think they bias their overall ranking simply because they use this reputational survey component, which is their heaviest-weighted factor,” explained Szathmáry. “One page asks you to rank all 47 universities that participate in this survey in the given year . . . I know a lot about universities, but do I know very much about the University of Cape Breton? . . . It’s entirely hearsay.”
UMSU president Gary Sran said he was very happy with the university’s decision to back out of the survey. Sran believed certain elements such as the University of Manitoba’s “vibrant atmosphere, the amount of research produced, a world-renowned architecture program, and one of the best business schools in Canada . . . gets overlooked to schools that probably have a higher operating budget because they have more students such as the U of T and McGill,” two schools that are usually ranked first or second.
Sran also expressed concerns about what these rankings indirectly do to students.
“There have been cases where university boards have decided on fee increases because they want to do better in these rankings,” Sran asserted, “but meanwhile, not really thinking of the impact this will have on students.”
Maclean’s has made some changes to the way the rankings will be published. These include publishing the individual scores that make up the final ranking and a tool available online that will allow users to alter the weight of the scores according to their definition of importance.
Maclean’s will continue to rank all 47 universities in Canada, including the institutions that have opted out of the survey. With respect to the universities that refuse to contribute, data will be collected using other sources. However, collecting data from other sources may still lead to a methodology that is questionable.
“We’re one of the only public and objective sources of information on the subject,” stated Keller.
“I think it just reinforces the point that the presidents made in their original letter about methodology,” said Szathmáry “There’s no quality control
on the data.” -- with files from William Wolfe-Wylie
