What do student satisfaction questionnaires measure?

Resumen

University students are asked to complete satisfaction questionnaires inwhich express their opinion on many aspects of their studies. It is hardto think what purpose such questionnaires may serve: it is very difficult todismiss university teacher on the basis of students’ perception of theirperformance; and staff are promoted on the basis of the success of their research rather than on the quality of their teaching. From a qualityenhancement point of view, one is led to conclude that studentsatisfaction surveys, by using valuable time staff time, produce moreharm than good. And yet, there are many important questions that couldbe answered with the help of student questionnaires. One obviousexample is whether teaching should be done by research active staff orby staff who do no research and specialise in teaching methods.Another aspect that is the impact on students’ perception of “sexing up”a subject. This would militate against intellectually challenging subjects,such as those that contain advanced quantitative analysis. Finally, onecan ponder on the question of class size. Up to what point is thestudent’s perception of the teaching quality influenced by personalcontact?In this study we collected summary information on questionnaire resultsfor all the undergraduate modules for which such information wasavailable in a management school in a major UK university for theacademic year 2008/2009. We also collected information on averagemarks achieved in the same modules, as well as a set of contextualvariables. The management school offered 56 undergraduate modulesbut, following a computer crash, information on student questionnaireswas lost for some of the modules, and full data is only available for 45modules. 37 lecturers were involved in the teaching. An estimate ofresearch activity was also available for each lecturer, having beendeveloped as a by-product of the 2008 research assessment exercise ofBritish universities.The data was analysed using various multivariate statistical tools,including multiple regression and principal components analysis. It wasfound that students’ perception of teaching quality was not related to theresearch rating of the lecturer, and that the only factors that influenced itwere class size and mathematical content. It was also found that anindex of teaching quality developed from the questionnaires was notrelated to results achieved.