The validity and reliability of the cross-national comparison of degree programme levels in European countries: What have students learnt?

Abstract

A cross-national comparison of degree programme levels became relevant when the borders of European countries opened for students and graduates, and higher education institutions were restructured into bachelor’s and master’s programmes. This new situation foregrounded the questions of what students are learning in the degree programmes of European countries and how to compare their achievements. Therefore, we conceptualised a valid and reliable ‘level’ construct that included a cognitive (‘disciplinary thinking’) and an affective aspect (‘professional attitude’). The main research question for our exploratory study was: ‘What procedure can lead to a valid and reliable cross-national comparison of degree programme levels?’ To achieve this comparison, we designed a Three-Step Procedure, in which level was operationalised (step 1), measured and analysed (step 2), and compared cross-nationally (step 3). The study was conducted in collaboration with four bachelor programmes in Hotel Management from four European countries; a total of 783 participants were involved. Four themes were generated to operationalise the concept of level: professional management, hospitality business research, leading management, and strategic management; their respective learning outcomes were measured with a questionnaire. Principal component analysis identified the conceptualised themes and measured their components with eigenvalues ≥1, which explained 66 % of the variance. The reliability of the components exceeded a Cronbach’s alpha coefficient of 0.70. Analysis of the components and of the single samples showed strong validity and reliability for the learning outcomes. Thus, we believe this study has produced a rigorous means to compare degree programme levels across countries.