Treffer: A Comparison between Two Systems of University Education: Years of Study versus Credit Accumulation
Postsecondary Education
1469-5782
Weitere Informationen
A recent major reform of higher education has changed a year-based system into a credit accumulation one. We use this exogenous shock to compare the two systems. Using a longitudinal dataset of undergraduate students from one of the largest French-speaking universities, a competing risks survival analysis is performed to study the probabilities and duration to graduation or dropout. The results indicate that the credit accumulation system encourages students to be less engaged in their studies, at the cost of late graduation in the best case, and in the worst case, late dropout which affects mostly students coming from disadvantaged backgrounds.
As Provided
AN0183842093;ede01apr.25;2025Mar21.03:44;v2.2.500
A comparison between two systems of university education: years of study versus credit accumulation
A recent major reform of higher education has changed a year-based system into a credit accumulation one. We use this exogenous shock to compare the two systems. Using a longitudinal dataset of undergraduate students from one of the largest French-speaking universities, a competing risks survival analysis is performed to study the probabilities and duration to graduation or dropout. The results indicate that the credit accumulation system encourages students to be less engaged in their studies, at the cost of late graduation in the best case, and in the worst case, late dropout which affects mostly students coming from disadvantaged backgrounds.
Keywords: Higher education; survival analysis; degree; dropout; credit accumulation system; competing risks
1. Introduction
For several decades, international organisations have been setting up comparisons to measure educational opportunities and outcomes across different countries (see for example https://www.enic-naric.net/; OECD Education at a Glance [34]). While these indicators based on accessibility, financial resources, teacher quality and inequalities in secondary education attempt to measure the performance of national education systems, it is increasingly recognised that the type of learners as well as the local structure of the system set up for the students' pathway are crucial elements in measuring the quality of a system.
With the increasing internationalisation of higher education, the European Bologna process was implemented to support student mobility between higher education institutions with the application of the ECTS (European Credit Transfer and Accumulation System) to help the recognition and transfer of credits earned by students during a mobility period abroad. This context has led to a structuring of educational systems into two types of systems well implemented elsewhere in the world: a year-based system[1] versus a credit accumulation system.[2]
The year-based system classically used in Europe has been modified in many countries. Indeed, although countries such as France or Ireland have kept this system, countries such as Germany, Italy, Finland, the Dutch speaking community of Belgium, and more recently the French-speaking community of Belgium have switched to a credit accumulation system. Furthermore, most countries outside of Europe are also aligned with these two types of systems. For example, the education systems in Canada, Australia, New Zealand, Singapore, Vietnam or Cambodia (see Chealy [15]; Nguyen and Ta [33]) are based on a credit accumulation system while systems in Japan, Egypt and Brazil have remained in a year of study system.
Because of these developments in higher education, it is important to analyse whether the switch from a year-based system to a credit accumulation system is a change that leads to greater efficiency and equality in a higher education system, and under what conditions. To the best of our knowledge, no other study has attempted to compare these two types of systems, applied in most countries, in terms of students' academic performance, especially for the most vulnerable students.
In the French-speaking community of Belgium (FWB), the 'Bologna' framework was implemented in 2004 in a year-based framework, while a second structural reform, called 'décret Paysage', changed the year-based logic into a credit accumulation one in 2014 with the aim to make higher education more inclusive and flexible by trying to compensate for the inequalities observed in secondary education and thus to give greater prominence to the 'social elevator' role of higher education.
The latter exogenous shock generated by the second Belgian higher education reform in the French community gives us the opportunity to empirically test the differences between the two systems: the year-based system versus the credit accumulation system. Analysing relatively similar populations at university entry over a few consecutive years greatly reduces potential unobserved heterogeneity. The higher education system in FWB has been relatively stable since 2004 which also implies that the characteristics of students, despite a constant progression in terms of access to higher education, are also very stable. Furthermore, it is important to mention that the same Minister of Higher Education has remained at the helm of the Ministry of Higher Education for over 10 years (from 2009 to 2019), with the only major educational policy change being the one discussed in this paper. Moreover, the Belgian economic remained relatively stable between the studied period (2010–2018) with a real GDP growth per capita percentage change on previous period never exceeding 2% according to Eurostat. This stability reinforces our argument that the differences observed between the two time periods analysed are indeed caused by the aforementioned reform.
Our study is based on student data from the Université libre de Bruxelles (ULB), one of the largest universities in the FWB. The events considered in this paper are dropout and graduation. Several control variables are used in the models including the holding of a scholarship used as a proxy for the socio-economic background of students. The present study has adopted the event history methodology as presented by Scott and Kennedy ([36]), applied in the empirical analysis of Arias Ortiz and Dehon ([5]). Survival analyses are undertaken to understand not only whether the event happens, but above all when it happens and how covariates are affected (Willett and Singer [45]).
The results of the survival models indicate that the odds of students dropping out have significantly decreased in the three first years after the first enrolment at university with the credit accumulation system. Similarly, the probability of obtaining a diploma on time or with a one-year delay has been significantly reduced. The reform negatively affected the academic outcomes of students coming from all socio-economic backgrounds. In particular, the findings suggest that the reform reduced the probability to earn a degree on time, especially for students from the most favourable socio-economic backgrounds while students from the lowest socio-economic backgrounds had a significantly lower probability of dropping out after three years, without any improvement regarding their time-to-graduation. These negative findings suggest that the flexibility in the success criterion allowed by the reform reduced the incentives of students to fully complete their annual course programme, at the cost of late graduation. What is worse, the accumulation of 'failed courses' together with longer study paths, seems to have led many students to drop out after many years spent at university.
This paper is organised in six sections. The literature is reviewed in Section 2. Section 3 describes the theoretical expectations in the Belgian context from the reform. The data and variables are described in Section 4. In Section 5, the notion of hazard is introduced in the exploratory analysis. Finally, the model, the estimates and the marginal effects are discussed in Section 6. We then summarise our main findings and conclude with some policy implications.
2. Literature review
From the literature, the theories most frequently used to analyse the behaviour of students in their academic career (dropout, perseverance, graduation) are related to economic,[3] organisational,[4] and interactional[5] approaches (Bean and Metzer [8]; Tinto [42], [43], [44]). These models highlight that several factors affect students' behaviour to pursue or quit higher education. Furthermore, they also show the difficulty for students to adapt intellectually and socially to higher education institutions. It should be noted that the influence of students' characteristics on their student path has been much more explored in the literature than the importance of the institutional system put in place even though the latter may also have significant impacts on students' career paths. One of the few examples focuses on degree completion based on measures related to the early years at university. The study of Haarala-Muhonen et al. ([24]) suggests that policies that help students in developing organisational and time management skills at the very beginning of their university studies were important determinants of 'on-time graduation' for students while Declercq and Verboven ([20]) and Bordon and Fu ([12]) have respectively shown that well-designed admission standards do not damage degree completion and improve student welfare.
In the empirical literature, many socio-economic and personal characteristics, as well as previous academic performance have been associated with academic success. Among the variables conventionally studied as factors influencing academic success are the age when enrolling into higher education (Bone and Reid [11]; Calcagno et al. [14]), gender (Sheard [37]; Sneyers and De Witte [39]), nationality (Bone and Reid [11]; Clerici, Giraldo, and Meggiolaro [17]; Jochems et al. [28]) and family income or parents' level of education (Allen [1]; Chen and DesJardins [16]; Hu and Kuh [26]). Although the access to higher education has become more democratic over the years, students from the lowest social strata still experience the lowest graduation rates (Arias Ortiz and Dehon [4], [5]). The level of education can be better understood on the basis of human capital theories. According to the Becker-Tomes model ([9]), the larger the commitment of parents to invest in their children, the greater the transmission of human capital. In other words, the model predicts that children from the most vulnerable families lag behind in terms of social mobility due to financial constraints or lower levels of education of parents. Nevertheless, numerous studies have also shown that financial assistance reduces the penalty for students coming from disadvantaged backgrounds (Arendt [2]; Baker and Velez [7]; Chen and DesJardins [16]; DesJardins, Ahlburg, and McCall [22]; Hu and John [25]; Ishitani and DesJardins [27]; Zarifa et al. [47]).
Regarding early educational background, it has been shown that low academic performance (Blankenberger et al. [10]; DesJardins, Ahlburg, and McCall [21], [23]), repetitions of a failed year and diplomas from vocational secondary schools (similar to vocational and technical secondary schools in the FWB) have a negative influence on university graduation (Clerici, Giraldo, and Meggiolaro [17]; Meggiolaro, Giraldo, and Clerici [31]). However, far from this deterministic view comes investment in study, intellectual abilities, student character and many other factors that are opportunities to compensate for initial socio-economic differences in educational attainment. For example, Bruffaerts, Dehon, and Guisset ([13]) showed that regularity and investment in studies can partly counter the burden of having unfavourable socio-economic characteristics that the student cannot control.
Literature that focuses on higher education in the FWB is in line with foreign literature. Nevertheless, to the best of our knowledge, no study in the international or in the FWB literature analyses the impact of the type of system (year-based
While remaining consistent with the findings of the literature regarding student characteristics of academic success, this study aims to make a novel contribution to the existing literature by examining, for the first time, the impact of an exogenous shift from a year-based system to a credit accumulation system on dropout and graduation in higher education.
3. Expected effects from theory in the Belgian context
To adequately describe the effects that would be expected from theory of the reform on academic success, it is important to understand the context of higher education in Belgium. The French-speaking community has opted for a mainly publicly funded higher education system with, relatively, very low tuition fees. Furthermore, except for some specialised programmes, access to higher education in the French-speaking Community of Belgium (FWB) is largely unrestricted, as it is granted solely upon possession of a certificate of upper secondary education (CESS).
In addition, several studies have also shown that secondary education in the FWB is one of the most unequal among OECD countries (Danhier and Jacobs [19]). It is therefore not surprising in this context to observe very high dropout rates at university, and relatively low graduation rates (41% of 25–34-year-olds graduated from higher education in FWB in 2015) despite a higher education access rate for secondary school graduates of 81% (Lambert [29]). The system opens the doors of universities to all types of student profiles in terms of skills and socio-economic background but at the same time, implies a large dropout rate, especially for the most fragile students. It is therefore clear that equality in opportunities do not ensure equality in success at university (Arias Ortiz and Dehon [5]).
With the 'Bologna' framework in the FWB, bachelor cycles were harmonised to consist of three study years that must each be successfully completed one after the other until graduation (year-based logic). When all the credits of a specific year were not earned, the entire year had to be retaken. The workload of a full year of study is the equivalent of 60 ECTS credits.
With the so-called 'Paysage' reform, the FWB switched to a 'credit accumulation' system, courses are no longer attached to years (credit accumulation logic) since students are allowed to progress in their student path, even when some courses have been failed. For instance, if 15 credits out of 60 are not passed at time
Graph
Graph
The effects that would be expected from theory of the reform from a year-based system to a credit-accumulation one on academic success would be bidirectional. They depend mainly on the behavioural response of different actors and specifically of the students with respect to the increased flexibility of the 'success rules' brought by the credit accumulation system.
On the one hand, the credit accumulation framework could be used by students as a solution in the case of some mishap. In other words, if a student that failed some courses to a limited extent is allowed to catch up the missing credits rapidly in their study cycle, they may be able to graduate on time while this would have been impossible under the year-based system. Hence, an improvement regarding time-to-graduation could be expected, making the investment in education more efficient.
On the other hand, students may not catch up the missing credits rapidly under the credit accumulation framework, forcing them to spread their bachelor cycle over more years. First, some students may have reduced their efforts since it is not necessary to pass every course in the programme to continue their student path. Second, incentives may have also changed for the remaining students i.e. those that would not have earned all the credits of their student's annual programme under the year-based system. While they would have either retaken a year, dropped out or switched to another study path, there would now be incentives to persist since students can progress in their student path in spite of failed credits. In this case, the spreading of the study cycle over more years may drive students to drop out at later stages due a lack of motivation coming from an accumulation of failed courses. Moreover, knowing that students switching to other study paths are more likely to complete a degree than those who repeat a failed year (Arias Ortiz and Dehon [5]), a lower incentive to switch combined with an unclear notion of success in a system of credit accumulation may be detrimental to degree completion.
From there, the effect of the credit accumulation policy on success for the students coming from the lowest socio-economic backgrounds could be amplified. Studies have shown that students from the lowest socio-economic background had lower probabilities of completing a degree and higher odds of dropping out (Arias Ortiz and Dehon [5]; Bruffaerts, Dehon, and Guisset [13]; DesJardins, Ahlburg, and McCall [22]). Thus, one can expect these students to be the most affected by the increased flexibility of the credit accumulation system since they are less likely to complete their bachelor cycle. Again, if these more vulnerable students catch up the missing credits and consider the flexibility of the credit-accumulation scheme as a positive in case of some mishap, there is some hope that their study outcomes will be improved. However, in the opposite case, inequalities towards success could be exacerbated.
Altogether, the efficiency of the decree seems to rely largely on the behavioural response of students with respect to the flexibility brought by the reform. The two systems will therefore be compared empirically at the level of student pathways in the following sections.
4. Data and variables
Our study is based on student data from the Université libre de Bruxelles (ULB), one of the largest universities in the FWB. Indeed, the ULB accounts each year for about 25% of the market share of new enrolments in university studies in the FWB, and almost 30% of the students enrolled in universities in the FWB. The ULB is a private university, but it is important to emphasise that the same regulations (registration fees, conditions of access, progression in the curriculum, etc.) and the same public funding are applied to all universities in the FWB. Moreover, even if the universities of the FWB are relatively comparable in terms of the composition of the student population, the ULB located in Brussels is a university with a slightly higher proportion of non-Belgian students (about 25% compared to 15% for all the FWB universities). The ULB also has a slightly higher proportion of socio-economically disadvantaged students (about 30% against 20% on average for all FWB universities). Moreover, Belgium being a relatively small and densely populated country, the distances between the universities are relatively small, which implies that about 50% of ULB students do not come from Brussels, but from the other provinces of French or Flemish-speaking regions as well as from abroad. The analyses carried out on the ULB can therefore be easily generalised to the overall situation in the FWB.
The longitudinal dataset[6] contains study outcomes and student characteristics. It is composed of 13,762 undergraduate students registering for the first time at the ULB from cohorts under the year-based (2010, 2011) and credit accumulation (2014, 2015) systems.[7] From 2014 on, all students enrolled for the first time in higher education were subject to the credit accumulation system. In addition, all students already enrolled at university before the introduction of the reform switched to the new system in 2015. The transition between the two systems therefore lasted only one year. As can be seen in Table 1, individuals from the year-based cohorts are observed for up to nine years (from 2010 to 2018 included[8]) while for the credit accumulation cohorts, observations are available for at most five years. Bold numbers refer to the number of students registered in each cohort; they are relatively similar across cohorts. Indeed, there are on average 3460 new students per year for the two years before the reform and 3420 new students for the two post-reform years. The number of undergraduate students enrolled at the ULB for the first time before the reform (2010 and 2011) and after the reform (2014 and 2015) is relatively stable (only a 1.2% increase).
Table 1. Distribution of the observations per cohort and academic year.
Furthermore, in our analysis we added several intrinsic and high school-related variables to control for these effects. Intrinsic characteristics refer to the dummies 'Female' (1 if female, 0 if male) and 'Belgian' (1 if Belgian, 0 if foreigner). Regarding high school features, high school repetition is a flaw in the FWB since roughly 13,5% of students have on average experienced at least one high school year repetition between the academic years 2010–2011 and 2016–2017 (ARES [3]). Hence 'Repeated HS' is a time-invariant dummy variable that equals 1 when the student repeated a year in high school, 0 otherwise. Finally, 'Scholarship' is used to proxy the socio-economic background of the student and will be fundamental to understand how the credit accumulation system affected the most vulnerable students. It is a time-invariant dummy equal to 1 if the student earned a scholarship in the first year of the bachelor cycle and 0 otherwise.
To ensure that the effects captured by the model are resulting from the reform change and not a change in covariates characteristics, we verify that the characteristics of the student population enrolling for the first time at university have not been significantly affected by the implementation of the reform, by comparing student compositions before (2010 and 2011) and after (2014 and 2015) the reform, given in Table 2.
Table 2. Student composition before and after the reform and results of proportion tests.
1 ***
Two other variables have been added for these descriptive statistics. The first is the field of study, and the second is the type of study in high school, the latter being the percentage of students who have followed the classical high school path to enter higher education. As can be seen, the changes in the percentages are relatively small. As these variations are so small, they do not allow us to reject the hypothesis of equality of propositions between the two periods for more than half of the variables (see Table 2). Nevertheless, to test this evolution more precisely, we carried out proportion tests with the null hypothesis that the difference in proportion between the two periods is greater than or equal to 3%, which represent a very small difference. In fact, the tests are equally conclusive for all variables with a difference of the 3% level, except for the proportion of students that were in the general schooling system in high school. For this latter variable, the null hypothesis would be rejected at the 1% level for a difference larger than 4%, which remains relatively limited. And as we reject this null hypothesis in all situations, we can assert that the difference in proportion of the characteristics of students between the two periods is very small. In addition, between 2009 and 2019, the minister of lower education (including high school) remained from the same political party, ensuring a relative stability during this period.
Regarding one of our key variables, it is also important to notice that the rules for obtaining a scholarship were set in a decree from 1983, and except for a slight adjustment in 2015 in relation to the timing of payment, no changes to this text were made before 2020. However, the criteria for determining the low-income status of applicants for a scholarship were modified in 2016 by broadening the population eligible for a scholarship. But this does not impact our study as the last cohort analysed entered university in 2015, and we test whether the students are disadvantaged by checking if they obtained a grant when they first enrolled at university.
5. Explanatory analysis
In survival analyses, students are followed until they experience an event. In other terms, each year, for every student, the outcome or non-outcome occurs. When a student is regularly enrolled, without experiencing a dropout nor a graduation, the non-outcome is assumed. However, when a student drops out or graduates, the event is said to have occurred; the student leaves the database and is no longer followed.[9] A last important category concerns censored individuals. These are subjects that do not experience any event during the observation period (right-censored data). All these cases are summarised by the variable 'Outcome' (0 if non-outcome, 1 if dropout, 2 if graduation). In the literature, the most studied outcomes are graduation, dropout and stop out (Chen and DesJardins [16]; DesJardins, Ahlburg, and McCall [21]; Willett and Singer [45]). In the FWB educational context, since less than 4% of students experience a stop out (i.e. temporary interruption from university), these individuals have been voluntarily censored to ensure no biases in the results.
Depending on when and whether the event happens, the effect of the reform and other control variables (intrinsic characteristics, high school variables) are explored to see to what extent these factors have influenced the event. Above all, the most important variable is 'Reform', a dummy equal to 1 for cohorts under the credit accumulation system and 0 under the year-based system. It captures the effect of the change from a year-based to a credit accumulation system.
To obtain a temporal vision of the outcomes, we analyse the hazard functions and the cumulative hazard functions. The hazard function is convenient to understand when graduation or dropout is most likely to occur. For
Graph
Graph
Graph
Graph
Graph
Graph
Graph
Graph
Graph
Graph
Graph
Graph
In Table 3, the discrete time hazard probabilities of students under the year-based system and the credit accumulation system frameworks are given. The column 'Population' contains the numbers of students at risk at time
Graph
Table 3. Discrete time hazard probabilities under Bologna and Paysage.
Figure 1 displays the hazard probabilities of dropouts and degree under both systems. Dropouts are initially higher under the year-based system, but this trend is reversed from the
Graph
Graph
Graph
Graph
Graph: Figure 1. Dropout and degree hazard functions: year-based vs credit accumulation system.
It is also interesting to analyse the cumulative hazard function
Graph
Graph
Graph
Graph
Graph
Graph
Graph: Figure 2. Cumulative Hazards: year-based vs credit accumulation system.
Figure 2 (right side) shows that for every time period, the cumulative hazard to have experienced a graduation or a dropout is always higher for the students that started university under the year-based framework. For instance, after five years from enrolment under the credit accumulation system, 11.51% of the students of the sample have not experienced any outcome, against 5.85% under the year-based system. In other words, the share of students that will either graduate or drop out after at least five years at university is larger post-reform. In the most optimistic scenario, these students will earn a degree. But in the least optimistic one, late dropouts would be the most damaging situation for students, universities, and public authorities.
One important aspect treated in this study concerns how the academic outcomes of the most vulnerable students, proxied by the holding of a scholarship, have evolved with the switch from a year-based to a credit accumulation system. Figure 3 shows that the cumulative hazards to dropout (to earn a degree), is as expected from the literature (Arias Ortiz and Dehon [5]), larger (lower) for scholarship holders compared to their counterparts. Regarding the effect of the reform on dropouts, scholarship and non-scholarship holders under the credit accumulation system are the less likely to have dropped out within 5 years from enrolment, especially for the scholarship holders. Figure 3 shows that the cumulative hazard to earn a diploma on time (i.e. in the
Graph
Graph
Graph
Graph: Figure 3. Cumulative Hazard Probabilities, Scholarship: year-based vs credit accumulation system.
6. Econometric models
Hazard and cumulative hazard probabilities were previously analysed. However, the descriptive statistics do not allow us to control simultaneously for multiple factors that might affect the probability of a given outcome. In this section, econometric models are used to analyse simultaneous factors to control for a possible difference in the cohort composition pre- and post-reform based on several observed variables.
Survival analysis models were first developed in continuous time (Cox [18]). Later, Willett and Singer ([45]) adapted survival analyses to discrete time that better fit education data. Their model copes with the academic timing, organised in semesters or years rather than in continuous time. Considering when the event occurs rather than whether the event has occurred (as in simple binary choice models) goes beyond a simple typology difference since the time of occurrence of the event is now considered. In survival analyses in the education context, students are followed over time until they graduate or drop out. When one of the events occurs, the student is removed from the sample immediately. Thereafter, the covariates associated with the individual experiencing the event are analysed based on the amount of time that has passed until the occurrence of an event.
The use of competing-risk event models is crucial to account for the multiple outcomes. In the earliest papers, single-risk models were used with education data, ignoring the interdependence between dropout and graduation (Murtaugh, Burns, and Schuster [32]; Ott [35]; Stage [40]). Censored individuals were thus far from being representative of the population at risk. For instance, by considering dropout as the unique event of interest, there can be some doubts that the characteristics of the students leaving the sample due to graduation are like the ones dropping out. Henceforth, DesJardins, Ahlburg, and McCall ([22]) showed that single-risk models were potentially leading to inaccurate estimations and further developed competing event models.
Considering this last remark, Scott and Kennedy ([36]) proved that multinomial logit models were effective tools to obtain the maximum likelihood estimates of the coefficients in a competing-event setting. Our study is in line with Scott and Kennedy ([36]) and Arias Ortiz and Dehon ([5]) but innovates by studying how a reform making the academic paths more flexible affected student success at university.
6.1. Model-based approach
In the first specification of the model where the impact of the reform is taken as time invariant, the hazard for student
Graph
Graph
Graph
Graph
where
Graph
Graph
Graph
Graph
Graph
Graph
Graph
Graph
Graph
Graph
Graph
Graph
Graph
Graph
Graph
Finally, interaction terms between the reform and all the covariates are added to the model to capture the eventual additive effect of this interaction on each outcome and thus to control for potential heterogeneity in the effects of the reform. These effects are captured by the parameters
Graph
Graph
Graph
Graph
Graph
Graph
These ratios are interesting for the interpretation of the magnitudes of the impacts of covariates. Indeed, the coefficient associated to an explanatory variable estimates the shift of the log of the outcome-specific hazard ratio due to a change of this explanatory variable, while controlling for the other variables. Moreover, marginal effects that depend on the values taken by all the other coefficients are computed in Section 6.3 to have a sense of the magnitude of the effect of the change from a year-based to a credit accumulation system on students coming from different socio-economic backgrounds and different personal variables.
Discrete time hazard models are based on three main assumptions: the linearity, the unobserved heterogeneity and the proportionality assumptions (Singer and Willett [38]). First, the linearity assumption requires that vertical displacements from the baseline level of hazard correspond to the equal changes in the values of a covariate. This assumption is automatically satisfied since all covariates have been transformed into a set of dummy variables.
The proportionality assumption imposes that hazards must be proportional over time (Cox [18]). For the assumption to hold, all logit hazard ratios (computed by Equation (3)) of all categories must be parallel over time. When violated, this assumption can be relaxed by interacting time dummy variables with the covariate in question. Nonetheless, interaction terms are costly when multinomial logit models are conducted since they increase the number of parameters to estimate and possibly lead to overfitting issues. According to Babyak ([6]), overfitting yields overly optimistic results if too many degrees of freedom are used relative to the number of observations. To limit this threat, minor violations of the proportionality assumption are allowed for the covariates. Clearly, this assumption is violated for the interest variable 'Reform'. To avoid this problem, we introduced a second model relaxing the strong assumption that the effect of the reform is constant over time (no interaction of 'Reform' with time). In the second specification, the impact of the reform is allowed to vary over time by including an interaction between the two variables 'Reform' and 'Time' with the row vector of parameters
Graph
Graph
Graph
Graph
The last assumption concerns unobserved heterogeneity. The variation in the hazard profiles of the students must only result from the observed variations in the values of the covariates. It is therefore important to introduce relevant variables from the literature into the model to explain dropout and graduation. Nevertheless, with observational data, it is never possible to completely rule out this problem. Fortunately for this type of situation, the literature provides empirical evidence that even in the presence of strong unobserved heterogeneity, standard survival models can give relatively robust parameter estimates and likelihood ratio statistics (Liu [30]). On the other hand, one must be more cautious when interpreting marginal effects.
6.2. Discussion of estimates
In this section, the two models proposed above (Equations (2) and (4)) are estimated and the coefficient estimates are given in Tables 4 and 5. The magnitudes of the impacts of covariates are interpreted in the text using odds ratios. In Table 4 (Equation (2)), the variable 'Reform' is included without time interaction in order to capture the effect of a switch from a year-based to a credit accumulation system under the assumption that the effect is constant over time (proportionality assumption). Time and control variables are added: the socio-economic background of the students (scholarship) and an interaction term between the reform and the holding of a scholarship in columns 1–2, intrinsic characteristics (gender and nationality) in columns 4–5, high school features (high-school repetition) in column 6–7 and interaction terms between the reform and the three control variables (gender, nationality and high school repetition) in columns 7–8.
Table 4. Multinomial logit model under the proportionality assumption (Equation (2)).
Table 5. Multinomial logit model with the proportionality assumption relaxed (Equation (4)).
To begin, time effects reflect the situation when all the other variables are set to zero (rows 1–6). Regarding the dropout time effects, all coefficients are significant at the 99% confidence level for the entire specification. The hazard of experiencing a dropout compared to the non-outcome is particularly strong in the 1st year from enrolment reaching 25.5% (for the last specification including all control variables) before decreasing to 12.3% in the 5th year from enrolment and 11.2% in the 6th year from enrolment.[11] Although hazard to earn a degree is ignored in the two first periods since graduation is observed after three time periods at the earliest, the hazards of experiencing a degree compared to the non-outcome 3, 4 and 5 years from enrolment are significant at the 95% confidence level for most of the specification. In the population at risk at time 3, the hazard of earning a degree compared to the non-outcome is particularly strong (72.8%) and the hazard of receiving a degree at time 4 compared to the non-outcome is even larger (86.9%).[12]
In all the specifications, the results suggest a significant (1% level) negative effect of the credit accumulation system on dropouts and the likelihood to earn a degree. This is in line with the intuition from the exploratory analysis suggesting that the credit accumulation framework disincentivised students to drop out. Indeed, we see that students have 23.3% (odd of
Graph
Graph
When the reform is associated with the holding of a scholarship without the other control variables (columns 1–2), the results indicate that scholarship holders under the credit accumulation system have an additional 10% (odd of
Graph
Graph
Regarding the effect of the other covariates on the probability to drop out or to earn a degree (columns 5-6), coefficients are generally significant (significance level at 1%) and consistent with the literature. For the dropout outcome (column 5), the results suggest a significant impact for the variable female (1% level), nationality (10% level), and high school repetition (1% level). Students that are female, Belgian, with no high school repetition are significantly less likely to drop out versus staying enrolled. For example, a female has a 7.5% (odd of
Graph
Graph
Relatively to degree outcome (column 6), students that are non-scholarship holders, female, Belgian, with no high school repetition are more likely (significance level at 1%) to earn a degree than their counterparts. A female has a 26.8% (odd of
Graph
Graph
Graph
Graph
In the last specification (columns 7–8), we control for potential heterogeneity in the effects of the reform across different types of audience. To achieve this, interactions between the 'Reform' variable and the three control variables (gender, nationality, and high school repetition) have been added to the model. There is no heterogeneity in the impact of the reform on the variable specifying that the student repeated or not at least one year in high school. But the situation is a little more complex for the other two variables. As far as gender is concerned, there is no change in behaviour before and after the reform for graduation. On the other hand, the introduction of the credit accumulation system has led to a greater difference in behaviour between male and female students in terms of dropout. This difference is not yet significant under the year-based system, but it is significant after the introduction of the reform. A female has a 11.6% (odd of
Graph
Graph
Graph
In the previous model, a strong assumption was made regarding the proportionality assumption, requiring that the effects of a change of framework be constant over time. In Table 5, this assumption is relaxed by interacting the variable 'Reform' with time. Time and its interaction with the reform, and the interaction between the reform and the holding of a scholarship can be found in columns 1–2, then, are gradually added intrinsic characteristics in columns 3–4, high-school features in columns 5–6 and interaction terms between the reform and the three control variables (gender, nationality and high-school repetition) in columns 7–8 to control for potential heterogeneity in the effects of the reform.
In this model, time dummies and all covariates that were included in Table 4 offer comparable coefficients in Table 5 in terms of significance and magnitude. Regarding dropouts, when all controls are included (column 7), the interaction of the three first time periods with the reform are significant at the 1% level. And the results indicate that the hazard of experiencing a dropout compared to the non-outcome in the first, second and third years from enrolment are respectively 17.1%, 33.4% and 4.1%[14] lower after the reform than before the reform. Concerning time-to-graduation (column 8), the interaction between the reform with the time dummies reveals significant (1% level) lower odds to earn a degree with one-year delay after the reform than prior to the reform, compared to the non-outcome. The hazard to earn a degree compared to the non-outcome is 52.8%[15] lower in the fourth time periods after the reform compared to prior to the reform.
In summary, the findings indicate that the reform reduced the odds to drop out, especially in the three first years at university, without improving graduation rates. In addition, the results demonstrate robustness as only marginal variations of the coefficients, and their significance can be observed when comparing Tables 4 and 5.
6.3. Marginal effects
Regarding the computation of the marginal effects, we have retained only the most complete model (Equation (4)) to minimise the possible presence of unobserved heterogeneity. Indeed, it is well known in the literature that this problem can damage the predictions (Liu [30]).
We computed marginal effects of the hazard at representative values using Williams ([46])'s methodology. Control variables are fixed to compute the magnitude of the effect of the reform for specific student profiles. Four socio-economic profiles of student are considered. First, a Belgian female without scholarship that never repeated a year in high school (Profile 1). Second, a Belgian female with a scholarship that never repeated a year in high school (Profile 2). Third, a Belgian male without scholarship that repeated at least one year in high school (Profile 3). Fourth, a Belgian male with a scholarship that repeated at least one year in high school (Profile 4). Following the literature, Profiles 1 and 4 have characteristics that are respectively favourable and unfavourable to academic success.
Under the assumption that the change of framework is not constant over time, the marginal probabilities resulting from Equation (4) without the interaction term between the reform and the high-school repetition variable[16] are estimated in Table 6. This most complete model is used to be able to discuss heterogeneity in the effects of the reform.
Table 6. Marginal effects w.r.t. 'Reform × Time' (Equation (4) without the interaction term between the reform and the high-school repetition variable).
With respect to dropouts, the coefficients are negative in the 3rd time period and stronger for the scholarship holders (profiles 2 and 4). At the 5% significance level in the
Graph
Looking at graduation in more depth, the credit accumulation system appears to have significantly reduced the probability to earn a degree in the
Graph
Furthermore, to better understand the links between the different variables, a multiple correspondence analysis[17] (Figure 4) is performed. This multivariate descriptive statistical method represents graphically underlying structures in the data using a projection of the data on the first two principal components, which are the directions that capture the maximum amount of information from the initial structure. This analysis highlights a strong association on the one side of the categories of the covariates of the students having a favourable background to succeed at university (no scholarship, female, Belgian, never repeated a year in high school) and on the other side, of students having a unfavourable background (scholarship, male, foreigner, having repeated at least one year in high school).
Graph: Figure 4. Multiple correspondence analysis where the two first dimensions explained 42.9% of the variability in the data.
Last, a robustness check can be carried out by reiterating the previous multinomial models without the 2015 cohort. This verifies that the 2015 cohort, that contains at most four time periods of observation after the first enrolment at university, has not introduced any biases which is coherent with the fact that survival models are able to control censored data. The findings of the multinomial models and their associated marginal effects are very stable and clearly robust since, the significance and magnitude of the effects barely change compared to the previous models.
7. Conclusion
To the best of our knowledge, this paper is the first to investigate the effect of a change from a year-based system to a credit accumulation framework on student academic outcomes (dropout and graduation), especially for socio-economic deprived students.
The econometric models have revealed that the odds of students to drop out have been significantly reduced at the early stages of their student path (i.e. in the three first time periods after the first enrolment at university), and their odds to earn a degree on time or with a one-year delay have been significantly reduced. Moreover, the results show that students from all socio-economic backgrounds were negatively affected by the credit-accumulation system. For the students with the most favourable backgrounds regarding academic success, the time to graduation is increased, while the most vulnerable students were mainly hit by a decrease in the probability to drop out without any improvement regarding graduation. Among other results, males, scholarship holders and students that repeated at least one year in high school tend to experience larger odds to drop out and are less likely to complete a degree. These results are consistent with the literature.
These findings suggest that the flexibility associated with the credit accumulation system prompted students to accumulate 'failed credits', forcing them to spread their bachelor cycle over more years. The lengthening of the duration of the studies may have led them to a lack of motivation, generating late dropouts, in particular for students coming from disadvantaged backgrounds. Concerning the postponing of graduation, a potential explanation could be that students (especially from favourable backgrounds) levelled down their efforts to succeed since the credit accumulation scheme allowed them to continue their student path in spite of failed courses. It should be kept in mind that these inefficiencies are morally and financially costly for all stakeholders, but especially for students, families, and the state.
A better structuring of the milestones to be reached at the beginning of the academic path seems to be an adequate direction to reduce the negative impacts of a credit accumulation system.
Acknowledgements
This study was made possible thanks to the tremendous data collection work of Alice McCathie (STEP, ULB). Thank you to ARES for having brought the initial master's thesis into light through the 'Prix Philippe Maystadt'. Finally, we would like to thank the two reviewers and the Editor for their careful reading of our manuscript and their useful comments.
Disclosure statement
No potential conflict of interest was reported by the author(s).
Notes
1 A year-based system is characterised by a yearly success of the yearly credits.
2 In a credit-accumulation system, courses are not linked to years and students are allowed to progress in their educational path, even when some courses have been failed.
3 Theories analysing student behaviour from the perspective of benefits (cost/effectiveness).
4 Theories analysing students' decisions by the impact of the organisational dimensions of the institution.
5 Theories analysing student's behaviour through their characteristics and the type of academic environment offered.
6 The students' names have been encrypted to preserve their anonymity.
7 In this study, each student path is unique in the sense that only the 'principal' study programme is retained (less than 4% of students are enrolled in a double degree).
8 The academic year 2019–2020 cannot be taken into account due to biases introduced by the COVID-19 pandemic.
9 Reorientation outside the ULB is considered as a dropout since no information is provided in the database when the reorientation occurs outside the university.
Only the 2014 cohort is displayed at time 5 under the credit accumulation system since the 2015 cohort follows only students up to 4 time periods.
To calculate odd ratios:
Graph
To calculate odd ratios:
Graph
To calculate odd ratios:
Graph
Relative odd ratios:
Graph
Relative odd ratios:
Graph
In Table 5, it is shown that there is no heterogeneity in the impact of the reform on the variable specifying that the student repeated or not at least one year in high school.
Multiple Correspondence Analysis (MCA) is a multivariate statistical technique that explores relationships between categorical variables by representing them in a low-dimensional space, emphasising correlation patterns. MCA aims to capture the maximum variance in the data while preserving the associations among variables, providing a comprehensive visualisation of complex categorical data structures.
References
Allen, David. 1999. " Desire to Finish College: An Empirical Link Between Motivation and Persistence." Research in Higher Education 40 : 461 – 485. https://doi.org/10.1023/A:1018740226006.
Arendt, Jacob Nielsen. 2013. " The Effect of Public Financial aid on Dropout from and Completion of University Education: Evidence from a Student Grant Reform." Empirical Economics 44 (3): 1545 – 1562. https://doi.org/10.1007/s00181-012-0638-5.
ARES. 2019. Indicateurs de l'enseignement supérieur.
Arias Ortiz, Elena, and Catherine Dehon. 2008. " What Are the Factors of Success at University? A Case Study in Belgium." CESifo Economic Studies 54 (2): 121 – 148. https://doi.org/10.1093/cesifo/ifn012.
Arias Ortiz, Elena, and Catherine Dehon. 2013. " Roads to Success in the Belgian French Community's Higher Education System: Predictors of Dropout and Degree Completion at the Université Libre de Bruxelles." Research in Higher Education 54 (6): 693 – 723. https://doi.org/10.1007/s11162-013-9290-y.
Babyak, Michael A. 2004. " What You See May Not Be What You Get: A Brief, Nontechnical Introduction to Overfitting in Regression-Type Models." Psychosomatic Medicine 66 (3): 411 – 421. https://doi.org/10.1097/01.psy.0000127692.23278.a9.
Baker, Therese L., and William Velez. 1996. " Access to and Opportunity in Postsecondary Education in the United States: A Review." Sociology of Education 69 : 82 – 101. https://doi.org/10.2307/3108457.
Bean, John P., and Barbara S. Metzer. 1985. " A Conceptual Model of Nontraditional Undergraduate Students' Attrition." Review of Educational Research 55 (4): 485 – 540. https://doi.org/10.3102/00346543055004485.
Becker, Gary S., and Nigel Tomes. 1976. " Child Endowments and the Quantity and Quality of Children." Journal of Political Economy 84 (4, Part 2): S143 – S162. https://doi.org/10.1086/260536.
Blankenberger, Bob, Eric Lichtenberger, M. Allison Witt, and Doug Franklin. 2017. " Diverse Students, High School Factors, and Completion Agenda Goals: An Analysis of the Illinois Class of 2003." Education and Urban Society 49 (5): 518 – 545. https://doi.org/10.1177/0013124516644047.
Bone, Elisa, and Robert Reid. 2013. " First Course at University: Assessing the Impact of Student age, Nationality and Learning Style." The International Journal of the First Year in Higher Education 4 (1): 95 – 107. https://doi.org/10.5204/intjfyhe.v4i1.138.
Bordon, P., and C. Fu. 2015. " College-Major Choice to College-Then-Major Choice." The Review OfEconomic Studies 82 (4): 1247 – 1128. https://doi.org/10.1093/restud/rdv023.
Bruffaerts, Christopher, Catherine Dehon, and Bertrand Guisset. 2011. "Can Schooling and Socio-Economic Level be a Millstone to a Student's Academic Success?" ECARES Working Paper 2011-016.
Calcagno, J. C., P. Crosta, T. Bailey, and D. Jenkins. 2007. " Does Age of Entrance Affect Community College Completion Probabilities? Evidence from a Discrete-Time Hazard Model." Educational Evaluation and Policy Analysis 29 (3): 218 – 235. https://doi.org/10.3102/0162373707306026.
Chealy, Chet. 2009. " Higher Education in Cambodia." In The Political Economy of Educational Reforms and Capacity Development in Southeast Asia: Cases of Cambodia, Laos and Vietnam, edited by Yasushi Hirosato and Yuto Kitamura, 153 – 165. Dordrecht : Springer Netherlands.
Chen, Rong, and Stephen L. DesJardins. 2008. " Exploring the Effects of Financial Aid on the Gap in Student Dropout Risks by Income Level." Research in Higher Education 49 (1): 1 – 18. https://doi.org/10.1007/s11162-007-9060-9.
Clerici, Renata, Anna Giraldo, and Silvia Meggiolaro. 2015. " The Determinants of Academic Outcomes in a Competing Risks Approach: Evidence from Italy." Studies in Higher Education 40 (9): 1535 – 1549. https://doi.org/10.1080/03075079.2013.878835.
Cox, David R. 1972. " Regression Models and Life-Tables." Journal of the Royal Statistical Society: Series B (Methodological) 34 (2): 187 – 202.
Danhier, Julien, and Dirk Jacobs. 2017. Aller au-delà de la ségrégation scolaire: Analyse des résultats à l'enquête PISA 2015 en Flandre et en Fédération Wallonie-Bruxelles. Bruxelles : Fondation Roi Baudouin.
Declercq, K., and F. Verboven. 2018. " Enrollment and Degree Completion in Higher Education Without Admission Standards." Economics of Education Review 66 : 223 – 244. https://doi.org/10.1016/j.econedurev.2018.08.008.
DesJardins, Stephen L., Dennis A. Ahlburg, and Brian P. McCall. 1999. " An Event History Model of Student Departure." Economics of Education Review 18 (3): 375 – 390. https://doi.org/10.1016/S0272-7757(98)00049-1.
DesJardins, Stephen L., Dennis A. Ahlburg, and Brian P. McCall. 2002. " Simulating the Longitudinal Effects of Changes in Financial Aid on Student Departure from College." Journal of Human Resources 37 (3): 653 – 679. https://doi.org/10.2307/3069685.
DesJardins, Stephen L., Dennis A. Ahlburg, and Brian P. McCall. 2006. " An Integrated Model of Application, Admission, Enrollment, and Financial Aid." The Journal of Higher Education 77 (3): 381 – 429. https://doi.org/10.1080/00221546.2006.11778932.
Haarala-Muhonen, A., M. Ruohoniemi, A. Parpala, E. Komulainen, and S. Lindblom-Ylänne. 2017. " How Do the Different Study Profiles of First-Year Students Predict Their Study Success, Study Progress and the Completion of Degrees? " Higher Education 74 (6): 949 – 962. https://doi.org/10.1007/s10734-016-0087-8.
Hu, Shouping, and Edward P. St. John. 2001. " Student Persistence in a Public Higher Education System: Understanding Racial and Ethnic Differences." The Journal of Higher Education 72 (3): 265 – 286.
Hu, Shouping, and George D. Kuh. 2002. " Being (Dis)Engaged in Educationally Purposeful Activities: The Influences of Student and Institutional Characteristics." Research in Higher Education 43 (5): 555 – 575. https://doi.org/10.1023/A:1020114231387.
Ishitani, Terry T., and Stephen L. DesJardins. 2002. " A Longitudinal Investigation of Dropout from College in the United States." Journal of College Student Retention: Research, Theory & Practice 4 (2): 173 – 201. https://doi.org/10.2190/V4EN-NW42-742Q-2NTL.
Jochems, Wim, Joke Snippe, Harm Jan Smid, and Agnes Verweij. 1996. " The Academic Progress of Foreign Students: Study Achievement and Study Behaviour." Higher Education 31 (3): 325 – 340. https://doi.org/10.1007/BF00128435.
Lambert, Jean-Paul. 2020. "L'enseignement supérieur peut-il être à la fois excellent et démocratique? Une analyse comparée des systems." CEREC Working Paper (2020) n° 2020/5. https://doi.org/10.25518/ciriec.wp202005
Liu, Xian. 2014. " Survival Models on Unobserved Heterogeneity and Their Applications in Analyzing Large-Scale Survey Data." Journal of Biometrics & Biostatistics 5 : 1000191.
Meggiolaro, Silvia, Anna Giraldo, and Renata Clerici. 2017. " A Multilevel Competing Risks Model for Analysis of University Students' Careers in Italy." Studies in Higher Education 42 (7): 1259 – 1274. https://doi.org/10.1080/03075079.2015.1087995.
Murtaugh, Paul A., Leslie D. Burns, and Jill Schuster. 1999. " Predicting the Retention of University Students." Research in Higher Education 40 (3): 355 – 371. https://doi.org/10.1023/A:1018755201899.
Nguyen, Huu Cuong, and Thi Thu Hien Ta. 2018. " Exploring Impact of Accreditation on Higher Education in Developing Countries: A Vietnamese View." Tertiary Education and Management 24 (2): 154 – 167. https://doi.org/10.1080/13583883.2017.1406001.
Organisation for Economic Co-operation and Development. 2022. Education at a Glance 2022; OECD Indicators. Paris, France : OECD Publishing.
Ott, Mary Diederich. 1988. " An Analysis of Predictors of Early Academic Dismissal." Research in Higher Education 28 (1): 34 – 48. https://doi.org/10.1007/BF00976858.
Scott, Marc A., and Benjamin B. Kennedy. 2005. " Pitfalls in Pathways: Some Perspectives on Competing Risks Event History Analysis in Education Research." Journal of Educational and Behavioral Statistics 30 (4): 413 – 442. https://doi.org/10.3102/10769986030004413.
Sheard, Michael. 2009. " Hardiness Commitment, Gender, and age Differentiate University Academic Performance." British Journal of Educational Psychology 79 (1): 189 – 204. https://doi.org/10.1348/000709908X304406.
Singer, Judith D., and John B. Willett. 1993. " It's About Time: Using Discrete-Time Survival Analysis to Study Duration and the Timing of Events." Journal of Educational Statistics 18 (2): 155 – 195. https://doi.org/10.3102/10769986018002155.
Sneyers, Eline, and Kristof De Witte. 2017. " The Effect of an Academic Dismissal Policy on Dropout, Graduation Rates and Student Satisfaction. Evidence from the Netherlands." Studies in Higher Education 42 (2): 354 – 389. https://doi.org/10.1080/03075079.2015.1049143.
Stage, Frances K. 1988. " University Attrition: LISREL with Logistic Regression for the Persistence Criterion." Research in Higher Education 29 (4): 343 – 357. https://doi.org/10.1007/BF00992775.
Tekle, Fetene B., and Jeroen K. Vermunt. 2012. " Event History Analysis." In APA Handbook of Research Methods in Psychology, Vol 3: Data Analysis and Research Publication, edited by H. Cooper, P. M. Camic, D. L. Long, A. T. Panter, D. Rindskopf, and K. J. Sher, 267 – 290. Washington, DC : American Psychological Association.
Tinto, Vincent. 1975. " Dropout from Higher Education: A Theoretical Synthesis of Recent Research." Review of Educational Research 45 (1): 89 – 125. https://doi.org/10.3102/00346543045001089.
Tinto, Vincent. 1992. " Student Attrition and Retention." The Encyclopedia of Higher Education 3 : 1697 – 1709.
Tinto, Vincent. 1993. Leaving College Rethinking the Causes and Cures of Student Attrition. Chicago : The University of Chicago Press.
Willett, John B., and Judith D. Singer. 1991. " From Whether to When: New Methods for Studying Student Dropout and Teacher Attrition." Review of Educational Research 61 (4): 407 – 450. https://doi.org/10.3102/00346543061004407.
Williams, Ronald. 2012. " Using the Margins Command to Estimate and Interpret Adjusted Predictions and Marginal Effects." The Stata Journal 12 (2): 308 – 331. https://doi.org/10.1177/1536867X1201200209.
Zarifa, David, Jeannie Kim, Brad Seward, and David Walters. 2018. " What's Taking You So Long? Examining the Effects of Social Class on Completing a Bachelor's Degree in Four Years." Sociology of Education 91 (4): 290 – 322. https://doi.org/10.1177/0038040718802258.
By Catherine Dehon and Léonore Lebouteiller
Reported by Author; Author