Predicting sequences of transactions from a limited first segment: Partial Entropy Gain Analysis for Segments of Observed Sequences (PEGASOS), with an application to youths' municipal social care trajectories

Abstract

Transactional databases are very common across industry and government. For example, the municipality of Utrecht registers transactions of social care use among youngsters. When using such databases for process insights, a common goal is to predict how a sequence of transactions will develop over time from a limited first segment, e.g. a new admission to the municipal social care system. In the present study a new method is introduced to predict how this segment will unfold. First, a standard sequence analysis procedure is used to create a clustering of sequences. Second, the partial entropy gain analysis for segments of sequences (PEGASOS) is introduced. PEGASOS uses the created clustering to make a distance based classification. Subsequently, a calibration procedure is used to take into account uncertainty about this classification. Finally, models with an increasing length of the first segment are compared to determine how long the municipality should wait for a good quality prediction. PEGASOS can be applied whenever analysts aim both at data interpretation through clustering and at prediction of further development from a limited number of transactions.