Effective and Efficient Classification of Topically-Enriched Domain-Specific Text Snippets

Abstract

In today’s society millions of new text snippets are created and generated every day in the form of tweets, search results, blog posts, and financial transactions. Due to the shortness and sparsity of text in these text snippets, classification is a complex matter. In this research a method is created that facilitates the efficient and effective classification of domain-specific text snippets.

Because of the sparsity of text in text snippets a literature study is performed on the enrichment of text snippets. Five different techniques are identified, all involving external data sources. Because involving external data sources increases the dimensionality of the data, and because millions of text snippets are created on a daily basis, additional research is performed to identify dimensionality reduction techniques.

Based on the results of the literature study and the evaluation of a previously created method for categorizing domain-specific text snippets a generic method, the Topically-Enriched Text Snippet Classification method (TETSC) is constructed. TETSC facilitates the topical enrichment of domain-specific text snippets through inferring topic distributions of enrichments originating from external data sources. In TETSC a distinction between text snippet types is made, because of which different enrichment strategies can be created for different types of text snippets. TETSC is validated through implementation at AFAS Personal.

This validation shows TETSC to be both effective and efficient in the classification of text snippets. Furthermore, TETSC shows to be very flexible in implementation, because of which the method and thus this research is a success.