TextMining

Electronic theses and dissertations (ETDs) contain valuable knowledge that can be useful in a wide range of research areas. Accordingly, we are building electronic infrastructure leveraging advanced work on digital libraries, for discovering and accessing the knowledge buried in ETDs. We focus on our work to incorporate topic modeling into digital libraries for ETDs. We present ETD-Topics, a framework that extracts topics from a large text corpus in an unsupervised way. The representations learnt from topic models can be useful for downstream tasks such as searching and/or browsing documents by topic, document recommendation, topic recommendation, and describing temporal topic trends (e.g., from the perspective of disciplines or universities). The characteristics of different models make the classification distinguished. We provide four modes (LDA, NeuralLDA, ProdLDA, and CTM) to serve user groups with different browsing and searching requirements. Our job was to import the preprocessed database and the trained models (four models with different topic numbers), and to accurately display key information (such as topics, document title, abstract, etc.) on web pages. We chose Python as the main language to implement the back-end, while using Flask as a bridge connecting the back-end and front-end. On the basis of using HTML for displaying data, we were able to use JavaScript and CSS to make the whole set of web pages look more fluent and comfortable by optimizing the UI, to include graphic bars, buttons (like “Submit”, “Show more”, etc.), and tables.