Text Data Mining Studio ETDs

Rielage, Robert; Brinkley, Hannah; Dy, Christian; Vaanjilnorov, Jason

Text Data Mining Studio ETDs

dc.contributor.author	Rielage, Robert	en
dc.contributor.author	Brinkley, Hannah	en
dc.contributor.author	Dy, Christian	en
dc.contributor.author	Vaanjilnorov, Jason	en
dc.date.accessioned	2020-05-13T23:17:40Z	en
dc.date.available	2020-05-13T23:17:40Z	en
dc.date.issued	2020-05-11	en
dc.description.abstract	The goal of the Text Data Mining Studio ETD project was to develop a software system that allows less technically-minded researchers to be able to easily access a vast amount of data to be used for text data analytics. Specifically, the problem that our team addressed was the lack of a centralized tool for analysis of a large amount of text files that would be valuable data to be used for machine learning and other forms of analytics for long form text documents. Our team created a centralized tool using ElasticSearch, JupyterHub, and Jupyter Notebooks, in the Docker Compose architecture, to provide this service to researchers. We envision this tool being used by researchers whose work involves the ingestion and analysis of large bodies of text. This work could involve producing deep learning based systems for textual analysis and search, for automatic abstract production, or perhaps for trend tracking across highly temporally spaced documents. The tool is intended to be a flexible foundation for these and other research tasks. During the process of producing this tool, our team learned a great deal about Docker systems, compartmentalization, and JupyterHub, as well as working in a widely distributed team using virtual meeting analogs. Unfortunately the tool is as of yet, incomplete, as the ElasticSearch systems are not linked to the JupyterHub frontend. We are hopeful that given the information outlined in the report that completing the tool will be possible with future work.	en
dc.description.notes	The following documents are included in this item: Project Report: This report includes an explanation of the requirements given to us by our client, the design of our project, specific implementation information, a User’s Manual, a Developer’s Manual, as well as the lessons we learned as a team in the process of working on this project, and a short explanation of recommended extensibility. Project Presentation: A short presentation on the project, containing a demonstration, and an overview of much of the content in the report. Project Demonstration: A short demo of the project, originally used as a prepackaged part of the presentation. Project Files: - Dockerfile: Docker execution file - Docker-compose.yml: Docker configuration file - load_metadata.ipynb: Base Python code for JupyterHub system.	en
dc.description.sponsorship	IMLS LG-37-19-0078-19	en
dc.identifier.uri	http://hdl.handle.net/10919/98252	en
dc.language.iso	en_US	en
dc.publisher	Virginia Tech	en
dc.rights	Creative Commons CC0 1.0 Universal Public Domain Dedication	en
dc.rights.uri	http://creativecommons.org/publicdomain/zero/1.0/	en
dc.subject	ETD	en
dc.subject	OCR	en
dc.subject	Elasticsearch	en
dc.subject	Jupyter	en
dc.subject	JupyterHub	en
dc.subject	Jupyter Notebooks	en
dc.subject	Docker	en
dc.subject	Docker Compose	en
dc.title	Text Data Mining Studio ETDs	en
dc.type	Presentation	en
dc.type	Report	en
dc.type	Video	en