%-------------Introduction
\chapter{Introduction}

\section{Objective}
The objective of this project is to build a state-of-the-art information retrieval system.
Our team's role is to process tobacco settlement documents and organize them in a way that would be suitable for ingestion into the system.
Then they would feed as input to the information retrieval system.
This project aims to provide a suitable interface to query tobacco settlement documents to aid Dr. Townsend’s research. 

The source of documents is UCSF’s Industry Documents Library (IDL). The IDL hosts documents pertaining to 5 different industries, namely tobacco, drug, chemical, food, and fossil fuel. Through the course of this project, we utilize the 14 million tobacco settlement documents housed by IDL. These documents were created by tobacco companies and come in varying formats like depositions, memos, company letters, etc.

The aim is to organize the documents, parse them, and provide metadata along with the data to the ElasticSearch (ELS) and the Text Analytics and Machine Learning (TML) teams.

We are working on the data ingestion pipeline, where we load the entire data, process all 14 million documents, and provide relevant data that can be processed by the ELS and front-end teams to provide the final product: an information retrieval system that can be used to query the tobacco dataset.

Our aim is to ensure that relevant and correct data is being returned while querying. We have worked closely with the above-mentioned teams to ensure that this happens.

\section{Client}
The client of this project is Dr. David M. Townsend, an assistant professor in the Virginia Polytechnic Institute and State University (Virginia Tech) Pamplin College of Business.
Dr. Townsend's research focuses on entrepreneurship, strategic management, and technology management. He is working on analyzing the documents publicly released by tobacco companies as a result of a number of lawsuits launched against them. As an active user of the document library, Dr. Townsend is personally interested in having a specialized information retrieval system that could assist him in his research endeavors.

\section{Project Management}
To provide effective communication between team members and representatives, we use Slack, an easy-to-use chat system with a user-friendly interface, which allows for creating and maintaining chat groups for instant sharing of updates and suggestions both within the team and among participants from different teams. This was extremely useful to collaborate with members of the ELS and TML teams. We were able to work closely with them to understand their requirements and deliver the requirements in a timely manner.

To effectively distribute tasks and track progress, we use Trello, a task management application that gives a visual overview of the progress made.

\section{Challenges}
There were a number of challenges faced during the course of this project.
One challenge was the nature of the data.
We started with 14 million documents.
There was little documentation regarding the structure of the data.
We initially used MySQL queries and a sample set of data to understand the attributes and metadata. We also reached out to the technical support team of UCSF library and they were extremely helpful. We were then able to fully understand the structure of data and their attributes.

Another challenge we faced was that we were unable to distribute the tasks among team members in a timely manner, given a range of levels of technical and programming expertise amongst the participants.
Notwithstanding, we constantly worked on effectively parallelizing work, addressing critical questions, and supporting each other in leveraging appropriate skills.

