\chapter{Requirements} \label{ch:require }                   
	    % Content Here %
        \section{General Requirements}
            Regarding the whole ETD project, our team should liaise with other teams and containerize all team's code that runs on cloud.cs.vt.edu. We should make sure we have continuous integration / continuous deployment (CI/CD), with a test routine for each service, and a test routine for each frequently run workflow, so when changes are checked in, there will be automated service and workflow testing, leading to immediate deployment if tests are successful. This CI/CD approach should lead to a considerable speedup in our system building,  testing, and experimentation. Following is a list of requirements/objectives.
            \begin{enumerate}
                \item Search will be performed on Electronic Theses and Documents.
                % \item This search can be performed on any digital object and/or its matching metadata object, including ETDs and anything, extracted or derived from an ETD or its parts. Adding the capability to search chapters and chapter summaries is a top priority.
                \item To facilitate this search, classification and other methods such as object detection will be used.
                \item Another workflow would be chapter or full-text summarization.
                % \item Data pre-processing such as identifying parts of speech, stemming, etc. will be used.
                \item Provide file systems to other teams to store the actual files and a database (Postgres) to store all the metadata.
                \item Provide appropriate levels of access to data.
                \item Tie all these together with the help of web servers running in Containers, Databases, Reasoner Engine, and Machine Learning models
                %running with the help of 
                aided by
                GPUs.
                \item Finally, using CI/CD to automatically deploy code and use automated test cases with the help of pipelines.
            \end{enumerate}
        \section{User Personas}\label{ch:user_personas}
	        User personas are built as a way to understand our requirements as we serve teams 1, 2, 3, and 4. These user personas will be more abstract and less ‘open’ 
         for Teams 3 and 4, as
         compared to Teams 1 and 2. Moreover, these personas are implicit, as in they do not reflect the exact nature of the user and their requirements.
	        Following are some of the user personas that we have collected from other teams.
	        \begin{itemize}                
	            \item \textbf{Team 1}
                    \begin{enumerate}
                        \item As an engineer, I want to deploy my backend/frontend code and any storage to cloud servers.
                        \item As an engineer, I want to automatically test and integrate my code with the one stored on the cloud.
                        \item As a curator, I should be able to curate digital objects such as ETDs and metadata objects.
                        \item As an engineer,  I should be able to make use of a Knowledge Graph to support question answering and managing the relationships between metadata and digital objects.
                    \end{enumerate}    
                \item \textbf{Team 2}
                    \begin{enumerate}
                        \item As an engineer, I want a cloud platform that helps me pre-process, index, and support the search of data.
                        \item As an engineer, I want to deploy my front-end web application for searching, and to make it available to my end user.
                        \item As an engineer, I should be able to set up the system in a way that it can answer questions.
                    \end{enumerate} 
                \item \textbf{Team 3 and Team 4}
                    \begin{enumerate}
                        \item As an Experimenter, I want a platform to train a machine-learning model to perform object detection and topic modeling.
                        \item As an Experimenter, I should be able to run experiments using these models and compare their results for optimal use.
                        \item As an Experimenter, I want access to the training data for my model.
                        \item As an Experimenter, I want to offload the results of my trained model to a persistent storage 
                        \item As an Experimenter, I should be able to perform chapter segmentation, text summarization, and classification of ETDs.
                        \item As an Experimenter, I should be able to perform object detection and topic modeling on ETDs.
                    \end{enumerate}
             \end{itemize}
	    \section{Workflow Automation Requirements}
	        The workflow automation system is centered around executing various tasks regarding Electronic Theses and Dissertations using services provided by other teams. The task execution will be abstracted into a Directed Acyclic Graph (DAG), fulfilled by a workflow engine called Apache Airflow. Following is a brief overview of Apache Airflow:
	        \begin{enumerate}
                \item Developers should be able to add goals, workflows, and services metadata, and test their functionalities.
                \item Experimenters and Developers should be able to see and execute the goals provided by other developers.
                \item Workflows (expressed as Directed Acyclic Graphs) should be automatically generated seamlessly. 
                \item Containerization of the Airflow engine and PostgreSQL database (that holds relevant workflow tables) should be in place.
                \item Appropriate data parsing should occur from the metadata tables to generate DAGs.
                \item Use of a scalable indexing engine would be desirable to facilitate optimized querying of information.
                \item The system should be straightforward to use.
                \item A Curator should be able to add a new ETD, so that its digital objects along with the document itself should be added to the already-existing collection of ETDs.
             \end{enumerate}
        \section{Performance}
            While we do not expect this system to be able to serve millions of people, we do expect that our basic architecture is scalable and parallel enough to prevent major bottlenecks.
            We will make sure, even if there are hundreds of services or workflows, that query speed will not be largely affected. 
            % We will do this by using "join" when querying.
            % By using Elastic Search, we expect that our system will already have an internal search engine that is optimized. 
            We have used an appropriate data tables design, and incorporated various optimizations, such as load balancing by Kubernetes and for indexing. These are just two of the key things implemented for better performance.
        \section{User Support}
            % By using various analytical tools, we can study user behavior and make certain predictions. A lot of statistical questions can also be answered with the help of these tools.
            The Workflow System consists of 5 elements: Reasoner, Apache Airflow, Registry UI, Registry Backend server, and the PostgreSQL Database that stores all goals and services (and their linkage). To generate or run workflows, various APIs have been exposed. We have built a UI that makes use of Workflow APIs such as `Add Workflow' and `Edit Workflow' for Experimenters to add new goals. This UI makes it easy to visualize and edit workflows and also to add new services. 

            Apart from this, we have also provided teams with an easy way to integrate this system into their frontend applications by the way of a code wrapper that is written in JavaScript, as is discussed more in Section \ref{workflow_automation_js_wrapper}.
	    % \section{Current Work}
	        % Our current work revolves mostly around fulfilling initial requirements for other teams. This includes mostly setting up containers and exploring some CI/CD for code deployment. We have also set up architecture diagrams for internal use.
            % Our current work 
	