	\chapter{Implementation} \label{ch:conclusions}
	\section{Workflow Automation} \label{ch:implementation_workflow_automation}
    In this section, we first describe how we used the 2020 INT team's code to generate the context-free-grammar file to represent DAG. Then we showcase all related reasoner APIs for workflow automation. After that, we show how the UI interacts with the reasoner APIs. Finally, we will give a detailed demo of how we automate team-4's services.
            \subsection {Context-Free Grammar} 
            
            A Context-Free Grammar (CFG) is a set of recursive rules used to generate patterns of strings. A context-free grammar in general can be described by four components V, ${\Sigma}$, R, and S, where V is a set of non-terminal symbols, ${\Sigma}$ is a set of terminal symbols, R is a set of production rules that defines a mapping from a non-terminal symbol to a string $s \in (V, E)$, and $S \in V$ is a start symbol. Context-Free Grammars find application in areas such as theoretical computer science, compiler design, and linguistics \cite{CFG}.
            
            In our approach, which is built on the reasoner implementation of Team INT-2020, a Context-Free Grammar is used as a way to represent goals and services as components of a workflow. 
            A goal identifies a result state, i.e., some stored data, that comes from running a service.
            The stored data herein is referred to as `input goal' since it can be an input to another service.
            Each service has a set of input goals and a single (result) goal. Input goals are starting directories or files that a service needs to use and work upon. Once completed, the output for such a service is a (result) goal which can either be a file or a directory. 
            
            The Python “nltk” package provides a “grammar” module which is used for generating a CFG from a registry of services. The advantage of generating a CFG is that it can help generate multiple possible workflows that can help to achieve a particular end goal. Although in our current approach each goal is mapped to a single workflow, in the future the implementation can be extended for cases where multiple workflows are possible for a given goal.

            How are workflows generated using a Context-Free Grammar? Let $G$ be a Goal, $s$ be a service, and $G_{I}$ be an Input Goal. Then, a production rule can be represented as $G \rightarrow s G_{I}$. We intentionally discard the start symbol as it has no use while generating workflows. For a general set of input goals, a generic production rule can be represented as $G \rightarrow s G_{I1}G_{I2}G_{I2}..G_{In}$. With the help of the reasoner table, the production rules can be generated for the entire table. Let's take an example:
            \begin{center}
                \begin{tabular}{||c c c||} 
                     \hline
                     Goal ID & Service ID & Input Goal ID \\ [0.5ex] 
                     \hline\hline
                     1 & Null & Null \\ 
                     \hline
                     2 & $S_{1}$ & 1 \\
                     \hline
                     3 & $S_{2}$ & 2 \\
                     \hline
                \end{tabular}
            \end{center}
            Before we attempt to construct the rules, let's give a brief explanation of the table. The tables state that:
            \begin{itemize}
              \item 1 is the start Goal (Service ID is Null).
              \item Goal 1's data is the input for Service $S_1$ and the output Goal is 2. Similarly for Service $S_2$.
            \end{itemize}
            This workflow, as shown, is linear, i.e., Service 3 executes after Service 2 which executes after Service 1.
            If by the above generic format we generate a production rule using the values, we obtain the following ($S$ is the start state):
            \begin{center}
                $S \rightarrow G_{1}$
               
                $G_{3} \rightarrow S_{2}G_{2}$
                
                $G_{2} \rightarrow S_{1}G_{1}$
                
                $G_{1} \rightarrow \epsilon$
            \end{center}
            If we now construct the string using these rules and consider $G_{3}$ as a final goal, then we obtain the following string, 
            \begin{center}
                $G_{3} \rightarrow S_{2}S_{1}\epsilon$
            \end{center}
             This string means that to achieve Goal $G_{3}$, Service $S_{1}$ and then $S_{2}$ need to be executed. If we now give Goal $G_{3}$ as the Goal ID to the generate goal API, then we will obtain a workflow with services $S_{1}$ and $S_{2}$. 

             Currently, the reasoner table, which helps generate these rules, is manually populated as per requirements from other teams. The following are the rules:
             \begin{center}
                 \begin{verbatim}
                                71 -> '37' 115
                                112 -> '38' 71
                                115 -> 
                                89 -> '48' 73
                                73 -> '47' 114
                                119 ->     117
                                116 ->     121
                                121 -> '60' 120
                                114 -> '39' 112
                                90 -> '49' 89
                                120 -> '59' 90
                                117 -> '62' 115
                                124 -> '62' 121
                                122 -> '63' 124
                                123 ->      122
                 \end{verbatim}
             \end{center}

             
            \subsection{Reasoner API} \label{reasoner_api}
            The reasoner API consists of four consecutive APIs, which are generate\_grammar\_API, generate\_workflow\_API, run\_workflow\_API, and get\_workflow\_status\_API. We explain each of them in detail here.
                    \subsubsection{Generate Grammar API}
                    This API sends a GET request to the reasoner server that queries the current reasoner table. Then it updates the cfg file. Required input: None. Output is shown in Figure~\ref{fig:gen_grammar_output}.
                    \begin{figure}[H]
                    	\centering
                    	\includegraphics[scale = 0.8]{img/gen_grammar_output.png}
                    	\caption{Example Output of Generate Grammar}
                    	\label{fig:gen_grammar_output}
                    \end{figure}
                    \subsubsection{ Generate Workflow API }
                    This API sends a POST request to the server, which reads the updated cfg file and generates the workflow DAG. Required input: \textit{workflowId} (which is also the end goal\_id of the workflow to execute, for example, the goal\_id of \textit{End} in Figure
                    \ref{fig:dag_sample}). Optional input: \textit{Env} for specified services within the workflow. We show an example in which all services need \textit{ETDID=7} set as one of the Environment Variables. See Figure~\ref{fig:gen_workflow_env} for the Code.
                    \begin{figure}[H]
                    	\centering
                    	\includegraphics[scale = 0.8]{img/gen_workflow_env.png}
                    	\caption{Example of Optional Environment variable passed to Generate Workflow}
                    	\label{fig:gen_workflow_env}
                    \end{figure}
                    Output: a JSON object that contains the requested goal ID, workflow ID, service metadata, goal metadata, and workflow metadata. The example output is shown in Figure~\ref{fig:gen_workflow_output}.
                    \begin{figure}[]
                    	\centering
                    	\includegraphics[scale = 0.6]{img/gen_workflow_output.png}
                    	\caption{Example Output of Generate Workflow}
                    	\label{fig:gen_workflow_output}
                    \end{figure}
                    \subsubsection{Run Workflow API}
                    Once the DAG is generated, this API can be called along with the workflow ID from the previous API, to execute the DAG. Required input: a JSON with \textit{args} and \textit{service\_names}. The first element in \textit{args} should be the workflow ID. The second is the \textit{execution\_date}. If the \textit{execution\_date} is before the current date, the DAG is triggered immediately. \textit{service\_names} is optional; all tasks will be executed if this field is left blank. An example input is in Figure~\ref{fig:run_workflow_input}
                    \begin{figure}[h]
                    	\centering
                    	\includegraphics[scale = 0.8]{img/run_workflow_input.png}
                    	\caption{Example of Request Body of Run Workflow}
                    	\label{fig:run_workflow_input}
                    \end{figure}
                    Output: a \textit{key} of current execution, a \textit{result\_url} that can be used to trace logs, and \textit{status} will show as \textit{running} (Figure~\ref{fig:run_workflow_output}).
                    \begin{figure}[h]
                    	\centering
                    	\includegraphics[scale = 0.6]{img/run_workflow_output.png}
                    	\caption{Example Output of Run Workflow}
                    	\label{fig:run_workflow_output}
                    \end{figure}
                    \subsubsection{Get Workflow Status API}
                    This API can be used to check the status of an execution. Required input: the \textit{key} produced by the previous API. Output: a JSON object that shows the information of the execution, such as error message, start time, end time, logs, and return code.
                    \subsubsection{API order}
                    The order to run each is as follows:
                    \begin{enumerate}
                        \item Generate Workflow API
                        \item Run Workflow API
                        \item Get Workflow Status API
                    \end{enumerate}
                    Note that the Generate Grammar API needs to be called only when services or goals are added/deleted or the reasoner table is modified in any way. 
                % \subsubsection{Pre-pushed Docker Image on Docker Hub with Postgres Database}
                % In our prototype, we pre-build and push the Postgres image that contains three metadata tables (as defined in Section~\ref{section:registryDatabasePrototype})  with the data definition language already baked, to Docker Hub (lyuze/postgres:latest). 
                
                % To enable Airflow to communicate with Postgres developers need to specify the same \textit{postgres\_conn\_id} in both DAG Python script and the Airflow webserver connection page. 
                % We use PgAdmin4 to enable easy management upon Postgres, see Figure~\ref{fig:pgadmin4} as example. 
                
                % \begin{figure}[!h]
                % 	\centering
                % 	\includegraphics[scale = 0.28]{img/pgadmin4.png}
                % 	\caption{PgAdmin Screenshot}
                % 	\label{fig:pgadmin4}
                % \end{figure}
                
                % \subsubsection{Containerizing Airflow}
                % We use \textit{docker-compose} to spawn the following containers. The Postgres container contains the workflow tables mentioned above. The Airflow Scheduler is the main Airflow Engine. The Airflow Webserver offers end users to execute DAGs easily. The Airflow Initializer is a helper daemon to spawn the Scheduler and Webserver. The detailed YML file is pushed to GitLab.
                
                % \begin{enumerate}
                %     \item Postgres
                %     \item PgAdmin ~\cite{pgAdmin}
                %     \item Airflow Scheduler
                %     \item Airflow Webserver 
                %     \item Airflow Initializer 
                % \end{enumerate}
                
                % We forward ports for containers to communicate with each other. Our \textit{docker-compose.yml} for our prototype is shown in the code section~\ref{lst:dockerCompose} below.
                
                % The Airflow webserver serves the created DAGs via a user interface to users, so that they may manage, configure or trigger DAGs on demand. 
                
                
                % \begin{lstlisting}[caption={\textit{docker-compose.yml} file},captionpos=b,label={lst:dockerCompose}]
                % ---
                % version: '3.4'
                
                % x-common:
                %   &common
                %   image: outerspace1920/airflow:latest
                %   user: "${AIRFLOW_UID}:0"
                %   env_file: 
                %     - .env
                %   volumes:
                %     - ./dags:/opt/airflow/dags
                %     - ./logs:/opt/airflow/logs
                %     - ./plugins:/opt/airflow/plugins
                
                % x-depends-on:
                %   &depends-on
                %   depends_on:
                %     postgres:
                %       condition: service_healthy
                %     airflow-init:
                %       condition: service_completed_successfully
                
                % services:
                %   postgres:
                %     image: lyuze/postgres:latest
                %     container_name: postgres_container
                %     ports:
                %       - "5434:5432"
                %     healthcheck:
                %       test: ["CMD", "pg_isready", "-U", "airflow"]
                %       interval: 5s
                %       retries: 5
                %     env_file:
                %       - .env
                %     # volumes:
                %     #   - ./postgres_mnt:/
                %     command: bash -c "/usr/lib/postgresql/13/bin/postgres -D /var/lib/postgresql/13/main -c config_file=/etc/postgresql/13/main/postgresql.conf"
                
                %   scheduler:
                %     <<: *common
                %     depends_on:
                %       - postgres
                %       - airflow-init
                %     container_name: airflow-scheduler
                %     command: scheduler
                %     restart: on-failure
                %     ports:
                %       - "8793:8793"
                
                %   webserver:
                %     <<: *common
                %     depends_on:
                %       - postgres
                %       - airflow-init
                %     container_name: airflow-webserver
                %     restart: always
                %     command: webserver
                %     ports:
                %       - "8080:8080"
                %     healthcheck:
                %       test: ["CMD", "curl", "--fail", "http://localhost:8080/health"]
                %       interval: 30s
                %       timeout: 30s
                %       retries: 5
                  
                %   airflow-init:
                %     <<: *common
                %     container_name: airflow-init
                %     entrypoint: /bin/bash
                %     command:
                %       - -c
                %       - |
                %         mkdir -p /sources/logs /sources/dags /sources/plugins
                %         chown -R "${AIRFLOW_UID}:0" /sources/{logs,dags,plugins}
                %         exec /entrypoint airflow version
                % \end{lstlisting}
                
                % \subsubsection{Object Detection Service}
                %     We will use the object detection work that was done by the Fall 2020 team. Their setup involves a containerized version of the object detection machine learning model that already contains the trained weights required. The Docker image that we will use is \href{https://hub.docker.com/layers/sampyash/yolov5/airflow_integration/images/sha256-44f3a2714690301e7c79671f6ee928609840912f33d341aab358f092e4d61d14?context=explore}{sampyash/yolov5:airflow\_integration}, which is based on the object detection open-source solution YOLOv5~\cite{yolov5} developed by Ultralytics\textsuperscript{\textcopyright}. This image takes in image files of various formats and outputs the coordinates of the objects detected in the image. An example taken from the authors themselves is shown in Figure~\ref{fig:object_detection_example}.
                
                % \begin{figure}[H]
                % 	\centering
                % 	\includegraphics[scale = 0.3]{img/object_detection_example.jpeg}
                % 	\caption{An example of object detection}
                % 	\label{fig:object_detection_example}
                % \end{figure}
                
                % \subsubsection{DAGs Auto-generation with Python Monitor}
                % In this section, we talk about how our Python monitor creates DAGs using the metadata stored in the Postgres database. The monitor includes several steps -
                % \begin{enumerate}
                %     \item Detect changes of workflow tables and run query
                %     \item Parse query result (2D array) into JSON objects
                %     \item Generate a DAG for each goal, depending on each goal's corresponding services
                % \end{enumerate}
                % First, we use \textit{join} queries to combine all 4 workflow tables to connect them, with optimized query performance. The query command is shown in Figure ~\ref{lst:workflow_query}.
                % A Directed Acyclic Graph (DAG) is a collection of all tasks that we want to run, organized such that we can conclude their relationships and dependencies. 
                
                % Steps for creating a DAG
                % \begin{enumerate}
                %     \item Importing the required modules
                %     \item Creating a DAG Object
                %     \item Creating a Task or Service
                %     \item Setting up Dependencies in DAG. Once we create multiple tasks as displayed above we can set their dependencies using '> >' or '< <'
                %     \item To execute our DAG file, we then use Apache Airflow
                % \end{enumerate}
                % with DAG(
                % 		dag_id='test-workflow',
                % 		schedule_interval="@once",
                % 		start_date=datetime.datetime.now()
                % 	) as dag:
                % 	    loadRegistry = PostgresOperator(
                %             task_id='loadRegistry',
                %             sql=";"
                %         )
                % 		  objectDetection = KubernetesPodOperator(
                % 		    ... See code section Listing 5.2 below
                % 	    )
                % 	    storeOutputAsJson = BashOperator(
                %             task_id='storeOutputAsJson',
                %             sql=";"
                %         )
                % 	)
                % loadRegistry >> objectDetection >> storeOutputAsJson
                
                % \lstset{upquote=true}
                % \begin{lstlisting}[caption={Query Command to Fetch Workflow Metadata},captionpos=b,label={lst:workflow_query}]
                %     select goal.goal_id, goal_name, goal_description, 
                %     goal_format, envs, reasoner.workflow_id, workflow_name, 
                %     workflow_description, service.service_id, service_name, 
                %     service_description, image_url, command, options, 
                %     service_input_location, service_output_location 
                %     from goal_metadata goal INNER JOIN goal_reasoner 
                %     ON goal.goal_id = goal_reasoner.goal_id INNER JOIN 
                %     reasoner_metadata reasoner ON goal_reasoner.workflow_id = 
                %     reasoner.workflow_id INNER JOIN service_metadata service 
                %     ON service.service_id = goal_reasoner.service_id 
                % \end{lstlisting}

                % Next, because the above-mentioned command returns a 2D-array Python list, each role contains one service metadata for each workflow, for each goal, we need to parse the list into several JSON objects, each containing the metadata for each goal/workflow, with several corresponding services (as a list). The example JSON for each goal is shown in~\ref{fig:af-json}.
                % \begin{figure}[!h]
                % 	\centering
                % 	\includegraphics[scale = 0.5]{img/af-json.png}
                % 	\caption{Example JSON Object parsed from queried result}
                % 	\label{fig:af-json}
                % \end{figure}
                
                % At last, we use one of Apache Airflow's operators called the \textit{KubernetesPodOperator} to spin up Docker containers on the CS Cloud Kubernetes cluster. We will be getting the details about the Docker image, and the input and output destinations from the registry, as mentioned before. A sample snippet with explanations of fields is given in Figure ~\ref{lst:sample_kpo}: 
                
                % \subsubsection{Creating Containers on the Cluster}
                % We will be using one of Apache Airflow's operators called the `KubernetesPodOperator' to spin up Docker containers on the CS Cloud Kubernetes cluster. We will be getting the details about the Docker image, and the input and output destinations from the registry, as mentioned before. A sample snippet with explanations of fields is given below: 

                % KubernetesPodOperator(
                % 		name='objectDetection',
                % 		# Name of the operator
                % 		namespace='etd',
                % 		# The namespace to use in the Kubernetes cluster
                % 		image='sampyash/yolov5:airflow\_integration',
                % 		# Name of the image to be used 
                % 		volume_mounts=[camelot_mount, ceph_mount],
                % 		# Volumes mounts on the K8s pod
                % 		volumes=[camelot_volume, ceph_volume],
                % 		# Volumes to be mounted on the K8s pod
                % 		task_id='objectDetection',
                % 		in_cluster=False,
                % 		config_file='/kube_config'
                % 		# This file tells Airflow how it can connect to the K8s cluster to spin up containers
                %   )
                % \lstset{upquote=true}
                % \begin{lstlisting}[caption={\textit{A sample KubernetesPodOperator item} file},captionpos=b,label={lst:sample_kpo}]
                % with dag:
                %     for service in goal.get('service_met'):
                %         serviceImageExt = KubernetesPodOperator(
                %             name='objectDetectionOpt',
                %             namespace='etd',
                %             image=service.get('image_url'),
                %             image_pull_policy='Always',
                %             task_id='serviceImageExt' + str(i),
                %             # cmds=["/bin/bash"],
                %             in_cluster=False,
                %             ports=[port],
                %             config_file='/opt/airflow/dags/kube_config',
                %             #need to be created 
                %         )
                % \end{lstlisting}
                
                % The `kube\_config' file used is given below: 
                % \begin{lstlisting}[caption={\textit{kube config} file},captionpos=b,label={lst:sample_kpo}]
                % apiVersion: v1
                % kind: Config
                % clusters:
                % - name: "discovery"
                %   cluster:
                %     server: "https://cloud.cs.vt.edu/<Redacted>"
                
                % users:
                % - name: "discovery"
                %   user:
                %     token: "<Redacted>"
                
                
                % contexts:
                % - name: "discovery"
                %   context:
                %     user: "discovery"
                %     cluster: "discovery"
                
                % current-context: "discovery"
                % \end{lstlisting}
                
                % Thus, running this operator will create Kubernetes pods in the `Discovery' cluster.

            \subsection{Interface} \label{ch:registry_ui}
            To allow for CRUD (Create, Read, Update and Delete) operations to be done on the registry by various team members for their respective services, we developed a frontend consisting of services, goals, and workflows.

            This page can be accessed from the navigation menu of the main frontend website given in Figure \ref{fig:workflow_menu}.
            \begin{figure}[h!]
	            \centering
	            \includegraphics[width=\textwidth]{img/Workflow UI Screenshots/workflow_ui_1.jpg}
	            \caption{Navigation to Workflow Automation Page}
	            \label{fig:workflow_menu}
	        \end{figure}

            Goals, Services, and Reasoner details and tables are described with their figures. Figure~\ref{fig:goals} displays the Goals table, Figure~\ref{fig:services} displays a screenshot of the services table, and Figure~\ref{fig:reasoner} displays a screenshot of the Reasoner table. The table metadata has been defined in the following section based on the schema of the database that helps populate these pages.

            \begin{figure}[H]
	            \centering
	            \includegraphics[width=\textwidth]{img/Workflow UI Screenshots/workflow_ui_2.jpg}
	            \caption{Goals Table}
	            \label{fig:goals}
	        \end{figure}

         \begin{figure}[H]
	            \centering
	            \includegraphics[width=\textwidth]{img/Workflow UI Screenshots/workflow_ui_3.png}
	            \caption{Services Table}
	            \label{fig:services}
	        \end{figure}

         \begin{figure}[H]
	            \centering
	            \includegraphics[width=\textwidth]{img/Workflow UI Screenshots/workflow_ui_4.png}
	            \caption{Reasoner Table}
	            \label{fig:reasoner}
	        \end{figure}

         For each of these tables, a team member can add or edit data. For example, Figure~\ref{fig:goal_addition} shows the pop-up menu that allows a user to fill in the required data for adding a new goal. Submitting this data populates the database with the provided Goal details and displays them on the Goals table accordingly.

     \begin{figure}[H]
            \centering
            \includegraphics[width=\textwidth]{img/Workflow UI Screenshots/workflow_ui_5.jpg}
            \caption{Addition of a new Goal}
            \label{fig:goal_addition}
        \end{figure}


        \subsubsection{Populating the registry with goals, services, and reasoner metadata}
            Ideally, the teams were supposed to use the registry UI to populate the data related to workflow automation. However, while the UI was being developed in parallel with the backend and other parts of workflow automation, we populated the database manually. Henceforth, since the UI is ready, any experimenter or team in the future will be able to populate the database with the registry UI, and manual data entry to the database will not be required. 

        \subsection{Facilitating easy adoption of the workflow automation system into frontend application}\label{workflow_automation_js_wrapper}
        With the interface, database, and database APIs in place, our next steps involved connecting these to allow users to be able to generate and run workflows directly from the user interface. This was accomplished by creating a code wrapper written in JavaScript, which consists of functions that make the required API calls. Thus, the developers who wish to integrate this system into their frontend applications need only to import these functions and call them.
        
        The three functions defined in this wrapper include the following which perform the API functionalities mentioned in Section ~\ref{reasoner_api}:
        \begin{enumerate}
            \item generateWorkflow()
            \item runWorkflow()
            \item getWorkflowStatus()
        \end{enumerate}

        The code for this wrapper and its functions can be found at \url{https://code.vt.edu/aaron2000/cs5604-front-end/-/blob/main/src/api/AirflowReasoner.js}
        
        \subsection{Registry} \label{ch:registry}
            The registry stores the workflow automation metadata which is referenced by the reasoner as well as Apache Airflow to construct and run workflow automations. For this purpose, we need to store the following information: 

            \begin{enumerate}
                \item Services
                \item Inputs and outputs (also called the Goal metadata)
                \item How the various services are connected via inputs and outputs (also called the reasoner metadata)
            \end{enumerate}

            We represent this schema in the form of three PostgreSQL database tables, whose Data Definition Language (DDL) is given below: 


            \begin{enumerate}
                \item Goal Metadata
                    \begin{lstlisting}[language=SQL]
CREATE TABLE goal_metadata (
    goal_id integer GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
    goal_name text NOT NULL,
    goal_description text NOT NULL,
    goal_format text NOT NULL,
    file_location text NOT NULL,
    environment_variable text NOT NULL,
    owned_by text NOT NULL
);

-- Indices -------------------------------------------------------

CREATE UNIQUE INDEX goal_metadata_pkey ON goal_metadata(goal_id int4_ops);
                        \end{lstlisting} 
                    \item Service Metadata
                        \begin{lstlisting}[language=SQL]
CREATE TABLE service_metadata (
    service_id integer GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
    service_name text NOT NULL,
    service_description text NOT NULL,
    image_url text NOT NULL,
    cluster_namespace text DEFAULT 'cs5604-int-test'::text,
    owned_by text NOT NULL
);

-- Indices -------------------------------------------------------

CREATE UNIQUE INDEX service_metadata_pkey ON service_metadata(service_id int4_ops);
                        \end{lstlisting} 
                    \item Reasoner
                        \begin{lstlisting}[language=SQL]
CREATE TABLE reasoner (
    goal_id integer REFERENCES goal_metadata(goal_id) ON DELETE CASCADE,
    service_id integer REFERENCES service_metadata(service_id) ON DELETE CASCADE,
    input_goal_id integer REFERENCES goal_metadata(goal_id) ON DELETE CASCADE
);
                        \end{lstlisting} 
                \end{enumerate}

                This Postgres database is hosted on the CS Cloud Infrastructure using Team 2020 INT's Docker image: \lstinline{container.cs.vt.edu/cs-5604-fall-2020/int/team-int-repo/postgres:latest} at the URL: \url{team2020db.discovery.cs.vt.edu}
               
                \subsection{Registry Backend} \label{ch:registry_backend}

                We have created a backend server responsible for exposing an interface to the registry, which stores the Goals, Workflows, and Services, i.e., the Knowledge Graph around workflow automation as described in Figure~\ref{fig:uml_diagram_airflow}. We have achieved this by creating a server that runs Python's Flask framework as a runtime and persists data into the PostgreSQL instance described in Section \ref{ch:registry}. The registry is exposed to the end-user in the form of a REST API containing CRUD (Create, Read, Update, Delete) operations on the Goals, Workflows, and Services. We used Postman~\cite{postman} as an API platform to design our API. This server can be accessed at \url{https://registry-backend.discovery.cs.vt.edu/} and the collection of REST API endpoints that can be used to perform operations on the registry can be found in the Postman collection at \url{https://documenter.getpostman.com/view/4087380/2s8YzL2kT9}. 

                The registry backend runs the following Docker image: \lstinline{ano2202/registry-backend:0.0.6-linux} and is based on the work done by Team 2020 INT. The codebase for the same can be found at \url{https://code.vt.edu/aaron2000/cs5604-f22-team-5-repo-2}.
            % \subsection{Example Workflow Collaborated with Team-4}
                % We have containerized and injected most of the services from all teams. Here we give an example of how we interact and automate team-4's services. This workflow involves adding new ETDs, segmentation, cleaning and parsing, summarization, classification, and uploading objects to the database.

            \subsection{Indexing - Workflow Automation}
                We show the indexing workflow in 
                Figure~\ref{fig:indexing_automation}. The indexing workflow performs indexing of the existing and newly added ETDs, and updates the index accordingly. The first service of this workflow involves fetching the ETDs and the metadata from the database of Team-1. The second service performs a transformation of the fetched data to a format suitable for the API of Team-2. In the final service, a call is made to the API developed by Team-2 for indexing the ETDs along with updating the existing index.

                 The following Docker image was created to facilitate this workflow:
               \begin{itemize}
                   \item ano2202/cs5604-team-2-update-index:0.8
               \end{itemize}

             \begin{figure}[h!]
	            \centering
	            \includegraphics[scale = 0.75]{img/indexing_1.png}
	            \caption{Indexing workflow}
	            \label{fig:indexing_automation}
            \end{figure}

            \subsection{Object Detection - Workflow Automation}
               The object detection workflow shown in Figure \ref{fig:object_detection_automation} detects objects in the input PDFs and stores the results as XML files. The ETDs are fetched from Team-1's database, and the objects in the PDFs are then detected by the service that hosts the Python script developed by Team-3 for object detection. The results of the detection, and other metadata related to each object, are stored in the form of XML files on the Ceph storage system under the path “/mnt/data/team3”.

            The following Docker images were created to facilitate this workflow:
               \begin{itemize}
                   \item ano2202/cs5604-curl-get-etd-pdf:0.2.9-linux
                   \item ano2202/cs5604-team3-processing-suite:0.5
               \end{itemize}

             \begin{figure}[h!]
	            \centering
	            \includegraphics[scale = 0.75]{img/object_detection_2.png}
	            \caption{Object detection workflow}
	            \label{fig:object_detection_automation}
            \end{figure}

            \subsection{Segmentation, Summarization and Classification - Workflow Automation}
               The segmentation, summarization, and classification of ETD chapters are combined into a single workflow, shown in Figure~\ref{fig:seg_sum_class}. The segmentation service segments the input PDFs into individual chapters. Each segmented chapter from the input PDF is then stored as a separate PDF in the file system. The clean and parse service extracts and cleans the textual content from the separated chapter PDFs which are then saved in the form of text files. In the summarization step, the extracted content is shortened and summarized and stored as JSON files. Finally, in the classification stage, a multi-label classification is performed on each chapter to identify the two fields to which they are most related. The outputs generated at the end of the segmentation, clean and parse, summarization, and classification steps are all uploaded to Team-1's database by making a call to their API.
               
               The following Docker images were created to facilitate this workflow:
               \begin{itemize}
                   \item ano2202/cs5604-curl-get-etd-pdf:0.2.13-linux
                   \item ano2202/cs5604-segmentation:0.4
                   \item lyuze/upload\_seg:0.7
                   \item outerspace1920/clean\_pdf:1.5 
                   \item outerspace1920/summarization:1.0
                   \item outerspace1920/classification:1.1
                   \item lyuze/upload\_summ:0.2
                   \item lyuze/upload\_class:0.1
                   \item outerspace1920/upload\_cleaned\_chapters:1.1
                   \item lyuze/update\_index:0.1
               \end{itemize}


             \begin{figure}[h!]
	            \centering
	            \includegraphics[scale = 0.5]{img/workflow_DAG.png}
	            \caption{Segmentation, Summarization and Classification Workflow}
	            \label{fig:seg_sum_class}
            \end{figure}


            \subsection{Alternate Workflow Automation}
               The workflows discussed above, and their development, can be generated in other ways supported by our system.
               As is explained in Section 6.1.4, the operations of other teams w.r.t. workflows involves their use of calls that include runWorkflow() and getWorkflowStatus().
               If, to achieve a goal, by running a workflow, other teams can cause Airflow to run
               a different yet suitable entry in the /dags folder, all should proceed properly.

               One way to produce suitable entries in the /dags folder is for a developer to edit an existing entry in the /dags folder. This allows a developer to take advantage of other features, like parallel operation, that Airflow  supports. The resulting benefit would then be achieved if the edited file can be given to Airflow so it can run the workflow that the edited entry describes.

               A second way to produce suitable entries in the /dags folder is for a developer to manually create and upload a suitable entry in the /dags folder for a workflow. They would not use the Generate Grammar API or the Generate Workflow API, since their hand-coding of the entry in the /dags folder for a workflow would just be a programming exercise aimed to create something appropriate that Airflow  supports. In this situation, the so-called Reasoner would not be employed, since the workflows would be hand-coded. Further, there would be no need for reasoner entries as discussed in Figure 7.10, and no need for the Reasoner table described in Section 6.1.5.

               Each of the above two variations could lead to system operation that bypasses the Reasoner.
               Each would result in having suitable entries in the /dags folder.
               The rest of the system could make use of a new API call, e.g., runDAGentry(), that would be given an identifier for a suitable entry in the /dags folder.
               This API would be similar to the existing runWorkflow() API.
               When a part of the overall system wanted to address some goal, instead of calling generateWorkflow() and runWorkflow(), it would call runDAGentry(), with argument being the /dags folder entry created as above.

           \subsection{Sample workflow Demonstration}

           A video demonstration for this workflow was created to be shown as part of the final presentation. It can be found as the video file named `Team5INTdemo.mp4' and shows this workflow automation being executed in its entirety. A tabular description of the demonstration video follows as Table~\ref{tab:demonstration_video_explanation} that contains timestamps in the video as one column and the actions or changes occurring at the corresponding timestamp as the other column. This demonstration combines the ``Segmentation, Summarization and Classification" and the ``Indexing" workflows as a single, chained workflow automation. 

               \begin{table}[H]
            \begin{tabular}{||p{0.12\linewidth} | p{0.88\linewidth}||}
             \hline
             Timestamp & Description\\ 
             \hline
             \hline
             00:01 & Navigating to the API request that generates a new workflow instance. The REST API client UI used here is `Postman'.\\
             \hline
             00:08 & API request to generate a new workflow instance of the ``Segmentation, Summarization and Classification" workflow automation is sent.\\ 
             \hline
             00:25 & API request to run the newly created workflow instance is sent (This will trigger Airflow and set the automation in motion).\\ 
             \hline
             00:30 & API request to check the status of the running workflow instance is sent.\\ 
             \hline
             00:38 & The first step of the workflow automation (`Get ETD') begins getting executed in the form of a new container that is created on Rancher.\\ 
             \hline
             00:45 & The ETD that was to be processed has been fetched via the `Get ETD' step and has been persisted in the file system.\\ 
             \hline
             00:51 & The `Segmentation' step begins execution in the form of a container being created on Rancher.\\ 
             \hline
             07:08 & The `Segmentation' step has finished running, which has persisted the segmented chapters in the file system. \\ 
             \hline
             07:14 & The segmented chapters are seen in the file system. There are three chapters that were generated by the automation for this ETD PDF. The segmented chapters have also been uploaded to Team-1's database.\\ 
             \hline
             07:18 & The `Clean and Parse' step of the workflow begins. \\ 
             \hline
             07:39 & The `Clean and Parse' step of the workflow finishes executing. \\ 
             \hline
             07:46 & The `Summarization' step of the automation begins. \\ 
             \hline
             08:15 & The `Summarization' step of the automation finishes executing. \\ 
             \hline
             08:18 & The `Classification' step of the automation begins. \\ 
             \hline
             09:13 & The `Classification' step of the automation finishes executing. \\ 
             \hline
             09:14 & The `Upload Summarization' step of the automation begins. \\ 
             \hline
             09:17 & The `Upload Summarization' step of the automation finishes executing. \\ 
             \hline
             09:20 & The `Upload Classification' step of the automation begins. \\ 
             \hline
             09:23 & The `Upload Classification' step of the automation finishes executing. \\ 
             \hline
             09:24 & The `Upload cleaned chapters' step of the automation begins. \\ 
             \hline
             09:28 & The `Upload Cleaned Chapters' step of the automation finishes executing. \\ 
             \hline
             09:30 & The `Fetch Metadata and Update Index' step of the automation begins. \\ 
             \hline
             09:43 & The `Fetch Metadata and Update Index' step of the automation finishes executing. \\ 
             \hline
             09:46 & API request to check the status of the running workflow instance is sent again. \\ 
             \hline
             09:48 & The API request is successful and shows the result of the run. The generated report contains the Docker container logs for all the steps that were run as part of the automation.  \\ 
             \hline
             10:42 & The artifacts created as part of the run are now displayed. These include, the segmented chapters, the parsed and cleaned chapters, classified chapters and the summarized chapters,  \\ 
             \hline
             \hline
            \end{tabular}
            \caption{\label{tab:demonstration_video_explanation}Explanation of the demonstration video: Team5INTdemo.mp4}
            \end{table}

               
                
        \section{Developer Operations}
        \subsection{Continuous Integration and Continuous Deployment}
    	   Pipelines in CI/CDs are nothing but a piece of hardware and software which executes the supplied code. 
        We are using GitLab as our main pipeline runner. GitLab's Runner is open-source software that can build, test, deploy, etc. and it is fully featured. 

    	   To set up CI/CD for each team, it is a prerequisite that we have a Virtual Machine running the GitLab Runner with Docker as its executor. The Developer Manual in Section \ref{gitlab_cicd_setup} has a guide to setting up the GitLab Runner on a Virtual Machine.
    	   
    	   Since the GitLab Runner needs access to each project, individual project tokens need to be registered. 
    	   
    	   Once registered, each team needs to set up a .gitlab-ci.yml file which contains the code to run the pipeline. 
    	   
    	   Following is a very simple .gitlab-ci.yml file that builds and pushes a Docker image to the Docker hub.
        	   \begin{minted}{yaml}
                stages:          
                  - build
                build-job: 
                  stage: build
                  image: docker:latest
                  script:
                    - docker build -t simplenode .
                    - echo “Build complete”
                    - echo “Deploying application...”
                    - docker push simplenode
        	   \end{minted}
    	   This yaml code assumes a simple Node.js application with the following Dockerfile:
        	   \begin{minted}{text}
                FROM node:14-alpine
                WORKDIR /app
                COPY package*.json /app/
                ENV NODE_ENV=development
                RUN npm i
                COPY . /app/
                CMD npm run start
        	   \end{minted}
    	   Since yaml is infinitely scriptable, exceedingly complex pipelines can be created.
            \subsection{CI/CD Pipeline setup}
                To deploy to the cloud, manually setting the tag for each deployment is a tedious process. To automate this and to inculcate automated testing, CI/CD pipelines are being set up for each team, taking into account their needs, and thus tailoring each container image.  
                
                Each Team Repository is set up to use the GitLab Runner which will run scripts as defined in a .gitlab-ci.yml

                The Runner is hosted at containers.cs.vt.edu and uses the Docker executor. The VM behind containers.cs.vt.edu is managed by the CS Department but to host the GitLab Runner, any Virtual Machine can be used. 
                
                The images that are built as a result of a Dockerfile, will be stored in GitLab's Container Registry. 

                Figure \ref{fig:gitlab_runner_frontend} shows the runner listed in the frontend team repository.
                \begin{figure}[h!]
    	            \centering
    	            \includegraphics[scale = 0.282]{img/gitlab_runner_frontend.png}
    	            \caption{Frontend Team's GitLab Runner}
    	            \label{fig:gitlab_runner_frontend}
                \end{figure}
                
                The following details each Team's container requirements:
                \begin{itemize}
                    \item Team 1 - uses Flask as their Python Server Framework wherein all APIs have been defined as well as their processing functions. A standalone PostgreSQL and Virtuoso container is set up which is the main repository of the class.
                    \item Team 2 - They have 2 containers, i.e., Recommendation System and the main search server. 
                    \item Team 3 - Team 3 uses an XML based preprocessing technique to display different objects of an ETD. This server also uses Flask as the framework. Topic Modelling also has its code containerized and deployed to Rancher via GitLab CI/CD pipelines.
                    \item Team 4 - Team 4 doesn't need a Flask server but their UI is integrated in the main frontend repository.
                    \item Lastly, the frontend team has a separate container that uses ReactJS as the Frontend Framework. The Dockerfile has all steps to create a production build of the App and to use Nginx as the Web Server.
                \end{itemize}
                Each of the 4 teams and the frontend team which has used Flask and ReactJS have their containers with associated code bases connected to a CI/CD pipeline. 
            \subsection{Troubleshooting for Other Teams}
                \begin{itemize}
                    \item Helped to set-up GPU backed containers. Initially, Teams 3 \& 4 faced the issue with connecting to GPUs and were unable to run nvidia-smi on their containers.
                    \item Helped people to troubleshoot whenever their Containers were inaccessible from the public internet.
                    \item Demonstrated how to build a Dockerfile and enabled people to create their own custom Docker images as per their requirements.
                    \item Supported Team 1 in setting up their Flask app and Jupyter Dev container.
                \end{itemize}
        
            \subsection{Containerization}
	        We have successfully spun up Kubernetes Deployments that contain Pods, which in turn run Docker containers based on the requirements specified by various teams. The containers are all running within the ‘Discovery’ Kubernetes cluster maintained by the CS Cloud Infrastructure. These pods are all healthy and are being used by the teams for their purposes. The steps followed for spinning up all of the containers mentioned here,
         involve 
         %have been spun up 
         using the user interface served by Rancher, as mentioned in Section \ref{ch:developer_manual_rancher}.
         
	        The following is a comprehensive list of all the running containers, along with their descriptions. See \ref{fig:containerization_overview} for a Visual Representation.
	        \begin{figure}[h!]
	            \centering
	            \includegraphics[width=\textwidth]{img/overall_containers.png}
	            \caption{Containerization overview}
	            \label{fig:containerization_overview}
	        \end{figure}
	        \begin{enumerate}
	            \item Team 1 - Team 1's focus is on data curation.
             Hence their first and foremost container would be a database container, i.e., PostgreSQL. Additionally, for computing, setting up APIs, etc. we have also installed a base Python image with other packages installed as required. Additionally, a Virtuoso container is created to provide the functionalities of a high-performance virtual database engine that can support a knowledge graph made with RDF triples.
	                \begin{enumerate}
	                    \item Container 1 - PostgreSQL
                        \item Container 2 - Base Image - Python, Jupyter, Pandas, Numpy, Matplotlib, Sqlalchemy, Flask
                        \item Container 3 - Virtuoso
	                \end{enumerate}
                    
                \item Team 2 -
                    Team 2 has 3 requirements. To run the recommendation system, a Jupyter notebook with GPU access is required. The backend\footnote{In order to transform queries that can be read by Elasticsearch, Team 2 uses a backend that handle queries from the frontend and query results by Elasticsearch. Moreover, it handles Authentication/Authorization using Google Oauth.} is written in Python using Flask as the framework. Finally, Elasticsearch is used as the search backend. The Python App is a custom Docker image that is hosted inside the Team 2 repository in \url{https://code.vt.edu/}. A pipeline will be used to build the Docker images and they will be pushed to the Container Registry of the same code repository. Elasticsearch runs inside a Pod on Rancher.
                    \begin{enumerate}
                        \item Container 1 - Jupyter notebook with GPU
                        \item Container 2 - Flask App
                        \item Container 3 - Elasticsearch
                    \end{enumerate}
                
                    
                \item Team 3 - We spun up GPU-backed containers for Team-3, and asked them about the list of dependencies that they require for running their workflows. However, they mentioned that their Subject Matter Expert had asked them to use their cluster for training the model and generating weights. After some clarifying questions were asked, they further clarified that they will still need containers to meet any further training demands that are generated by the workflow automation that our team will work on. For this purpose, we provisioned GPU-backed Docker containers running a custom Docker image (discussed in Section \ref{ch:gpu_backed_containers}) which included Jupyter. The descriptions of the containers and their URLs are as follows:
                    \begin{enumerate}
                        \item Container 1 - GPU-backed container running Jupyter Notebooks+Pytorch (URL: \url{https://team3-container-1.discovery.cs.vt.edu})
                        \item Container 2 - GPU-backed container running Jupyter Notebooks+Pytorch (URL: \url{https://team3-container-2.discovery.cs.vt.edu})
                    \end{enumerate}
                
                \item Team 4 - Team 4 requires three Docker containers running on Nodes that have native GPU support for supporting Machine Learning tasks that require hardware acceleration. These containers were provisioned on the     CS Cloud Infrastructure. They are running Jupyter notebooks based on       custom images (discussed in Section \ref{ch:gpu_backed_containers}) and are exposed through the URLs mentioned alongside the container names and their intended use cases below:
                    \begin{enumerate}
                        \item Container 1 - For segmenting ETDs into chapters (URL: \url{https://team4-container-1.discovery.cs.vt.edu})
                        \item Container 2 - For generating summaries of ETDs (URL: \url{https://team4-container-2.discovery.cs.vt.edu})
                        \item Container 3 - For classification (URL: \url{https://team4-container-3.discovery.cs.vt.edu})
                    \end{enumerate}

                \item Team 5 - Team 5 has 4 containers that it uses to orchestrate workflow automation. These are listed below. More information about the same is also available in Section \ref{ch:implementation_workflow_automation}.
                    \begin{enumerate}
                        \item PostgreSQL server (Registry), which houses the data about all the services, goals, and reasoner metadata (URL: \url{https://team2020db.discovery.cs.vt.edu})
                        \item Registry API server, which is the interface between the user and the registry database (URL: \url{https://team2020-api.discovery.cs.vt.edu})
                        \item Reasoner backend, which handles the generation and maintenance of the Context Free Grammar used to represent workflows (URL: \url{http://reasoner.discovery.cs.vt.edu/})
                        \item Apache Airflow, which is used for the execution of workflows that are represented as DAG files (URL: \url{http://team2020-airflow.discovery.cs.vt.edu/})
                    \end{enumerate}
	        \end{enumerate}
	     
                \subsection{Spinning up GPU Backed Containers} \label{ch:gpu_backed_containers}
	        We faced many challenges while spinning up GPU-backed containers for Teams 3 and 4. The following steps were taken. 
	        
	            \subsubsection{Cloud Hardware}
	            CS Cloud Infrastructure has provisioned physical nodes running NVidia GPUs for our use, on which we can run Docker containers. These nodes are connected to the Kubernetes cluster and are available to us through the Kubernetes control plane.
	            
	            \subsubsection{Specifying usage of GPU Nodes}
	            We use Kubernetes' NodeAffinity constraint to tie the deployment to a particular node. These nodes are selected by the labels that are assigned to each node. By specifying a nodeSelector label while defining a deployment, we can tie a particular deployment so that it only gets deployed on that node. The GPU-backed nodes in the CS Cloud infrastructure are assigned the label ‘gpu’. The following YAML snippet allows us to do this: 
	            
                    \begin{lstlisting}[language=YAML]
spec:
  affinity:
    nodeAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: type
                operator: In
                values:
                - gpu
                    \end{lstlisting}
	            
	            \subsubsection{Building a custom Docker Image}
	            Team-4 communicated their requirements to us and mentioned that they required the NVidia base Docker image
             (\url{https://catalog.ngc.nvidia.com/orgs/nvidia/containers/pytorch})
             along with the following dependencies: 
	            
	           \begin{enumerate}
	               \item nltk
	               \item tensorflow-gpu
	               \item pandas
	               \item scikit-learn
	               \item pickle
	               \item lxml
	               \item argparse
	               \item BeautifulSoup
	               \item nlp
	               \item transformers
	               \item spacy
	               \item seaborn
	               \item Matplotlib
	           \end{enumerate}      
	            
	            To help the team get started quickly, we decided to package all of the dependencies along with the base Docker image in a custom Docker image (\url{https://hub.docker.com/r/ano2202/cs5604\_pytorch}) that we published to Docker Hub. This image requires that the following hardware dependencies be specified when trying to run this image on our hardware~\cite{ngc_requirements}:
	            
	            \begin{enumerate}
	                \item Docker Engine (\url{https://docs.docker.com/get-docker/})
                    \item NVIDIA GPU Drivers (\url{https://docs.nvidia.com/datacenter/tesla/tesla-installation-notes/index.html})
                    \item NVIDIA Container Toolkit (\url{https://github.com/NVIDIA/nvidia-docker})
	            \end{enumerate} 
	            
	            \subsubsection{Deployment}
	            Following the steps that are mentioned in Section \ref{ch:developer_manual_rancher}, in the image section of deploying the workload, we specify the following image name when entering the details: `ano2202/cs5604\_pytorch'. Since this image is publicly deployed on Docker Hub, it can be accessed by all containers in any cluster.
	            
	            \subsubsection{Testing}
	            Once the containers are up we need to test whether the image and the container that it is running are indeed running on a GPU-backed node as well as able to leverage the capabilities of hardware acceleration. NVidia offers a simple command to test this called `nvidia-smi'. To test this, navigate to the container in question and click on `Execute Shell' as shown in  Figure \ref{fig:execute_shell}. 
	            
    	        \begin{figure}[h!]
    	            \centering
    	            \includegraphics[scale = 0.37]{img/rancher_container_execute_shell.png}
    	            \caption{The option to execute a shell inside the container, present in the Rancher UI}
    	            \label{fig:execute_shell}
    	        \end{figure}
    	        
    	        If successful, it should display the  result shown in Figure \ref{fig:nvidia_smi}.
    	        \begin{figure}[h!]
    	            \centering
    	            \includegraphics[scale = 0.73]{img/nvidia_smi.png}
    	            \caption{Successful execution of the `nvidia-smi' command}
    	            \label{fig:nvidia_smi}
    	        \end{figure}
	       
            
	      
        \section{Project Management}
        \label{section:projectManagement}
        We decided to follow Beyer's User Centered Agile Methods~\cite{user_centered_agile} for project management and decided to employ Scrum~\cite{scrum_definition} to manage our project. The monograph prescribes an extensive study of end-user requirements in the way of user interviews at their workplace, describing user personas, story-boarding, etc., and describes ways in which a typical software project can be managed such as sprint planning, the product owner role, user stories, etc. While the approach defined in the monograph is extensive, we decided to pick up a few concepts from it, for project management and user experience design. The following points highlight our project management approach: 
        
        \begin{enumerate}
            \item Sprints: We decided to have one-week sprints. This was done because we wanted quick feedback while keeping in mind the overall duration of the course. A sprint board was created to track the progress of our tickets. Figure \ref{fig:sprint_board} displays the sprint board as of 09/15/2022. 
            
            \begin{figure}[h!]
	            \centering
	            \includegraphics[scale = 0.272]{img/sprint_board.png}
	            \caption{Sprint Board on GitLab as of 09/15/2022}
	            \label{fig:sprint_board}
            \end{figure}
            
            \item User stories: We created rudimentary user stories out of the requirements as mentioned in Section \ref{ch:require }. We suggest as part of Future Work to create more detailed user stories, based on the user personas that we have identified and listed in Section \ref{ch:user_personas} and any other incoming requirements from the other teams.
            
            \item Team meetings: Our team convenes in three fifty minutes long meetings on Tuesday and Thursday mornings as well as Sunday afternoons. While we discuss the day-to-day items on the weekdays, we reserve the Sunday meeting for sprint planning and also overall goal planning.
            
            \item Product owner: Rather than appointing a person from our team as a representative for the user, we treat the liaisons from other teams as product owners. Their input is our source of user personas and stories.
        \end{enumerate}
        
        \subsection{Interviews}
	        \subsubsection{Interview Methodology - Liaisoning}
	        We have appointed members to be liaisons of each team. 
	        The effort of liaisons is two-fold. For one, these liaisons will talk to their respective teams and understand their ideas and requirements to coordinate the creation of Docker containers, web servers, etc. Secondly, these liaisons will also help formulate the overall architecture, so that components with inter-dependency can be easily identified and built upon. 
	      
        \section{Timeline}
        
	        % Content Here %

	       % \begin{center}
	       %{||c|c|c|c||}
	       \begin{table}[H]
            \begin{tabular}{||p{0.02\linewidth} | p{0.7\linewidth} | p{0.06\linewidth} | p{0.09\linewidth}||}
             \hline
             \# & Description & Week & Status \\ 
             \hline
             \hline
             1 & Conducting interviews with other teams & 1 & Done \\
             \hline
             2 & Conducting onboarding and reading sessions to address knowledge gaps & 1 & Done \\ 
             \hline
             3 & Finalizing team requirements & 1 & Done \\ 
             \hline
             4 & Literature review of tools, technologies, and best practices to identify possible solution approaches & 2 & Done \\ 
             \hline
             5 & Design and plan system and architecture diagram from requirements & 3 & Done \\ 
             \hline
             6 & Setting up containers with the specific requirements & 3 & Done \\ 
             \hline
             7 & Setting up training containers & 4-5 & Done \\ 
             \hline
             8 & Setting up service containers & 4-5 & Done \\ 
             \hline
             9 & Setting up a Continuous Integration pipeline to ensure code integration and testing practices are managed as a prototype within Team 5 & 6-7 & Done \\ 
             \hline
             10 & Setting up a Continuous Deployment pipeline prototype within Team 5 & 6-7 & Done \\ 
             \hline
             11 & Conducting literature review and understanding the workflow automation completed thus far & 6 & Done \\
             \hline
             12 & Setting up a Continuous Integration and Continuous Deployment pipeline for each team & 8-9 & Done \\ 
             
            %  \hline
            %  16 & Show the latest architecture and updates from our team & 13-15 & To-Do \\
             
             \hline
             13 & Developing a prototype of the workflow automation platform using Airflow and a singular goal of object detection & 8-9 & Done \\
             \hline
             14 & Setting up a functioning repository for the knowledge graph, goals, workflows, and services & 10-11 & Done \\
              \hline
             15 & Planning and designing our content for the registry User Interface & 12 & Done \\ 
             \hline
             16 & Implementing the registry User Interface & 12 & Done \\ 
             \hline
             17 & Liaison with teams to get their services containerized - We will be starting with Team-4 & 12 & Done \\
             \hline
             18 & Setting up the reasoner/optimizer & 12 & Done \\
             \hline
             19 & Deploying the entire set-up on Rancher and testing it & 15 & Done \\ 
             \hline
             20 & End-to-end tweaking/testing along with final demonstration & 17 & Done \\ 
             \hline
             21 & Making PostgreSQL highly available and redundant with the use of persistent storage such as Ceph or a VM & NA & Future work \\ 
             \hline
             22 & Enabling observability and monitoring of entities within the cluster using Grafana & NA  & Future work \\ 
             \hline
             \hline
            \end{tabular}
            \caption{\label{tab:table-name}Project Timeline}
            \end{table}
        
	    \section{Milestones and Deliverables}
	        % Content Here %

	        \begin{enumerate}
	            \item Manual deployment of all teams' containers 
	            \item Provide version control for all teams 
	            \item Containers should be up and running with reliability, speed, and resilience 
                \item Team should be able to version control using Git; integrate, test, and deploy their business logic to the cloud, seamlessly
                \item Create container and version control repository for the front-end team
                \item Team artifacts (containers) should be able to communicate with each other and teams should be able to integrate their responsibilities
                \item Develop Continuous Integration and Continuous Deployment pipeline that helps to test, build and deploy the codes of the various teams
                \item Integrate the workflow management system based on user-defined goals 
                \item Identify and understand the pre-existing work done concerning workflow management
                \item Design an efficient architecture for the workflow management pipeline
                \item Set up the database with the required services, goals, knowledge graph data
                \item End-to-end prototype of the workflow management system
                \item User Interface that helps to create and interact with the workflow management system
                \item Be available to provide support in case of systemic anomalies/failures and technical difficulties faced by other teams
	        \end{enumerate}