Goel, HarshilChoudhary, VarunDhondi, AnishDesai, Parth2023-07-052023-07-052023-04-13http://hdl.handle.net/10919/115645Electronic Theses and Dissertations (ETDs) are digital versions of academic papers of graduate students. ETDs are highly complicated and lengthy texts: these include multimedia elements and other forms of information. A digital library with summaries for each ETD would enable more people to explore all these texts to learn about various domains. However, most chapters of ETDs don’t have summaries so the solution was to create AI generated summaries for users across disciplines to read. Before these summaries are accessible to the public, Bipasha will find researchers in various disciplines to evaluate the summaries. Incorporating human feedback into AI-generated summaries results in improved accuracy, relevance, originality, engagement and satisfaction. Quantitative measures for evaluating AI-generated content are great but qualitative feedback is important too. Subject matter experts can detect errors and inconsistencies in these summaries: this feedback provides guidance. Our team has developed a website that enables users to view and rank AI-generated summaries of texts against the ground truth (provided) chapter summaries. The users will not know beforehand which is which. The scholars (those with an education level of graduate school and beyond) should be able to accurately evaluate the summaries for the ETDs in their field. The purpose of this project is to allow human evaluation of these texts. The ranking feature serves as a form of feedback to perfect the AI generated summaries. The domain experts will use this website to determine the model that serves as a gold standard for summarization. For now, the intended users for this application are subject matter experts at Virginia Tech, so that they can evaluate the summaries. The evaluation will provide an idea of which model performs best to serve as a gold standard for AI generated summaries. Eventually, the final model will be used to serve users outside of Virginia Tech who want to know more about the domain that the ETD is associated with. Providing these summaries allows those outside the domain to grasp the basic concept of the ETD and its associated chapters without having to read the entire paper.enSummarizationEvaluationSummaryAI GeneratedReport