Goal-Conditioned Evaluation of Sustainable Development Goal Contributions in Theses and Dissertations

Abstract

Evaluating whether a research document contributes to a United Nations Sustainable Development Goal (SDG) requires reasoning about how the document's claims, methods, and findings relate to the targets that define the goal, a teleological judgment that depends on the evaluator's interpretive framework rather than on the document's content alone. Different evaluators applying the same criteria to the same document can produce different yet equally defensible assessments, because the SDG framework does not prescribe a single pathway through which research contributes to a goal's targets. This interpretive structure makes the task incompatible with conventional annotation-based approaches and motivates a different computational formulation. This dissertation proposes Goal-Conditioned Multi-Label Distillation, a methodology for computationally operationalizing this form of evaluation. A large language model is prompted once per SDG under goal-conditioned inputs, and the logit margin between the two constrained output tokens is used to derive a Bernoulli parameter reflecting the model's assessed probability that the document contributes to that goal. Applied iteratively to each of the seventeen SDGs, this procedure produces a continuous 17-dimensional assessment vector for each document. A DeBERTa-based student model is then trained to approximate this mapping directly from document text, without requiring the goal specification at inference time. Empirical analysis of the teacher signal identifies a boundary region where causal proximity between document content and SDG targets becomes sufficiently attenuated that the evaluation loses stability. This region, located at approximately |zt|<1.10, is confirmed by prompt-format perturbation analysis, repeated inference analysis, and cross-model disagreement analysis. The confidence-aware training objective developed in this dissertation accounts for this boundary by reducing the loss contribution of low-margin teacher outputs, concentrating learning on the portions of the corpus where the teacher's signal is most stable. The methodology is applied to electronic theses and dissertations (ETDs), a large and institutionally grounded corpus for which no SDG classification method previously existed. The trained student model transfers to ETD abstracts without degradation in discrimination quality, and continued adaptation on ETD teacher signals further refines its target-domain calibration. Per-document assessment vectors are aggregated into department-level SDG profiles that align with disciplinary expectations for engineering, natural sciences, social sciences, and other academic units, demonstrating that the methodology supports interpretable institutional research intelligence from ETD abstracts alone.