Statistical Inference for Image Classification, Language Reasoning, and Counterfactual Estimation
Files
TR Number
Date
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
This dissertation addresses three interconnected challenges in statistical inference and machine learning: uncertainty quantification under distribution shift, robust causal estimation in panel data, and reliable confidence estimation for large language models.
The first contribution introduces DPI-RG, a distribution-free framework for constructing predictive sets for high-dimensional data under distribution shifts. Using a conditional Wasserstein round-trip generative model, DPI-RG maps data to a lower-dimensional latent space where test statistics asymptotically follow a Chi-square distribution, yielding theoretically guaranteed p-values. The framework exhibits strong robustness to outliers and target shifts without requiring outlier labels during training.
The second contribution presents CONCORD, a confidence estimation framework for large language models that represents responses as semantic distributions over constituent clauses rather than single embedding vectors. By combining Sentence Mover's Distance with Symbolic Answer Match, CONCORD captures both structural divergence and discrete answer agreement, producing calibrated confidence scores that substantially outperform logit-based and embedding-based baselines on mathematical reasoning benchmarks.
The third contribution proposes an