Scholarly Works, Virginia Tech National Security Institute

Permanent URI for this collection

https://hdl.handle.net/10919/110084

Browse

Now showing 1 - 20 of 49

Bot Automation Using Large Language Models (LLMs) and Plugins
Ramakrishnan, Naren; Butler, Patrick; Mayer, Brian B.; Neeser, Andrew (2024-07)
The aim of this research study was to create tools that automate information extraction pipelines to support business processes in contract and procurement management. The research team was specifically asked to explore opportunities to use Large Language Models (LLMs) to accomplish this task. After reviewing the problem space and the potential solutions, the team designed and created a tool to generate reports on the status of entries from the Contractor Performance Assessment Reporting System (CPARS), broken down by contracting division. This tool automates the extraction of the Contracting Officer’s Representative (COR) status information. The team also explored methods for using LLM pipelines to automate other potential contractual management tasks and presented some demonstrations of possible uses. The research indicated that LLMs have significant potential to enhance contract and procurement management processes, e.g., automating field extraction from existing contracts, assisting contract generation and customization, rapid contract analysis, and streamlining routine document processing tasks. Based on demonstrations the sponsor agreed on their potential. Yet, while the potential benefits are substantial there are concerns with data privacy and security, accuracy and reliability, legal and compliance issues, and integration with existing systems. To mitigate these concerns and maximize benefits, the research team suggests focusing on local, open-source LLM solutions like LLaMA or Phi. These models can be deployed on-premises, ensuring data privacy and security while providing powerful LLM capabilities including customization and specialization.
AI-Based DPCAP FAR/DFARS Change Support Tool
Ramirez-Marquez, Jose; Gorman, Joshua; Akram, Amer; Buettner, Douglas J.; Mayer, Brian B.; Butler, Patrick; Ramakrishnan, Naren; Freedman, Bradley (2025-04-02)
The Department of Defense’s Defense Pricing, Contracting, and Acquisition Policy Contract Policy Directorate in the Office of the Assistant Secretary of Defense is responsible for periodic updates to the Federal Acquisition Regulation (FAR) and Defense FAR Supplement (DFARS) based on changes in the National Defense Authorization Act (NDAA), Small Business Administration rule changes, U.S. Department of Labor rule changes, or from executive orders. Reading through and assessing these documents for changes that require corresponding changes to acquisition regulations is labor-intensive. Further, when rule changes are proposed to the public for comments, reading and summarizing these public comments can range from straightforward to very labor-intensive. In this paper, we report our initial research results to greatly improve the efficiency of analyzing the NDAA language for required updates of the FAR and DFARS, and issuance of memoranda and guidance using artificial intelligence, including large language models and advanced natural language processing techniques to provide an improvement in staff efficiency for these laborious tasks.
Test and Evaluation of Large Language Models to Support Informed Government Acquisition
Chandrasekaran, Jaganmohan; Mayer, Brian B.; Frase, Heather; Lanus, Erin; Butler, Patrick; Adams, Stephen C.; Gregersen, Jared; Ramakrishnan, Naren; Freeman, Laura J. (2025-04-02)
As large language models (LLMs) continue to advance and find applications in critical decision-making systems, robust and thorough test and evaluation (T&E) of these models will be necessary to ensure we reap their promised benefits without the risks that often come with LLMs. Most existing applications of LLMs are in specific areas like healthcare, marketing, and customer support and thus these domains have influenced their T&E processes. When investigating LLMs for government acquisition, we encounter unique challenges and opportunities. Key challenges include managing the complexity and novelty of Artificial Intelligence (AI) systems and implementing robust risk management practices that can pass muster with the stringency of government regulatory requirements. Data management and transparency are critical concerns, as is the need for ensuring accuracy (performance). Unlike traditional software systems developed for specific functionalities, LLMs are capable of performing a wide variety of functionalities (e.g., translation, generation). Furthermore, the primary mode of interaction with an LLM is through natural language. These unique characteristics necessitate a comprehensive evaluation across diverse functionalities and accounting for the variability in the natural language inputs/outputs. Thus, the T&E for LLMs must support evaluating the model’s linguistic capabilities (understanding, reasoning, etc.), generation capabilities (e.g., correctness, coherence, and contextually relevant responses), and other quality attributes (fairness, security, lack of toxicity, robustness). T&E must be thorough, robust, and systematic to fully realize the capabilities and limitations (e.g., hallucinations and toxicity) of LLMs and to ensure confidence in their performance. This work aims to provide an overview of the current state of T&E methods for ascertaining the quality of LLMs and structured recommendations for testing LLMs, thus resulting in a process for assuring warfighting capability.
Privacy Risks of Cybersquatting Attacks
Kolenbrander, Jack; Rheault, Elliott; Michaels, Alan J. (MDPI, 2026-02-19)
Cybersquatting is a collection of methods commonly used by malicious actors to mislead or trick internet users into accessing fraudulent or malicious content. Much of the current research has concentrated on the specific techniques used by attackers in this domain, such as typosquatting, combosquatting, and sound squatting. Some research has explored the financial and time impacts of cybersquatting; however, an understanding of user privacy impacts is limited. Prior research into privacy implications has primarily relied on passive techniques such as analyzing DNS records, HTML content, and domain registrations. These passive approaches limit the ability to interact with these domains and track the downstream impact of sharing personally identifiable information (PII). This research develops an active open-source intelligence (OSINT) collection system capable of rapidly collecting and analyzing squatting domains through both passive and active techniques, with a particular emphasis on identifying those that solicit user information. Synthetic identities are then registered with these domains, and their associated communications are collected and analyzed to identify privacy-related risks and determine whether shared PII propagates.
Demo: CLOUD-D RF - Cloud-based Distributed Spectrum Sensing with Heterogeneous Hardware Testbed
Ghaleb, Rami; Cox, Jason; Risi, Joseph; Jones, Alyse M. (ACM, 2025-12-03)
Collaborative spectrum sensing enables distributed RF devices to share observations and improve classification performance in congested or contested environments. While most prior research assumes homogeneous sensors, real-world systems involve heterogeneous devices with diverse hardware, bandwidth, and channel conditions. This work introduces CLOUD-D RF, a framework that leverages convolutional neural networks at edge radios to extract learned features that are sent to a cloud-based fusion center for improved modulation classification. Phase I validated the concept with synthetic datasets, achieving over 97% classification accuracy. Phase II implemented a hardware testbed using GNU Radio with USRP radios, demonstrating real-time modulation classification over a distributed edge network of radios. Results confirm that cloud-based fusion reduces bandwidth requirements while maintaining accuracy, offering a practical path toward resilient and scalable spectrum sensing systems.
Introducing WebPdL2Ork
Bukvic, Ivica Ico; Furgerson, William; Kerobo, Justin; Davis, Bradley A. (ACM, 2025-06-30)
In the following paper, we present WebPdL2Ork, a WebAssemblybased runtime environment for the Pd-L2Ork ecosystem that enables easy deployment of patches or code snippets inside a Web browser. Below, we outline and compare its approach to other similar implementations and highlight its unique affordances, some that are inherited from Pd-L2Ork, and others developed to address idiosyncrasies associated with the Web browser’s sandboxed environment. Lastly, we evaluate its implementation using learning modules developed as part of the VT Waves project designed to promote connections between wave physics concepts and music, and the L2Ork Tweeter Pd-L2Ork app, the most complex patch that is included with Pd-L2Ork and is designed to enable EDM-style musicking over the internet while maintaining perfect sync.
Flexible cost-penalized Bayesian model selection: Developing inclusion paths with an application to diagnosis of heart disease
Porter, Erica M.; Franck, Christopher T.; Adams, Stephen C. (Wiley, 2024-07-20)
We propose a Bayesian model selection approach that allows medical practitioners to select among predictor variables while taking their respective costs into account. Medical procedures almost always incur costs in time and/or money. These costs might exceed their usefulness for modeling the outcome of interest. We develop Bayesian model selection that uses flexible model priors to penalize costly predictors a priori and select a subset of predictors useful relative to their costs. Our approach (i) gives the practitioner control over the magnitude of cost penalization, (ii) enables the prior to scale well with sample size, and (iii) enables the creation of our proposed inclusion path visualization, which can be used to make decisions about individual candidate predictors using both probabilistic and visual tools. We demonstrate the effectiveness of our inclusion path approach and the importance of being able to adjust the magnitude of the prior's cost penalization through a dataset pertaining to heart disease diagnosis in patients at the Cleveland Clinic Foundation, where several candidate predictors with various costs were recorded for patients, and through simulated data.
Controlled ion transport in the subsurface: A coupled advection-diffusion-electromigration system
Tang, Kunning; Bo, Zhenkai; Li, Zhe; Da Wang, Ying; McClure, James; Su, Hongli; Mostaghimi, Peyman; Armstrong, Ryan T. (AIP Publishing, 2024-06-01)
Ion transport within saturated porous media is an intricate process in which efficient ion delivery is desired in many engineering problems. However, controlling the behavior of ion transport proves challenging, as ion transport is influenced by a variety of driving mechanisms, which requires a systematic understanding. Herein, we study a coupled advection-diffusion-electromigration system for controlled ion transport within porous media using the scaling analysis. Using the Lattice-Boltzmann-Poisson method, we establish a transport regime classification based on an Advection Diffusion Index (ADI) and a novel Electrodiffusivity Index (EDI) for a two-dimensional (2D) microchannel model under various electric potentials, pressure gradients, and concentration conditions. The resulting transport regimes can be well controlled by changing the applied electric potential, the pressure field, and the injected ions concentration. Furthermore, we conduct numerical simulations in a synthetic 2D porous media and an x-ray microcomputed tomography sandstone image to validate the prevailing transport regime. The simulation results highlight that the defined transport regime observed in our simple micromodel domain is also observed in the synthetic two- and three-dimensional domains, but the boundary between each transport regime differs depending on the variation of the pore size within a given domain. Consequently, the proposed ADI and EDI emerge as dimensionless indicators for controlled ion transport. Overall, our proof-of-concept for ion transport control in porous media is demonstrated under advection-diffusion-electromigration transport, demonstrating the richness of transport regimes that can develop and provide future research directions for subsurface engineering applications.
Quantifying water effluent violations and enforcement impacts using causal AI
Wang, Yingjie; Sobien, Dan; Kulkarni, Ajay; Batarseh, Feras A. (Wiley, 2024-06-01)
In the landscape of environmental governance, controlling water pollution through the regulation of point sources is vital as it preserves ecosystems, protects human health, ensures legal compliance, and fulfills global environmental responsibilities. Under the Clean Water Act, the integrated compliance information system monitors the compliance and enforcement status of facilities regulated by the National Pollutant Discharge Elimination System (NPDES) permit program. This study assesses temporal and geographic trends for effluent violations within the United States and introduces a novel metric for quantifying violation trends at the facility level. Furthermore, we utilize a linear parametric approach for Conditional Average Treatment Effect (CATE) causal analysis to quantify the heterogeneous effects of EPA and state enforcement actions on effluent violation trends at facilities with NPDES permits. Our research reveals insights into national pollutant discharge trends, regional clustering of all pollutant violation types in Ohio (G(i)* Z-score of 2.15), and priority pollutants in West Virginia (G(i)* Z-score of 3.07). The trend metric identifies regulated facilities that struggle with severe and recurring violations. The causal model highlights variations in state compliance and enforcement effectiveness, underscoring the successful moderation of violation trends by states such as Montana and Maryland, among others.
The Impact of Wettability on the Co-moving Velocity of Two-Fluid Flow in Porous Media
Alzubaidi, Fatimah; McClure, James E.; Pedersen, Hakon; Hansen, Alex; Berg, Carl Fredrik; Mostaghimi, Peyman; Armstrong, Ryan T. (Springer, 2024-09-01)
The impact of wettability on the co-moving velocity of two-fluid flow in porous media is analyzed herein. The co-moving velocity, developed by Roy et al. (Front Phys 8:4, 2022), is a novel representation of the flow behavior of two fluids through porous media. Our study aims to better understand the behavior of the co-moving velocity by analyzing simulation data under various wetting conditions. We analyzed 46 relative permeability curves based on the Lattice-Boltzmann color fluid model and two experimentally determined relative permeability curves. The analysis of the relative permeability data followed the methodology proposed by Roy et al. (Front Phys 8:4, 2022) to reconstruct a constitutive equation for the co-moving velocity. Surprisingly, the coefficients of the constitutive equation were found to be nearly the same for all wetting conditions. On the basis of these results, a simple approach was proposed to reconstruct the relative permeability of the oil phase using only the co-moving velocity relationship and the relative permeability of the water phase. This proposed method provides new information on the interdependence of the relative permeability curves, which has implications for the history matching of production data and the solution of the associated inverse problem. The research findings contribute to a better understanding of the impact of wettability on fluid flow in porous media and provide a practical approach for estimating relative permeability based on the co-moving velocity relationship, which has never been shown before.
A systematic literature review on the mathematical underpinning of model-based systems engineering
Wach, Paul; Topcu, Taylan G.; Jung, Sukhwan; Sandman, Brandt; Kulkarni, Aditya U.; Salado, Alejandro (Wiley, 2025-01-01)
The International Council on Systems Engineering (INCOSE) has initiated a Future of Systems Engineering (FuSE) program that includes a stream for advancing the theoretical foundations of the discipline of Systems Engineering (SE). A near-term goal of FuSE is to assess the adequacy of current theoretical foundations of SE. The discipline of SE is converging toward model-based practices (i.e., MBSE) that have not yet reached the maturity of model-based practices in other engineering domains. For example, finite element analysis and computational fluid dynamics are grounded in mathematical theory, while, generally, MBSE is not. However, some attempts have been made to underpin MBSE with theoretical richness. This article presents a systematic literature study that surveyed state of the art on providing MBSE with mathematical foundations. Our protocol collected over 2000 publications that were reviewed for inclusion/exclusion, categorized, and analyzed. We provide insights to the type of mathematical theories used, domains of applications, and areas of SE to which the math was applied to, among other analysis. We also provide a synthesized discussion about the field moving forward, emphasizing positive trends along with the negatives and areas of concern. Overall, we found the field to be nascent.
Probabilistic Models for Military Kill Chains
Adams, Stephen; Kyer, Alex; Lee, Brian; Sobien, Dan; Freeman, Laura J.; Werner, Jeremy (MDPI, 2025-10-20)
Military kill chains are the sequence of events, tasks, or functions that must occur to successfully accomplish a mission. As the Department of Defense moves towards Combined Joint All-Domain Command and Control, which will require the coordination of multiple networked assets with the ability to share data and information, kill chains must evolve into kill webs with multiple paths to achieve a successful mission outcome. Mathematical frameworks for kill webs provide the basis for addressing the complexity of this system-of-systems analysis. A mathematical framework for kill chains and kill webs would provide a military decision maker a structure for assessing several key aspects to mission planning including the probability of success, alternative chains, and parts of the chain that are likely to fail. However, to the best of our knowledge, a generalized and flexible mathematical formulation for kill chains in military operations does not exist. This study proposes four probabilistic models for kill chains that can later be adapted to kill webs. For each of the proposed models, events in the kill chain are modeled as Bernoulli random variables. This extensible modeling scaffold allows flexibility in constructing the probability of success for each event and is compatible with Monte Carlo simulations and hierarchical Bayesian formulations. The probabilistic models can be used to calculate the probability of a successful kill chain and to perform uncertainty quantification. The models are demonstrated on the Find–Fix–Track–Target–Engage–Assess kill chain. In addition to the mathematical framework, the MIMIK (Mission Illustration and Modeling Interface for Kill webs) software package has been developed and publicly released to support the design and analysis of kill webs.
Hardware Validation for Semi-Coherent Transmission Security
Fletcher, Michael; McGinthy, Jason; Michaels, Alan J. (MDPI, 2025-09-05)
The rapid growth of Internet-connected devices integrating into our everyday lives has no end in sight. As more devices and sensor networks are manufactured, security tends to be a low priority. However, the security of these devices is critical, and many current research topics are looking at the composition of simpler techniques to increase overall security in these low-power commercial devices. Transmission security (TRANSEC) methods are one option for physical-layer security and are a critical area of research with the increasing reliance on the Internet of Things (IoT); most such devices use standard low-power Time-division multiple access (TDMA) or frequency-division multiple access (FDMA) protocols susceptible to reverse engineering. This paper provides a hardware validation of previously proposed techniques for the intentional injection of noise into the phase mapping process of a spread spectrum signal used within a receiver-assigned code division multiple access (RA-CDMA) framework, which decreases an eavesdropper’s ability to directly observe the true phase and reverse engineer the associated PRNG output or key and thus the spreading sequence, even at high SNRs. This technique trades a conscious reduction in signal correlation processing for enhanced obfuscation, with a slight hardware resource utilization increase of less than 2% of Adaptive Logic Modules (ALMs), solidifying this work as a low-power technique. This paper presents the candidate method, quantifies the expected performance impact, and incorporates a hardware-based validation on field-programmable gate array (FPGA) platforms using arbitrary-phase phase-shift keying (PSK)-based spread spectrum signals.
MENTORPDM: Learning Data-Driven Curriculum for Multi-Modal Predictive Maintenance
Zhang, Shuaicheng; Wang, Tuo; Adams, Stephen; Bhattacharya, Sanmitra; Tiyyagura, Sunil; Bowen, Edward; Veeramani, Balaji; Zhou, Dawei (ACM, 2025-07-20)
Predictive Maintenance (PDM) systems are essential for preemptive monitoring of sensor signals to detect potential machine component failures in industrial assets such as bearings in rotating machinery. Existing PDM systems face two primary challenges: 1) Irregular Signal Acquisition, where data collection from the sensors is intermittent, and 2) Signal Heterogeneity, where the full spectrum of sensor modalities is not effectively integrated. To address these challenges, we propose a Curriculum Learning Framework for Multi-Modal Predictive Maintenance – MentorPDM. MentorPDM consists of 1) a graph-augmented pretraining module that captures intrinsic and structured temporal correlations across time segments via a temporal contrastive learning objective and 2) a bi-level curriculum learning module that captures task complexities for weighing the importance of signal modalities and samples via modality and sample curricula. Empirical results from MentorPDM show promising performance with better generalizability in PDM tasks compared to existing benchmarks. The efficacy of the MentorPDM model will be further demonstrated in real industry testbeds and platforms.
The Impact of Generative AI on Test & Evaluation: Challenges and Opportunities
Freeman, Laura J.; Robert, John; Wojton, Heather (ACM, 2025-06-23)
Generative Artificial Intelligence (GenAI) is transforming software development processes, including test and evaluation (T&E). From automating test case design to enabling continuous testing in DevOps pipelines, AI-driven tools enhance the efficiency, accu-racy, and speed of software testing. At the same time, the integra-tion of AI components into software-reliant systems introduces new challenges for verification and validation (V&V). Traditional T&E methodologies must evolve to address issues such as AI bias, hal-lucinated outputs, and the complexity of validating non-determin-istic behaviors. This position paper examines how existing T&E methods must evolve to account for AI’s stochastic nature, and con-versely how GenAI is transforming T&E practices across the soft-ware development lifecycle (SDLC).
Vulnerabilities Caused by Metric-based Policies in Reinforcement Learning Based Covert Communication Under Steering Attack
Jones, Alyse M.; Costa, Maice (ACM, 2025-06-30)
This paper explores the concept of timeliness in covert communications when faced with eavesdropping and jamming. We consider a transmitter-receiver pair communicating over a wireless channel where the choice of a resource block (frequency, time) to transmit is the result of a Reinforcement Learning policy. The eavesdropper aims to detect a transmission to perform a steering attack. Using two multiarmed bandit systems, we investigate the problem of minimizing the Age of Information (AoI) regret at the legit receiver, while maximizing the AoI regret at the adversary. We present an upper bound for regret and demonstrate through simulations the validity of the bound and the vulnerabilities introduced by the use of metric-guided policies such as age-aware policies.
Evaluation of Confusion Behaviors in SEI Models
Olds, Brennan; Maas, Ethan; Michaels, Alan J. (MDPI, 2025-06-27)
Radio Frequency Machine Learning (RFML) has in recent years become a popular method for performing a variety of classification tasks on received signals. Among these tasks is Specific Emitter Identification (SEI), which seeks to associate a received signal with the physical emitter that transmitted it. Many different model architectures, including individual classifiers and ensemble methods, have proven their capabilities for producing high accuracy classification results when performing SEI. Though the works studying different model architectures report on successes, there is a notable absence regarding the examination of systemic failures and negative traits associated with learned behaviors. This work studies those failure patterns for a 64-radio SEI classification problem by isolating common patterns in incorrect classification results across multiple model architectures and two distinct control variables: Signal-to-Noise Ratio (SNR) and the quantity of training data utilized. This work finds that many of the RFML-based models devolve to selecting from amongst a small subset of classes (≈10% of classes) as SNRs decrease and that observed errors are reasonably consistent across different SEI models and architectures. Moreover, our results validate the expectation that ensemble models are generally less brittle, particularly at a low SNR, yet they appear not to be the highest-performing option at a high SNR.
Personality Emulation Utilizing Large Language Models
Kolenbrander, Jack; Michaels, Alan J. (MDPI, 2025-06-12)
Fake identities have proven to be an effective methodology for conducting privacy and cybersecurity research; however, existing models are limited in their ability to interact with and respond to received communications. To perform privacy research in more complex Internet domains, withstand enhanced scrutiny, and persist long-term, fake identities must be capable of automatically generating responses while maintaining consistent behavior and personality. This work proposes a method for assigning personality to fake identities using the widely accepted psychometric Big Five model. Leveraging this model, the potential application of large language models (LLMs) to generate email responses that emulate human personality traits is investigated to enhance fake identity capabilities for privacy research at scale.
Land subsidence risk to infrastructure in US metropolises
Ohenhen, Leonard O.; Zhai, Guang; Lucy, Jonathan; Werth, Susanna; Carlson, Grace; Khorrami, Mohammad; Onyike, Florence; Sadhasivam, Nitheshnirmal; Tiwari, Ashutosh; Ghobadi-Far, Khosro; Sherpa, Sonam F.; Lee, Jui-Chi; Zehsaz, Sonia; Shirzaei, Manoochehr (Springer Nature, 2025-05-08)
Land subsidence is a slow-moving hazard with adverse environmental and socioeconomic consequences worldwide. While often considered solely a coastal hazard due to relative sea-level rise, subsidence also threatens inland urban areas, causing increased flood risks, structural damage and transportation disruptions. However, spatially dense subsidence rates that capture granular variations at high spatial density are often lacking, hindering assessment of associated infrastructure risks. Here we use space geodetic measurements from 2015 to 2021 to create high-resolution maps of subsidence rates for the 28 most populous US cities. We estimate that at least 20% of the urban area is sinking in all cities, mainly due to groundwater extraction, affecting ~34 million people. Additionally, more than 29,000 buildings are located in high and very high damage risk areas, indicating a greater likelihood of infrastructure damage. These datasets and information are crucial for developing ad hoc policies to adapt urban centers to these complex environmental challenges.
An Application of Explainable Multi-Agent Reinforcement Learning for Spectrum Situational Awareness
Perini, Dominick J.; Muller, Braeden P.; Kopacz, Justin; Michaels, Alan J. (MDPI, 2025-04-10)
Allocating low-bandwidth radios to observe a wide portion of a spectrum is a key class of search-optimization problems that requires system designers to leverage limited resources and information efficiently. This work describes a multi-agent reinforcement learning system that achieves a balance between tuning radios to newly observed energy while maintaining regular sweep intervals to yield detailed captures of both short- and long-duration signals. This algorithm, which we have named SmartScan, and system implementation have demonstrated live adaptations to dynamic spectrum activity, persistence of desirable sweep intervals, and long-term stability. The SmartScan algorithm was also designed to fit into a real-time system by guaranteeing a constant inference latency. The result is an explainable, customizable, and modular approach to implementing intelligent policies into the scan scheduling of a spectrum monitoring system.

Browse

Recent Submissions