Scholarly Works, Virginia Tech National Security Institute

Permanent URI for this collection

https://hdl.handle.net/10919/110084

Browse

Now showing 1 - 20 of 36

MENTORPDM: Learning Data-Driven Curriculum for Multi-Modal Predictive Maintenance
Zhang, Shuaicheng; Wang, Tuo; Adams, Stephen; Bhattacharya, Sanmitra; Tiyyagura, Sunil; Bowen, Edward; Veeramani, Balaji; Zhou, Dawei (ACM, 2025-07-20)
Predictive Maintenance (PDM) systems are essential for preemptive monitoring of sensor signals to detect potential machine component failures in industrial assets such as bearings in rotating machinery. Existing PDM systems face two primary challenges: 1) Irregular Signal Acquisition, where data collection from the sensors is intermittent, and 2) Signal Heterogeneity, where the full spectrum of sensor modalities is not effectively integrated. To address these challenges, we propose a Curriculum Learning Framework for Multi-Modal Predictive Maintenance – MentorPDM. MentorPDM consists of 1) a graph-augmented pretraining module that captures intrinsic and structured temporal correlations across time segments via a temporal contrastive learning objective and 2) a bi-level curriculum learning module that captures task complexities for weighing the importance of signal modalities and samples via modality and sample curricula. Empirical results from MentorPDM show promising performance with better generalizability in PDM tasks compared to existing benchmarks. The efficacy of the MentorPDM model will be further demonstrated in real industry testbeds and platforms.
The Impact of Generative AI on Test & Evaluation: Challenges and Opportunities
Freeman, Laura; Robert, John; Wojton, Heather (ACM, 2025-06-23)
Generative Artificial Intelligence (GenAI) is transforming software development processes, including test and evaluation (T&E). From automating test case design to enabling continuous testing in DevOps pipelines, AI-driven tools enhance the efficiency, accu-racy, and speed of software testing. At the same time, the integra-tion of AI components into software-reliant systems introduces new challenges for verification and validation (V&V). Traditional T&E methodologies must evolve to address issues such as AI bias, hal-lucinated outputs, and the complexity of validating non-determin-istic behaviors. This position paper examines how existing T&E methods must evolve to account for AI’s stochastic nature, and con-versely how GenAI is transforming T&E practices across the soft-ware development lifecycle (SDLC).
Vulnerabilities Caused by Metric-based Policies in Reinforcement Learning Based Covert Communication Under Steering Attack
Jones, Alyse M.; Costa, Maice (ACM, 2025-06-30)
This paper explores the concept of timeliness in covert communications when faced with eavesdropping and jamming. We consider a transmitter-receiver pair communicating over a wireless channel where the choice of a resource block (frequency, time) to transmit is the result of a Reinforcement Learning policy. The eavesdropper aims to detect a transmission to perform a steering attack. Using two multiarmed bandit systems, we investigate the problem of minimizing the Age of Information (AoI) regret at the legit receiver, while maximizing the AoI regret at the adversary. We present an upper bound for regret and demonstrate through simulations the validity of the bound and the vulnerabilities introduced by the use of metric-guided policies such as age-aware policies.
Evaluation of Confusion Behaviors in SEI Models
Olds, Brennan; Maas, Ethan; Michaels, Alan J. (MDPI, 2025-06-27)
Radio Frequency Machine Learning (RFML) has in recent years become a popular method for performing a variety of classification tasks on received signals. Among these tasks is Specific Emitter Identification (SEI), which seeks to associate a received signal with the physical emitter that transmitted it. Many different model architectures, including individual classifiers and ensemble methods, have proven their capabilities for producing high accuracy classification results when performing SEI. Though the works studying different model architectures report on successes, there is a notable absence regarding the examination of systemic failures and negative traits associated with learned behaviors. This work studies those failure patterns for a 64-radio SEI classification problem by isolating common patterns in incorrect classification results across multiple model architectures and two distinct control variables: Signal-to-Noise Ratio (SNR) and the quantity of training data utilized. This work finds that many of the RFML-based models devolve to selecting from amongst a small subset of classes (≈10% of classes) as SNRs decrease and that observed errors are reasonably consistent across different SEI models and architectures. Moreover, our results validate the expectation that ensemble models are generally less brittle, particularly at a low SNR, yet they appear not to be the highest-performing option at a high SNR.
Personality Emulation Utilizing Large Language Models
Kolenbrander, Jack; Michaels, Alan J. (MDPI, 2025-06-12)
Fake identities have proven to be an effective methodology for conducting privacy and cybersecurity research; however, existing models are limited in their ability to interact with and respond to received communications. To perform privacy research in more complex Internet domains, withstand enhanced scrutiny, and persist long-term, fake identities must be capable of automatically generating responses while maintaining consistent behavior and personality. This work proposes a method for assigning personality to fake identities using the widely accepted psychometric Big Five model. Leveraging this model, the potential application of large language models (LLMs) to generate email responses that emulate human personality traits is investigated to enhance fake identity capabilities for privacy research at scale.
Land subsidence risk to infrastructure in US metropolises
Ohenhen, Leonard O.; Zhai, Guang; Lucy, Jonathan; Werth, Susanna; Carlson, Grace; Khorrami, Mohammad; Onyike, Florence; Sadhasivam, Nitheshnirmal; Tiwari, Ashutosh; Ghobadi-Far, Khosro; Sherpa, Sonam F.; Lee, Jui-Chi; Zehsaz, Sonia; Shirzaei, Manoochehr (Springer Nature, 2025-05-08)
Land subsidence is a slow-moving hazard with adverse environmental and socioeconomic consequences worldwide. While often considered solely a coastal hazard due to relative sea-level rise, subsidence also threatens inland urban areas, causing increased flood risks, structural damage and transportation disruptions. However, spatially dense subsidence rates that capture granular variations at high spatial density are often lacking, hindering assessment of associated infrastructure risks. Here we use space geodetic measurements from 2015 to 2021 to create high-resolution maps of subsidence rates for the 28 most populous US cities. We estimate that at least 20% of the urban area is sinking in all cities, mainly due to groundwater extraction, affecting ~34 million people. Additionally, more than 29,000 buildings are located in high and very high damage risk areas, indicating a greater likelihood of infrastructure damage. These datasets and information are crucial for developing ad hoc policies to adapt urban centers to these complex environmental challenges.
An Application of Explainable Multi-Agent Reinforcement Learning for Spectrum Situational Awareness
Perini, Dominick J.; Muller, Braeden P.; Kopacz, Justin; Michaels, Alan J. (MDPI, 2025-04-10)
Allocating low-bandwidth radios to observe a wide portion of a spectrum is a key class of search-optimization problems that requires system designers to leverage limited resources and information efficiently. This work describes a multi-agent reinforcement learning system that achieves a balance between tuning radios to newly observed energy while maintaining regular sweep intervals to yield detailed captures of both short- and long-duration signals. This algorithm, which we have named SmartScan, and system implementation have demonstrated live adaptations to dynamic spectrum activity, persistence of desirable sweep intervals, and long-term stability. The SmartScan algorithm was also designed to fit into a real-time system by guaranteeing a constant inference latency. The result is an explainable, customizable, and modular approach to implementing intelligent policies into the scan scheduling of a spectrum monitoring system.
Blind Interference Suppression with Uncalibrated Phased-Array Processing
Lusk, Lauren O.; Gaeddert, Joseph D. (MDPI, 2025-03-27)
As the number of devices using wireless communications increases, the amount of usable radio frequency spectrum becomes increasingly congested. As a result, the need for robust, adaptive communications to improve spectral efficiency and ensure reliable communication in the presence of interference is apparent. One solution is using beamforming techniques on digital phased-array receivers to maximize the energy in a desired direction and steer nulls to remove interference; however, traditional phased-array beamforming techniques used for interference removal rely on perfect calibration between antenna elements and precise knowledge of the array configuration. Consequently, if the exact array configuration is not known (unknown or imperfect assumption of element locations, unknown mutual coupling between elements, etc.), these traditional beamforming techniques are not viable, so a beamforming approach with relaxed requirements (blind beamforming) is required. This paper proposes a novel blind beamforming approach to address complex narrowband interference in spectrally congested environments where the precise array configuration is unknown. The resulting process is shown to suppress numerous interference sources, all without any knowledge of the primary signal of interest. The results are validated through wireless laboratory experimentation conducted with a two-element array, verifying that the proposed beamforming approach achieves a similar performance to the theoretical performance bound of receiving packets in additive white Gaussian noise (AWGN) with no interference present.
Trust at Your Own Peril: A Mixed Methods Exploration of the Ability of Large Language Models to Generate Expert-Like Systems Engineering Artifacts and a Characterization of Failure Modes
Topcu, Taylan G.; Husain, Mohammed; Ofsa, Max; Wach, Paul (Wiley, 2025-02-21)
Multi-purpose large language models (LLMs), a subset of generative artificial intelligence (AI), have recently made significant progress. While expectations for LLMs to assist systems engineering (SE) tasks are paramount; the interdisciplinary and complex nature of systems, along with the need to synthesize deep-domain knowledge and operational context, raise questions regarding the efficacy of LLMs to generate SE artifacts, particularly given that they are trained using data that is broadly available on the internet. To that end, we present results from an empirical exploration, where a human expert-generated SE artifact was taken as a benchmark, parsed, and fed into various LLMs through prompt engineering to generate segments of typical SE artifacts. This procedure was applied without any fine-tuning or calibration to document baseline LLM performance. We then adopted a two-fold mixed-methods approach to compare AI generated artifacts against the benchmark. First, we quantitatively compare the artifacts using natural language processing algorithms and find that when prompted carefully, the state-of-the-art algorithms cannot differentiate AI-generated artifacts from the human-expert benchmark. Second, we conduct a qualitative deep dive to investigate how they differ in terms of quality. We document that while the two-material appear very similar, AI generated artifacts exhibit serious failure modes that could be difficult to detect. We characterize these as: premature requirements definition, unsubstantiated numerical estimates, and propensity to overspecify. We contend that this study tells a cautionary tale about why the SE community must be more cautious adopting AI suggested feedback, at least when generated by multi-purpose LLMs.
Number Recognition Through Color Distortion Using Convolutional Neural Networks
Henshaw, Christopher; Dennis, Jacob; Nadzam, Jonathan; Michaels, Alan J. (MDPI, 2025-01-22)
Machine learning applied to image-based number recognition has made significant strides in recent years. Recent use of Large Language Models (LLMs) in natural language search and generation of text have improved performance for general images, yet performance limitations still exist for data subsets related to color blindness. In this paper, we replicated the training of six distinct neural networks (MNIST, LeNet5, VGG16, AlexNet, and two AlexNet modifications) using deep learning techniques with the MNIST dataset and the Ishihara-Like MNIST dataset. While many prior works have dealt with MNIST, the Ishihara adaption addresses red-green combinations of color blindness, allowing for further research in color distortion. Through this research, we applied pre-processing to accentuate the effects of red-green and monochrome colorblindness and hyper-parameterized the existing architectures, ultimately achieving better overall performance than currently published in known works.
Wireless Mobile Distributed-MIMO for 6G
Bondada, Kumar Sai; Jakubisin, Daniel J.; Said, Karim; Buehrer, R. Michael; Liu, Lingjia (IEEE, 2024-01-01)
The paper proposes a new architecture for Distributed MIMO (D-MIMO) in which the base station (BS) jointly transmits with wireless mobile nodes to serve users (UEs) within a cell for 6G communication systems. The novelty of the architecture lies in the wireless mobile nodes participating in joint D-MIMO transmission with the BS (referred to as D-MIMO nodes), which are themselves users on the network. The D-MIMO nodes establish wireless connections with the BS, are generally near the BS, and ideally benefit from higher SNR links and better connections with edge-located UEs. These D-MIMO nodes can be existing handset UEs, Unmanned Aerial Vehicles (UAVs), or Vehicular UEs. Since the D-MIMO nodes are users sharing the access channel, the proposed architecture operates in two phases. First, the BS communicates with the D-MIMO nodes to forward data for the joint transmission, and then the BS and D-MIMO nodes jointly serve the UEs through coherent D-MIMO operation. Capacity analysis of this architecture is studied based on realistic 3GPP channel models, and the paper demonstrates that despite the two-phase operation, the proposed architecture enhances the system's capacity compared to the baseline where the BS communicates directly with the UEs.
Programmable quantum emitter formation in silicon
Jhuria, K.; Ivanov, Vsevolod; Polley, D.; Zhiyenbayev, Y.; Liu, W.; Persaud, A.; Redjem, W.; Qarony, W.; Parajuli, P.; Ji, Q.; Gonsalves, A. J.; Bokor, J.; Tan, L. Z.; Kante, B.; Schenkel, T. (Nature Portfolio, 2024-05-27)
Silicon-based quantum emitters are candidates for large-scale qubit integration due to their single-photon emission properties and potential for spin-photon interfaces with long spin coherence times. Here, we demonstrate local writing and erasing of selected light-emitting defects using femtosecond laser pulses in combination with hydrogen-based defect activation and passivation at a single center level. By choosing forming gas (N2/H2) during thermal annealing of carbon-implanted silicon, we can select the formation of a series of hydrogen and carbon-related quantum emitters, including T and Ci centers while passivating the more common G-centers. The Ci center is a telecom S-band emitter with promising optical and spin properties that consists of a single interstitial carbon atom in the silicon lattice. Density functional theory calculations show that the Ci center brightness is enhanced by several orders of magnitude in the presence of hydrogen. Fs-laser pulses locally affect the passivation or activation of quantum emitters with hydrogen for programmable formation of selected quantum emitters.
CLOUD-D RF: Cloud-based Distributed Radio Frequency Heterogeneous Spectrum Sensing
Green, Dylan; McIrvin, Caleb; Thaboun, River; Wemlinger, Cora; Risi, Joseph; Jones, Alyse; Toubeh, Maymoonah; Headley, William (ACM, 2024-12-04)
In wireless communications, collaborative spectrum sensing is a process that leverages radio frequency (RF) data from multiple RF sensors to make more informed decisions and lower the overall risk of failure in distributed settings. However, most research in collaborative sensing focuses on homogeneous systems using identical sensors, which would not be the case in a real world wireless setting. Instead, due to differences in physical location, each RF sensor would see different versions of signals propagating in the environment, establishing the need for heterogeneous collaborative spectrum sensing. Hence, this paper explores the implementation of collaborative spectrum sensing across heterogeneous sensors, with sensor fusion occurring in the cloud for optimal decision making. We investigate three different machine learning-based fusion methods and test the fused model’s ability to perform modulation classification, with a primary goal of optimizing for network bandwidth in regard to next-generation network applications. Our analysis demonstrates that our fusion process is able to optimize the number of features extracted from the heterogeneous sensors according to their varying performance limitations, simulating adverse conditions in a real-world wireless setting.
Uncertainty Quantification in Data Fusion Classifier for Ship-Wake Detection
Costa, Maice; Sobien, Daniel; Garg, Ria; Cheung, Winnie; Krometis, Justin; Kauffman, Justin A. (MDPI, 2024-12-14)
Using deep learning model predictions requires not only understanding the model’s confidence but also its uncertainty, so we know when to trust the prediction or require support from a human. In this study, we used Monte Carlo dropout (MCDO) to characterize the uncertainty of deep learning image classification algorithms, including feature fusion models, on simulated synthetic aperture radar (SAR) images of persistent ship wakes. Comparing to a baseline, we used the distribution of predictions from dropout with simple mean value ensembling and the Kolmogorov—Smirnov (KS) test to classify in-domain and out-of-domain (OOD) test samples, created by rotating images to angles not present in the training data. Our objective was to improve the classification robustness and identify OOD images during the test time. The mean value ensembling did not improve the performance over the baseline, in that there was a –1.05% difference in the Matthews correlation coefficient (MCC) from the baseline model averaged across all SAR bands. The KS test, by contrast, saw an improvement of +12.5% difference in MCC and was able to identify the majority of OOD samples. Leveraging the full distribution of predictions improved the classification robustness and allowed labeling test images as OOD. The feature fusion models, however, did not improve the performance over the single SAR-band models, demonstrating that it is best to rely on the highest quality data source available (in our case, C-band).
Training from Zero: Forecasting of Radio Frequency Machine Learning Data Quantity
Clark, William H.; Michaels, Alan J. (MDPI, 2024-07-18)
The data used during training in any given application space are directly tied to the performance of the system once deployed. While there are many other factors that are attributed to producing high-performance models based on the Neural Scaling Law within Machine Learning, there is no doubt that the data used to train a system provide the foundation from which to build. One of the underlying heuristics used within the Machine Learning space is that having more data leads to better models, but there is no easy answer to the question, “How much data is needed to achieve the desired level of performance?” This work examines a modulation classification problem in the Radio Frequency domain space, attempting to answer the question of how many training data are required to achieve a desired level of performance, but the procedure readily applies to classification problems across modalities. The ultimate goal is to determine an approach that requires the lowest amount of data collection to better inform a more thorough collection effort to achieve the desired performance metric. By focusing on forecasting the performance of the model rather than the loss value, this approach allows for a greater intuitive understanding of data volume requirements. While this approach will require an initial dataset, the goal is to allow for the initial data collection to be orders of magnitude smaller than what is required for delivering a system that achieves the desired performance. An additional benefit of the techniques presented here is that the quality of different datasets can be numerically evaluated and tied together with the quantity of data, and ultimately, the performance of the architecture in the problem domain.
Assessing the Value of Transfer Learning Metrics for Radio Frequency Domain Adaptation
Wong, Lauren J.; Muller, Braeden P.; McPherson, Sean; Michaels, Alan J. (MDPI, 2024-07-25)
The use of transfer learning (TL) techniques has become common practice in fields such as computer vision (CV) and natural language processing (NLP). Leveraging prior knowledge gained from data with different distributions, TL offers higher performance and reduced training time, but has yet to be fully utilized in applications of machine learning (ML) and deep learning (DL) techniques and applications related to wireless communications, a field loosely termed radio frequency machine learning (RFML). This work examines whether existing transferability metrics, used in other modalities, might be useful in the context of RFML. Results show that the two existing metrics tested, Log Expected Empirical Prediction (LEEP) and Logarithm of Maximum Evidence (LogME), correlate well with post-transfer accuracy and can therefore be used to select source models for radio frequency (RF) domain adaptation and to predict post-transfer accuracy.
Use and Abuse of Personal Information, Part I: Design of a Scalable OSINT Collection Engine
Rheault, Elliott; Nerayo, Mary; Leonard, Jaden; Kolenbrander, Jack; Henshaw, Christopher; Boswell, Madison; Michaels, Alan J. (MDPI, 2024-08-13)
In most open-source intelligence (OSINT) research efforts, the collection of information is performed in an entirely passive manner as an observer to third-party communication streams. This paper describes ongoing work that seeks to insert itself into that communication loop, fusing openly available data with requested content that is representative of what is sent to second parties. The mechanism for performing this is based on the sharing of falsified personal information through one-time online transactions that facilitate signup for newsletters, establish online accounts, or otherwise interact with resources on the Internet. The work has resulted in the real-time Use and Abuse of Personal Information OSINT collection engine that can ingest email, SMS text, and voicemail content at an enterprise scale. Foundations of this OSINT collection infrastructure are also laid to incorporate an artificial intelligence (AI)-driven interaction engine that shifts collection from a passive process to one that can effectively engage with different classes of content for improved real-world privacy experimentation and quantitative social science research.
Use & Abuse of Personal Information, Part II: Robust Generation of Fake IDs for Privacy Experimentation
Kolenbrander, Jack; Husmann, Ethan; Henshaw, Christopher; Rheault, Elliott; Boswell, Madison; Michaels, Alan J. (MDPI, 2024-08-11)
When personal information is shared across the Internet, we have limited confidence that the designated second party will safeguard it as we would prefer. Privacy policies offer insight into the best practices and intent of the organization, yet most are written so loosely that sharing with undefined third parties is to be anticipated. Tracking these sharing behaviors and identifying the source of unwanted content is exceedingly difficult when personal information is shared with multiple such second parties. This paper formulates a model for realistic fake identities, constructs a robust fake identity generator, and outlines management methods targeted towards online transactions (email, phone, text) that pass both cursory machine and human examination for use in personal privacy experimentation. This fake ID generator, combined with a custom account signup engine, are the core front-end components of our larger Use and Abuse of Personal Information system that performs one-time transactions that, similar to a cryptographic one-time pad, ensure that we can attribute the sharing back to the single one-time transaction and/or specific second party. The flexibility and richness of the fake IDs also serve as a foundational set of control variables for a wide range of social science research questions revolving around personal information. Collectively, these fake identity models address multiple inter-disciplinary areas of common interest and serve as a foundation for eliciting and quantifying personal information-sharing behaviors.
Multi-Hop User Equipment (UE) to UE Relays for MANET/Mesh Leveraging 5G NR Sidelink
Shyy, DJ; Luu, Cuong; Xu, John D.; Liu, Lingjia; Erpek, Tugba; Gabay, David; Bate, David (ACM, 2023-12-06)
This paper provides use cases to adapt 5G sidelink technology to enable multi-hop User Equipment (UE)-to-UE (U2U) and UE-to- Network relaying in 3GPP standards. Such a capability could enable groups of users to communicate with each other when operating at the periphery or outside a network’s coverage area, with commercial and public safety benefits. This paper compares routing protocols to enable sidelink with U2U relay to support a Mobile Ad hoc Network (MANET). A gap analysis of current 3rd Generation Partnership Project (3GPP) Release 18 (R-18) specifications is performed to determine the missing procedures to enable multi-hop U2U relaying, along with a proposed candidate protocol to fill the gap. The candidate protocol can be submitted as a contribution to 3GPP TSG Service and System Aspects (SA) Working Group 2 (WG2) as proposed changes to the 5G architecture in 3GPP Release 19 (R-19).
A Combinatorial Approach to Hyperparameter Optimization
Khadka, Krishna; Chandrasekaran, Jaganmohan; Lei, Yu; Kacker, Raghu N.; Kuhn, D. Richard (ACM, 2024-04-14)
In machine learning, hyperparameter optimization (HPO) is essential for effective model training and significantly impacts model performance. Hyperparameters are predefined model settings which fine-tune the model’s behavior and are critical to modeling complex data patterns. Traditional HPO approaches such as Grid Search, Random Search, and Bayesian Optimization have been widely used in this field. However, as datasets grow and models increase in complexity, these approaches often require a significant amount of time and resources for HPO. This research introduces a novel approach using 𝑡-way testing—a combinatorial approach to software testing used for identifying faults with a test set that covers all 𝑡-way interactions—for HPO. 𝑇 -way testing substantially narrows the search space and effectively covers parameter interactions. Our experimental results show that our approach reduces the number of necessary model evaluations and significantly cuts computational expenses while still outperforming traditional HPO approaches for the models studied in our experiments.

Browse

Recent Submissions