Journal Articles, Association for Computing Machinery (ACM)

Permanent URI for this collection

https://hdl.handle.net/10919/105038

Browse

Now showing 1 - 20 of 573

Evaluating Assessment Practices in Team-Based Computing Capstone Projects
Hooshangi, Sara; Shakil, Asma; Riddle, Steve; Aydin, Ilknur; Nasir, Nayla; Parupudi, Tejasvi; Rehman, Attiqa; Scott, Michael James; Vahrenhold, Jan; Weerasinghe, Amali; Wu, Xi (ACM, 2025-06-27)
Team-based capstone projects are vital in preparing computer science students for real-world work by developing teamwork, communication, and industry-relevant technical skills. Their assessment, however, is challenging, requiring alignment between academic criteria and external stakeholder expectations, fair evaluation of individual contributions, recognition of diverse skills, and clarity on external partners' involvement in the evaluation process. The high stakes of these projects further demand transparent and equitable assessment methods that are perceived as fair by all involved. Our working group (WG) addresses the challenges of capstone project assessment by examining the perspectives of instructors, students, and external stakeholders to support fair and effective evaluation. Building on insights from our previous WG and a comprehensive review of the literature, we used a mixed-methods approach combining online surveys (quantitative) and in-depth interviews (qualitative) with instructors, students, and external stakeholders. In total, we collected 66 survey responses and conducted 30 interviews across multiple countries and institutions, capturing a diverse range of global perspectives on capstone course assessments. Insights from instructors and students revealed several commonalities, for example, in the types of assessed components and the challenges of identifying and addressing non-contributing group members. Our findings also revealed clear variation between instructor and student perspectives on how contributions are measured and weighted. Instructors were reluctant to rely heavily on peer or self-evaluation due to concerns about reliability, preferring scaffolded assessments and early-warning systems to gather contribution data and moderate team dynamics. They viewed contribution-based grading as positive but resource-intensive. Students, in contrast, emphasized the need for more transparency, formative feedback, and accurate recognition of individual contributions. They also expressed concerns about the lack of recognition for hidden labor (e.g., project management, team coordination), assessor inconsistency, and a reluctance to critique peers. Instructors treated peer input as supplementary evidence, whereas students perceived it as high-stakes and socially risky. Stakeholder involvement in assessment was generally limited to providing formative feedback and participating in final showcase events. We also identified generative AI as a rapidly evolving challenge, with both students and instructors seeking guidance on acceptable use and exploring opportunities to automate aspects of assessment. Our results offer actionable evidence-based guidance for designing transparent and equitable assessment practices in team-based computing capstones.
Enabling Open Educational Resource Adoption through Integrated Sharing in PrairieLearn
Poulsen, Seth; Herman, Geoffrey; Silva, Mariana; Fowler, Max; Smith, David H. IV; Porter, Leo; Ritschel, Nico; Zilles, Craig; West, Matthew (ACM, 2026-02-18)
This paper introduces the PrairieLearn Question Sharing System (PQSS), which enables instructors to share question generators with other instructors, either as open educational resources or privately. PQSS is integrated into PrairieLearn, an open-source, problemdriven online learning platform. PQSS addresses a critical need for more open-source assessments by making it easier for instructors to share assessments and for instructors to use those assessments. Instructors often do not share questions due to the time it takes to publish them and the lack of recognition for their work. Because it is directly integrated into PrairieLearn, PQSS reduces the aforementioned friction of sharing and using shared questions, and we can report usage statistics to help question authors receive recognition for their work. In this paper, we share design and implementation details of the system, as well as experiences using it to share course content across courses and between universities.
A Call for Critical Technology to Enable Innovative and Alternative Grading Practices
Decker, Adrienne; Edwards, Stephen H.; Edmison, Bob; Pérez-Quiñones, Manuel; Rorrer, Audrey (ACM, 2026-02-18)
The call for alternative grading practices has been made both inside and outside the computing education community. Various practices exist to provide assessment and feedback to students that do not rely strictly on points out of one hundred percent, weighted averages, high stakes assignments, and grading for behaviors instead of learning. However, modern classrooms, especially computer science classrooms, rely on a myriad of digital tools to organize and maintain the course structure. Tools like learning management systems, automatic grading systems, submission systems, and practice systems all exist for computing students and faculty to use to help support the learning of programming concepts. By and large, these systems all rely on an underlying mechanism of points and aggregating points for scoring. In the face of such technological choices, adopting alternative grading practices can prove challenging for instructors and confusing for students. In this position paper, we advocate addressing key research problems to make these systems easier to use with alternative grading practices. These include comprehensive support for categorical grading, comprehensive support for rework and resubmission, and improved protocols for communication of scores and feedback. We propose an extension to LTI to support the needs of alternative grading practices, and we provide an initial design for this LTI extension. We discuss current problems and potential solutions and challenge the community to work on these problems and consider the design of future systems to embrace grading approaches that go beyond just points-based scoring.
A Multi-Institutional Study on Peer Instruction: Evaluating Text-Chat with Assigned Group Members vs Verbal Discussion
Gu, Xingjian; Ericson, Barbara; Wu, Zihan; Ellis, Margaret O'Neil; Pearce, Janice; Rodger, Susan; Velasco, Yesenia (ACM, 2026-02-18)
In Peer Instruction (PI) an instructor displays a challenging multiplechoice question during lecture that students answer individually, discuss verbally with nearby peers, answer individually again, and finally, the instructor leads a discussion of the question. Peer Instruction typically increases student learning and motivation over traditional lecture. We added a text-chat mode to improve PI for remote synchronous learning. This feature assigns students to discussion groups to maximize the number of groups that have members with different answers. The tool was pilot tested in Winter 2022 and revised. In Fall 2022 and Winter 2023, it was tested at one institution. In Fall 2024, it was tested at four institutions. We conducted a log file analysis of student data from 1394 students and analyzed surveys with 848 student responses. We found that questions answered using the text-chat had a significantly higher improvement than those using traditional verbal discussion, although the two modes were tested with different questions. Interestingly, most of the students preferred to discuss the question verbally, although some preferred the text-chat discussion. These results inform efforts to improve the effectiveness of Peer Instruction and increase its adoption.
Creating Exercises with Generative AI for Teaching Introductory Secure Programming: Are We There Yet?
St. Amour, Leo; Tilevich, Eli (ACM, 2026-02-18)
Despite ongoing efforts to integrate security concepts into computer science curricula, many graduates still lack practical software security skills. Active learning strategies—such as drill-and-practice— offer a promising approach to bridging this educational gap. To implement these strategies effectively, educators must design and deliver hands-on exercises focusing specifically on secure programming. However, creating effective secure programming exercises is difficult, requiring substantial time and in-depth expertise. This paper examines the potential of generative AI to aid in creating drill-and-practice exercises for introductory secure programming settings. Specifically, we prompt several large language models (LLMs) to assist in generating exercises targeting three common software vulnerability classes, with tasks aligned to the advanced beginner stage of the Dreyfus skills acquisition model. We systematically evaluate the generated exercises for correctness and instructional viability. Our results show that, for some vulnerabilities, LLMs can produce technically sound and useful exercises for advanced beginners. While many generated exercises were near classroom-ready, minor fine-tuning is often necessary to ensure quality and pedagogical alignment. These findings suggest that effective exercise generation in secure programming is best achieved through a symbiosis between generative AI and human educators.
Constructing Commonsense Knowledge Graph for Persona Consistency
Xia, Lei; Chen, Yuyan; Chen, Xiangqin; Fan, Jixiang; Dai, Weinan; Li, Xiaomei; Li, Zhixu (ACM, 2026-02-22)
Ensuring consistent persona in interactive AI systems presents a significant challenge, especially in diverse application scenarios ranging from virtual assistants to customer service bots. Such capability is often constrained by the system's understanding of direct and explicit persona conflicts. Traditional approaches primarily focus on detecting discrepancies between machine responses and its predefined profile, or the contextual inconsistencies between the responses at the semantic level rather than the persona level. Due to the lack of a comprehensive persona-specific Commonsense Knowledge Graph, some indirect and implicit persona inconsistencies between machine responses can hardly be identified. In this paper, we build the first persona commonsense knowledge graph (PersonaKG), based on which we then construct a large-scale persona consistency dialogue dataset (PersonaCOM) containing both explicit and implicit persona conflicts between machine responses. With the guidance of the persona commonsense knowledge, we propose a Recognize-Rewrite framework (R2) which first recognizes the responses that are inconsistent in persona with the previous responses, and then rewrites them into consistent ones. The empirical study demonstrates that utilizing R2 method on PersonaCOM with PersonaKG results in a significant improvement of 12.20% in automatic metrics and 10.09% in manual evaluation compared to not using the R2 method and PersonaKG.
Together or Apart: Designing Boundaries for Personal Intelligent Agents
Kang, Hyunmin; Lee, Seul Chan; Jeong, JiHyun; Kim, Hyochang; Cha, Minchul; Jeon, Myounghoon (ACM, 2025-11-10)
Personal intelligent agents (IAs) are increasingly embedded in everyday life, a trend accelerated by generative AI technologies. Despite their growing presence, these agents often remain fragmented across different life domains and environments. This workshop explores how to design integrated IA ecosystems emphasizing continuity, coordination, and human-centered values. Participants with varied perspectives will collaboratively develop frameworks, scenarios, and guidelines for cohesive personal agent systems that enrich user experiences holistically. By examining factors that shape users’ preferences for information integration or separation, we aim to inform the design of coherent, user-aligned multi-agent systems.
Scalable and Fault-Tolerant Storage and File System Services with Non-Blocking Synchronization for Private Clouds
Sung, Mincheol; Nikolaev, Ruslan; Ravindran, Binoy (ACM, 2025-11-19)
We present two system services – the storage service and the file system service designed for private cloud environments to facilitate file sharing across different virtual machines (VMs). Our services are scalable, fault-tolerant, and deliver excellent performance. These system servers are implemented as unikernels running atop of the Xen hypervisor. Additionally, our storage service can leverage NetBSD code, enabling support for a wide range of both legacy and modern storage devices, such as NVMe. Furthermore, the storage service addresses the challenge of transparent fault recovery for storage, a complex task for stateful subsystems, without incurring significant overhead – a well-known challenge in storage systems. Our file system service is designed to be copy-free, enhancing overall performance. We have also designed an inter-VM communication (IVMC) mechanism that fosters scalability and reliability by leveraging lock-free concurrent ring buffers. Since this mechanism is lock-free, our system services communicate with application VMs in a more scalable manner compared to traditional ring buffers used in hypervisors such as Xen. Our lock-free design also aids in restoring storage states during the fault recovery process of the storage server. Our evaluation results demonstrate that our system services achieve performance comparable to that of Linux.
10CACHE: Heterogeneous Resource-Aware Tensor Caching and Migration for LLM Training
Afroz, Sabiha; Khan, Redwan Ibne Seraj; Albahar, Hadeel; Han, Jingoo; Butt, Ali R. (ACM, 2025-11-19)
Training large language models (LLMs) in the cloud faces growing memory bottlenecks due to the limited capacity and high cost of GPUs. While GPU memory offloading to CPU and NVMe has made large-scale training more feasible, existing approaches suffer from high tensor migration latency and suboptimal device memory utilization, ultimately increasing training time and cloud costs. To address these challenges, we present 10Cache, a resource-aware tensor caching and migration system that accelerates LLM training by intelligently coordinating memory usage across GPU, CPU, and NVMe tiers. 10Cache profiles tensor execution order to construct prefetch policies, allocates memory buffers in pinned memory based on tensor size distributions, and reuses memory buffers to minimize allocation overhead. Designed for cloud-scale deployments, 10Cache improves memory efficiency and reduces reliance on high-end GPUs. Across diverse LLM workloads, it achieves up to 2× speedup in training time, improves GPU cache hit rate by up to 86.6×, and increases CPU/GPU memory utilization by up to 2.15× and 1.33×, respectively, compared to state-of-the-art offloading methods. These results demonstrate that 10Cache is a practical and scalable solution for optimizing LLM training throughput and resource efficiency in cloud environments.
Hybrid Learning and Optimization-Based Dynamic Scheduling for DL Workloads on Heterogeneous GPU Clusters
Dongare, Shruti; Khan, Redwan Ibne Seraj; Albahar, Hadeel; Zhao, Nannan; Meléndez-Maita, Diego; Butt, Ali R. (ACM, 2025-11-19)
Modern cloud platforms increasingly host large-scale deep learning (DL) workloads, demanding high-throughput, low-latency GPU scheduling. However, the growing heterogeneity of GPU clusters and limited visibility into application characteristics pose major challenges for existing schedulers, which often rely on offline profiling or application-specific assumptions. We present RLTune, an application-agnostic reinforcement learning (RL)–based scheduling framework that dynamically prioritizes and allocates DL jobs on heterogeneous GPU clusters. RLTune integrates RL–driven prioritization with MILP–based job-to-node mapping to optimize systemwide objectives such as job completion time (JCT), queueing delay, and resource utilization. Trained on large-scale production traces from Microsoft Philly, Helios, and Alibaba, RLTune improves GPU utilization by up to 20%, reduces queueing delay by up to 81%, and shortens JCT by as much as 70%. Unlike prior approaches, RLTune generalizes across diverse workloads without requiring per-job profiling, making it practical for cloud providers to deploy at scale for more efficient, fair, and sustainable DLworkload management.
iMIA: Assessing Mission Risk in Uncertain, Interdependent AI Systems
Yoon, Han Jun; Thukkaraju, Ashrith; Cho, Jin-Hee; Matsumoto, Shou; Ferrari, Jair; Costa, Paulo; Lee, Donghwan; Ahn, Myung Kil (ACM, 2026)
Mission Impact Assessment (MIA) is critical for enhancing system effectiveness and ensuring mission success. This paper presents iMIA, an interdependent MIA framework that models relationships among mission components and enables probabilistic reasoning under uncertainty. Designed for AI-driven mission systems operating in dynamic, low-data, or poorly observable environments, iMIA addresses the limitations of traditional methods that often rely on overly confident assumptions about adversary behavior. While conventional hypergame theory (HGT) captures perceptual uncertainty from asymmetric or inaccurate views, it overlooks epistemic uncertainty arising from limited knowledge. To bridge this gap, we introduce a hybrid SL-based HGT model (SLHG), integrating Subjective Logic (SL) to represent epistemic uncertainty and HGT to account for misperceptions. This integration supports informed decision-making under both uncertain strategy beliefs and divergent environmental views. iMIA evaluates mission impact using multidimensional system quality metrics, security, trust, resilience, and agility, across diverse attacker-defender interactions. It identifies critical nodes influencing mission outcomes and quantifies performance gains from asset capacity reinforcement and asset vulnerability mitigation. Applied to a vehicle-assisted AI-based mission system, iMIA with SLHG improves performance by 16% in ASR, 20% in MTBF, 11% in TSA, and 14% in P_ACC. Designed for incremental development, iMIA supports continuous feedback and iterative refinement. Our results show that feedback-driven adjustments improve overall system performance by up to 18% in the accuracy performance.
Understanding Tradeoffs of Replicated Data Library Integration Strategies in Multilingual Environments
Mondal, Provakar; Tilevich, Eli (ACM, 2025-12-15)
Modern distributed systems replicate data across multiple execution sites by means of special-purpose replicated data libraries (RDLs), which provide read-write data access and synchronization. Programming languages often need to be mixed across replica sites to meet business requirements and resource constraints. Because RDLs are typically written in a single language, integrating them in multilingual environments requires special-purpose code, whose characteristics are poorly understood. We aim to bridge this knowledge gap by reviewing two key strategies for integrating RDLs in multilingual environments: (1) foreign-function interface (FFI) and (2) common data format (CDF). Our preliminary results indicate performance and implementation tradeoffs: CDF offers latency and memory consumption advantages, while incurring an additional implementation burden. With modern distributed systems utilizing multiple languages, our findings can inform the design of RDLs for multilingual replicated data systems.
Toward Thorough and Practical Integration Testing of Replicated Data Systems
Mondal, Provakar (ACM, 2025-12-15)
Highly available applications rely on replicated data, but complex event interleavings between application logic and replicated data libraries (RDLs) often cause subtle integration bugs. Detecting such bugs is challenging due to the inherent nondeterminism of distributed execution, as certain bugs can only manifest under specific interleavings. Correctness testing, therefore, requires replaying all possible interleavings—a challenging task due to the combinatorial explosion of the interleaving space. My doctoral dissertation addresses this challenge with ER-𝜋, a middleware framework that exercises all possible interleavings between the application code and RDL; it also eliminates redundant and impossible interleavings via novel pruning techniques. Initial results show that ER-𝜋 successfully reproduces 12 real-world bugs across multiple opensource RDLs while significantly reducing the interleaving search space. Our ongoing work extends this foundation with interleaving prioritization, ranking interleavings execution by their likelihood of exposing faults—particularly those introduced by recent code changes, thus accelerating bug discovery. This research supports developers responsible for ensuring the correctness and reliability of replicated data systems.
RailEstate: An Interactive System for Metro Linked Property Trends
Chang, Chen-Wei; Cheng, Yu-Chieh; Tsai, Yun-En; Chen, Fanglan; Lu, Chang-Tien (ACM, 2025-11-03)
Access to metro systems plays a critical role in shaping urban housing markets by enhancing neighborhood accessibility and driving property demand.We present RailEstate, a novel web-based system that integrates spatial analytics, natural language interfaces, and interactive forecasting to analyze how proximity to metro stations influences residential property prices in the Washington metropolitan area. Unlike static mapping tools or generic listing platforms, RailEstate combines 25 years of historical housing data with transit infrastructure to support low-latency geospatial queries, time-series visualizations, and predictive modeling. Users can interactively explore ZIP-code-level price patterns, investigate longterm trends, and forecast future housing values around any metro station. A key innovation is our natural language chatbot, which translates plain-English questions (e.g., “What is the highest price in Falls Church in the year 2000?”) into executable SQL over a spatial database. This unified and interactive platform empowers urban planners, investors, and residents to derive actionable insights from metro-linked housing data—without requiring technical expertise. A demonstration video of the system is available at https://www.youtube.com/watch?v=ZLiz8S1UXsc.
LLM-Based Multi-Agent System and Simplicial Self-Supervised Learning Model for Regional Cancer Prevalence Estimation Using Satellite Imagery
Yang, Jiue-An; Chen, Yuzhou; Tribby, Calvin; Lee, Huikyo; Erhunmwunsee, Loretta; Benmarhnia, Tarik; Thompson, Caroline; Gel, Yulia; Jankowska, Marta (ACM, 2025-11-03)
Traditional cancer rate estimations are often limited in spatial resolutions and lack considerations of environmental factors. Satellite imagery has become a vital data source for monitoring diverse urban environments, supporting applications across environmental, socio-demographic, and public health domains. However, while deep learning (DL) tools, particularly convolutional neural networks, have demonstrated strong performance in extracting features from high-resolution imagery, their reliance on local spatial cues often limits their ability to capture complex, non-local, and higher-order structural information. To overcome this limitation, we propose a novel LLM-based multi-agent coordination system for satellite image analysis, which integrates visual and contextual reasoning through a simplicial contrastive learning framework (Agent- SNN). Our Agent-SNN contains two augmented superpixel-based graphs and maximizes mutual information between their latent simplicial complex representations, thereby enabling the system to learn both local and global topological features. The LLM-based agents generate structured prompts that guide the alignment of these representations across modalities. Experiments with satellite imagery of Los Angeles and San Diego demonstrate that Agent-SNN achieves significant improvements over state-of-the-art baselines in regional cancer prevalence estimation tasks.
Sparsity-aware Kernel Selection for Edge-Connected Jaccard Similarity in Graph Datasets
Gondhalekar, Atharva; Sathre, Paul; Chaudhury, Nabayan; Feng, Wu-chun (ACM, 2025-09-08)
Performance of graph algorithms often depends both on the underlying hardware architecture and on structural properties of the input graph. Optimizations that deliver high performance on one class of graphs, such as hypersparse graphs with low average degree, can degrade performance on other classes, for example denser graphs with high average degree. In this work, we investigate sparsityaware GPU kernel selection for computing the Jaccard similarity index, a measure of neighborhood overlap in graph datasets. In our kernel selection approach, we use the vertex-centric Jaccard similarity implementation from the cuGraph library as the baseline and include both vertex- and edge-centric variants of this kernel, with set-intersection algorithms varying between two-pointer linear search, binary search, and adaptive dynamic search. We use 80 real-world graphs in our evaluation with variation in average degree, maximum degree, Gini index, and average intersection cost. A random forest classifier, trained on a subset of these graphs on an NVIDIA A100 GPU, achieves 88.8% inference accuracy in predicting the fastest kernel. Kernels selected by the classifier achieve a 4.37× mean speedup over the vertex-centric cuGraph baseline from NVIDIA.
Are We on the Same Page? Examining Developer Perception Alignment in Open Source Code Reviews
Alebachew, Yoseph Berhanu; Ko, Minhyuk; Brown, Chris (ACM, 2025-06-17)
Code reviews are a critical aspect of open-source software (OSS) development, ensuring quality and fostering collaboration. This study examines perceptions, challenges, and biases in OSS code review processes, focusing on the perspectives of Contributors and Maintainers. Through surveys (𝑛 = 289), interviews (𝑛 = 23), and repository analysis (𝑛 = 81), we identify key areas of alignment and disparity. While both groups share common objectives, differences emerge in priorities, e.g, with Maintainers emphasizing alignment with project goals while Contributors overestimated the value of novelty. Bias, particularly familiarity bias, disproportionately affects underrepresented groups, discouraging participation and limiting community growth. Misinterpretation of approach differences as bias further complicates reviews. Our findings underscore the need for improved documentation, better tools, and automated solutions to address delays and enhance inclusivity. This work provides actionable strategies to promote fairness and sustain the long-term innovation of OSS ecosystems.
Investigating Seamless Transitions Between Immersive Computational Notebooks and Embodied Data Interactions
In, Sungwon; Krokos, Eric; Whitley, Kirsten; North, Christopher L.; Yang, Yalong (ACM, 2025-11-12)
A growing interest in Immersive Analytics (IA) has led to the extension of computational notebooks (e.g., Jupyter Notebook) into an immersive environment to enhance analytical workflows. However, existing solutions rely on the WIMP (windows, icons, menus, pointer) metaphor, which remains impractical for complex data exploration. Although embodied interaction offers a more intuitive alternative, immersive computational notebooks and embodied data exploration systems are implemented as standalone tools. This separation requires analysts to invest considerable effort to transition from one environment to an entirely different one during analytical workflows. To address this, we introduce ICoN, a prototype that facilitates a seamless transition between computational notebooks and embodied data explorations within a unified, fully immersive environment. Our findings reveal that unification improves transition efficiency and intuitiveness during analytical workflows, highlighting its potential for seamless data analysis.
The Importance of Cueing While Visually Searching a 360 Degree Environment for Multiple Targets in the Presence of Distractors
Kelley, Brendan; McMahan, Ryan P.; Wickens, Christopher; Clegg, Benjamin; Ortega, Francisco (ACM, 2025-11-12)
Visually searching for objects is an everyday task. In many contexts, people must visually search for multiple objects at the same time while avoiding distractor objects, such as triage during a mass casualty incident. While many prior augmented reality (AR) and virtual reality (VR) studies have investigated cues to aid in visual search tasks, few have investigated cues in contexts involving multiple targets and distractors with a full 360° effective field of regard (EFOR). Individually, multiple targets, distractors, and a full 360° EFOR each add complexity to visual search; when combined, they compound the difficulty even further. In this paper, we present such a study that compares three common types of visual cues (2D Wedge, 3D Arrow, and Gaze Line) to a baseline condition with no cueing for a 360° visual search task. Our results reinforce the importance of providing some type of cue, with the Gaze Line design being particularly beneficial. We discuss the potential implications of these findings for designing cues specifically for such complex visual search tasks.
Towards an Embodied Composition Framework for Organizing Immersive Computational Notebooks
In, Sungwon; Krokos, Eric; Whitley, Kirsten; North, Christopher L.; Yang, Yalong (ACM, 2025-11-12)
As immersive technologies evolve, immersive computational notebooks offer new opportunities for interacting with code, data, and outputs. However, scaling these environments remains a challenge, particularly when analysts manually arrange large numbers of cells to maintain both execution logic and visual coherence. To address this, we introduce an embodied composition framework, facilitating organizational processes in the context of immersive computational notebooks. To evaluate the effectiveness of the embodied composition framework, we conducted a controlled user study comparing manual and embodied composition frameworks in an organizational process. The results show that embodied composition frameworks significantly reduced user effort and decreased completion time. However, the design of the triggering mechanism requires further refinement. Our findings highlight the potential of embodied composition frameworks to enhance the scalability of the organizational process in immersive computational notebooks.

Browse

Recent Submissions