Browsing by Author "Jin, Xianhao"

Now showing 1 - 2 of 2
  • Thumbnail Image
    Cost-saving in Continuous Integration: Development, Improvement, and Evaluation of Build Selection Approaches
    Jin, Xianhao (Virginia Tech, 2022-05-24)
    Continuous integration (CI) is a widely used practice in modern software engineering. Unfortunately, it is also an expensive practice — Google and Mozilla estimate their CI systems in millions of dollars. In this dissertation, I propose a collection of novel build selection approaches that are able to save the cost of CI. I also propose the first exhaustive comparison of techniques to improve CI including build and test granularity approaches. I firstly design a build selection approach (SMARTBUILDSKIP) for CI cost reduction in a balanceable way. The evaluation of SMARTBUILDSKIP shows that it can save a median of 30% of builds by only incurring a median delay of 1 build in a median of 15% of failing builds under its most conservative configuration. To minimize the delayed failure observation, I then propose the second build selection approach (PRECISEBUILDSKIP) that can save cost without delaying failure observation. We find that PRECISEBUILDSKIP can save a median of 5.5% of builds while capturing the majority of failing builds (100% in median) from the evaluation. After that, I evaluate the strengths and weaknesses of 10 techniques that can improve CI including SMARTBUILDSKIP. The findings of the comparison motivate my next work to design a hybrid technique (HYBRIDBUILDSKIP) that combines these techniques to produce more cost-saving while keeping a low proportion of failing builds that are delayed in observation. Finally, I design an experiment to understand how different weights of test duration among the whole build duration can influence the cost-saving of build and test selection techniques.
  • Thumbnail Image
    HybridCISave: A Combined Build and Test Selection Approach in Continuous Integration
    Jin, Xianhao; Servant, Francisco (ACM, 2023)
    Continuous integration (CI) is a popular practice in modern software engineering. Unfortunately, it is also a high cost practice --- Google and Mozilla estimate their CI systems in millions of dollars. To reduce the computational cost in CI, researchers developed approaches to selectively execute builds or tests that are likely to fail (and skip those likely to pass). In this paper, we present a novel hybrid technique (HybridCISave) to improve on the limitations of existing techniques: to provide higher cost savings and higher safety. To provide higher cost savings, HybridCISave combines techniques to predict and skip executions of both full builds that are predicted to pass and partial ones (only the tests in them predicted to pass). To provide higher safety, HybridCISave combines the prediction of multiple techniques to obtain stronger certainty before it decides to skip a build or test. We evaluated HybridCISave by comparing its effectiveness with the existing build selection techniques over 100 projects, and found that it provided higher cost savings at the highest safety. We also evaluated each design decision in HybridCISave and found that skipping both full and partial builds increased its cost savings and that combining multiple test selection techniques made it safer.