Understanding Software Merge Conflicts for Java Programs

Abstract

In collaborative software development, developers create branches to add features and fix bugs tentatively to work in parallel. Periodically developers need to merge the branches to integrate edits. However, merge conflicts can arise when edits overlap or interfere with each other. Resolving merge conflicts is challenging and tedious. Prior work proposed tools to detect and solve conflicts. However, many questions are still not fully investigated, such as what types of conflicts exist in real-world applications and how developers or tools handle them. To solve this problem, we conducted an empirical study to investigate merge conflicts in 208 open-source repositories. We manually inspected 538 conflicts with three types (textual, build and test) and their resolutions to characterize merge conflicts from different angles. We also designed a benchmark, ConflictBench, to test various tools in terms of their capabilities of conflict detection and resolution. ConflictBench consists of 180 merging scenarios extracted from 180 open-source Java projects. We defined three novel metrics to evaluate merge tools. By applying five state-of-the-art tools to ConflictBench, we observed that ConflictBench is effective to characterize different tools. It helps reveal limitations of existing tools and sheds light on future research. Lastly, to measure the ability of large language model(LLM) in the area of merge conflicts, we apply two state-of-art LLMs, ChatGPT and Gemini, to our benchmark. This experiment reveal the advantage and limitation of LLMs in resolving merge conflicts. In summary, our research characterizes real-world merge conflicts as well as software merge tools, and test with LLMs in resolving conflicts. Our work will help developers better tackle the technical challenges posed by software merge conflicts, and shed light on future research in that area.