Supplementary data for: Assessing the effectiveness of large language models in detecting semantic conflicts

This repository contains the artifacts from the experiments using Large Language Models (LLMs) in SMAT. The data associated with the initial experiments were used in the paper titled 'Detecção de Conflitos Semânticos com Testes Gerados por LLM' (DOI: https://doi.org/10.5753/sast.2025.13826). For this specific publication, the files are organized into three main directories:

-the inputs/ folder contains the JSON input files utilized in the SMAT experiments; -the runs/ folder holds the experimental results, including the generated tests for the analyzed projects, as well as the resulting reports, charts, and tables; -and the scripts/ folder provides the Python scripts used to execute the experiments and generate the visual and textual reports.

Furthermore, the repository includes the artifacts for the extended research published in the paper 'Assessing the Effectiveness of Large Language Models in Detecting Semantic Conflicts'. The data for this extended evaluation can be found in the reports-jserd directory, which contains the artifacts from the experiments conducted with multiple LLMs.

Mots clés

• Large Language Models
• LLM
• Semantic Conflicts
• Software Testing
• Test Generation
• SMAT
• LUCIA