Flaky_Fault_Localization

Scripts for the Bachelors-Thesis: "Flaky Fault Localization" by Christian Kasberger. The thesis examines the usefulness of spectrum based fault localization methods for finding the root causes of flakiness in python tests.

Contents

Contains a pytest plugin that writes the information needed to a coverage.csv, a script to calculate the suspiciousness-scores for the project using said coverage.csv and a script, that automatically installs dependencies needed to run python projects as well as a bash script, executing all the above mentioned scripts to produce suspiciousness-scores for multiple projects at once.

It is recommended to use the bash-script in some sort of container when executing unfamiliar code.

All the python scripts can be used for themselves to execute the corresponding steps i.e., to calculate the suspiciousness-scores using an already existing coverage.csv you can use the script calculate_scores.py.

requirements_installation.py

Installs all dependencies documented in a project for the following file-formats: requirements.txt

• dev-requirements.txt
• dev_requiredments.txt
• test-requirements.txt
• test_requirements.txt
• requirements-dev.txt
• requirements_dev.txt
• requirements-test.txt
• requirements_test.txt
• Pipfile

calculate_scores.py

Calculates suspiciousness-scores using a given coverage.csv. Note that multiple test runs are necessary for the calculation to work, since both a failing and passing run for the flaky test(s) are necessary. Call the script with

$ ./calculate_scores.py <directory_with_coverage.csv> <results_directory> --flaky_tests <flaky_tests> --sfl_method <sfl_method>

flaky tests takes multiple arguments separated by spaces and the sfl-method can be chosen from tarantula, ochiai or dstar.

pytest plugin

To use the plugin, the file conftest.py needs to be located in the directory from where the following command is executed.

$ pytest <project_directory> --cov_per_test <project_directory> --results_dir <results_dir>

flaky_fault_localization.sh

Takes a csv file as input, ignores the first line and then executes the other scripts to calculate the suspiciousness-scores for each line of python code in the projects specified in the input file.