4.0

Release Notes

• based on the artefacts of this release, you can:
  • execute the experiment on your own machine with our input data
  • use your own data as input
  • explore our results either manually or with IncrementalAnalysesHelpers (use Version 4.0)

File	Description
diffs-ordered-list-of-commits.txt	ordered commit hashes line by line containing all commits that were used for the evaluation
diffs.zip	Diff files used as input for the analysis executions
data-CONFIG.zip	data generated during execution with configuration CONFIG(=configurationblockchange/blockchange/artifactchange/baseline). During the baseline execution for commit 410, the wall clock jumped back an hour. We therefore include an adjusted version of the log-file for commit 410 where the timestamps do not jump an hour.
investigation-cached-models.zip	ten executions of the non-incremental dead code analysis with cached models
investigation-reextracted-models.zip	ten executions of the non-incremental dead code analysis with reextracted models
evaluator-logs.zip	Logs from IncrementalAnalysesHelpers that include performance results as well as a consistency checks for the identified dead code blocks. For the performance, we provide two sets of results for each incremental configuration: one with (*-incl1.log) and one without (*-excl1.log) the first commit.
linux-vm.7z	Virtual machine with prerequisites for running the evaluation installed. User: sse Password: sse
evaluation-results.zip	Evaluation results containing the intensity of variability changes in the commits sequence (directory “comani-check”), the performance results and related calculations (directory “performance-check”), the correlation between execution times and number of blocks (directory “performance-commits-check”), and a general summary in “performance-evaluation-summary.xlsx”

3.0

Release Notes

• based on the artefacts of this release, you can:
  • execute the experiment on your own machine with our input data
  • use your own data as input
  • explore our results either manually or with [IncrementalAnalysesHelpers] (https://github.com/moritzfl/IncrementalAnalysesHelpers) (use Version 3.0)
• This release includes the results of a ComAn execution for our commit range

incremental-experiment

Release Notes

• based on the artefacts of this release, you can:
  • execute the experiment on your own machine with our input data
  • use your own data as input
  • explore our results either manually or with [IncrementalAnalysesHelpers] (https://github.com/moritzfl/IncrementalAnalysesHelpers). The version we used is contained in the replication package.

Overview over Published Artefacts

File	Description
system-information.txt	technical specifications of the reference system
diffs.zip	Diff files used as input for the analysis executions
results-plot.[xlsx/pdf]	plots and excel worksheet of performance results
linux-vm.7z	Virtual machine with prerequisites for running the evaluation installed. User: sse Password: sse
replication-package.zip	Our result data and a complete replication package (excluding the VM due to file size limits for individual files on github releases, you can download that in a separate download)

2.1: remove evaluation-diffs from git

Release Notes

• based on the artefacts of this release, you can:
  • execute the experiment on your own machine with our input data
  • use your own data as input
  • explore our results either manually or with IncrementalAnalysesHelpers
• This release includes the results of a ComAn execution for our commit range

Overview over Published Artefacts

File	Description
reference-system.png	technical specifications of the reference system
commit-hashes-for-evaluation.txt	commit hashes for commits used for evaluation
diffs.zip	Diff files used as input for the analysis executions
data-CONFIG.zip	data generated during execution with configuration CONFIG(=change/varchange/baseline), each file also includes the template of the configuration file for KernelHaven
results-performance-plots.zip	plots and excel worksheet of performance results
coman-results.zip	results of the execution of the ComAn tool on our commit range
linux-vm.7z	Virtual machine with prerequisites for running the evaluation installed. User: sse Password: sse

2.0

• now matches version 2.0 of IncrementalAnalysesInfrastructure.

Submitted Version (for thesis)

This release contains evaluation runs that were made right before the thesis was finalized. Due to the fact that they were produced shortly before the thesis deadline, the written thesis does not describe them.

Final Evaluation (for written thesis)

Release Notes

• Repeated evaluation after fixing issues identified during early evaluation

Overview over Published Artifacts

File	Description
reference-system.png	technical specs of reference system
commit-hashes-for-evaluation.txt	commit hashes for commits used for evaluation
configuration-CONFIG.properties	configuration-file representing the configuration CONFIG as described in thesis
data-CONFIG-ref.zip	data generated during execution with configuration CONFIG, also contains the data of the reference execution
data-change-outlier.zip	data of repeated evaluation for the outlier of the change execution
results-CONFIG.zip	results of the comparison of the execution data of CONFIG with ref
results-performance-plots.zip	plots and excel worksheet of performance results

Early Evaluation (for written thesis)

1.0

Update kernelhaven-evaluation-reference.sh