Highly-configurable systems allow users to achieve their specific needs. Such software systems are flexible, but come with difficulties for program analyses, as it is usually not possible to test all combinations of options separately due to the configuration space explosion. Variability-aware approaches exploit redundancies among configurations to share analysis efforts. Variability-aware execution aggressively shares computations and data in program execution across multiple configurations, for example, when executing a test over all configurations. By maximizing sharing, the approach can outperform traditional approaches by orders of magnitude on many highly-configurable systems, without falling back to incomplete sampling strategies. We investigate and evaluate the impact of interactions on sharing and scalability to current testing approaches using several small benchmark programs. We show that sharing reduces the number of executed instructions significantly, while the effort to execute them usually stays low. We applied VarexJ to several larger real-world applications to illustrate typical sharing potential in real-world applications. Finally, we found that options high interact on data in these systems, but only local, orthogonal and rare.
VarexJ is based on JavaPathfinder v7.0 (rev 1155+).
Use gradle to build the project (e.g., ./gradlew build)
Within Eclipse you can use the gradle plugin: gradle
factory=BDDchoice=TreeChoicefeaturemodel="path"\model.dimacsinvocation=true
import gov.nasa.jpf.annotation.Conditional;
@Conditional
static A = true;
@Conditional
static B = true;
void run() {
int i = 0;
i = i + 2;
if (A) {
i++;
}
i = i * 2;
if (B) {
i = deci(i);
}
i = i - 1;
}
int deci(int k) {
return k / 10;
}
A and B are used as if they have both values true and false.
a) as test: see test package “cmu.*”, it contains several examples for variability-aware execution
b) as JVM via command line:
java -jar ..\RunJPF.jar +native_classpath=.."path to VarexJ"\lib\* +search.class=.search.RandomSearch +featuremodel="path to the feature model"\model.dimacs +choice=TreeChoice +factory=BDD +classpath="path to the application"\bin\ A.B.Main args
We did several experiments on small bechmark programms to analyze the scalability of variability-aware execution compared to other approaches. All benchmarks are adjusteble to the number of options from 0 to 100. The performance measures are shown in the three right diagrams. The most diagramms in the second column show how the options interact on data and on the program flow, and the resulting essential configuration complexity (i.e., essential effort to execute instructions cuased by interactions).
We compared the scalability of VarexJ with the following tools:
JavaPathfinder (JPF), JPF-symbolic, JPF-bdd, SPLat(source code)
µJVM, average execution time with the JVM - µVarexJ - average execution time
for VarexJ without conditional options, overhead to JVM in parentheses - maxVarexJ - maximal execution time for VarexJ
without conditional options - VA VarexJ, time for variability-aware execution, overhead to µVarexJ and maxVarexJ in
parentheses.
Distributions of essential configuration complexity (blue bars), interaction degree on data (green bars), and interaction degree on the control flow (red line):
Feature Interaction Study:
Jens Meinicke, Chu-Pan Wong, Christian Kästner, Thomas Thüm, and Gunter Saake. On Essential Configuration Complexity: Measuring Interactions In Highly-Configurable Systems. In Proceedings of the International Conference on Automated Software Engineering (ASE), volume 31. ACM, September 2016. accepted.
Tool Description:
Jens Meinicke. VarexJ: A Variability-Aware Interpreter for Java Application. Master’s thesis, University of Magdeburg, Germany, December 2014.
Thesis:
Jens Meinicke. Variational Debugging: Understanding Differences among Executions. PhD thesis, University of Magdeburg, January 2019.
gh-pages/README.md