Fork me on GitHub

VarexJ

Highly-configurable systems allow users to achieve their specific needs. Such software systems are flexible, but come with difficulties for program analyses, as it is usually not possible to test all combinations of options separately due to the configuration space explosion. Variability-aware approaches exploit redundancies among configurations to share analysis efforts. Variability-aware execution aggressively shares computations and data in program execution across multiple configurations, for example, when executing a test over all configurations. By maximizing sharing, the approach can outperform traditional approaches by orders of magnitude on many highly-configurable systems, without falling back to incomplete sampling strategies. We investigate and evaluate the impact of interactions on sharing and scalability to current testing approaches using several small benchmark programs. We show that sharing reduces the number of executed instructions significantly, while the effort to execute them usually stays low. We applied VarexJ to several larger real-world applications to illustrate typical sharing potential in real-world applications. Finally, we found that options high interact on data in these systems, but only local, orthogonal and rare.

VarexJ is based on JavaPathfinder v7.0 (rev 1155+).

Usage

Build

Use gradle to build the project (e.g., ./gradlew build)

Within Eclipse you can use the gradle plugin: gradle

VarexJ options

Specify conditional boolean fields

VAriability-Aware Trace

import gov.nasa.jpf.annotation.Conditional;

@Conditional
static A = true;
@Conditional
static B = true;

void run() {
	int i = 0;
	i = i + 2;
	if (A) {
		i++; 
	}
	i = i * 2;
	if (B) {
		i = deci(i);
	}
	i = i - 1;
}
int deci(int k) {
	return k / 10;
}

A and B are used as if they have both values true and false.

Run VarexJ

a) as test: see test package “cmu.*”, it contains several examples for variability-aware execution

b) as JVM via command line:

java -jar ..\RunJPF.jar +native_classpath=.."path to VarexJ"\lib\* +search.class=.search.RandomSearch +featuremodel="path to the feature model"\model.dimacs +choice=TreeChoice +factory=BDD +classpath="path to the application"\bin\ A.B.Main args

Scalability

Interaction Benchmarks

We did several experiments on small bechmark programms to analyze the scalability of variability-aware execution compared to other approaches. All benchmarks are adjusteble to the number of options from 0 to 100. The performance measures are shown in the three right diagrams. The most diagramms in the second column show how the options interact on data and on the program flow, and the resulting essential configuration complexity (i.e., essential effort to execute instructions cuased by interactions).

Benchmarks

We compared the scalability of VarexJ with the following tools:

JavaPathfinder (JPF), JPF-symbolic, JPF-bdd, SPLat(source code)

Study on overhead and efficiency of VarexJ

Teable µJVM, average execution time with the JVM - µVarexJ - average execution time for VarexJ without conditional options, overhead to JVM in parentheses - maxVarexJ - maximal execution time for VarexJ without conditional options - VA VarexJ, time for variability-aware execution, overhead to µVarexJ and maxVarexJ in parentheses.

Understanding Interactions

Distributions of essential configuration complexity (blue bars), interaction degree on data (green bars), and interaction degree on the control flow (red line):

Elevator

Elevator

Mine Pump

Elevator

E-Mail

E-Mail

GPL

GPL

ZipMe

ZipMe

QuEval

QuEval

Prevayler

Prevayler

Checkstyle

Checkstyle

Jetty

Jetty

Credits

Publications

Feature Interaction Study:

Jens Meinicke, Chu-Pan Wong, Christian Kästner, Thomas Thüm, and Gunter Saake. On Essential Configuration Complexity: Measuring Interactions In Highly-Configurable Systems. In Proceedings of the International Conference on Automated Software Engineering (ASE), volume 31. ACM, September 2016. accepted.

Tool Description:

Jens Meinicke. VarexJ: A Variability-Aware Interpreter for Java Application. Master’s thesis, University of Magdeburg, Germany, December 2014.

gh-pages/README.md

gh-pages/index.html