Posted by: Morten Nobel-Jørgensen | July 16, 2008

Using Eclipse compiler to create dynamic Java objects


In this blog I’ll describe how I used the Eclipse compiler for compiling Java source on-the-fly to create instant ‘view’ of the result.

More precisely I’ll show how to compile and invoke the following simple HalloWorld class on the fly:

public class HalloWorldTest{   
	public static void main(String[] args) {       
		System.out.println("Hallo world");   
	}
}


Choosing compiler

Actually there are several Java compilers available; In Java 1.6 the javax.tools.JavaCompiler interface was introduced to give a uniform way to access a Java compiler. The following factory method gives access the Suns javacompiler (when launched from Suns JDK).

JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();

However to gain access to the JDK compiler you need to run your application from the JDK, and since this is not default behaviour, I choose to use the Eclipse compiler instaid. (Besides the Eclipse compiler share the same interface, so the two compilers should behave similar).

Using Eclipse compiler

The first step is to download the Eclipse compiler. The Eclipse project is an extremely large project, but you are able to download the compiler as a part of the Eclipse Development Tools Core package.

The package is found at http://download.eclipse.org/eclipse/downloads/. Click on latest release – and scroll down to the JDT Core Batch Compiler section. The file you are looking for is called ecj-3.4.jar (where 3.4 is the version number).

With this file in your classpath, you are able access the Eclipse compiler using the simple assignment:

JavaCompiler javac = new EclipseCompiler();

The JavaCompiler interface

You start the compilation using creating a CompilationTask object that you invoke the method call on it:

JavaCompiler.CompilationTask compile = javac.getTask(out, fileManager, dianosticListener, options, classes, compilationUnits);
Boolean res = compile.call();

If the res is true, then all files have been compiled without any errors.

Feeding the source to the compiler

I want to compile the source code in memory, and have therefore created a MemorySource class that implements the JavaFileObject (that is passed to the java compiler as compilationUnits).

class MemorySource extends SimpleJavaFileObject {   
	private String src;   
	public MemorySource(String name, String src) {       
		super(URI.create("file:///" + name + ".java"), Kind.SOURCE);       
		this.src = src;  
	}   
	public CharSequence getCharContent(boolean ignoreEncodingErrors) {       
		return src;   
	}   
	public OutputStream openOutputStream() {       
		throw new IllegalStateException();   
	}   
	public InputStream openInputStream() {       
		return new ByteArrayInputStream(src.getBytes());   
	}
}

Note the URI must start with the prefix ‘file:’ for the Eclipse compiler to accept the file.

Receiving the compiled bytecode

The default behaviour of the compiler is to create the result in actual files, but I want to keep the compiled classes in memory and be able to load these classes as well.

For this I need two things:

  • A custom class loader, that stores compiled classes in memory and loads the compiled classes from memory
  • A custom JavaFileManager that deletages the storage of the compiled classes to the custom classloader
class SpecialClassLoader extends ClassLoader {   
	private Map<String,MemoryByteCode> m = new HashMap<String, MemoryByteCode>();

	protected Class<?> findClass(String name) throws ClassNotFoundException {       
		MemoryByteCode mbc = m.get(name);       
		if (mbc==null){           
			mbc = m.get(name.replace(".","/"));           
			if (mbc==null){               
				return super.findClass(name);           
			}       
		}       
		return defineClass(name, mbc.getBytes(), 0, mbc.getBytes().length);   
	}

	public void addClass(String name, MemoryByteCode mbc) {       
		m.put(name, mbc);   
	}
}

This simple classloader adds the compiled classes in a Map using the addClass(…) method, and loads class from memory in findClass(…) method.

class SpecialJavaFileManager extends ForwardingJavaFileManager<StandardJavaFileManager> {   
	private SpecialClassLoader xcl;   
	public SpecialJavaFileManager(StandardJavaFileManager sjfm, SpecialClassLoader xcl) {       
		super(sjfm);       
		this.xcl = xcl;   
	}   
	public JavaFileObject getJavaFileForOutput(Location location, String name, JavaFileObject.Kind kind, FileObject sibling) throws IOException {       
		MemoryByteCode mbc = new MemoryByteCode(name);       
		xcl.addClass(name, mbc);       
		return mbc;   
	}

	public ClassLoader getClassLoader(Location location) {       
		return xcl;   
	}
}

Putting it all together

The following method will use the classes described above to compile java source code in memory and return a class with the result.

private static Class compileClass(String halloWorldProgram, String className) {   
	try{       
		JavaCompiler javac = new EclipseCompiler();

		StandardJavaFileManager sjfm = javac.getStandardFileManager(null, null, null);                 
		SpecialClassLoader cl = new SpecialClassLoader();       
		SpecialJavaFileManager fileManager = new SpecialJavaFileManager(sjfm, cl);       
		List<String> options = Collections.emptyList();       
		List<MemorySource> compilationUnits = Arrays.asList(new MemorySource(className, halloWorldProgram));
		DiagnosticListener<JavaFileObject> dianosticListener = null;       
		Iterable<String> classes = null;       
		Writer out = new PrintWriter(System.err);       
		JavaCompiler.CompilationTask compile = javac.getTask(out, fileManager, dianosticListener, options, classes, compilationUnits);       
		boolean res = compile.call();       
		if (res){           
			return cl.findClass(className);        
		}   
	}    catch (Exception e){       
		e.printStackTrace();   
	}   
	return null;
}

I don’t use the DiagnosticListener – instead I use System.err (wrapped in a PrintWriter) for error reporting.

Finally I’m ready for my testprogram to run:

private final static String HALLO_WORLD_CLASS_NAME = "HalloWorldTest";
private final static String HALLO_WORLD_SOURCE =
	"public class "+HALLO_WORLD_CLASS_NAME+"{\n" +
	"    public static void main(String[] args) {\n" +       
	"        System.out.println(\"Hallo world\");\n" +       
	"    }\n" +       
	"}";

public static void main(String[] args) {   
	Class compiledClass = compileClass(HALLO_WORLD_SOURCE, HALLO_WORLD_CLASS_NAME);   
	if (compiledClass==null){       
		return;   
	}   
	try{       
		Method m = compiledClass.getMethod("main",String[].class);       
		m.invoke(null, new Object[]{null});   
	} catch (Exception e) {       
		e.printStackTrace();   
	}
}

When invoked the famous “Hallo world” is printed on my screen.

The full sourcecode to the program is listed here:

import org.eclipse.jdt.internal.compiler.tool.EclipseCompiler;

import javax.tools.*;
import java.lang.reflect.Method;
import java.util.*;
import java.io.*;
import java.net.URI;

public class DynamicHalloWorld {   
	private final static String HALLO_WORLD_CLASS_NAME = "HalloWorldTest";   
	private final static String HALLO_WORLD_SOURCE =
		"public class "+HALLO_WORLD_CLASS_NAME+"{\n" +           
		"    public static void main(String[] args) {\n" +           
		"        System.out.println(\"Hallo world\");\n" +           
		"    }\n" +           
		"}";

	public static void main(String[] args) {       
		Class compiledClass = compileClass(HALLO_WORLD_SOURCE, HALLO_WORLD_CLASS_NAME);       
		if (compiledClass==null){           
			return;       
		}       
		try{           
			Method m = compiledClass.getMethod("main",String[].class);           
			m.invoke(null, new Object[]{null});       
		}
		catch (Exception e) {           
			e.printStackTrace();       
		}   
	}

	private static Class compileClass(String halloWorldProgram, String className) {       
		try{           
			JavaCompiler javac = new EclipseCompiler();

			StandardJavaFileManager sjfm = javac.getStandardFileManager(null, null, null);
			SpecialClassLoader cl = new SpecialClassLoader();           
			SpecialJavaFileManager fileManager = new SpecialJavaFileManager(sjfm, cl);           
			List options = Collections.emptyList();

			List compilationUnits = Arrays.asList(new MemorySource(className, halloWorldProgram));           
			DiagnosticListener dianosticListener = null;           
			Iterable classes = null;           
			Writer out = new PrintWriter(System.err);           
			JavaCompiler.CompilationTask compile = javac.getTask(out, fileManager, dianosticListener, options, classes, compilationUnits);           
			boolean res = compile.call();           
			if (res){               
				return cl.findClass(className);           
			}       
		} catch (Exception e){           
			e.printStackTrace();       
		}       
		return null;   
	}
}

class MemorySource extends SimpleJavaFileObject {   
	private String src;   
	public MemorySource(String name, String src) {       
		super(URI.create("file:///" + name + ".java"), Kind.SOURCE);       
		this.src = src;   
	}   
	public CharSequence getCharContent(boolean ignoreEncodingErrors) {       
		return src;   
	}   
	public OutputStream openOutputStream() {       
		throw new IllegalStateException();   
	}   
	public InputStream openInputStream() {       
		return new ByteArrayInputStream(src.getBytes());   
	}
}

class SpecialJavaFileManager extends ForwardingJavaFileManager {   
	private SpecialClassLoader xcl;   
	public SpecialJavaFileManager(StandardJavaFileManager sjfm, SpecialClassLoader xcl) {       
		super(sjfm);       
		this.xcl = xcl;   
	}   
	public JavaFileObject getJavaFileForOutput(Location location, String name, JavaFileObject.Kind kind, FileObject sibling) throws IOException {       
		MemoryByteCode mbc = new MemoryByteCode(name);       
		xcl.addClass(name, mbc);       
		return mbc;   
	}

	public ClassLoader getClassLoader(Location location) {       
		return xcl;   
	}
}

class MemoryByteCode extends SimpleJavaFileObject {   
	private ByteArrayOutputStream baos;   
	public MemoryByteCode(String name) {       
		super(URI.create("byte:///" + name + ".class"), Kind.CLASS);   
	}   
	public CharSequence getCharContent(boolean ignoreEncodingErrors) {       
		throw new IllegalStateException();   
	}   
	public OutputStream openOutputStream() {       
		baos = new ByteArrayOutputStream();       
		return baos;   
	}   
	public InputStream openInputStream() {       
		throw new IllegalStateException();   
	}   
	public byte[] getBytes() {       
		return baos.toByteArray();   
	}
}

class SpecialClassLoader extends ClassLoader {   
	private Map<String,MemoryByteCode> m = new HashMap<String, MemoryByteCode>();

	protected Class<?> findClass(String name) throws ClassNotFoundException {       
		MemoryByteCode mbc = m.get(name);       
		if (mbc==null){           
			mbc = m.get(name.replace(".","/"));           
			if (mbc==null){               
				return super.findClass(name);           
			}       
		}       
		return defineClass(name, mbc.getBytes(), 0, mbc.getBytes().length);   
	}

	public void addClass(String name, MemoryByteCode mbc) {       
		m.put(name, mbc);   
	}
}

More advanced example

For a more advanced example checkout my Scenegraph Shell 0.2. The Scenegraph Shell is a code editor with a visual representation of what you code. Read more about Java Scenegraph
here: https://scenegraph.dev.java.net/ and read more about the project in my two previous blogs: Scenegraph Shell and Scenegraph Shell 0.2.

(Sources: I used a post on a Danish programming forum as inspiration for this blog. www.eksperten.dk )

Advertisements

Responses

  1. […] using the Eclipse compiler in Java Web Start. If you didn’t read (or disliked) my blog about Using Eclipse compiler to create dynamic Java objects I suggest that you skip the last […]

  2. Hello!
    I tried your sourcecodes and got an error:

    Exception in thread “main” java.lang.Error: Unresolved compilation problem:
    The constructor Object(URI, JavaFileObject.Kind) is undefined

    at MemorySource.(MemorySource.java:20)
    at DynamicHelloWorld.compileClass(DynamicHelloWorld.java:52)
    at DynamicHelloWorld.main(DynamicHelloWorld.java:27)

    super(URI.create(“file:///” + name + “.java”), Kind.SOURCE) might be the problem because Eclipse says that the constructor Object(URI, JavaFileObject.Kind) is undefined.

    How can I solve this problem?

  3. Thanks a lot for this post! It helped me on my latest project (the user will enter code that needs to be compiled on-the-fly).

  4. […] Many times you’ll probably want to load the class too, without saving the bytecode to memory. In this case, you need to use a custom <code>ClassLoader</code> to define the class for you and then instantiate it. I didn’t come up with this by myself, most of the code I used is from here: https://blog.nobel-joergensen.com/2008/07/16/using-eclipse-compiler-to-create-dynamic-java-objects-2/&#8230;. […]

  5. Thanks for such a complete example! Even now, 7 years since you wrote this, you’re still helping people.
    I’ve used it in an automated tests suite to exercise the output from my code generator utility.
    FYI, since it’s a tool for developers, I can be sure there will always be a JDK. So I used getSystemJavaCompiler(), and it all worked fine.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

%d bloggers like this: