Kengo's blog

Technical articles about original projects, JVM, Static Analysis and JavaScript.

Creating tree walker for accessing parsed words

 In the last article I generated a simple parser, but I cannot access parsed words. It means that I cannot know 'what is provided as S'. It's very big defect so I had to try to solve this problem.
 To access parsed words, dyn.js uses 'tree walker'. I researched how to use it, and found that it's easy :)

Create grammar file in src/main/antlr

 At first, I created a simple grammar file for tree walker. I named this file 'StatementWalker.g'.

tree grammar StatementWalker;

options
{
	ASTLabelType = CommonTree;
}

tokens
{
	S = 'Tomcat';
	V = 'runs';
}

@header
{
	package jp.skypencil.antlr;
}

statement returns [java.util.Map<String, String> map = new java.util.HashMap<String, String>();]
	: (s=S v=V)
	{
		$map.put("S", s.getText());
		$map.put("V", v.getText());
	};

 Key point is [java.util.Map ... ] and '{ $map.put... }'. They create a HashMap instance and put values into this instance. It realizes accessing parsed words via generated code.
 BTW, there are some deprecated definitions of token like S and V. I already wrote them in Statement.g which was created in last article. You can delete them by using tokenVocab option if you need.

 ANTLR3 creates StatementWalker class from this grammar file. Then, I can create its instance like this.

statement_return statement = parser.statement();
CommonTree tree = (CommonTree) statement.getTree();
CommonTreeNodeStream treeNodeStream = new CommonTreeNodeStream(tree);
treeNodeStream.setTokenStream(tokens);

StatementWalker walker = new StatementWalker(treeNodeStream);
Map<String, String> map = walker.statement();
assertThat(map.get("S"), is(equalTo("Tomcat")));
assertThat(map.get("V"), is(equalTo("runs")));

 It works, but using Map isn't cool, is it? So I want to change it to original class.

Use original class for returned value

 How to use your original class is very intuitive. Just changing Map and HashMap to yours.

statement returns [Statement st = new Statement();]
	: (s=S v=V)
	{
		$st.setS(s.getText());
		$st.setV(v.getText());
	};

 Definition of Statement class is here. It is a simple POJO.

public class Statement {
	private String s;
	private String v;
	// setters and getters...
}

 Finally I reached how to handle parsed words with ANTLR3. Next, I'll try to enhance my grammar.