Develop Reference

Logical problems causing multiple instances of a note to be created

Here is the (without most of the functions) definition of a class called note.
public class Note
{
private String text;
String fileName = "";
NoteManager noteManager = null;
List<String> hyperlinks = new ArrayList<String>();
public static final int BUFFER_SIZE = 512;
public Note(NoteManager noteManager) {
this.noteManager = noteManager;
this.text = "";
}
public Note(NoteManager noteManager, String content) {
this(noteManager);
if (content == null)
setText("");
else
setText(content);
}
public Note(NoteManager noteManager, CharSequence content) {
this(noteManager, content.toString());
}
....some functions....
public static Note newFromFile(NoteManager noteManager, Context context,
String filename) throws IOException
{
FileInputStream inputFileStream = context.openFileInput(filename);
StringBuilder stringBuilder = new StringBuilder();
byte[] buffer = new byte[BUFFER_SIZE];
int len;
while ((len = inputFileStream.read(buffer)) > 0)
{
String line = new String(buffer, 0, len);
stringBuilder.append(line);
buffer = new byte[Note.BUFFER_SIZE];
}
Note n = new Note(noteManager, stringBuilder.toString().trim());
n.fileName = filename;
inputFileStream.close();
return n;
}
.... some functions attributed to this class
}
These notes are managed by a class called NoteManager.java, which I have abbreviated below:
public class NoteManager
{
Context context=null;
ArrayList<Note> notes = new ArrayList<Note>();
..... some functions...
public void addNote(Note note)
{
if (note == null || note.noteManager != this || notes.contains(note)) return;
note.noteManager = this;
notes.add(note);
try
{
note.saveToFile(context);
} catch (IOException e)
{
e.printStackTrace();
}
}
....some functions....
public void loadNotes()
{
String[] files = context.fileList();
notes.clear();
for (String fname:files)
{
try
{
notes.add(Note.newFromFile(this, context, fname));
} catch (IOException e)
{
e.printStackTrace();
}
}
}
}
public void addNote(Note note)
{
if (note == null || notes.contains(note)) return;
note.noteManager = this;
notes.add(note);
try
{
note.saveToFile(context);
} catch (IOException e)
{
e.printStackTrace();
}
}
I am trying to work out why this notepad app creates random new notes when the app is fully shutdown and then reopened, however I just cannot see what the problem is. I have cut out all the functions which didnt seem to relate to the problem, so the logical error must be here somewhere.
How does one go about finding what I am guessing to be some kind of circular reference or lack of checks?

Android typically uses UTF-8, with multi-byte characters. Creating a new String on a arbitrary byte sub-array can have issues at begin and end, if you deviate from ASCII.
public static Note newFromFile(NoteManager noteManager, Context context,
String filename) throws IOException
{
Path path = Paths.get(filename);
byte[] bytes = Files.readAllBytes(path);
String content = new String(bytes, "UTF-8");
Note n = new Note(noteManager, content.trim());
n.fileName = filename;
noteManager.add(n); // One registration?
return n;
}
The problem of having multiple instances of a node might need the addition within newFromFile or maybe an extra check:
public void addNote(Note note)
{
if (note == null || note.noteManager != this || notes.contains(note)) {
return;
}
note.noteManager = this;
notes.add(note);
And finally a Note must be well defined.
public class Note extends Comparable<Note> {
private NoteManager noteManager:
private final String content; // Immutable.
public NoteManager(NoteManager noteManager, String content) {
this.noteManager = noteManager;
this.content = content;
}
... compare on the immutable content
... hashCode on content
Not being to be able to change the content, and comparing on the string content, means notes cannot be doubled, change in the set, mixing up the set ordering.

Is the following code thread safe?

I think I have implemented the Double-checked locking pattern but not sure if it safe or it works as intended. Any other logic to implement the same would be really helpful.
public class OnProperties {
private static String dfltPropertyFile = "on.properties";
private static long refreshSecs = 120L;
private static Properties props;
private static long lastReadTimestamp = 0;
public static String getProperty(String propertyName, String dfltValue) {
long currentTimestamp = System.currentTimeMillis() / 1000L;
if (props == null
|| (refreshSecs > 0 && (currentTimestamp - lastReadTimestamp) > refreshSecs)) {
synchronized (props) {
if (props == null
|| (refreshSecs > 0 && (currentTimestamp - lastReadTimestamp) > refreshSecs)) {
lastReadTimestamp = currentTimestamp;
try {
loadProperties(dfltPropertyFile);
refreshSecs = getProperty("on.properties.refresh", 120L);
if (refreshSecs < 0L) {
refreshSecs = 0L;
}
} catch (Exception e) {
refreshSecs = 600L;
}
}
}
}
if (props == null) {
return dfltValue;
}
String propertyValue = props.getProperty(propertyName, dfltValue);
return propertyValue;
}
public static boolean getProperty(String propertyName, boolean dfltValue) {
boolean value = dfltValue;
String strValue = getProperty(propertyName, (String) null);
if (strValue != null) {
try {
value = Boolean.parseBoolean(strValue);
} catch (NumberFormatException e) {
// just keep the default
}
}
return value;
}
private static void loadProperties(String p_propertiesFile)
throws java.io.IOException, java.io.FileNotFoundException {
InputStream fileStream = new FileInputStream(p_propertiesFile);
props = new Properties();
props.load(fileStream);
fileStream.close();
}
}
Generally multiple threads running often access the "getProperty" method as follows:
extDebug = OnProperties.getProperty("on.extdebug", false);

Atomic values guarantee to always return the complete latest value to all threads. This prevents a number of multi-threading issues in this case. A bit of synchronization is still required, but it can be limited to a minimum. See my implementation below:
import java.io.File;
import java.io.FileInputStream;
import java.util.Properties;
import java.util.concurrent.atomic.AtomicLong;
import java.util.concurrent.atomic.AtomicReference;
public class OnProperties {
private static int refreshIntervalDefaultSecs;
private static int refreshIntervalOnErrorSecs;
static {
setRefreshInterval(120);
}
private static final AtomicReference<Properties> propsRef = new AtomicReference<Properties>(new Properties());
private static final AtomicLong nextPropsLoad = new AtomicLong(0L);
private static final Object loadLock = new Object();
private static String dfltPropertyFile = "on.properties";
public static String getProperty(String key, String defaultValue) {
String value = getProperty(key);
if (value == null) {
value = defaultValue;
}
return value;
}
private static String getProperty(String key) {
reloadWhenNeeded();
return propsRef.get().getProperty(key);
}
private static void reloadWhenNeeded() {
long now = System.currentTimeMillis();
if (now > nextPropsLoad.get()) {
boolean reload = false;
synchronized(loadLock) {
if (now > nextPropsLoad.get()) {
// need loadLock because there is time between previous get()
// and next set()
updateNextPropsLoad(now, refreshIntervalDefaultSecs);
reload = true;
}
}
if (reload) {
reloadProps(now);
}
}
}
private static void updateNextPropsLoad(long now, int nextRefreshSecs) {
nextPropsLoad.set(now + nextRefreshSecs * 1000);
}
private static void reloadProps(long now) {
Properties p = new Properties();
FileInputStream in = null;
System.out.println("Reloading from " + new File(dfltPropertyFile).getAbsolutePath());
try {
p.load(in = new FileInputStream(new File(dfltPropertyFile)));
propsRef.set(p);
setRefreshInterval(getProperty("on.properties.refresh", 120));
updateNextPropsLoad(now, refreshIntervalDefaultSecs);
} catch (Exception e) {
updateNextPropsLoad(now, refreshIntervalOnErrorSecs);
} finally {
try { if (in != null) in.close(); } catch (Exception e) {
updateNextPropsLoad(now, refreshIntervalOnErrorSecs);
}
}
}
private static void setRefreshInterval(int refreshSecs) {
if (refreshSecs < 1) {
refreshSecs = 120;
}
refreshIntervalDefaultSecs = refreshSecs;
refreshIntervalOnErrorSecs = 5 * refreshSecs;
}
public static boolean getProperty(String key, boolean defaultValue) {
boolean value = defaultValue;
String svalue = getProperty(key);
if (svalue != null) {
try {
value = Boolean.valueOf(svalue);
} catch (Exception ignored) {}
}
return value;
}
public static int getProperty(String key, int defaultValue) {
int value = defaultValue;
String svalue = getProperty(key);
if (svalue != null) {
try {
value = Integer.valueOf(svalue);
} catch (Exception ignored) {}
}
return value;
}
public static void main(String[] args) {
System.out.println("Refresh value from file: " + getProperty("on.properties.refresh", 120));
System.out.println("No reload " + getProperty("does.not.exist", true));
System.out.println("Next reload after " + ((nextPropsLoad.get() - System.currentTimeMillis()) / 1000) + " seconds.");
}
}
One drawback of the implementation is that one thread will get slowed down when it is selected to reload the properties from file. A better approach would be to create a 'watchdog' thread/scheduled task that checks every (for example) five seconds if the properties-file has a changed modification date and then trigger a reload (in which case the AtomicReference for the Properties still comes in handy).
Also keep in mind that there is a logical threading issue: if property values are interrelated (i.e. one value is only correct if another value is also updated), a reload could present a thread with old and new values that should not be mixed. The only way around that is to keep a reference to one set of properties in methods that use the interrelated values of the properties (and a class like this with static methods and variables is not handy in such a situation).

It is not safe as you have multiple variables which are read in a way which is not thread safe (i.e. access is not synchronized and they are not volatile).
It appears the workflow is mostly reads with a few writes. I would suggest using a ReentrantReadWriteLock to synchronize access.

To have this working correctly with double-checked locking you must do two things:
private static Properties props must be declared volatile;
as already mentioned, synchronised(props) won't work in case props are null - you need to declare a special lock object field:
.
private static final Object propsLockObject = new Object();
...
synchronized(propsLockObject) {
...
P.S. The lastReadTimestamp won't work also unless declared volatile. Though this is not about double-checked locking anymore.

To reload the properties, you don't need to re-initialize the props variable. Initialize the properties during the declaration statement itself will do. This will solve the problem of synchronizing with null.
Remove the initialization code in the loadProperties block.
remove the prop==null check outside and inside the synchronized block.
Once that is done, your code will work exactly the way you want.
public class OnProperties {
private static String dfltPropertyFile = "on.properties";
private static long refreshSecs = 120L;
private static Properties props = new Properties();
private static long lastReadTimestamp = 0;
public static String getProperty(String propertyName, String dfltValue) {
long currentTimestamp = System.currentTimeMillis() / 1000L;
if (refreshSecs > 0 && (currentTimestamp - lastReadTimestamp) > refreshSecs) {
synchronized (props) {
if (refreshSecs > 0 && (currentTimestamp - lastReadTimestamp) > refreshSecs) {
lastReadTimestamp = currentTimestamp;
try {
loadProperties(dfltPropertyFile);
refreshSecs = getProperty("on.properties.refresh", 120L);
if (refreshSecs < 0L) {
refreshSecs = 0L;
}
} catch (Exception e) {
refreshSecs = 600L;
}
}
}
}
String propertyValue = props.getProperty(propertyName, dfltValue);
return propertyValue;
}
public static boolean getProperty(String propertyName, boolean dfltValue) { boolean value = dfltValue;
String strValue = getProperty(propertyName, (String) null);
if (strValue != null) {
try {
value = Boolean.parseBoolean(strValue);
} catch (NumberFormatException e) {
// just keep the default
}
}
return value;
}
private static void loadProperties(String p_propertiesFile) throws java.io.IOException, java.io.FileNotFoundException { InputStream fileStream = new FileInputStream(p_propertiesFile); props.load(fileStream); fileStream.close(); } }

Please accept that the double-checked locking idiom is broken and does not work (i.e. does not synchronize properly). Even if you make it work using volatile (at the right place), it is far too complex for what you get.
So my suggestion: Simply synchronize everything. Then try and measure. If you find out that OnProperties is the bottleneck, consider more powerful/clever synchronization techniques and come back if necessary:
public class OnProperties {
/* some private fields here */
public static synchronized String getProperty(String propertyName, String dfltValue) {
reloadPropertiesIfNecessary();
return props.getProperty(propertyName, dfltValue);
}
/* other public methods using getProperty come here */
private static void reloadPropertiesIfNecessary() {
// check timestamp etc.
if (/* check timestamp etc. */) {
loadProperties(dfltPropertyFile);
// update timestamp etc.
}
}
private static void loadProperties(String filename) throws IOException {
try (InputStream stream = new FileInputStream(filename)) {
props = new Properties();
props.load(fileStream);
}
}
}

using dbpedia spotlight in java or scala

Does anyone know where to find a little how to on using dbpedia spotlight in java or scala? Or could anyone explain how it's done? I can't find any information on this...

The DBpedia Spotlight wiki pages would be a good place to start.
And I believe the installation page has listed the most popular ways (using a jar, or set up a web service) to use the application.
It includes instructions on using the Java/Scala API with your own installation, or calling the Web Service.
There are some additional data needed to be downloaded to run your own server for full service, good time to make a coffee for yourself.

you need download dbpedia spotlight (jar file) after that u can use next two classes ( author pablomendes ) i only make some change .
public class db extends AnnotationClient {
//private final static String API_URL = "http://jodaiber.dyndns.org:2222/";
private static String API_URL = "http://spotlight.dbpedia.org:80/";
private static double CONFIDENCE = 0.0;
private static int SUPPORT = 0;
private static String powered_by ="non";
private static String spotter ="CoOccurrenceBasedSelector";//"LingPipeSpotter"=Annotate all spots
//AtLeastOneNounSelector"=No verbs and adjs.
//"CoOccurrenceBasedSelector" =No 'common words'
//"NESpotter"=Only Per.,Org.,Loc.
private static String disambiguator ="Default";//Default ;Occurrences=Occurrence-centric;Document=Document-centric
private static String showScores ="yes";
#SuppressWarnings("static-access")
public void configiration(double CONFIDENCE,int SUPPORT,
String powered_by,String spotter,String disambiguator,String showScores){
this.CONFIDENCE=CONFIDENCE;
this.SUPPORT=SUPPORT;
this.powered_by=powered_by;
this.spotter=spotter;
this.disambiguator=disambiguator;
this.showScores=showScores;
}
public List<DBpediaResource> extract(Text text) throws AnnotationException {
LOG.info("Querying API.");
String spotlightResponse;
try {
String Query=API_URL + "rest/annotate/?" +
"confidence=" + CONFIDENCE
+ "&support=" + SUPPORT
+ "&spotter=" + spotter
+ "&disambiguator=" + disambiguator
+ "&showScores=" + showScores
+ "&powered_by=" + powered_by
+ "&text=" + URLEncoder.encode(text.text(), "utf-8");
LOG.info(Query);
GetMethod getMethod = new GetMethod(Query);
getMethod.addRequestHeader(new Header("Accept", "application/json"));
spotlightResponse = request(getMethod);
} catch (UnsupportedEncodingException e) {
throw new AnnotationException("Could not encode text.", e);
}
assert spotlightResponse != null;
JSONObject resultJSON = null;
JSONArray entities = null;
try {
resultJSON = new JSONObject(spotlightResponse);
entities = resultJSON.getJSONArray("Resources");
} catch (JSONException e) {
//throw new AnnotationException("Received invalid response from DBpedia Spotlight API.");
}
LinkedList<DBpediaResource> resources = new LinkedList<DBpediaResource>();
if(entities!=null)
for(int i = 0; i < entities.length(); i++) {
try {
JSONObject entity = entities.getJSONObject(i);
resources.add(
new DBpediaResource(entity.getString("#URI"),
Integer.parseInt(entity.getString("#support"))));
} catch (JSONException e) {
LOG.error("JSON exception "+e);
}
}
return resources;
}
}
second class
/**
* #author pablomendes
*/
public abstract class AnnotationClient {
public Logger LOG = Logger.getLogger(this.getClass());
private List<String> RES = new ArrayList<String>();
// Create an instance of HttpClient.
private static HttpClient client = new HttpClient();
public List<String> getResu(){
return RES;
}
public String request(HttpMethod method) throws AnnotationException {
String response = null;
// Provide custom retry handler is necessary
method.getParams().setParameter(HttpMethodParams.RETRY_HANDLER,
new DefaultHttpMethodRetryHandler(3, false));
try {
// Execute the method.
int statusCode = client.executeMethod(method);
if (statusCode != HttpStatus.SC_OK) {
LOG.error("Method failed: " + method.getStatusLine());
}
// Read the response body.
byte[] responseBody = method.getResponseBody(); //TODO Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommended.
// Deal with the response.
// Use caution: ensure correct character encoding and is not binary data
response = new String(responseBody);
} catch (HttpException e) {
LOG.error("Fatal protocol violation: " + e.getMessage());
throw new AnnotationException("Protocol error executing HTTP request.",e);
} catch (IOException e) {
LOG.error("Fatal transport error: " + e.getMessage());
LOG.error(method.getQueryString());
throw new AnnotationException("Transport error executing HTTP request.",e);
} finally {
// Release the connection.
method.releaseConnection();
}
return response;
}
protected static String readFileAsString(String filePath) throws java.io.IOException{
return readFileAsString(new File(filePath));
}
protected static String readFileAsString(File file) throws IOException {
byte[] buffer = new byte[(int) file.length()];
#SuppressWarnings("resource")
BufferedInputStream f = new BufferedInputStream(new FileInputStream(file));
f.read(buffer);
return new String(buffer);
}
static abstract class LineParser {
public abstract String parse(String s) throws ParseException;
static class ManualDatasetLineParser extends LineParser {
public String parse(String s) throws ParseException {
return s.trim();
}
}
static class OccTSVLineParser extends LineParser {
public String parse(String s) throws ParseException {
String result = s;
try {
result = s.trim().split("\t")[3];
} catch (ArrayIndexOutOfBoundsException e) {
throw new ParseException(e.getMessage(), 3);
}
return result;
}
}
}
public void saveExtractedEntitiesSet(String Question, LineParser parser, int restartFrom) throws Exception {
String text = Question;
int i=0;
//int correct =0 ; int error = 0;int sum = 0;
for (String snippet: text.split("\n")) {
String s = parser.parse(snippet);
if (s!= null && !s.equals("")) {
i++;
if (i<restartFrom) continue;
List<DBpediaResource> entities = new ArrayList<DBpediaResource>();
try {
entities = extract(new Text(snippet.replaceAll("\\s+"," ")));
System.out.println(entities.get(0).getFullUri());
} catch (AnnotationException e) {
// error++;
LOG.error(e);
e.printStackTrace();
}
for (DBpediaResource e: entities) {
RES.add(e.uri());
}
}
}
}
public abstract List<DBpediaResource> extract(Text text) throws AnnotationException;
public void evaluate(String Question) throws Exception {
evaluateManual(Question,0);
}
public void evaluateManual(String Question, int restartFrom) throws Exception {
saveExtractedEntitiesSet(Question,new LineParser.ManualDatasetLineParser(), restartFrom);
}
}
main()
public static void main(String[] args) throws Exception {
String Question ="Is the Amazon river longer than the Nile River?";
db c = new db ();
c.configiration(0.0, 0, "non", "CoOccurrenceBasedSelector", "Default", "yes");
System.out.println("resource : "+c.getResu());
}

I just add one little fix for your answer.
Your code is running, if you add the evaluate method call:
public static void main(String[] args) throws Exception {
String question = "Is the Amazon river longer than the Nile River?";
db c = new db ();
c.configiration(0.0, 0, "non", "CoOccurrenceBasedSelector", "Default", "yes");
c.evaluate(question);
System.out.println("resource : "+c.getResu());
}
Lamine

In the request method of the second class (AnnotationClient) in Adel's answer, the author Pablo Mendes hasn't finished
TODO Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommended.
which is an annoying warning that needs to be removed by replacing
byte[] responseBody = method.getResponseBody(); //TODO Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommended.
// Deal with the response.
// Use caution: ensure correct character encoding and is not binary data
response = new String(responseBody);
with
Reader in = new InputStreamReader(method.getResponseBodyAsStream(), "UTF-8");
StringWriter writer = new StringWriter();
org.apache.commons.io.IOUtils.copy(in, writer);
response = writer.toString();

Can't add a value to ArrayList because it's in an Inner Class?

I am writing a file that can parse rdf and owl files. I am using SAX and Java.
My problem is on the line activeObject.add(file);
I get the error "Syntax error on tokens, Misplaced construct(s)" - I don't know what this means. And it doesn't seem to make sense, any help would be much appreciated.
PS: I might be completely wrong about what is causing the error, it might have nothing to do with an inner class.
public static void main(String args[]) throws URISyntaxException, MalformedURLException, IOException {
// String file =
// "http://www.srdc.metu.edu.tr/ubl/contextOntology/cpc.owl";
// final String file = "http://onto.eva.mpg.de/obo/image.owl";
// final String file =
// "http://www.informatik.uni-ulm.de/ki/Liebig/owl/SUMO-LiteTB.rdf";
// final String file =
// "http://www.csd.abdn.ac.uk/~apreece/research/TowardsAnIntelligentWeb/PUB_findallbyKen_result.rdf";
// final String file =
// "http://www.srdc.metu.edu.tr/ubl/contextOntology/naics.owl";
final String file = "http://www.w3.org/1999/02/22-rdf-syntax-ns#";
URI uri = new URI(file);
InputStream is = uri.toURL().openStream();
try {
SAXParserFactory factory = SAXParserFactory.newInstance();
SAXParser saxParser = factory.newSAXParser();
DefaultHandler handler = new DefaultHandler() {
ArrayList<String> activeProperty = new ArrayList<String>();
ArrayList<Triple> triplesList = new ArrayList<Triple>();
ArrayList<String> activeObject = new ArrayList<String>();
boolean name = false;
activeObject.add(file);
public void startElement(String uri, String localName,String qName, Attributes attributes) throws SAXException {
activeProperty.add(qName);
int attrLength = attributes.getLength();
for (int i = 0; i < attrLength; i++) {
String attrName = attributes.getQName(i).toLowerCase();
String attrValue = attributes.getValue(i).toLowerCase();
if (attrName.equals("rdf:about") || attrName.equals("rdf:resource") || attrName.equals("rdf:id")) {
activeObject.add(attrValue);
System.out.println(activeObject);
}
else{
String subject = activeObject.get(activeObject.size()-1);
String predicate = attrName;
String object = attrValue;
Triple newTriple = new Triple(subject, predicate, object);
}
}
}
public void characters(char[] ch, int start, int length) throws SAXException {
// tempVal = new String(ch, start, length);
}
public void endElement(String uri, String localName, String rawName) {
String subject = activeObject.get(activeObject.size()-2);
String predicate = activeProperty.get(activeProperty.size()-1);
String object = activeObject.get(activeObject.size()-1);
Triple newTriple = new Triple(subject,predicate, object);
if (rawName.equals(activeProperty.get(activeProperty.size() - 1))) {
activeProperty.remove(activeProperty.size() - 1);
}
else{
System.out.println("Something is seriosuly wrong ...sdf.sdf.we8ryw98fsydh");
}
// System.out.println(activeProperty);
}
// if (qName.equalsIgnoreCase("NAME")) {
// name = true;
// }
// public void characters(char ch[], int start, int length)
// throws SAXException {
// if (name) {
// System.out.println("Name: "
// + new String(ch, start, length));
// name = false;
// }
// }
};
saxParser.parse(is, handler);
} catch (Exception e) {
e.printStackTrace();
}
}

You're trying to write normal statements within an anonymous class, as if you'd tried this:
class Foo extends DefaultHandler {
ArrayList<String> activeProperty = new ArrayList<String>();
ArrayList<Triple> triplesList = new ArrayList<Triple>();
ArrayList<String> activeObject = new ArrayList<String>();
boolean name = false;
activeObject.add(file);
}
You can't do that. If you want this to be performed on construction, you can put it in an initializer block, like this:
ArrayList<String> activeProperty = new ArrayList<String>();
ArrayList<Triple> triplesList = new ArrayList<Triple>();
ArrayList<String> activeObject = new ArrayList<String>();
boolean name = false;
{
activeObject.add(file);
}
Or you could populate the list in some other way, perhaps - for instance with Guava you could write:
ArrayList<String> activeObject = Lists.newArrayList(file);

How can I introspect a freemarker template to find out what variables it uses?

I'm not at all sure that this is even a solvable problem, but supposing that I have a freemarker template, I'd like to be able to ask the template what variables it uses.
For my purposes, we can assume that the freemarker template is very simple - just "root level" entries (the model for such a template could be a simple Map). In other words, I don't need to handle templates that call for nested structures, etc.

one other way to get the variables from java. This just tries to process the template and catch the InvalidReferenceException to find all the variables in a freemarker-template
/**
* Find all the variables used in the Freemarker Template
* #param templateName
* #return
*/
public Set<String> getTemplateVariables(String templateName) {
Template template = getTemplate(templateName);
StringWriter stringWriter = new StringWriter();
Map<String, Object> dataModel = new HashMap<>();
boolean exceptionCaught;
do {
exceptionCaught = false;
try {
template.process(dataModel, stringWriter);
} catch (InvalidReferenceException e) {
exceptionCaught = true;
dataModel.put(e.getBlamedExpressionString(), "");
} catch (IOException | TemplateException e) {
throw new IllegalStateException("Failed to Load Template: " + templateName, e);
}
} while (exceptionCaught);
return dataModel.keySet();
}
private Template getTemplate(String templateName) {
try {
return configuration.getTemplate(templateName);
} catch (IOException e) {
throw new IllegalStateException("Failed to Load Template: " + templateName, e);
}
}

This is probably late, but in case anyone else encountered this problem : you can use 'data_model' and 'globals' to inspect the model - data_model will only contain values provided by the model while globals will also contain any variables defined in the template.
You need to prepend the special variables with a dot - so to access globals, use ${.globals}
For other special variables see http://freemarker.sourceforge.net/docs/ref_specvar.html

I had the same task to get the list of variables from template on java side and don't found any good approaches to that except using reflection. I'm not sure whether there is a better way to get this data or not but here's my approach:
public Set<String> referenceSet(Template template) throws TemplateModelException {
Set<String> result = new HashSet<>();
TemplateElement rootTreeNode = template.getRootTreeNode();
for (int i = 0; i < rootTreeNode.getChildCount(); i++) {
TemplateModel templateModel = rootTreeNode.getChildNodes().get(i);
if (!(templateModel instanceof StringModel)) {
continue;
}
Object wrappedObject = ((StringModel) templateModel).getWrappedObject();
if (!"DollarVariable".equals(wrappedObject.getClass().getSimpleName())) {
continue;
}
try {
Object expression = getInternalState(wrappedObject, "expression");
switch (expression.getClass().getSimpleName()) {
case "Identifier":
result.add(getInternalState(expression, "name").toString());
break;
case "DefaultToExpression":
result.add(getInternalState(expression, "lho").toString());
break;
case "BuiltinVariable":
break;
default:
throw new IllegalStateException("Unable to introspect variable");
}
} catch (NoSuchFieldException | IllegalAccessException e) {
throw new TemplateModelException("Unable to reflect template model");
}
}
return result;
}
private Object getInternalState(Object o, String fieldName) throws NoSuchFieldException, IllegalAccessException {
Field field = o.getClass().getDeclaredField(fieldName);
boolean wasAccessible = field.isAccessible();
try {
field.setAccessible(true);
return field.get(o);
} finally {
field.setAccessible(wasAccessible);
}
}
Sample project that I made for demonstrating template introspection can be found on github: https://github.com/SimY4/TemplatesPOC.git

I solved this for my very simple usecase (only using flat datastructure, no nesting (for example: ${parent.child}), lists or more specific) with dummy data provider:
public class DummyDataProvider<K, V> extends HashMap<K, V> {
private static final long serialVersionUID = 1;
public final Set<String> variables = new HashSet<>();
#SuppressWarnings("unchecked")
#Override
public V get(Object key) {
variables.add(key.toString());
return (V) key;
}
}
You can give this for processing to a template and when it finishes Set variables contains your variables.
This is very simplistic approach, which certainly needs improvement, but you get the idea.

public static Set<String> getNecessaryTemplateVariables(String templateName) throws TemplateModelException {
Set<String> result = new HashSet<>();
TemplateElement rootTreeNode = getTemplate(templateName).getRootTreeNode();
if ("IteratorBlock".equals(rootTreeNode.getClass().getSimpleName())) {
introspectFromIteratorBlock(result, rootTreeNode);
return result;
}
for (int i = 0; i < rootTreeNode.getChildCount(); i++) {
TemplateModel templateModel = rootTreeNode.getChildNodes().get(i);
if (!(templateModel instanceof StringModel)) {
continue;
}
Object wrappedObject = ((StringModel) templateModel).getWrappedObject();
if ("DollarVariable".equals(wrappedObject.getClass().getSimpleName())) {
introspectFromDollarVariable(result, wrappedObject);
} else if ("ConditionalBlock".equals(wrappedObject.getClass().getSimpleName())) {
introspectFromConditionalBlock(result, wrappedObject);
} else if ("IfBlock".equals(wrappedObject.getClass().getSimpleName())) {
introspectFromIfBlock(result, wrappedObject);
} else if ("IteratorBlock".equals(wrappedObject.getClass().getSimpleName())) {
introspectFromIteratorBlock(result, wrappedObject);
}
}
return result;
}
private static void introspectFromIteratorBlock(Set<String> result, Object wrappedObject) {
try {
Object expression = getInternalState(wrappedObject, "listExp");
result.add(getInternalState(expression, "name").toString());
} catch (NoSuchFieldException | IllegalAccessException ignored) {
}
}
private static void introspectFromConditionalBlock(Set<String> result, Object wrappedObject)
throws TemplateModelException {
try {
Object expression = getInternalState(wrappedObject, "condition");
if (expression == null) {
return;
}
result.addAll(dealCommonExpression(expression));
String nested = getInternalState(wrappedObject, "nestedBlock").toString();
result.addAll(getNecessaryTemplateVariables(nested));
} catch (NoSuchFieldException | IllegalAccessException ignored) {
}
}
private static Set<String> dealCommonExpression(Object expression)
throws NoSuchFieldException, IllegalAccessException {
Set<String> ret = Sets.newHashSet();
switch (expression.getClass().getSimpleName()) {
case "ComparisonExpression":
String reference = dealComparisonExpression(expression);
ret.add(reference);
break;
case "ExistsExpression":
reference = dealExistsExpression(expression);
ret.add(reference);
break;
case "AndExpression":
ret.addAll(dealAndExpression(expression));
default:
break;
}
return ret;
}
private static String dealComparisonExpression(Object expression)
throws NoSuchFieldException, IllegalAccessException {
Object left = getInternalState(expression, "left");
Object right = getInternalState(expression, "right");
String reference;
if ("Identifier".equals(left.getClass().getSimpleName())) {
reference = getInternalState(left, "name").toString();
} else {
reference = getInternalState(right, "name").toString();
}
return reference;
}
private static String dealExistsExpression(Object expression) throws NoSuchFieldException, IllegalAccessException {
Object exp = getInternalState(expression, "exp");
return getInternalState(exp, "name").toString();
}
private static Set<String> dealAndExpression(Object expression) throws NoSuchFieldException,
IllegalAccessException{
Set<String> ret = Sets.newHashSet();
Object lho = getInternalState(expression, "lho");
ret.addAll(dealCommonExpression(lho));
Object rho = getInternalState(expression, "rho");
ret.addAll(dealCommonExpression(rho));
return ret;
}
private static void introspectFromIfBlock(Set<String> result, Object wrappedObject)
throws TemplateModelException {
result.addAll(getNecessaryTemplateVariables(wrappedObject.toString()));
}
private static void introspectFromDollarVariable(Set<String> result, Object wrappedObject)
throws TemplateModelException {
try {
Object expression = getInternalState(wrappedObject, "expression");
switch (expression.getClass().getSimpleName()) {
case "Identifier":
result.add(getInternalState(expression, "name").toString());
break;
case "DefaultToExpression":
result.add(getInternalState(expression, "lho").toString());
break;
case "BuiltinVariable":
break;
default:
break;
}
} catch (NoSuchFieldException | IllegalAccessException e) {
throw new TemplateModelException("Unable to reflect template model");
}
}
private static Object getInternalState(Object o, String fieldName) throws NoSuchFieldException, IllegalAccessException {
Field [] fieldArray = o.getClass().getDeclaredFields();
for (Field field : fieldArray) {
if (!field.getName().equals(fieldName)) {
continue;
}
boolean wasAccessible = field.isAccessible();
try {
field.setAccessible(true);
return field.get(o);
} finally {
field.setAccessible(wasAccessible);
}
}
throw new NoSuchFieldException();
}
private static Template getTemplate(String templateName) {
try {
StringReader stringReader = new StringReader(templateName);
return new Template(null, stringReader, null);
} catch (IOException e) {
throw new IllegalStateException("Failed to Load Template: " + templateName, e);
}
}
I optimized SimY4's answer, and supported <#list> and <#if> block. Code is not fully tested

Execute following regex on the template:
(?<=\$\{)([^\}]+)(?=\})
(?<=\$\{) Matches everything followed by ${
([^\}]+) Matches any string not containing }
(?=\}) Matches everything before }

I had the same problem and none of posted solution made sense to me. What I came with is plugging custom implementation of
TemplateExceptionHandler. Example:
final private List<String> missingReferences = Lists.newArrayList();
final Configuration cfg = new Configuration(Configuration.VERSION_2_3_23);
cfg.setTemplateExceptionHandler(new TemplateExceptionHandler() {
#Override
public void handleTemplateException(TemplateException arg0, Environment arg1, Writer arg2) throws TemplateException {
if (arg0 instanceof InvalidReferenceException) {
missingReferences.add(arg0.getBlamedExpressionString());
return;
}
throw arg0;
}
}
Template template = loadTemplate(cfg, templateId, templateText);
StringWriter out = new StringWriter();
try {
template.process(templateData, out);
} catch (TemplateException | IOException e) {
throw new RuntimeException("oops", e);
}
System.out.println("Missing references: " + missingReferences);
Read more here: https://freemarker.apache.org/docs/pgui_config_errorhandling.html