Develop Reference

Retrieving list of Tags from file properties

I would like to retrieve the list of Tags attached to a file in Windows 7 programatically. I am trying to create a mapping of file->tags that I can move across different platforms.
Is anyone aware of a library, or a way to get the 'Tags' values from command line? So far I have only been able to find ways to get basic file attributes such as Author, Date Created, etc.
I am unable to load PowerShell scripts on the computer unfortunately so am not able to make use of those features.
I tried using 'UserDefinedFileAttributeView' but that did not return any values, like so:
private LinkedList<String> windowsGetAllFileTags(File file) {
UserDefinedFileAttributeView fileAttributeView = Files.getFileAttributeView(file.toPath().toAbsolutePath(), UserDefinedFileAttributeView.class);
List<String> allAttributes = null;
try {
allAttributes = fileAttributeView.list();
} catch (IOException e) {
e.printStackTrace();
}
for(String attribute : allAttributes) {
System.out.println("Attribute = " + attribute);
}
return null;
}
An image of the Windows 7 Properties View

There is a Java library written and called as PE/COFF 4J on Github.
import java.io.IOException;
import org.boris.pecoff4j.PE;
import org.boris.pecoff4j.ResourceDirectory;
import org.boris.pecoff4j.ResourceEntry;
import org.boris.pecoff4j.constant.ResourceType;
import org.boris.pecoff4j.io.PEParser;
import org.boris.pecoff4j.io.ResourceParser;
import org.boris.pecoff4j.resources.StringFileInfo;
import org.boris.pecoff4j.resources.StringTable;
import org.boris.pecoff4j.resources.VersionInfo;
import org.boris.pecoff4j.util.ResourceHelper;
public class Main {
public static void main(String[] args) throws IOException {
PE pe = PEParser.parse("C:/windows/system32/notepad.exe");
ResourceDirectory rd = pe.getImageData().getResourceTable();
ResourceEntry[] entries = ResourceHelper.findResources(rd, ResourceType.VERSION_INFO);
for (int i = 0; i < entries.length; i++) {
byte[] data = entries[i].getData();
VersionInfo version = ResourceParser.readVersionInfo(data);
StringFileInfo strings = version.getStringFileInfo();
StringTable table = strings.getTable(0);
for (int j = 0; j < table.getCount(); j++) {
String key = table.getString(j).getKey();
String value = table.getString(j).getValue();
System.out.println(key + " = " + value);
}
}
}
}
Will print:
CompanyName = Microsoft Corporation
FileDescription = Notepad
FileVersion = 6.1.7600.16385 (win7_rtm.090713-1255)
InternalName = Notepad
LegalCopyright = © Microsoft Corporation. All rights reserved.
OriginalFilename = NOTEPAD.EXE
ProductName = Microsoft® Windows® Operating System
ProductVersion = 6.1.7600.16385
If you mention of obtaining tags of images or videos, #Drew Noakes has written Java library called as metadata-extractor for it.
Metadata metadata = ImageMetadataReader.readMetadata(imagePath);
To iterate all values in the file:
for (Directory directory : metadata.getDirectories()) {
for (Tag tag : directory.getTags()) {
System.out.println(tag);
}
}
You can also read specific values from specific directories:
// obtain the Exif SubIFD directory
ExifSubIFDDirectory directory
= metadata.getFirstDirectoryOfType(ExifSubIFDDirectory.class);
// query the datetime tag's value
Date date = directory.getDate(ExifSubIFDDirectory.TAG_DATETIME_ORIGINAL);
The library is available for Maven users too.

In Windows PowerShell, you could grab it with a bit of help from PresentationCore.dll:
function Get-ImageTags {
param(
[string]$Path
)
Add-Type -AssemblyName PresentationCore
try {
$FileStream = (Get-Item $Path).Open('Open','Read')
$BitmapFrame = [System.Windows.Media.Imaging.BitmapFrame]::Create($FileStream)
$Tags = #($BitmapFrame.Metadata.Keywords |%{ $_ })
}
catch {
throw
return
}
finally {
if($FileStream){
$FileStream.Dispose()
}
}
return $Tags
}
Then use like:
$Tags = Get-ImageTags -Path path\to\file.jpeg
The $Tags variable will now contain an array of tags

what about Files.getAttribute
I didn't tried that but probably this could work:
Files.getAttribute(Paths.get("/some/dir","file.txt"), "description:tags")

Parsing XML with StAX with non-unique tag paths, design suggestions

I need to parse a large XML file (probably going to use StAX in Java) and output it into a delimited text file and I have a couple of design questions. First here is an example of the XML
<demographic>
<value>001</value>
<question>Name?</question>
<value>Bob</value>
<question>Last Name?</question>
<value>Smith</value>
<followUpQuestions>
<question>Middle Init.</question>
<value>J</value>
</followUpQuestions>
</demographic>
this would need to be outputted (in the delimited output file) as
001~Bob~Smith~J
so here are my questions:
How can I distinguish between all the different "value" tags, since the tag names are not unique. Currently I tried to resolve this by having 'state' variables that turn on once they pass question-text such as "Name?", however this approach doesnt really work for the first value since I have to check to make sure the 'name' and 'lastName' states are off to ensure I'm getting the first value.
Everytime the client changes the text of the questions (which happens) I have to change the code and recompile it. Is there anyway to avoid this? Maybe save the questions-text in a text file that the program reads in?
Can this be scalable? I need to extract over 100 values and the XML files are usually about 2 gigs large.
Thank you, in advance, for your help (from a Java and XML newbie)!!
UPDATE: here is my attempt to code the solution, can someone please help to streamline? There has to be a less messy way to do this:
import javax.xml.stream.XMLInputFactory;
import javax.xml.stream.XMLStreamConstants;
import javax.xml.stream.XMLStreamException;
import javax.xml.stream.XMLStreamReader;
import java.io.*;
class TestJavaForStackOverflow{
boolean nameState = false,
lastNameState = false,
middleInitState = false;
String name = "",
lastName = "",
middleInit = "",
value = "";
public void parse() throws IOException, XMLStreamException{
XMLInputFactory factory = XMLInputFactory.newInstance();
XMLStreamReader streamReader = factory.createXMLStreamReader(
new FileReader("/n04/data/revmgmt/anthony/scripts/Java_Programs/TestJavaForStackOverflow.xml"));
while(streamReader.hasNext()){
streamReader.next();
if(streamReader.getEventType() == XMLStreamReader.START_ELEMENT){
if("demographic".equals(streamReader.getLocalName())){
parseDemographicInformation(streamReader);
}
}
}
System.out.println(value + "~" + name + "~" + lastName + "~" + middleInit);
}
public void parseDemographicInformation(XMLStreamReader streamReader) throws XMLStreamException {
while(streamReader.hasNext()){
streamReader.next();
if(streamReader.getEventType() == XMLStreamReader.END_ELEMENT){
if("demographic".equals(streamReader.getLocalName())){
return;
}
}
else if(streamReader.getEventType() == XMLStreamReader.START_ELEMENT){
if("question".equals(streamReader.getLocalName())){
streamReader.next();
if("Name?".equals(streamReader.getText())){
nameState = true;
}
else if("Last Name?".equals(streamReader.getText())){
lastNameState = true;
}
else if("Middle Init.".equals(streamReader.getText())){
middleInitState = true;
}
}
else if("value".equals(streamReader.getLocalName())){
streamReader.next();
if(nameState){
name = streamReader.getText();
nameState = false;
}
else if (lastNameState){
lastName = streamReader.getText();
lastNameState = false;
}
else if (middleInitState){
middleInit = streamReader.getText();
middleInitState = false;
}
else {
value = streamReader.getText();
}
}
}
}
}
public static void main(String[] args){
TestJavaForStackOverflow t = new TestJavaForStackOverflow();
try{t.parse();}
catch(IOException e1){}
catch(XMLStreamException e2){}
}
}

I think the flags are not very scalable if you have a lot of different questions to parse, and neither are the global variables to hold the results... if you have 100 questions then you'll need 100 variables, and when they change over time it will be a bear to keep them up to date. I would use a map structure to hold the result, and another one to hold the correspondence between each question text and the corresponding field you are trying to capture (this is not actual Java, just an approximation):
public Map parseDemographicInformation(XmlStream xml, Map questionMap) {
Map record = new Map();
String field = "id";
while((elem = xml.getNextElement())) {
if(elem.tagName == "question") {
field = questionMap[elem.value];
} else if(elem.tagName == "value") {
record[field] = elem.value;
}
}
return record;
}
Then you have something like this to output the result:
String[] fieldsToOutput = { "id", "firstName", "lastName" }; // ideally read this from a file too so it can be changed dynamically
// ...
for(int i=0; i < fieldsToOutput.length; i++){
if(i > 0)
System.out.print("~");
System.out.print(record[fieldsToOutput[i]]);
}
System.out.println();

Fast CSV parsing

I have a java server app that download CSV file and parse it. The parsing can take from 5 to 45 minutes, and happens each hour.This method is a bottleneck of the app so it's not premature optimization. The code so far:
client.executeMethod(method);
InputStream in = method.getResponseBodyAsStream(); // this is http stream
String line;
String[] record;
reader = new BufferedReader(new InputStreamReader(in), 65536);
try {
// read the header line
line = reader.readLine();
// some code
while ((line = reader.readLine()) != null) {
// more code
line = line.replaceAll("\"\"", "\"NULL\"");
// Now remove all of the quotes
line = line.replaceAll("\"", "");
if (!line.startsWith("ERROR"){
//bla bla
continue;
}
record = line.split(",");
//more error handling
// build the object and put it in HashMap
}
//exceptions handling, closing connection and reader
Is there any existing library that would help me to speed up things? Can I improve existing code?

Apache Commons CSV
Have you seen Apache Commons CSV?
Caveat On Using split
Bear in mind is that split only returns a view of the data, meaning that the original line object is not eligible for garbage collection whilst there is a reference to any of its views. Perhaps making a defensive copy will help? (Java bug report)
It also is not reliable in grouping escaped CSV columns containing commas

opencsv
Take a look at opencsv.
This blog post, opencsv is an easy CSV parser, has example usage.

The problem of your code is that it's using replaceAll and split which are very costly operation. You should definitely consider using a csv parser/reader that would do a one pass parsing.
There is a benchmark on github
https://github.com/uniVocity/csv-parsers-comparison
that unfortunately is ran under java 6. The number are slightly different under java 7 and 8. I'm trying to get more detail data for different file size but it's work in progress
see https://github.com/arnaudroger/csv-parsers-comparison

Apart from the suggestions made above, I think you can try improving your code by using some threading and concurrency.
Following is the brief analysis and suggested solution
From the code it seems that you are reading the data over the network (most possibly apache-common-httpclient lib).
You need to make sure that bottleneck that you are saying is not in the data transfer over the network.
One way to see is just dump the data in some file (without parsing) and see how much does it take. This will give you an idea how much time is actually spent in parsing (when compared to current observation).
Now have a look at how java.util.concurrent package is used. Some of the link that you can use are (1,2)
What you ca do is the tasks that you are doing in for loop can be executed in a thread.
Using the threadpool and concurrency will greatly improve your performance.
Though the solution involves some effort, but at the end this will surly help you.

opencsv
You should have a look at OpenCSV. I would expect that they have performance optimizations.

A little late here, there is now a few benchmarking projects for CSV parsers. Your selection will depend on the exact use-case (i.e. raw data vs data binding etc).
SimpleFlatMapper
uniVocity
sesseltjonna-csv (disclaimer: I wrote this parser)

Quirk-CSV
The new kid on the block. It uses java annotations and is built on apache-csv which one of the faster libraries out there for csv parsing.
This library is also thread safe as well if you wanted to re-use the CSVProcessor you can and should.
Example:
Pojo
#CSVReadComponent(type = CSVType.NAMED)
#CSVWriteComponent(type = CSVType.ORDER)
public class Pojo {
#CSVWriteBinding(order = 0)
private String name;
#CSVWriteBinding(order = 1)
#CSVReadBinding(header = "age")
private Integer age;
#CSVWriteBinding(order = 2)
#CSVReadBinding(header = "money")
private Double money;
#CSVReadBinding(header = "name")
public void setA(String name) {
this.name = name;
}
#Override
public String toString() {
return "Name: " + name + System.lineSeparator() + "\tAge: " + age + System.lineSeparator() + "\tMoney: "
+ money;
}}
Main
import java.io.IOException;
import java.io.StringReader;
import java.io.StringWriter;
import java.util.*;
public class SimpleMain {
public static void main(String[] args) {
String csv = "name,age,money" + System.lineSeparator() + "Michael Williams,34,39332.15";
CSVProcessor processor = new CSVProcessor(Pojo.class);
List<Pojo> list = new ArrayList<>();
try {
list.addAll(processor.parse(new StringReader(csv)));
list.forEach(System.out::println);
System.out.println();
StringWriter sw = new StringWriter();
processor.write(list, sw);
System.out.println(sw.toString());
} catch (IOException e) {
}
}}
Since this is built on top of apache-csv you can use the powerful tool CSVFormat. Lets say the delimiter for the csv are pipes (|) instead of commas(,) you could for Example:
CSVFormat csvFormat = CSVFormat.DEFAULT.withDelimiter('|');
List<Pojo> list = processor.parse(new StringReader(csv), csvFormat);
Another benefit are inheritance is also consider.
For other examples on handling reading/writing non-primitive data

For speed you do not want to use replaceAll, and you don't want to use regex either. What you basically always want to do in critical cases like that is making a state-machine character by character parser. I've done that having rolled the whole thing into an Iterable function. It also takes in the stream and parses it without saving it out or caching it. So if you can abort early that's likely going to go fine as well. It should also be short enough and well coded enough to make it obvious how it works.
public static Iterable<String[]> parseCSV(final InputStream stream) throws IOException {
return new Iterable<String[]>() {
#Override
public Iterator<String[]> iterator() {
return new Iterator<String[]>() {
static final int UNCALCULATED = 0;
static final int READY = 1;
static final int FINISHED = 2;
int state = UNCALCULATED;
ArrayList<String> value_list = new ArrayList<>();
StringBuilder sb = new StringBuilder();
String[] return_value;
public void end() {
end_part();
return_value = new String[value_list.size()];
value_list.toArray(return_value);
value_list.clear();
}
public void end_part() {
value_list.add(sb.toString());
sb.setLength(0);
}
public void append(int ch) {
sb.append((char) ch);
}
public void calculate() throws IOException {
boolean inquote = false;
while (true) {
int ch = stream.read();
switch (ch) {
default: //regular character.
append(ch);
break;
case -1: //read has reached the end.
if ((sb.length() == 0) && (value_list.isEmpty())) {
state = FINISHED;
} else {
end();
state = READY;
}
return;
case '\r':
case '\n': //end of line.
if (inquote) {
append(ch);
} else {
end();
state = READY;
return;
}
break;
case ',': //comma
if (inquote) {
append(ch);
} else {
end_part();
break;
}
break;
case '"': //quote.
inquote = !inquote;
break;
}
}
}
#Override
public boolean hasNext() {
if (state == UNCALCULATED) {
try {
calculate();
} catch (IOException ex) {
}
}
return state == READY;
}
#Override
public String[] next() {
if (state == UNCALCULATED) {
try {
calculate();
} catch (IOException ex) {
}
}
state = UNCALCULATED;
return return_value;
}
};
}
};
}
You would typically process this quite helpfully like:
for (String[] csv : parseCSV(stream)) {
//<deal with parsed csv data>
}
The beauty of that API there is worth the rather cryptic looking function.

Apache Commons CSV ➙ 12 seconds for million rows
Is there any existing library that would help me to speed up things?
Yes, the Apache Commons CSV project works very well in my experience.
Here is an example app that uses Apache Commons CSV library to write and read rows of 24 columns: An integer sequential number, an Instant, and the rest are random UUID objects.
For 10,000 rows, the writing and the read each take about half a second. The reading includes reconstituting the Integer, Instant, and UUID objects.
My example code lets you toggle on or off the reconstituting of objects. I ran both with a million rows. This creates a file of 850 megs. I am using Java 12 on a MacBook Pro (Retina, 15-inch, Late 2013), 2.3 GHz Intel Core i7, 16 GB 1600 MHz DDR3, Apple built-in SSD.
For a million rows, ten seconds for reading plus two seconds for parsing:
Writing: PT25.994816S
Reading only: PT10.353912S
Reading & parsing: PT12.219364S
Source code is a single .java file. Has a write method, and a read method. Both methods called from a main method.
I opened a BufferedReader by calling Files.newBufferedReader.
package work.basil.example;
import org.apache.commons.csv.CSVFormat;
import org.apache.commons.csv.CSVParser;
import org.apache.commons.csv.CSVPrinter;
import org.apache.commons.csv.CSVRecord;
import java.io.*;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.time.Duration;
import java.time.Instant;
import java.util.UUID;
public class CsvReadingWritingDemo
{
public static void main ( String[] args )
{
CsvReadingWritingDemo app = new CsvReadingWritingDemo();
app.write();
app.read();
}
private void write ()
{
Instant start = Instant.now();
int limit = 1_000_000; // 10_000 100_000 1_000_000
Path path = Paths.get( "/Users/basilbourque/IdeaProjects/Demo/csv.txt" );
try (
Writer writer = Files.newBufferedWriter( path, StandardCharsets.UTF_8 );
CSVPrinter printer = new CSVPrinter( writer , CSVFormat.RFC4180 );
)
{
printer.printRecord( "id" , "instant" , "uuid_01" , "uuid_02" , "uuid_03" , "uuid_04" , "uuid_05" , "uuid_06" , "uuid_07" , "uuid_08" , "uuid_09" , "uuid_10" , "uuid_11" , "uuid_12" , "uuid_13" , "uuid_14" , "uuid_15" , "uuid_16" , "uuid_17" , "uuid_18" , "uuid_19" , "uuid_20" , "uuid_21" , "uuid_22" );
for ( int i = 1 ; i <= limit ; i++ )
{
printer.printRecord( i , Instant.now() , UUID.randomUUID() , UUID.randomUUID() , UUID.randomUUID() , UUID.randomUUID() , UUID.randomUUID() , UUID.randomUUID() , UUID.randomUUID() , UUID.randomUUID() , UUID.randomUUID() , UUID.randomUUID() , UUID.randomUUID() , UUID.randomUUID() , UUID.randomUUID() , UUID.randomUUID() , UUID.randomUUID() , UUID.randomUUID() , UUID.randomUUID() , UUID.randomUUID() , UUID.randomUUID() , UUID.randomUUID() , UUID.randomUUID() , UUID.randomUUID() );
}
} catch ( IOException ex )
{
ex.printStackTrace();
}
Instant stop = Instant.now();
Duration d = Duration.between( start , stop );
System.out.println( "Wrote CSV for limit: " + limit );
System.out.println( "Elapsed: " + d );
}
private void read ()
{
Instant start = Instant.now();
int count = 0;
Path path = Paths.get( "/Users/basilbourque/IdeaProjects/Demo/csv.txt" );
try (
Reader reader = Files.newBufferedReader( path , StandardCharsets.UTF_8) ;
)
{
CSVFormat format = CSVFormat.RFC4180.withFirstRecordAsHeader();
CSVParser parser = CSVParser.parse( reader , format );
for ( CSVRecord csvRecord : parser )
{
if ( true ) // Toggle parsing of the string data into objects. Turn off (`false`) to see strictly the time taken by Apache Commons CSV to read & parse the lines. Turn on (`true`) to get a feel for real-world load.
{
Integer id = Integer.valueOf( csvRecord.get( 0 ) ); // Annoying zero-based index counting.
Instant instant = Instant.parse( csvRecord.get( 1 ) );
for ( int i = 3 - 1 ; i <= 22 - 1 ; i++ ) // Subtract one for annoying zero-based index counting.
{
UUID uuid = UUID.fromString( csvRecord.get( i ) );
}
}
count++;
if ( count % 1_000 == 0 ) // Every so often, report progress.
{
//System.out.println( "# " + count );
}
}
} catch ( IOException e )
{
e.printStackTrace();
}
Instant stop = Instant.now();
Duration d = Duration.between( start , stop );
System.out.println( "Read CSV for count: " + count );
System.out.println( "Elapsed: " + d );
}
}

Run piece of code contained in a String

I have a piece of Java code in a String.
String javaCode = "if(polishScreenHeight >= 200 && " +
"polishScreenHeight <= 235 && polishScreenWidth >= 220) { }";
Is it possible to convert this Java String to a Java statement and run it? Possibly using Java reflection?

As has already been suggested you can compile, save and run code on the fly using the Compiler API.
Another neat alternative would be to use beanshell. Beanshell is no longer actively developed, but I can vouch for it's reliability, I've used it successfully in multiple production projects.

Use BeanShell. There's a page on how to use it from Java.

Beanshell (as Boris suggested) is a way to "execute" java source code. But it looks like, you want to "execute" fragments that can interact with the compiled classes. Your example contains variabe names.
Reflection will definitly not help, because reflection targets classes ("classfiles").
You could try to define a complete class ("valid java source file"), compile it and load it (url classloader). Then you should be able to use the methods from that "live generated class". But once a class is loaded, you can't get rid of it (unload), so this will work only once (AFAIK).

As far as I know there is no simple way to do this.
However, in Java 6 onwards, you can compile source code for complete classes using javax.tools.Compiler. The compiled classes can then be loaded and executed. But I don't think this will achieve what you want.

Another way would be to execute your code as Groovy code, see this for an example.

you can use this code to run method from using this code
new Statement(Object target, String methodName, Object[] arguments).execute();
import java.beans.Statement;
public class HelloWorld {
public void method_name(String name) {
System.out.println(name);
}
public static void main(String[] args) throws Exception {
HelloWorld h = new HelloWorld();
new Statement(h, "method_name", new Object[]{"Hello world"}).execute();
}
}

Please reevaluate your design and this should be your last alternative.
You should validate the sanity of the String which will be executed to avoid future injection attack.
Now If you can have the String as JavaScript then below code should help,
public class EvalScript {
public static void main(String[] args) throws Exception {
// create a script engine manager
ScriptEngineManager factory = new ScriptEngineManager();
// create a JavaScript engine
ScriptEngine engine = factory.getEngineByName("JavaScript");
// below JS function is executed.
/*
* student object value will be provided by the program as the JSON String.
function checkStudentElgibility(student){
if(student.age >= 10 && student.currentGrade >= 5){
return true;
}
}
// student object value will be provided by the program as the JSON String
checkStudentElgibility(student);
*/
String studentJsonString = "{\n" +
" \"age\" : 10,\n" +
" \"currentGrade\" : 5\n" +
"}";
String javaScriptFunctionString = "function checkStudentElgibility(student){\n" +
" if(student.age >= 10 && student.currentGrade >= 5){\n" +
" return true;\n" +
" }\n" +
"}\n" +
"checkStudentElgibility(student);";
StringBuilder javaScriptString = new StringBuilder();
javaScriptString.append("student=");
javaScriptString.append(studentJsonString);
javaScriptString.append("\n");
javaScriptString.append(javaScriptFunctionString);
Object object = engine.eval(javaScriptString.toString());
System.out.println(object);
// You can also pass the object as follows,
// evaluate JavaScript code that defines an object with one method
engine.eval("var obj = new Object()");
engine.eval("obj.hello = function(name) { print('Hello, ' + name)
}");
// expose object defined in the script to the Java application
Object obj = engine.get("obj");
// create an Invocable object by casting the script engine object
Invocable inv = (Invocable) engine;
// invoke the method named "hello" on the object defined in the
// in js Object with "Script Method" as parameter
inv.invokeMethod(obj, "hello", "Script Method!");
// You can also use Java Objects as Java script object and you can also pass Objects as reference inside Java script function
String jsString = " var obj = new Object()\n" +
" var ArrayList = Java.type(\"java.util.ArrayList\");\n" +
" var customSizeArrayList = new ArrayList(16);\n" +
" obj.hello = function(name) { \n" +
" customSizeArrayList(name); \n" +
" print('Hello, ' + name) \n" +
" }";
}
}
Reference :
https://docs.oracle.com/javase/8/docs/technotes/guides/scripting/prog_guide/javascript.html#A1147187

Try the JavaCompiler API.
Someone else answered this way better than I could, though:
Convert String to Code
Be careful before actually using something like this...

It is not Java, but as pgras has already suggested you could use GrooyScript like so :
Binding binding = new Binding();
GroovyShell shell = new GroovyShell(binding);
String[] strings = new String[]{"World", "Groovy"};
shell.setVariable("args", strings);
String script = "return \"Hello \" + args[1]";
String value = (String) shell.evaluate(script);
System.out.println(value); // Hello Groovy

Creating classes dynamically with Java

I have tried to find information about this but have come up empty handed:
I gather it is possible to create a class dynamically in Java using reflection or proxies but I can't find out how. I'm implementing a simple database framework where I create the SQL queries using reflection. The method gets the object with the database fields as a parameter and creates the query based on that. But it would be very useful if I could also create the object itself dynamically so I wouldn't have the need to have a simple data wrapper object for each table.
The dynamic classes would only need simple fields (String, Integer, Double), e.g.
public class Data {
public Integer id;
public String name;
}
Is this possible and how would I do this?
EDIT: This is how I would use this:
/** Creates an SQL query for updating a row's values in the database.
*
* #param entity Table name.
* #param toUpdate Fields and values to update. All of the fields will be
* updated, so each field must have a meaningful value!
* #param idFields Fields used to identify the row(s).
* #param ids Id values for id fields. Values must be in the same order as
* the fields.
* #return
*/
#Override
public String updateItem(String entity, Object toUpdate, String[] idFields,
String[] ids) {
StringBuilder sb = new StringBuilder();
sb.append("UPDATE ");
sb.append(entity);
sb.append("SET ");
for (Field f: toUpdate.getClass().getDeclaredFields()) {
String fieldName = f.getName();
String value = new String();
sb.append(fieldName);
sb.append("=");
sb.append(formatValue(f));
sb.append(",");
}
/* Remove last comma */
sb.deleteCharAt(sb.toString().length()-1);
/* Add where clause */
sb.append(createWhereClause(idFields, ids));
return sb.toString();
}
/** Formats a value for an sql query.
*
* This function assumes that the field type is equivalent to the field
* in the database. In practice this means that this field support two
* types of fields: string (varchar) and numeric.
*
* A string type field will be escaped with single parenthesis (') because
* SQL databases expect that. Numbers are returned as-is.
*
* If the field is null, a string containing "NULL" is returned instead.
*
* #param f The field where the value is.
* #return Formatted value.
*/
String formatValue(Field f) {
String retval = null;
String type = f.getClass().getName();
if (type.equals("String")) {
try {
String value = (String)f.get(f);
if (value != null) {
retval = "'" + value + "'";
} else {
retval = "NULL";
}
} catch (Exception e) {
System.err.println("No such field: " + e.getMessage());
}
} else if (type.equals("Integer")) {
try {
Integer value = (Integer)f.get(f);
if (value != null) {
retval = String.valueOf(value);
} else {
retval = "NULL";
}
} catch (Exception e) {
System.err.println("No such field: " + e.getMessage());
}
} else {
try {
String value = (String) f.get(f);
if (value != null) {
retval = value;
} else {
retval = "NULL";
}
} catch (Exception e) {
System.err.println("No such field: " + e.getMessage());
}
}
return retval;
}

There are many different ways to achieve this (e.g proxies, ASM), but the simplest approach, one that you can start with when prototyping is:
import java.io.*;
import java.util.*;
import java.lang.reflect.*;
public class MakeTodayClass {
Date today = new Date();
String todayMillis = Long.toString(today.getTime());
String todayClass = "z_" + todayMillis;
String todaySource = todayClass + ".java";
public static void main (String args[]){
MakeTodayClass mtc = new MakeTodayClass();
mtc.createIt();
if (mtc.compileIt()) {
System.out.println("Running " + mtc.todayClass + ":\n\n");
mtc.runIt();
}
else
System.out.println(mtc.todaySource + " is bad.");
}
public void createIt() {
try {
FileWriter aWriter = new FileWriter(todaySource, true);
aWriter.write("public class "+ todayClass + "{");
aWriter.write(" public void doit() {");
aWriter.write(" System.out.println(\""+todayMillis+"\");");
aWriter.write(" }}\n");
aWriter.flush();
aWriter.close();
}
catch(Exception e){
e.printStackTrace();
}
}
public boolean compileIt() {
String [] source = { new String(todaySource)};
ByteArrayOutputStream baos= new ByteArrayOutputStream();
new sun.tools.javac.Main(baos,source[0]).compile(source);
// if using JDK >= 1.3 then use
// public static int com.sun.tools.javac.Main.compile(source);
return (baos.toString().indexOf("error")==-1);
}
public void runIt() {
try {
Class params[] = {};
Object paramsObj[] = {};
Class thisClass = Class.forName(todayClass);
Object iClass = thisClass.newInstance();
Method thisMethod = thisClass.getDeclaredMethod("doit", params);
thisMethod.invoke(iClass, paramsObj);
}
catch (Exception e) {
e.printStackTrace();
}
}
}

It is possible to generate classes (via cglib, asm, javassist, bcel), but you shouldn't do it that way. Why?
the code that's using the library should expect type Object and get all the fields using reflection - not a good idea
java is statically typed language, and you want to introduce dynamic typing - it's not the place.
If you simply want the data in an undefined format, then you can return it in an array, like Object[], or Map<String, Object> if you want them named, and get it from there - it will save you much trouble with unneeded class generation for the only purpose of containing some data that will be obtained by reflection.
What you can do instead is have predefined classes that will hold the data, and pass them as arguments to querying methods. For example:
public <T> T executeQuery(Class<T> expectedResultClass,
String someArg, Object.. otherArgs) {..}
Thus you can use reflection on the passed expectedResultClass to create a new object of that type and populate it with the result of the query.
That said, I think you could use something existing, like an ORM framework (Hibernate, EclipseLink), spring's JdbcTemplate, etc.

This is possible, but (I believe) you need something like ASM or BCEL.
Alternately, you could use something with more power (like Groovy).

It will take a couple of minutes to create a data model class for each table, which you can easily map to the database with an ORM like Hibernate or by writing your own JDBC DAOs. It is far easier than delving deeply into reflection.
You could create a utility that interrogates the database structure for a table, and creates the data model class and DAO for you. Alternatively you could create the model in Java and create a utility to create the database schema and DAO from that (using reflection and Java 5 Annotations to assist). Don't forget that javaFieldNames are different from database_column_names typically.

Recently I needed to create about 200 simple classes from medatata (objects filled with static data) and I did it through the open source burningwave library, with the following scenario:
The classes needed to have a certain prefix in the name, for example "Registro "*.java;
The classes needed to extend from a superclass Registro.java
The classes needed to contain JPA annotations like #Entity, #Column (in attributes), Lombok annotations and custom annotations.
Here is the link to the repository with the complete project: https://github.com/leandrosoares6/criacao-classes-entidade-efd
Here is the code snippet responsible for creating the classes:
public class RegistrosClassFactory {
private static final String PACOTE = "com.example.demo.model.registros";
private static final String SCHEMA = "MY_SCHEMA";
private static final String PREFIXO = "Registro";
static void criaRegistros() {
List<RegistroTest> registros = RegistroMetadataFactory.criaMetadados();
criaClasses(registros);
}
private static void criaClasses(List<RegistroTest> registros) {
for (RegistroTest registroTest : registros) {
UnitSourceGenerator gerador = UnitSourceGenerator.create(PACOTE);
ClassSourceGenerator registro = ClassSourceGenerator
.create(TypeDeclarationSourceGenerator.create(PREFIXO + registroTest.getNome()))
.addModifier(Modifier.PUBLIC)
.addAnnotation(AnnotationSourceGenerator.create(Getter.class))
.addAnnotation(AnnotationSourceGenerator.create(Setter.class))
.addAnnotation(AnnotationSourceGenerator.create(NoArgsConstructor.class))
.addAnnotation(AnnotationSourceGenerator.create(ToString.class))
.addAnnotation(AnnotationSourceGenerator.create(Entity.class))
.addAnnotation(AnnotationSourceGenerator.create(Table.class)
.addParameter("name",
VariableSourceGenerator.create(String.format("\"%s\"",
registroTest.getNomeTabelaBd())))
.addParameter("schema", VariableSourceGenerator
.create(String.format("\"%s\"", SCHEMA))));
criaColunas(registroTest.getCampos(), registro);
registro.addConstructor(FunctionSourceGenerator.create().addModifier(Modifier.PUBLIC)
.addParameter(VariableSourceGenerator.create(String.class, "linha"))
.addBodyCodeLine("super(linha);")).expands(Registro.class);
gerador.addClass(registro);
// System.out.println("\nRegistro gerado:\n" + gerador.make());
String caminhoPastaRegistros = System.getProperty("user.dir") + "/src/main/java/";
gerador.storeToClassPath(caminhoPastaRegistros);
}
}
private static void criaColunas(List<Campo> campos, ClassSourceGenerator registro) {
for (Campo campo : campos) {
VariableSourceGenerator field = VariableSourceGenerator
.create(TypeDeclarationSourceGenerator.create(String.class),
campo.getNomeAtributo())
.addModifier(Modifier.PRIVATE)
.addAnnotation(AnnotationSourceGenerator.create(Column.class)
.addParameter("name", VariableSourceGenerator
.create(String.format("\"%s\"", campo.getNome())))
)
.addAnnotation(AnnotationSourceGenerator.create(Indice.class).addParameter(
"valor",
VariableSourceGenerator.create(String.valueOf(campo.getSequencial()))));
if (campo.getNome().equals("ID")) {
field.addAnnotation(AnnotationSourceGenerator.create(Id.class));
}
if (campo.getEId()) {
field.addAnnotation(AnnotationSourceGenerator.create(CampoTipoId.class));
}
if (campo.getEData()) {
field.addAnnotation(AnnotationSourceGenerator.create(CampoTipoData.class));
}
if (campo.getEDataPart()) {
field.addAnnotation(AnnotationSourceGenerator.create(CampoTipoDataPart.class));
}
registro.addField(field);
}
}
}

I'm aware of the performance drawback of reflection but for my little project I needed this and I created a project lib which converts JSON to Java and then finally .class in JVM context.
Anyone need such thing can have a look into my open source solution, which requires JDK to compile the code.
https://medium.com/#davutgrbz/the-need-history-c91c9d38ec9?sk=f076487e78a1ff5a66ef8eb1aa88f930