I am executing groovy script in java:
final GroovyClassLoader classLoader = new GroovyClassLoader();
Class groovy = classLoader.parseClass(new File("script.groovy"));
GroovyObject groovyObj = (GroovyObject) groovy.newInstance();
groovyObj.invokeMethod("main", null);
this main method println some information which I want to save in some variable. How can I do it ?
You would have to redirect System.out into something else..
Of course, if this is multi-threaded, you're going to hit issues
final GroovyClassLoader classLoader = new GroovyClassLoader();
Class groovy = classLoader.parseClass(new File("script.groovy"));
GroovyObject groovyObj = (GroovyObject) groovy.newInstance();
ByteArrayOutputStream buffer = new ByteArrayOutputStream() ;
PrintStream saveSystemOut = System.out ;
System.setOut( new PrintStream( buffer ) ) ;
groovyObj.invokeMethod("main", null);
System.setOut( saveSystemOut ) ;
String output = buffer.toString().trim() ;
It's probably better (if you can) to write our scripts so they return something rather than dump to system.out
Related
I wonder what's the proper way to reuse a normalizer in ND4J/DL4J. Currently, I save it follows:
final DataNormalization normalizer = new NormalizerStandardize();
normalizer.fit( trainingData );
normalizer.transform( trainingData );
normalizer.transform( testData );
try {
final NormalizerSerializer normalizerSerializer = new NormalizerSerializer();
normalizerSerializer.addStrategy( new StandardizeSerializerStrategy() );
normalizerSerializer.write( normalizer, path );
} catch ( final IOException e ) {
// ...
}
And load it via:
try {
final NormalizerSerializer normalizerSerializer = new NormalizerSerializer();
normalizerSerializer.addStrategy( new StandardizeSerializerStrategy() );
final DataNormalization normalizer = normalizerSerializer.restore( path );
} catch ( final Exception e ) { // Throws Exception instead of IOException.
// ...
}
Is that OK? Unfortunately, I wasn't able to find more information in the docs.
This is what I do...
DataNormalization normalizer = new NormaizerStandardize();
normalizer.fit(trainingData);
normalizer.transform(trainingData);
save it
NormalizerSerializer saver = NormalizerSerializer.getDefaults();
File normalsFile = new File("fileName");
saver.write(normalizer,normalsFile);
restore it
NormalizerSerializer loader = NormalizerSerializer.getDefaults();
DataNormalization restoredNormalizer = loader.restore(normalsFile);
restoredNormalizer.transform(testData);
The ND4J Java Docs say that .getDefaults() gets a serializer, configured with strategies for the built-in normalizer implementations. As you are using NormalizerStandardize the getDefaults() offers a short-hand way of achieving the same end without explicitly adding the strategy.
I am trying to create a Groovy script that takes a list of change lists from our trunk and merges them one at a time into a release branch. I would like to have all the change lists locally because I want to run a test build before submitting upstream. However, whenever I run the script I find that when I look in P4V all the merges have been placed in the default change list. How can I keep them separate?
My code (in Groovy but using the Java API) is as follows:
final changeListNumbers = [ 579807, 579916, 579936 ]
final targetBranch = "1.0.7"
changeListNumbers.each { changeListNumber ->
final existingCl = server.getChangelist( changeListNumber )
final cl = new Changelist(
IChangelist.UNKNOWN,
client.getName(),
server.userName,
ChangelistStatus.NEW,
new Date(),
"${existingCl.id} - ${existingCl.description}",
false,
server
);
cl.fileSpecs = mergeChangeListToBranch( client, cl, changeListNumber, targetBranch )
}
def List<IFileSpec> mergeChangeListToBranch( final IClient client, final IChangelist changeList, final srcChangeListNumber, final String branchVersion ){
final projBase = '//Clients/Initech'
final trunkBasePath = "$projBase/trunk"
final branchBasePath = "$projBase/release"
final revisionedTrunkPath = "$trunkBasePath/...#$srcChangeListNumber,$srcChangeListNumber"
final branchPath = "$branchBasePath/$branchVersion/..."
println "trunk path: $revisionedTrunkPath\nbranch path is: $branchPath"
mergeFromTo( client, changeList, revisionedTrunkPath, branchPath )
}
def List<IFileSpec> mergeFromTo( final IClient client, final IChangelist changeList,final String sourceFile, final String destFile ){
mergeFromTo(
client,
changeList,
new FileSpec( new FilePath( FilePath.PathType.DEPOT, sourceFile ) ),
new FileSpec( new FilePath( FilePath.PathType.DEPOT, destFile ) )
)
}
def List<IFileSpec> mergeFromTo( final IClient client, final IChangelist changeList, final FileSpec sourceFile, final FileSpec destFile ){
final resolveOptions = new ResolveFilesAutoOptions()
resolveOptions.safeMerge = true
client.resolveFilesAuto(
client.integrateFiles( sourceFile, destFile, null, null ),
// client.integrateFiles( changeList.id, false, null, null, sourceFile, destFile ),
resolveOptions
)
}
If I try to IChangeList.update() I get the following error:
Caught: com.perforce.p4java.exception.RequestException: Error in change specification.
Error detected at line 7.
Invalid status 'new'.
If instead of using IChangelist.UNKNOWN to existingCl.id + 10000 (which is larger than any existing change list number currently in use) then I get
Caught: com.perforce.p4java.exception.RequestException: Tried to update new or default changelist
To create the changelist in the server, call IClient.createChangelist():
final existingCl = server.getChangelist( changeListNumber )
cl = new Changelist(
IChangelist.UNKNOWN,
... snip ...
);
cl = client.createChangelist(cl);
cl.fileSpecs = mergeChangeListToBranch( client, cl, ...
Then to integrate into this particular change:
IntegrateFilesOptions intOpts = new IntegrateFilesOptions()
intOpts.setChangelistId( cl.getId())
client.integrateFiles( sourceFile, destFile, null, intOpts)
That integrateFiles() returns the integrated file(s), so check that the returned IFileSpec.getOpStatus() is FileSpecOpStatus.VALID.
Let's say we have a String like this:
String string2code = "variable = 'hello';";
How could we convert that String to a piece of code like this?:
variable = "hello";
GroovyShell is the answer:
String string2code = "variable = 'hello'; return variable.toUpperCase()";
def result = new GroovyShell().evaluate string2code
assert result == "HELLO"
If you're into more complex stuff later, you can compile whole classes using GroovyClassLoader.
private static Class loadGroovyClass( File file ) throws MigrationException {
try {
GroovyClassLoader gcl = new GroovyClassLoader( ExternalMigratorsLoader.class.getClassLoader() );
GroovyCodeSource src = new GroovyCodeSource( file );
Class clazz = gcl.parseClass( src );
return clazz;
}
catch( CompilationFailedException | IOException ex ){
...
}
}
Maybe you can take a look a Janino
Janino is a small java compiler than not only can compile source files, it can compile expressions like the one you have.
I am doing following to do compression of o/p files from reducer:
OutputStream out = ipFs.create( new Path( opDir + "/" + fileName ) );
CompressionCodec codec = new GzipCodec();
OutputStream cs = codec.createOutputStream( out );
BufferedWriter cout = new BufferedWriter( new OutputStreamWriter( cs ) );
cout.write( ... )
But got null pointer exception in line 3:
java.lang.NullPointerException
at org.apache.hadoop.io.compress.zlib.ZlibFactory.isNativeZlibLoaded(ZlibFactory.java:63)
at org.apache.hadoop.io.compress.GzipCodec.createOutputStream(GzipCodec.java:92)
at myFile$myReduce.reduce(myFile.java:354)
I also got following JIRA for the same.
Can you please suggest if I am doing something wrong?
You should use the CompressionCodecFactory if you want to use compression outside of the standard OutputFormat handling (as detailed in #linker answer):
CompressionCodecFactory ccf = new CompressionCodecFactory(conf)
CompressionCodec codec = ccf.getCodecByClassName(GzipCodec.class.getName());
OutputStream compressedOutputSream = codec.createOutputStream(outputStream)
You're doing it wrong. The standard way to do this would be:
TextOutputFormat.setOutputCompressorClass(job, GzipCodec.class);
The GzipCodec is a Configurable, you have to initialize it properly if you instantiate it directly (setConf, ...)
Try this and let me know if that works.
Is there any way to run program compiled by JavaCompiler? [javax.tools.JavaCompiler]
My code:
JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
DiagnosticCollector<JavaFileObject> diagnostics = new DiagnosticCollector<JavaFileObject>();
CompilationTask task = compiler.getTask(null, null, diagnostics, null, null, prepareFile(nazwa, content));
task.call();
List<String> returnErrors = new ArrayList<String>();
String tmp = new String();
for (Diagnostic diagnostic : diagnostics.getDiagnostics()) {
tmp = String.valueOf(diagnostic.getLineNumber());
tmp += " msg: "+ diagnostic.getMessage(null);
returnErrors.add(tmp.replaceAll("\n", " "));
}
Now i want to run that program with lifetime 1 sec and get output to string variable. Is there any way i could do that?
You need to use reflection, something like that:
// Obtain a class loader for the compiled classes
ClassLoader cl = fileManager.getClassLoader(CLASS_OUTPUT);
// Load the compiled class
Class compiledClass = cl.loadClass(CLASS_NAME);
// Find the eval method
Method myMethod = compiledClass.getMethod("myMethod");
// Invoke it
return myMethod.invoke(null);
Adapt it of course to suit your needs
You have to add the compiled class to the current classpath.
There's a number of ways to do that, depending on how your project is set up. Here's one instance using java.net.URLClassLoader:
ClassLoader cl_old = Thread.currentThread().getContextClassLoader();
ClassLoader cl_new = new URLClassLoader(urls.toArray(new URL[0]), cl_old);
where "urls" is a list of urls pointing to the filesystem.