is there a built-in support in Groovy to handle Zip files (the groovy way)?
Or do i have to use Java's java.util.zip.ZipFile to process Zip files in Groovy ?
Maybe Groovy doesn't have 'native' support for zip files, but it is still pretty trivial to work with them.
I'm working with zip files and the following is some of the logic I'm using:
def zipFile = new java.util.zip.ZipFile(new File('some.zip'))
zipFile.entries().each {
println zipFile.getInputStream(it).text
}
You can add additional logic using a findAll method:
def zipFile = new java.util.zip.ZipFile(new File('some.zip'))
zipFile.entries().findAll { !it.directory }.each {
println zipFile.getInputStream(it).text
}
In my experience, the best way to do this is to use the Antbuilder:
def ant = new AntBuilder() // create an antbuilder
ant.unzip( src:"your-src.zip",
dest:"your-dest-directory",
overwrite:"false" )
This way you aren't responsible for doing all the complicated stuff - ant takes care of it for you. Obviously if you need something more granular then this isn't going to work, but for most 'just unzip this file' scenarios this is really effective.
To use antbuilder, just include ant.jar and ant-launcher.jar in your classpath.
AFAIK, there isn't a native way. But check out this article on how you'd add a .zip(...) method to File, which would be very close to what you're looking for. You'd just need to make an .unzip(...) method.
The Groovy common extension project provides this functionality for Groovy 2.0 and above: https://github.com/timyates/groovy-common-extensions
The below groovy methods will unzip into specific folder (C:\folder). Hope this helps.
import org.apache.commons.io.FileUtils
import java.nio.file.Files
import java.nio.file.Paths
import java.util.zip.ZipFile
def unzipFile(File file) {
cleanupFolder()
def zipFile = new ZipFile(file)
zipFile.entries().each { it ->
def path = Paths.get('c:\\folder\\' + it.name)
if(it.directory){
Files.createDirectories(path)
}
else {
def parentDir = path.getParent()
if (!Files.exists(parentDir)) {
Files.createDirectories(parentDir)
}
Files.copy(zipFile.getInputStream(it), path)
}
}
}
private cleanupFolder() {
FileUtils.deleteDirectory(new File('c:\\folder\\'))
}
This article expands on the AntBuilder example.
http://preferisco.blogspot.com/2010/06/using-goovy-antbuilder-to-zip-unzip.html
However, as a matter of principal - is there a way to find out all of the properties, closures, maps etc that can be used when researching a new facet in groovy/java?
There seem to be loads of really useful things, but how to unlock their hidden treasures? The NetBeans/Eclipse code-complete features now seem hopelessly limited in the new language richness that we have here.
Unzip using AntBuilder is good way.
Second option is use an third party library - I recommend Zip4j
Although taking the question a bit into another direction, I started off using Groovy for a DSL that I was building, but ended up using Gradle as a starting point to better handle a lot of the file-based tasks that I wanted to do (eg., unzip and untar files, execute other programs, etc). Gradle builds on what groovy can do, and can be extended further via plugins.
// build.gradle
task doUnTar << {
copy {
// tarTree uses file ext to guess compression, or may be specific
from tarTree(resources.gzip('foo.tar.gz'))
into getBuildDir()
}
}
task doUnZip << {
copy {
from zipTree('bar.zip')
into getBuildDir()
}
}
Then, for example (this extracts the bar.zip and foo.tgz into the directory build):
$ gradle doUnZip
$ gradle doUnTar
def zip(String s){
def targetStream = new ByteArrayOutputStream()
def zipStream = new GZIPOutputStream(targetStream)
zipStream.write(s.getBytes())
zipStream.close()
def zipped = targetStream.toByteArray()
targetStream.close()
return zipped.encodeBase64()
}
Related
I'm trying to understand a comment that a colleague made. We're using testcontainers to create a fixture:
import org.testcontainers.containers.GenericContainer;
import org.testcontainers.utility.DockerImageName;
public class SalesforceFixture extends GenericContainer<SalesforceFixture> {
private static final String APPLICATION_NAME = "salesforce-emulator";
public SalesforceFixture() {
// super(ImageResolver.resolve(APPLICATION_NAME));
super(DockerImageName.parse("gcr.io/ad-selfserve/salesforce-emulator:latest"));
...
}
...
The commented code is what it used to be. The next line is my colleague's suggestion. And on that line he commented:
This is the part I don't know. The [ImageResolver] gets the specific version of the emulator, rather than the latest. You need a docker-info file for that though, which jib doesn't automatically generate (but I think it can).
This is what I know or have figured so far:
SalesforceFixture is a class that will be used by other projects to write tests. It spins up a container in Docker, running a service that emulates the real service's API. It's like a local version of the service that behaves enough like the real thing that if one writes code and tests using the fixture, it should work the same in production. (This is where my knowledge ends.)
I looked into ImageResolver—it seems to be a class we wrote that searches a filesystem for something:
public static String resolve(String applicationName, File... roots) {
Stream<File> searchPaths = Arrays.stream(roots).flatMap((value) -> {
return Stream.of(new File(value, "../" + applicationName), new File(value, applicationName));
});
Optional<File> buildFile = searchPaths.flatMap((searchFile) -> {
if (searchFile.exists()) {
File imageFile = new File(searchFile + File.separator + "/target/docker/image-name");
if (imageFile.exists()) {
return Stream.of(imageFile);
}
}
return Stream.empty();
}).findAny();
InputStream build = (InputStream)buildFile.map(ImageResolver::fileStream).orElseGet(() -> {
return searchClasspath(applicationName);
});
if (build != null) {
try {
return IOUtils.toString(build, Charset.defaultCharset()).trim();
} catch (IOException var6) {
throw new RuntimeException("An exception has occurred while reading build file", var6);
}
} else {
throw new RuntimeException("Could not resolve target image for application: " + applicationName);
}
}
But I'm confused. What filesystem? Like, what is the present working directory? My local computer, wherever I ran the Java program from? Or is this from within some container? (I don't think so.) Or maybe the directory structure inside a .jar file? Or somewhere in gcr.io?
What does he mean about a "specific version number" vs. "latest"? I mean, when I build this project, whatever it built is all I have. Isn't that equivalent to "latest"? In what case would an older version of an image be present? (That's what made me think of gcr.io.)
Or, does he mean, that in the project using this project's image, one will not be able to specify a version via Maven/pom.xml—it will always spin up the latest.
Sorry this is long, just trying to "show my work." Any hints welcome. I'll keep looking.
I can't comment on specifics of your own internal implementations, but ImageResolver seems to work on your local filesystem, e.g. it looks into your target/ directory and also touches the classpath. I can imagine this code was just written for resolving an actual image name (not an image), since it also returns a String.
Regarding latest, using a latest tag for a Docker image is generally considered an anti-pattern, so likely your colleague is commenting about this. Here is a random article from the web explaining some of the issues with latest tag:
https://vsupalov.com/docker-latest-tag/
Besides, I don't understand why you ask these questions which are very specific to your project here on SO rather than asking your colleague.
Is there any code (Java, for example) that creates a file in .xml format and saves it in a folder?
I can't find...
I've searched in several places, but to no success. =//
Thanks in advance!
Have you really tried to search? There is an official documentation for Java, i.e. check Writing Out a DOM as an XML File guide
Also be aware that there is no Java support in JMeter (unless you decide to develop your own plugin) and it's recommended to use Groovy for scripting so it might be a better idea to look for code to copy and paste in Groovy documentation. A quote from the above page:
def writer = new StringWriter()
def xml = new MarkupBuilder(writer)
xml.records() {
car(name: 'HSV Maloo', make: 'Holden', year: 2006) {
country('Australia')
record(type: 'speed', 'Production Pickup Truck with speed of 271kph')
}
car(name: 'Royale', make: 'Bugatti', year: 1931) {
country('France')
record(type: 'price', 'Most Valuable Car at $15 million')
}
}
def records = new XmlSlurper().parseText(writer.toString())
To my knowledge, there aren't any out of box features in JMeter to process XML. You can use Groovy XML objects/operations to read/write XML
I'm quite new to Gradle so the answer might be simple, so I apologize if the answer is simple: I have a testing tool that needs to fetch it's version and compare it to the version of the application it is testing. However , the version of my tool is in my build.graddle as
version '1.0'
I tried different way to access it ( such as) :
task generateSources {
File outDir
outDir = file("$buildDir/classes/test/tests/Version.java")
doFirst {
outDir.exists() || outDir.mkdirs()
new File(outDir).write("public class Version { public static final String VERSION = \"$project.version\"; }")
}
}
compileJava.dependsOn generateSources
compileJava.source generateSources.outputs.files, sourceSets.main.java
I found this piece of code to output the version to another java file, but I fail to see how I'd be able to retrieve that info afterwards ( I mean, my tests are defined in src and I would need to point to a file that doesn't exist at compilation -- correct me if I'm wrong here).
Any idea on how I could accomplish this task?
First of all, you are trying to create java source file in your build/classes (it should contain compiled classes, not sources) directory, but you have to do it in your sources, otherwise it won't be compiled. And if you need this new class to be vailable not for tests, then use src/main/java, not src/test/java/
But anyway, I suppose for your case it's much easier to use some properties file for that and replace some token within it during build. That will allow you to make some static logic to get this property value and use it yet before running the build. So all you need is:
1- to have some properties file in your resources src/main/resources (for example app.properties), where should version variable be stored, with it's value like APP_VERSION_TOKEN
version=%APP_VERSION_TOKEN%
2- configure you Gradle processResources to replace tokens, something like this:
processResources {
filesMatching('**/app.properties') {
filter {
it.replace('%APP_VERSION_TOKEN%', version)
}
}
}
3- make some method to read this file and return the value of the property and use it where you need.
And that's all. For unit tests you can have another file with the same name under src/test/resource with the unchanging value you need for testing.
I got a little project where I have to compute a list. The computation depends on serveal factors.
The point is that these factors change from time to time and the user should be allowed to change this by it's self.
Up to now, the factors are hard-coded and no changes can be done without recompiling the code.
At the moment the code looks like this:
if (someStatement.equals("someString")) {
computedList.remove("something");
}
My idea is to make an editable and human readable textfile, configfile, etc. which is loaded at runtime/ at startup? This file should hold the java code from above.
Any ideas how to do that? Please note: The targeted PCs do not have the JDK installed, only an JRE.
An effective way of going about this is using a static initializer. Static Block in Java A good and concise explanation can be found under this link.
One option here that would allow this would be to use User Input Dialogs from the swing API - then you could store the users answer's in variables and export them to a text file/config file, or just use them right in the program without saving them. You would just have the input dialogs pop up at the very beginning of the program before anything else happens, and then the program would run based off those responses.
You could use Javascript for the configuration file language, instead of java. Java 7 SE and later includes a javascript interpreter that you can call from Java. it's not difficult to use, and you can inject java objects into the javascript environment.
Basically, you'd create a Javascript environment, insert the java objects into it which the config file is expected to configure, and then run the config file as javascript.
Okay, here we go... I found an quite simple solution for my problem.
I am using Janino by Codehaus (Link). This library has an integrated Java compiler and seems to work like the JavaCompiler class in Java 7.
BUT without having the JDK to be installed.
Through Janino you can load and compile *.java files(which are human readable) at runtime, which was exactly what I needed.
I think the examples and code-snippets on their homepage are just painful, so here's my own implementation:
Step one is to implement an interface with the same methods your Java file has which is loaded at runtime:
public interface ZuordnungInterface {
public ArrayList<String> Zuordnung(ArrayList<String> rawList);}
Then you call the Janino classloader when you need the class:
File janinoSourceDir = new File(PATH_TO_JAVAFILE);
File[] srcDir = new File[] { janinoSourceDir };
String encoding = null;
ClassLoader parentClassLoader = this.getClass().getClassLoader();
ClassLoader cl = new JavaSourceClassLoader(parentClassLoader, srcDir,
encoding);
And create an new instance
ZuordnungsInterface myZuordnung = (ZuordnungInterface) cl.loadClass("zuordnung")
.newInstance();
Note: The class which is loaded is named zuordnung.java, so there is no extension needed in the call cl.loadClass("zuordnung").
And finaly the class I want to load and compile at runtime of my program, which can be located wherever you want it to be (PATH_TO_JAVAFILE):
public class zuordnung implements ZuordnungInterface {
public ArrayList<String> Zuordnung(ArrayList<String> rawList){
ArrayList<String> computedList = (ArrayList<String>) rawList.clone();
if (Model.getSomeString().equals("Some other string")) {
computedList.add("Yeah, I loaded an external Java class");
}
return computedList;
}}
That's it. Hope it helps others with similar problems!
I want to collect a list of all files under a directory, in particular including subdirectories. I like not doing things myself, so I'm using FileUtils.listFiles from Apache Commons IO. So I have something like:
import java.io.File;
import java.util.Collection;
import org.apache.commons.io.FileUtils;
import org.apache.commons.io.filefilter.TrueFileFilter;
public class TestListFiles {
public static void main(String[] args) {
Collection<File> found = FileUtils.listFiles(new File("foo"),
TrueFileFilter.INSTANCE, TrueFileFilter.INSTANCE);
for (File f : found) {
System.out.println("Found file: " + f);
}
}
}
Problem is, this only appears to find normal files, not directories:
$ mkdir -p foo/bar/baz; touch foo/one_file
$ java -classpath commons-io-1.4.jar:. TestListFiles
Found file: foo/one_file
I'm already passing TrueFileFilter to both of the filters, so I can't think of anything more inclusive. I want it to list: "foo", "foo/one_file", "foo/bar", "foo/bar/baz" (in any order).
I would accept non-FileUtils solutions as well, but it seems silly to have to write my own BFS, or even to collect the set of parent directories from the list I do get. (That would miss empty subdirectories anyway.) This is on Linux, FWIW.
An old answer but this works for me:
FileUtils.listFilesAndDirs(new File(dir), TrueFileFilter.INSTANCE, DirectoryFileFilter.DIRECTORY);
shows both:
I use:
FileUtils.listFilesAndDirs(new File(dir), new NotFileFilter(TrueFileFilter.INSTANCE), DirectoryFileFilter.DIRECTORY)
Only shows directories and not files...
Have you tried simply:
File rootFolder = new File(...);
File[] folders = rootFolder.listFiles((FileFilter) FileFilterUtils.directoryFileFilter());
It seems to work for me.
You will need recursion, of course.
Hope it helps.
I avoid the Java IO libraries in most of my non-trivial applications, preferring Commons VFS instead. I believe a call to this method with the appropriate params will accomplish your goal, but I'll grant its a long way to go for the functionality.
Specifically, this code will do what you want:
FileObject[] files = fileObject.findFiles(new FileSelector() {
public boolean includeFile(FileSelectInfo fileInfo) {
return fileInfo.getFile().getType() == FileType.FOLDER; }
public boolean traverseDescendents(FileSelectInfo fileInfo) {
return true;
}
});
where fileObject is an instance of FileObject.
If you look at the source code and read between the lines in the JavaDoc, you will see that -- unfortunately -- this API is not designed to do what you want. It will return a list of files (not a list of files and directories) that match the provided arguments. In the source code -- look at the method innerListFiles -- you will see that directories are searched and not added to the result list.
I am not aware of any public API that will do what you want. Hopefully someone else will know of one. Most will probably be a DFS, not a BFS, which may or may not matter for your purposes. (So far, all Java code I've ever looked at that did a directory tree traversal did it via a depth-first search. Which doesn't mean that BFS's aren't out there, of course.)
If you really want a list of everything under a given directory, it's easy enough to roll your own. But I understand your wish to not reinvent the wheel.
Note: It's possible that Apache Commons Finder will support what you need, but this library is in The Commons Sandbox, which means it is more experimental at this stage. It may or may not be complete and it may or may not be maintained. It also may be heavyweight for what you are looking for.
An easier+complete Commons VFS solution:
FileSystemManager fsManager = VFS.getManager();
FileObject fileObject = fsManager.resolveFile( "yourFileNameHere" );
FileObject[] files = fileObject.findFiles( new FileTypeSelector( FileType.FOLDER ) )
It should work, based on their API.
Here is my own version of FileUtils, not as complete as Commons IO, it contains only what I need. Search for findFiles or you can use iterate to avoid creating huge lists(sometime/most of the time you just want to do something with those files so collecting them in a List it doesn't makes sense).