Develop Reference

Spring SFTP Adapter InboundAdapter with RotatingServerAdvice - Getting full path

Using spring-integration-sftp: 5.1.3
I have an inbound setup with RotatingServerAdvice that is watching two directories:
--inbox
-----dir1
-----dir2
Watching: inbox/dir1 and inbox/dir2
Local directory: temp/sftp
When I put a file in inbox/dir1/file1.txt, my handler is called as expected and file1.txt is copied to a temp/sftp/file1.txt (<- this is the problem, details below)
Problem:
My use case is that when I get the file, I want to know which remote subdirectory it's from. If the local file was transferred to temp/sftp/inbox/dir1/file1.txt then I could tell that it came from /inbox/dir1 and I can perform some operations on the remote sftp on this directory.
Why is the file being transferred flat and not in subdirectories on the local directory? Any help would be appreciated.
Inbound Adapter:
#Bean
public IntegrationFlow sftpInboundsFlow(DelegatingSessionFactory<LsEntry> delegatingSessionFactory,
SftpServiceActivator serviceActivator) throws JSchException {
SftpConnection config = sftpConnectionConfig.getSftpConnectionConfig(delegatingSessionFactory);
return IntegrationFlows
.from(Sftp.inboundAdapter(delegatingSessionFactory)
.preserveTimestamp(true)
.autoCreateLocalDirectory(true)
.preserveTimestamp(true)
.remoteDirectory("*")
.filter(new CopartFileFilter(Pattern.compile(".*")))
.localDirectory(new File( System.getProperty("java.io.tmpdir") + "/" + config.getLocalDirectory())),
e -> e.id("inboundSftpChannel")
.autoStartup(true)
.poller(Pollers
.fixedDelay(config.getPollerInterval())
.advice(advice(delegatingSessionFactory))
.maxMessagesPerPoll(1)))
.handle(m -> serviceActivator.handleMessage(m))
.get();
}
File info on handler:
file: file1.txt, parent: /var/folders/sd/5k6jwzgs2hj2q165b6x9pql55mp8t7/T/sftp, headers {file_originalFile=/var/folders/sd/5k6jwzgs2hj2q165b6x9pql55mp8t7/T/sftp/file1.txt, id=d2620539-ab0d-2590-9b51-f4dfb442a74a, file_name=file1.txt, file_relativePath=file1.txt, timestamp=1581371879819}
Try 1:
I think this is similar to the first approach mentioned.
.localFilenameExpression("#remoteDirectory + '/' + #this")
It correctly puts the file under temp/sftp/inbox/dir1/file1.txt. The problem now is that the message I get is for this directory:
temp/sftp/inbox
Not the file1.txt

Something like this should work...
Subclass the StandardRotationPolicy, overriding
#Override
public void beforeReceive(MessageSource<?> source) {
...
}
Call super.beforeReceive(source) then cast the source to an AbstractInboundFileSynchronizingMessageSource<?>.
Then call getCurrent() to get the current KeyDirectory.
Then abstractInboundFileSynchronizingMessageSource.setLocalDirectory(new File(...));
Add your custom implementation to the advice.
However, you will have to use a RecursiveDirectoryScanner in the message source so the tree will be scanned.
Alternatively, use an outbound gateway configured to use the MGET command using alternate directories each time (or just inbox in recursive mode). The gateway can be configured to recreate the remote tree.

Had some trouble implementing the solutions suggested because of me being new to this. I went with a different approach because it worked fine for my requirement:
In the sftpInboundsFlow
.localFilenameExpression("#remoteDirectory.replaceAll('/', '-') + '_' + #this")
This places the files in a flat structure in the local temp directory.
temp/sftp:
inbox-dir1_file1.txt
Then in the handler:
Extract remote directory from file name:
public String extractRemoteDir(File ackFile) {
String fileName = ackFile.getName();
return fileName.split("_")[0].replaceAll("-", "/");
}
Custom logic on the remote dir
Delete ackFile
One thing that would make this less hacky would be to to know the remote directory in the local file handler (maybe in the headers), but I didn't find a way to find that.

JavaFx MediaPlayer behaves differently in unit test vs application, why?

I want to load meta data from an MP3 file, to be played by a JavaFx MediaPlayer. This works fine in the unit test, but not in the application. In the unit test, 6 items of metaData reported, but zero in the application. The method that "does the work" is the same.
The main class of the application extends Application. The test class extends ApplicationTest from TestFx. Could that affect the behavior?
The application:
public class MediaMain extends Application {
#Override
public void start(Stage primaryStage) throws Exception {
Map<String, Object> meta = metaData();
System.out.printf("Number of meta data: %d.%n", meta.size());
System.out.println(meta);
}
Map<String, Object> metaData() {
File audioFile = new File("src/main/resources", "beingBoiled.mp3");
final URI uri = audioFile.toURI();
final String source = uri.toString();
Media media = new Media(source);
new MediaPlayer(media);
return media.getMetadata();
}
}
The unit test:
class MediaMainTest extends ApplicationTest {
#Test
void testMeta() {
MediaMain main = new MediaMain();
Map<String, Object> metaData = main.metaData();
assertNotEquals(0, metaData.size());
System.out.printf("Number of meta data: %d.%n", metaData.size());
System.out.println(metaData);
}
}
Printout from the application:
Number of meta data: 0.
{}
Printout from the unit test:
Number of meta data: 6.
{year=1980, artist=The Human League, raw metadata={ID3=java.nio.HeapByteBufferR[pos=254 lim=3214 cap=3214]}, album=Travelogue, genre=(52), title=Being Boiled}
What could be the reason? It's a mystery to me. Written with Java 11, JavaFx 11.0.2 and TestFx 4.0.15-alpha.

You are referencing a file with a location of src/main/resources, this is probably not a good idea as your deployed application likely won't have a src/main/resources directory, plus the resource might be bundled within the application jar rather than as a file on disk, so using a file protocol to access it won't work.
It is probably best to use something like below:
String mediaLoc = getClass().getResource("/beingBoiled.mp3").toExternalForm()
Media media = new Media(mediaLoc)
Like in How load css file in javafx8. The exact location of the resource to be loaded may differ based on build and project structure. If you don't want to load from the class path, but instead via a File or over network http call, then you would need to use something else.
The above code assumes that your build system is setup to copy the media from the src/main/resources to your target packaging location and package the resource into the application distributable (e.g. an application jar file) in the root of the jar file.
Make sure that your build system is actually copying the file to the target location. You can check if it is there by running your build, looking at the resultant jar and running jar tvf <myjarfilename>.jar to see if the mp3 resource is in the correct location at the root of the jar file.

Read local file in Android Studio JUnit test [duplicate]

In a unit test, how can I read data from a json file on my (desktop) file system, without hardcoding the path?
I would like to read test input (for my parsing methods) from a file instead of creating static Strings.
The file is in the same location as my unit testing code, but I can also place it somewhere else in the project if needed. I am using Android Studio.

Depending on android-gradle-plugin version:
1. version 1.5 and higher:
Just put json file to src/test/resources/test.json and reference it as
classLoader.getResource("test.json").
No gradle modification is needed.
2. version below 1.5: (or if for some reason above solution doesn't work)
Ensure you're using at least Android Gradle Plugin version 1.1. Follow the link to set up Android Studio correctly.
Create test directory. Put unit test classes in java directory and put your resources file in res directory. Android Studio should mark them like follow:
Create gradle task to copy resources into classes directory to make them visible for classloader:
android{
...
}
task copyResDirectoryToClasses(type: Copy){
from "${projectDir}/src/test/res"
into "${buildDir}/intermediates/classes/test/debug/res"
}
assembleDebug.dependsOn(copyResDirectoryToClasses)
Now you can use this method to get File reference for the file resource:
private static File getFileFromPath(Object obj, String fileName) {
ClassLoader classLoader = obj.getClass().getClassLoader();
URL resource = classLoader.getResource(fileName);
return new File(resource.getPath());
}
#Test
public void fileObjectShouldNotBeNull() throws Exception {
File file = getFileFromPath(this, "res/test.json");
assertThat(file, notNullValue());
}
Run unit test by Ctrl+Shift+F10 on whole class or specyfic test method.

For local unit tests (vs. instrumentation tests), you can put files under src/test/resources and read them using classLoader. For example, following code opens myFile.txt file in the resources directory.
InputStream in = this.getClass().getClassLoader().getResourceAsStream("myFile.txt");
It worked with
Android Studio 1.5.1
gradle plugin 1.3.1

In my case, the solution was to add to the gradle file
sourceSets {
test.resources.srcDirs += 'src/unitTests/resources'
}
After it everything was found by AS 2.3.1
javaClass.classLoader.getResourceAsStream("countries.txt")

I though I should add my findings here. I know this is a little old but for the newer versions of Gradle, where there is NO src/test/resources directory, but only one single resources directory for the whole project, you have to add this line to your Gradle file.
android {
testOptions {
unitTests {
includeAndroidResources = true
}
}
}
By doing this you can access your resource with:
this.getClass().getClassLoader().getResourceAsStream(fileName);
I've been searching for this and could not find an answer, so I decided to help others here.

I've had plenty of problems with test resources in Android Studio so I set up a few tests for clarity. In my
mobile (Android Application) project I added the following files:
mobile/src/test/java/test/ResourceTest.java
mobile/src/test/resources/test.txt
mobile/src/test/resources/test/samePackage.txt
The test class (all tests passes):
assertTrue(getClass().getResource("test.txt") == null);
assertTrue(getClass().getResource("/test.txt").getPath().endsWith("test.txt"));
assertTrue(getClass().getResource("samePackage.txt").getPath().endsWith("test/samePackage.txt"));
assertTrue(getClass().getResource("/test/samePackage.txt").getPath().endsWith("test/samePackage.txt"));
assertTrue(getClass().getClassLoader().getResource("test.txt").getPath().endsWith("test.txt"));
assertTrue(getClass().getClassLoader().getResource("test/samePackage.txt").getPath().endsWith("test/samePackage.txt"));
In the same root project I have a Java (not Android) project called data. If I add the same files to the data project:
data/src/test/java/test/ResourceTest.java
data/src/test/resources/test.txt
data/src/test/resources/test/samePackage.txt
Then all the tests above will fail if I execute them from Android Studio, but they pass on the command line with ./gradlew data:test.
To get around it I use this hack (in Groovy)
def resource(String path) {
getClass().getResource(path) ?:
// Hack to load test resources when executing tests from Android Studio
new File(getClass().getClassLoader().getResource('.').path
.replace('/build/classes/test/', "/build/resources/test$path"))
}
Usage: resource('/test.txt')
Android Studio 2.3, Gradle 3.3

If you go to Run -> Edit configurations -> JUnit and then select the run configuration for your unit tests, there is a 'Working directory' setting. That should point to wherever your json file is. Keep in mind this might break other tests.

Actually, this is what worked for me with Instrumentation Tests (Running Android Studio version 3.5.3, Android Gradle Plugin version 3.5.3, Gradle version 6.0.1):
Put your files in src/androidTest/assets folder
In your test file:
InputStream is = InstrumentationRegistry.getInstrumentation().getContext().getAssets().open("filename.txt");

In my case I only needed to create resources folder into test folder and put my resource file in that folder.
Then, in the test simply load the file as resource stream with:
val inputStream =
this.javaClass.classLoader?.getResourceAsStream("gallery.xml")
Reference on medium: https://medium.com/mobile-app-development-publication/android-reading-a-text-file-during-test-2815671e8b3b

Step(1) open Android Studio select Project view
Step(2) and create resources directory under test.
Please look at attached screenshot for Step(2) result
Step(3) put json file into resources folder(app/src/test/resources)
Please look at attached screenshot for Step(3) result
Step(4) Create a common class to handle reading local json files and converting to respective models using Gson
Example :-
object TestHelper {
private val gson = Gson()
fun loadJsonAsString(fileName: String): String {
val inputStream = javaClass.getResourceAsStream("/$fileName")
return getStringFromInputStream(inputStream)
}
#Throws(IOException::class)
private fun getStringFromInputStream(stream: InputStream?): String {
var n = 0
val buffer = CharArray(1024 * 4)
val reader = InputStreamReader(stream, "UTF8")
val writer = StringWriter()
while (-1 != reader.read(buffer).also { n = it }) writer.write(buffer, 0, n)
return writer.toString()
}
fun <T> convertJsonToModel(jsonString: String, classT: Class<T>): T{
return gson.fromJson(jsonString, classT)
}
}
Step(5) load the json file stored locally in resources directory created in Step(2)
val GET_USER_INFORMATION_RESPONSE_FILE_NAME = "user_response.json"
val jsonString = loadJsonAsString(GET_USER_INFORMATION_RESPONSE_FILE_NAME)
val networkStatusResponse =
convertJsonToModel(jsonString, UserResponse::class.java)
Step(6) at the end of Step(5) you would have converted local json file into required model class that can be used to write your unit tests.

Parameter 'directory' is not a directory for a parameter which is a directory

I'm getting a strange error where the parameter I supply to a method complains that it's not a directory but it IS in fact a directory with files in it...I don't understand what's wrong...
Toplevel:
public static File mainSchemaFile = new File("src/test/resources/1040.xsd");
public static File contentDirectory = new File("src/test/resources/input");
public static File outputDirectory = new File("src/test/resources/output");
DecisionTableBuilder builder =constructor.newInstance(log, contentDirectory, outputDirectory);
// Here is where the error occurs
builder.compile(mainSchemaFile);
The class I'm using:
public class DecisionTableBuilder {
public void compiler(File schemaFile) {
...
// It's complaining about contentDirectory, it goes to FileUtils class for this
Collection<File> flowchartFiles = FileUtils.listFiles(contentDirectory, mapExtension, true);
...
}
}
Here is the apache FileUtils class:
public class FileUtils {
private static void validateListFilesParameters(File directory, IOFileFilter fileFilter) {
if (!directory.isDirectory()) {
throw new IllegalArgumentException("Parameter 'directory' is not a directory");
}
if (fileFilter == null) {
throw new NullPointerException("Parameter 'fileFilter' is null");
}
}
}
Output: Parameter 'directory' is not a directory
Which is the error output I am getting...
Anyone have any idea what is happening here I'm super confused...any help will be greatly appreciated.
EDIT:
In my toplevel I added the following line:
if(contentDirectory.isDirectory()) {
System.out.println("Content Directory: "+contentDirectory);
}
Output: src/test/resources/input

You're pointing to the file and not a directory in mainSchemaFile variable. Reduce the path to the folder containing 1040.xsd - it should resolve the issue.

Error is thrown if paths cannot be reached
The file paths that you show do not tell where you try to run the code. If you are in your workspace, but you want to run it on a server, and the paths are meant to be on the server, see as follows:
I saw during debugging in the error logs of the console output of my own project that the code tried to get the data from my workspace. While coding, I thought that it would reach the files on the production server, but it did not.
Exception in thread "my_project" java.lang.IllegalArgumentException: Parameter 'directory' is not a directory
at org.apache.commons.io.FileUtils.validateListFilesParameters(FileUtils.java:545)
at org.apache.commons.io.FileUtils.listFiles(FileUtils.java:521)
at org.apache.commons.io.FileUtils.listFiles(FileUtils.java:691)
With the needed file copied to my workspace and the right path in the code, the error was gone since it found the directory.
In my program, my working directory was the repository from where I ran the code. I had to pull the repository on the server to run it with the working directory on the server, so that it could find the production directory for the input files.

Arquillian: path relative to project instead of server

How can I use a project-relative path when using Arquillian/Shrinkwrap to test my testcases?
IDataSet dataSet = new FlatXmlDataSetBuilder().build(new File("src\\ZZZZZ.xml"));
insertFromXML(dataSet);
Will give me this exception when testing
java.io.FileNotFoundException: C:\Uprogs\jboss-eap-6.2.4\bin\src\ZZZZZ.xml
It tries to locate the file within the folders of the server on which I deploy the test onto.
Rather than that, I want him to look in the folders relative to my project (e.g. C:\Users\xy\workspaces\MyProject\src\ZZZZZ.xml). Searched the internet but found nothing
Shrinkwrap gets deployed like this:
#Deployment
public static Archive<?> createDeployment() {
File[] libs = Maven.resolver()
.loadPomFromFile("pom.xml").resolve(
"jcifs:jcifs"
, "org.dbunit:dbunit"
, "com.ibm:db2jcc_license_cisuz"
, "com.ibm:db2jcc"
)
.withTransitivity()
.asFile();
return ShrinkWrap.create(WebArchive.class, "test.war")
.addAsLibraries(libs)
.addPackage("de.abc.RuleEditor")
.addAsResource("de/abc/RuleEditor/messages.properties", "messages.properties")
.addAsManifestResource("test-jboss-deployment-structure.xml","jboss-deployment-structure.xml")
.addAsWebInfResource("test-beans.xml", "beans.xml")
.addAsWebInfResource(
new StringAsset("<faces-config version=\"2.0\"/>"), "faces-config.xml")
.merge(ShrinkWrap.create(GenericArchive.class).as(ExplodedImporter.class)
.importDirectory("src/main/webapp").as(GenericArchive.class), "/", Filters.include(".*\\.xhtml$"));
}

How can I use a project-relative path when using Arquillian/Shrinkwrap (...)?
It's a ridiculous approach, don't go that way :]
The idea behind Arquillian is to create micro-deployments (it means: jar/war/ear archive using ShrinkWrap tool) and include everything inside that archive.
So please modify your deployment:
#Deployment
public static Archive<?> createDeployment() {
return ShrinkWrap.create(WebArchive.class, "test.war")
(...)
.addAsLibraries(libs)
// Add in below way any additional file to the archive
.addAsResource("path-to-testset.xml", "dbunit/testset.xml")
.addPackage("de.abc.RuleEditor")
.addAsResource("de/abc/RuleEditor/messages.properties", "messages.properties")
.addAsManifestResource("test-jboss-deployment-structure.xml","jboss-deployment-structure.xml")
(...)
}
And then use getResourceAsStream() to load file from the class path:
#Test
public void test_something() {
FlatXmlDataSetBuilder builder = new FlatXmlDataSetBuilder();
builder.setColumnSensing(true);
FlatXmlDataSet xmlDataSet = builder.build(
this.getClass().getResourceAsStream("/dbunit/testset.xml"));
// ...
It is much better, as everything what is needed to test, is already included inside archive. No relative, nor absolute path names.
My personal advice is: be careful about DbUnit, as perhaps you notice in the future, crafting and managing many xml data sets may become bigger and bigger problem as your project will grow. That is why I prefer DbSetup.