Develop Reference

AWS S3 upload file name mismatch

I am able to upload multiple files to s3 bucket at once. However there is a mismatch in the file name the one I provided and uploaded file. I am interested in file name as I need to generate cloud front signed url based on that.
File generation code
final String fileName = System.currentTimeMillis() + pictureData.getFileName();
final File file = new File(fileName); //fileName is -> 1594125913522_image1.png
writeByteArrayToFile(img, file);
AWS file upload code
public void uploadMultipleFiles(final List<File> files) {
final TransferManager transferManager = TransferManagerBuilder.standard().withS3Client(amazonS3).build();
try {
final MultipleFileUpload xfer = transferManager.uploadFileList(bucketName, null, new File("."), files);
xfer.waitForCompletion();
} catch (InterruptedException exception) {
if (LOGGER.isInfoEnabled()) {
LOGGER.info("InterruptedException occurred=>" + exception);
}
} catch (AmazonServiceException exception) {
if (LOGGER.isInfoEnabled()) {
LOGGER.info("AmazonServiceException occurred =>" + exception);
}
throw exception;
}
}
Uploaded file name is 94125913522_image1.png. As you can see first two characters disappeared. What am I missing here. I am not able to figure out. Kindly advice.
private static void writeByteArrayToFile(final byte[] byteArray, final File file) {
try (OutputStream outputStream = new BufferedOutputStream(Files.newOutputStream(Paths.get(file.getName())))) {
outputStream.write(byteArray);
} catch (IOException exception) {
throw new FileIllegalStateException("Error while writing image to file", exception);
}
}

The reason of the problem
You lose the first two charecters of the file names because of the third argument of this method:
transferManager.uploadFileList(bucketName, null, new File("."), files);
What happens in this case
So, what is the third argument:
/**
...
* #param directory
* The common parent directory of files to upload. The keys
* of the files in the list of files are constructed relative to
* this directory and the virtualDirectoryKeyPrefix.
...
*/
public MultipleFileUpload uploadFileList(... , File directory, ...){...}
And how will it be used:
...
int startingPosition = directory.getAbsolutePath().length();
if (!(directory.getAbsolutePath().endsWith(File.separator)))
startingPosition++;
...
String key = f.getAbsolutePath().substring(startingPosition)...
Thus, the directory variable is used to define a starting index to trim file paths to get file keys.
When you pass new File(".") as a directory, the parent directory for your files will be {your_path}.
But this is a directory, and you need to work with files inside it. So the common part, retrieved from your directory file, is {your_path}./
That is 2 symbols more than you actually need. And for this reason this method trims the 2 extra characters - an extra shift of two characters when trimming the file path.
The solution
If you only need to work with the current directory, you can pass the current directory as follows:
MultipleFileUpload upload = transferManager.uploadFileList(bucketName, "",
System.getProperty("user.dir"), files);
But if you start working with external sources, it won't work. So you can use this code, which creates one MultipleFileUpload per group of files from one directory.
private final String PATH_SEPARATOR = File.separator;
private String bucketName;
private TransferManager transferManager;
public void uploadMultipleFiles(String prefix, List<File> filesToUpload){
Map<File, List<File>> multipleUploadArguments =
getMultipleUploadArguments(filesToUpload);
for (Map.Entry<File, List<File>> multipleUploadArgument:
multipleUploadArguments.entrySet()){
try{
MultipleFileUpload upload = transferManager.uploadFileList(
bucketName, prefix,
multipleUploadArgument.getKey(),
multipleUploadArgument.getValue()
);
upload.waitForCompletion();
} catch (InterruptedException ex) {
throw new RuntimeException(ex);
}
}
}
private Map<File, List<File>> getMultipleUploadArguments(List<File> filesToUpload){
return filesToUpload.stream()
.collect(Collectors.groupingBy(this::getDirectoryPathForFile));
}
private File getDirectoryPathForFile(File file){
String filePath = file.getAbsolutePath();
String directoryPath = filePath.substring(0, filePath.lastIndexOf(PATH_SEPARATOR));
return new File(directoryPath);
}

SnakeYAML Dump nested key

I am using SnakeYAML as my YAML parser for a project, and I don't know how to set keys that are nested. For instance, here is a YAML file with nested keys in it.
control:
advertising:
enabled: true
logging:
chat: true
commands: true
format: '%id% %date% %username% | %value%'
My goal is to be able to easily set the path control.advertising.enabled or any other path to any value.
When I use
void Set(String key, Object value, String configName){
Yaml yaml = new Yaml();
OutputStream oS;
try {
oS = new FileOutputStream(main.getDataFolder() + File.separator + configName);
} catch (FileNotFoundException e) {
e.printStackTrace();
return;
}
Map<String, Object> data = new HashMap<String, Object>();
// set data based on original + modified
data.put(key, value);
String output = yaml.dump(data);
try {
oS.write(output.getBytes());
} catch (IOException e) {
e.printStackTrace();
}
}
to set the value, instead of getting
logging:
chat: true
commands: true
format: '%id% %date% %username% | %value%'
the entire yaml file is cleared and I only get
{logging.chat: false}
Thank you!

Define your structure with Java classes:
public class Config {
public static class Advertising {
public boolean enabled;
}
public static class Control {
public Advertising advertising;
}
public static class Logging {
public boolean chat;
public boolean commands;
public String format;
}
Control control;
Logging logging;
}
Then you can modify it like this:
Yaml yaml = new Yaml();
Config config = yaml.loadAs(inputStream, Config.class);
config.control.advertising.enabled = false;
String output = yaml.dump(config);
Note that loading & saving YAML data this way might mess with the order of mapping keys, because this order is not preserved. I assume that the output order will be according to the field order in the Java classes, but I'm not sure.

Relative path to file | Springboot

I am new to Spring-boot/Java and trying to read the contents of a file in a String.
What's the issue:
I'm getting "File not found exception" and unable to read the file. Apparently, I'm not giving the correct file path.
i've attached the directory structure and my code. I'm in FeedProcessor file and want to read feed_template.php (see image)
public static String readFileAsString( ) {
String text = "";
try {
// text = new String(Files.readAllBytes(Paths.get("/src/main/template/feed_template_head.php")));
text = new String(Files.readAllBytes(Paths.get("../../template/feed_template_head.php")));
} catch (IOException e) {
e.printStackTrace();
}
return text;
}

You need to put template folder inside resource folder. And then use following code.
#Configuration
public class ReadFile {
private static final String FILE_NAME =
"classpath:template/feed_template_head.php";
#Bean
public void initSegmentPerformanceReportRequestBean(
#Value(FILE_NAME) Resource resource,
ObjectMapper objectMapper) throws IOException {
new BufferedReader(resource.getInputStream()).lines()
.forEach(eachLine -> System.out.println(eachLine));
}
}
I suggest you to go though once Resource topic in spring.
https://docs.spring.io/spring/docs/3.0.x/spring-framework-reference/html/resources.html

Error on setup config properties with Java/Selenium/TestNG/Maven

I tried best couldn't find a complete instructions on how to config a properties file with Maven,Testng.
Here are what I did and the exception I got:
from TestNG for suite, added
content of the config file:
user=testuser
password=pswd
pom.xml
src/test/resources
true
in code:
#BeforeTest #Parameters(value = { "config-file" })
public void initFramework(String configfile) throws Exception
{
InputStream stream = Config.class.getResourceAsStream("/config.properties");
Properties properties = new Properties();
try {
properties.load(stream);
String user = properties.getProperty("user");
String password = properties.getProperty("password");
System.out.println("\nGot User FirstName+LastName shows as:"+ user +"\n" + password + "===========");
} catch (IOException e) {
e.printStackTrace();
// You will have to take some action here...
}
}
Here is what I got when compile:
org.testng.TestNGException:
Parameter 'config-file' is required by #Configuration on method initFramework but has not been marked #Optional or defined
Question:
I think I got all options mixed but really wanted a working way to read the parameter for Java/Selenium/TestNG/Maven.

Properties CONFIG= new Properties();
FileInputStream ip = new FileInputStream("C://config.properties");
CONFIG.load(ip);
//Now simply read through property file:-
String user = CONFIG.getProperty("user");
String password = CONFIG.getProperty("password");
//To write property file:-
CONFIG.setProperty("user","newbie1");
CONFIG.setProperty("password","secret123");

Loading a properties file from Java package

I need to read a properties files that's buried in my package structure in com.al.common.email.templates.
I've tried everything and I can't figure it out.
In the end, my code will be running in a servlet container, but I don't want to depend on the container for anything. I write JUnit test cases and it needs to work in both.

When loading the Properties from a Class in the package com.al.common.email.templates you can use
Properties prop = new Properties();
InputStream in = getClass().getResourceAsStream("foo.properties");
prop.load(in);
in.close();
(Add all the necessary exception handling).
If your class is not in that package, you need to aquire the InputStream slightly differently:
InputStream in =
getClass().getResourceAsStream("/com/al/common/email/templates/foo.properties");
Relative paths (those without a leading '/') in getResource()/getResourceAsStream() mean that the resource will be searched relative to the directory which represents the package the class is in.
Using java.lang.String.class.getResource("foo.txt") would search for the (inexistent) file /java/lang/String/foo.txt on the classpath.
Using an absolute path (one that starts with '/') means that the current package is ignored.

To add to Joachim Sauer's answer, if you ever need to do this in a static context, you can do something like the following:
static {
Properties prop = new Properties();
InputStream in = CurrentClassName.class.getResourceAsStream("foo.properties");
prop.load(in);
in.close()
}
(Exception handling elided, as before.)

The following two cases relate to loading a properties file from an example class named TestLoadProperties.
Case 1: Loading the properties file using ClassLoader
InputStream inputStream = TestLoadProperties.class.getClassLoader()
.getResourceAsStream("A.config");
properties.load(inputStream);
In this case the properties file must be in the root/src directory for successful loading.
Case 2: Loading the properties file without using ClassLoader
InputStream inputStream = getClass().getResourceAsStream("A.config");
properties.load(inputStream);
In this case the properties file must be in the same directory as the TestLoadProperties.class file for successful loading.
Note: TestLoadProperties.java and TestLoadProperties.class are two different files. The former, .java file, is usually found in a project's src/ directory, while the latter, .class file, is usually found in its bin/ directory.

public class Test{
static {
loadProperties();
}
static Properties prop;
private static void loadProperties() {
prop = new Properties();
InputStream in = Test.class
.getResourceAsStream("test.properties");
try {
prop.load(in);
in.close();
} catch (IOException e) {
e.printStackTrace();
}
}

public class ReadPropertyDemo {
public static void main(String[] args) {
Properties properties = new Properties();
try {
properties.load(new FileInputStream(
"com/technicalkeeda/demo/application.properties"));
System.out.println("Domain :- " + properties.getProperty("domain"));
System.out.println("Website Age :- "
+ properties.getProperty("website_age"));
System.out.println("Founder :- " + properties.getProperty("founder"));
// Display all the values in the form of key value
for (String key : properties.stringPropertyNames()) {
String value = properties.getProperty(key);
System.out.println("Key:- " + key + "Value:- " + value);
}
} catch (IOException e) {
System.out.println("Exception Occurred" + e.getMessage());
}
}
}

Assuming your using the Properties class, via its load method, and I guess you are using the ClassLoader getResourceAsStream to get the input stream.
How are you passing in the name, it seems it should be in this form: /com/al/common/email/templates/foo.properties

I managed to solve this issue with this call
Properties props = PropertiesUtil.loadProperties("whatever.properties");
Extra, you have to put your whatever.properties file in /src/main/resources

Nobody mentions the similar but even simpler solution than above with no need to deal with the package of the class. Assuming myfile.properties is in the classpath.
Properties properties = new Properties();
InputStream in = ClassLoader.getSystemResourceAsStream("myfile.properties");
properties.load(in);
in.close();
Enjoy

use the below code please :
Properties p = new Properties();
StringBuffer path = new StringBuffer("com/al/common/email/templates/");
path.append("foo.properties");
InputStream fs = getClass().getClassLoader()
.getResourceAsStream(path.toString());
if(fs == null){
System.err.println("Unable to load the properties file");
}
else{
try{
p.load(fs);
}
catch (IOException e) {
e.printStackTrace();
}
}