Develop Reference

How can I modify a particular line in a file without deleting the existing data using Java?

I need to change the values in a file which has more than 30 lines and each line has a data like:
ENABLE_TLS=true
PSWD_MIN_LENGTH=8
Here, let us consider this as a key and value pair, and I needed to change only the value for the 2nd line alone, without deleting the 1st line. Can someone help me how can I do this??
I have tried bufferedwriter, but it is replacing all the lines.
My expectation is:
I need to modify only a particular key's value and the remaining lines should not get deleted

Your description of the data sounds like Java Properties. If you are certain that all the data in that file takes the form key=value you could read it in as a Properties object, update the value for the key in question, and write it back to the file.
Properties properties = new Properties();
try (FileInputStream inputStream = new FileInputStream("/path/to/file")) {
properties.load(inputStream);
}
properties.put("PSWD_MIN_LENGTH", 12);
try (FileOutputStream outputStream = new FileOutputStream("/path/to/file")) {
properties.store(outputStream, null);
}
BEWARE: there is no guarantee that the order of the key/value entries in the file will be maintained (they probably won't). If you are looking for a Properties implementation that will maintain the order, maybe this SO answer will do the trick (UNTESTED!) How maintain the order of keys in Java properties file?

Amazon S3 Client NOT listing all folders in the bucket

I'm trying to list all so-called folders and sub-folders in an s3 bucket.
Now, as I am trying to list all the folders in a path recursively I am not using withDelimeter() function.
All the so-called folder names should end with / and this is my logic to list all the folders and sub-folders.
Here's the scala code (Intentionally not pasting the catch code here):
val awsCredentials = new BasicAWSCredentials(awsKey, awsSecretKey)
val client = new AmazonS3Client(awsCredentials)
def listFoldersRecursively(bucketName: String, fullPath: String): List[String] = {
try {
val objects = client.listObjects(bucketName).getObjectSummaries
val listObjectsRequest = new ListObjectsRequest()
.withPrefix(fullPath)
.withBucketName(bucketName)
val folderPaths = client
.listObjects(listObjectsRequest)
.getObjectSummaries()
.map(_.getKey)
folderPaths.filter(_.endsWith("/")).toList
}
}
Here's the structure of my bucket through an s3 client
Here's the list I am getting using this scala code
Without any apparent pattern, many folders are missing from the list of retrieved folders.
I did not use
client.listObjects(listObjectsRequest).getCommonPrefixes.toList
because it was returning empty list for some reason.
P.S: Couldn't add photos in post directly because of being a new user.

Without any apparent pattern, many folders are missing from the list of retrieved folders.
Here's your problem: you are assuming there should always be objects with keys ending in / to symbolize folders.
This is an incorrect assumption. They will only be there if you created them, either via the S3 console or the API. There's no reason to expect them, as S3 doesn't actually need them or use them for anything, and the S3 service does not create them spontaneously, itself.
If you use the API to upload an object with key foo/bar.txt, this does not create the foo/ folder as a distinct object. It will appear as a folder in the console for convenience, but it isn't there unless at some point you deliberately created it.
Of course, the only way to upload such an object with the console is to "create" the folder unless it already appears -- but appears in the console does not necessarily equate to exists as a distinct object.
Filtering on endsWith("/") is invalid logic.
This is why the underlying API includes CommonPrefixes with each ListObjects response if delimiter and prefix are specified. This is a list of the next level of "folders", which you have to recursively drill down into in order to find the next level.
If you specify a prefix, all keys that contain the same string between the prefix and the first occurrence of the delimiter after the prefix are grouped under a single result element called CommonPrefixes. If you don't specify the prefix parameter, the substring starts at the beginning of the key. The keys that are grouped under the CommonPrefixes result element are not returned elsewhere in the response.
https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketGET.html
You need to access this functionality with whatever library you or using, or, you need to iterate the entire list of keys and discover the actual common prefixes on / boundaries using string splitting.

Well, in case someone faces the same problem in future, the alternative logic I used is as suggested by #Michael above, I iterated through all the keys, splat them at last occurrence of /. The first index of the returned list + / was the key of a folder, appended it to another list. At the end, returned the unique list I was appending into. This gave me all the folders and sub-folders in a certain prefix location.
Note that I didn't use CommonPrefixes because I wasn't using any delimiter and that's because I didn't want the list of folders at a certain level but instead recursively get all the folders and sub-folders
def listFoldersRecursively(bucketName: String, fullPath: String): List[String] = {
try {
val objects = client.listObjects(bucketName).getObjectSummaries
val listObjectsRequest = new ListObjectsRequest()
.withPrefix(fullPath)
.withBucketName(bucketName)
val folderPaths = client.listObjects(listObjectsRequest)
.getObjectSummaries()
.map(_.getKey)
.toList
val foldersList: ArrayBuffer[String] = ArrayBuffer()
for (folderPath <- folderPaths) {
val split = folderPath.splitAt(folderPath.lastIndexOf("/"))
if (!split._1.equals(""))
foldersList += split._1 + "/"
}
foldersList.toList.distinct
P.S: Catch block is intentionalyy missing due to irrelevancy.

The listObjects function (and others) is paginating, returning up to 100 entries every time.
From the doc:
Because buckets can contain a virtually unlimited number of keys, the
complete results of a list query can be extremely large. To manage
large result sets, Amazon S3 uses pagination to split them into
multiple responses. Always check the ObjectListing.isTruncated()
method to see if the returned listing is complete or if additional
calls are needed to get more results. Alternatively, use the
AmazonS3Client.listNextBatchOfObjects(ObjectListing) method as an easy
way to get the next page of object listings.

JAVA Rewrite/store only specified key value of properties

Let say i have property file test.properties.
There are already defined some key/values pairs e.g:
key1=value1
key2=value2
key3=value3
I change in memory some value of these properties (let say only one key's value). I would like to store changes into property file, but to store really only changed key/value => not rewrite whole file.
Is that possible?
Any implementation of some library to I could achieve something like that?

String fileName = "C:\\test\\test.txt";
File f = new File(fileName);
InputStream is = new FileInputStream(f);
Properties p = new Properties();
p.load(is);
p.setProperty("key3","value4");
OutputStream os = new FileOutputStream(f);
p.store(os,"comments");
But I think this will overwrite the entire properties file.

Look at java.util.prefs.Preferences
EDIT:
This is a Java utility class that does what you seem to want -- store key/value pairs (only strings as keys) without having to (re)write an entire file of them to change one value. Java has implemented them with system-dependent backing so they're portable.

How to change the Properties.store() divider symbol from "=" to ":"?

I recently found out about java.util.Properties, which allows me to write and read from a config without writing my own function for it.
I was excited since it is so easy to use, but later noticed a flaw when I stored the modified config file.
Here is my code, quite simple for now:
FileWriter writer = null;
Properties configFile = new Properties();
configFile.load(ReadFileTest.class.getClassLoader().getResourceAsStream("config.txt"));
String screenwidth = configFile.getProperty("screenwidth");
String screenheight = configFile.getProperty("screenheight");
System.out.println(screenwidth);
System.out.println(screenheight);
configFile.setProperty("screenwidth", "1024");
configFile.setProperty("screenheight", "600");
try {
writer = new FileWriter("config.txt" );
configFile.store(writer, null);
} catch (IOException e) {
e.printStackTrace();
}
writer.flush();
writer.close();
The problem I noticed was that the config file I try to edit is stored like this:
foo: bar
bar: foo
foobar: barfoo
However, the output after properties.store(writer, null) is this:
foo=bar
bar=foo
foobar=barfoo
The config file I edit is not for my program, it is for an other application that needs the config file to be in the format shown above with : as divider or else it will reset the configuration to default.
Does anybody know how to easily change this?
I searched through the first 5 Google pages now but found noone with a similar problem.
I also checked the Javadoc and found no function that allows me to change it without writing a class for myself.
I would like to use Properties for now since it is there and quite easy to use.
I also got the idea of just replacing all = with : after I saved the file but maybe someone got a better suggestion?

Don't use a tool that isn't designed for the task - don't use Properties here. Instead, I'd just write your own - should be easy enough.
You can still use a Properties instance as your "store", but don't use it for serializing the properties to text. Instead, just use a FileWriter, iterate through the properties, and write the lines yourself - as key + ": " + value.

New idea here
Your comment about converting the = to : got me thinking: Properties.store() writes to a Stream. You could use an in-memory ByteArrayOutputStream, convert as appropriate in memory before you write to a file, then write the file. Likewise for Properties.load(). Or you could insert FilterXXXs instead. (I'd probably do it in memory).
I was looking into how hard it would be to subclass. It's nearly impossible. :-(
If you look at the source code for Properties, (I'm looking at Java 6) store() calls store0(). Now, unfortunately, store0 is private, not protected, and the "=" is given as a magic constant, not something read from a property. And it calls another private method called saveConvert() that also has a lot of magic constants.
Overall, I rate this code as D- quality. It breaks almost all the rules of good code and good style.
But, it's open source, so, theoretically, you could copy and paste (and improve!) a bunch of code into your own BetterProperties class.

Creating and reading from a custom file type in Java

I'm creating an installer and there are some resource files (.xmls, .zip files, a .jar file, etc) that must be read during installation, but I'd like to pack them into a custom file (i.e., a .dat file) so that when distributed, users don't get to mess around with them too much. The problem is that the installer must be written in Java and I've never done this sort of thing before in any programming language. Is it even possible? If so, how can I pack it in a way that can be read by my Java app afterwards and how can I make my Java app read it?

There are a lot of questions you'll need to answer for yourself about the requirements of this filetype. Does it need to be compressed? Encrypted? Does it need to support random access reading, or is stream-reading good enough?
I could be wrong, but I don't think that's what you're asking in this question. If I'm reading you correctly, I think you're asking "how do I read & write arbitrary file data?"
So that's the question I'll answer. Update your question if that's not quite what you're looking for.
Custom filetypes can easily be implemented using the DataInputStream and DataOutputStream classes. These will let you read & write primitives (boolean, char, byte, int, long, float, double) to the stream. There are also some convenience methods for reading & writing UTF-8 encoded Strings, byte-arrays, and a few other goodies.
Let's get started.
For the sake of argument, let's pretend that all my data elements are byte arrays. And each of them has a name. So my filetype can be modeled logically as a Map<String, byte[]>. I'd implement my custom filetype reader/writer class like this:
public class MyFileTypeCodec {
public static void writeToFile(File f, Map<String, byte[]> map)
throws IOException {
// Create an output stream
DataOutputStream stream = new DataOutputStream(
new BufferedOutputStream(new FileOutputStream(f))
);
// Delegate writing to the stream to a separate method
writeToStream(stream, map);
// Always be sure to flush & close the stream.
stream.flush();
stream.close();
}
public static Map<String, byte[]> readFromFile(File f)
throws IOException {
// Create an input stream
DataInputStream stream = new DataInputStream(
new BufferedInputStream(new FileInputStream(f))
);
// Delegate reading from the stream to a separate method
Map<String, byte[]> map = readFromStream(stream);
// Always be sure to close the stream.
stream.close();
return map;
}
public static void writeToStream(DataOutputStream stream, Map<String, byte[]> map)
throws IOException {
// First, write the number of entries in the map.
stream.writeInt(map.size());
// Next, iterate through all the entries in the map
for (Map.Entry<String, byte[]> entry : map.entrySet()) {
// Write the name of this piece of data.
stream.writeUTF(entry.getKey());
// Write the data represented by this name, making sure to
// prefix the data with an integer representing its length.
byte[] data = entry.getValue();
stream.writeInt(data.length);
stream.write(data);
}
}
public static Map<String, byte[]> readFromStream(DataInputStream stream)
throws IOException {
// Create the data structure to contain the data from my custom file
Map<String, byte[]> map = new HashMap<String, byte[]>();
// Read the number of entries in this file
int entryCount = stream.readInt();
// Iterate through all the entries in the file, and add them to the map
for (int i = 0; i < entryCount; i++) {
// Read the name of this entry
String name = stream.readUTF();
// Read the data associated with this name, remembering that the
// data has an integer prefix representing the array length.
int dataLength = stream.readInt();
byte[] data = new byte[dataLength];
stream.read(data, 0, dataLength);
// Add this entry to the map
map.put(name, data);
}
return map;
}
}
The basic idea is that you can write any data to an output stream (and read it back again) if you can represent that data as some combination of primitives. Arrays (or other collections) can be prefixed with their length, like I've done here. Or you can avoid writing the length prefix if you put a TERMINUS sentinel at the end (kind of like null-terminated strings).
I always use this kind of setup when I implement a custom filetype codec, with file IO methods delegating down into stream IO methods. Usually, I discover later that the object I'm reading & writing from this stream could be just as easily written into some larger & more complex file.
So I might have a SuperFancyCodec for reading/writing the data for my whole system, and it calls down into my TinySpecialPurposeCodec. As long as the stream reading & writing methods are public, then I can assemble new filetypes using a component-oriented methodology.

The extension usually have very little to do with how the file is interpreted.
If you'd like to have just config.dat instead of config.xml you just rename the file. (You'd typically give an xml-parser an InputStream or a Reader as input, which may read any file, regardless of extension)
If the problem you're describing is about combining multiple files, (.zip, .jar, etc) into a single .dat file, you could for instance zip them together, and name the zip file with a .dat extension. Java has good support for zip-files and can handle the zip file regardless of filename / extension.
Related link: Reading the Contents of a ZIP File

When creating/reading files in Java (or anything else), the file extension is not strictly tyed to the actual structure of the file's data. If I wanted, I could make an XML files file.gweebz. OS's and applications would not know what to do with it, but once opened, it would be clear that it is XML.
That being said, it is often good to follow the conventions already established and usually .dat files are files in a binary format. You can use .dat for what you want, but be warned that some users may have OS bindings for the file type and clicking on your file may cause different-than-expected behavior on their systems.
As for how to do it in Java. Grabbing a file handle in Java is easy...
File myFile = new File("/dir/file.gweebz");
It is as simple as that and you can name it whatever you want. You will need other classes to write and read from the file or to do compression, but I will assume you know how to do that. If not, this site will have the answer.