Develop Reference

Split large files using Java [duplicate]

This question already has answers here:
Java - Read file and split into multiple files
(11 answers)
Closed 3 years ago.
How can I split a file into parts larger than 2GB?
An array of bytes accepts an int instead of a long as the size. any solution?
public void splitFile(SplitFile file) throws IOException {
int partCounter = 1;
int sizeOfFiles = (int)value;
byte[] buffer = new byte[sizeOfFiles];
File f = file.getFile();
String fileName = f.getName();
try (FileInputStream fis = new FileInputStream(f);
BufferedInputStream bis = new BufferedInputStream(fis)) {
int bytesAmount = 0;
while ((bytesAmount = bis.read(buffer)) > 0) {
String filePartName = fileName + partCounter + file.config.options.getExtension();
partCounter++;
File newFile = new File(f.getParent(), filePartName);
try (FileOutputStream out = new FileOutputStream(newFile)) {
out.write(buffer, 0, bytesAmount);
}
}
}
}

Don't read the entire file into memory, obviously, or even an entire 'part file'.
Your code as pasted will split the file into as many parts as the read method partitions; this seems very silly; after all, the read() method is specced to allow it to partition into single byte increments.
Don't make a new part-file for every call to read. Instead, separate this out: Your read call gets anywhere from 1 to <BUFFER_SIZE> bytes, and your part's size is <PART_SIZE> large; these two things do not have to be the same and you shouldn't write the code that way.
Once you have an open FileOutputStream you can call .write(buffer, 0, bytesAmount) on it any number of times; you can even call .write(buffer, 0, theSmallerOfBytesLeftToWriteInThisPartAndBytesAmount) followed by opening up the next part file FileOutputStream and calling .write(buffer, whereWeLeftOff, remainder) on that one.

Read bytes from file Java

I'm trying to parse my file which keeps all data in binary form. How to read N bytes from file with offset M? And then I need to convert it to String using new String(myByteArray, "UTF-8");. Thanks!
Here's some code:
File file = new File("my_file.txt");
byte [] myByteArray = new byte [file.lenght];
UPD 1: The answers I see are not appropriative. My file keeps strings in byte form, for example: when I put string "str" in my file it actually prints smth like [B#6e0b... in my file. Thus I need to get from this byte-code my string "str" again.
UPD 2: As it's found out the problem appears when I use toString():
PrintWriter writer = new PrintWriter(new BufferedWriter(new OutputStreamWriter(new FileOutputStream(new File(System.getProperty("db.file")), true), "UTF-8")));
Iterator it = storage.entrySet().iterator();//storage is a map<String, String>
while (it.hasNext()){
Map.Entry pairs = (Map.Entry)it.next();
String K = new String(pairs.getKey().toString());
String V = new String(pairs.getValue().toString);
writer.println(K.length() + " " + K.getBytes() + " " + V.length() + " " + V.getBytes());//this is just the format I need to have in file
it.remove();
}
May be there're some different ways to perform that?

As of Java 7, reading the whole of a file really easy - just use Files.readAllBytes(path). For example:
Path path = Paths.get("my_file.txt");
byte[] data = Files.readAllBytes(path);
If you need to do this more manually, you should use a FileInputStream - your code so far allocates an array, but doesn't read anything from the file.
To read just a portion of a file, you should look at using RandomAccessFile, which allows you to seek to wherever you want. Be aware that the read(byte[]) method does not guarantee to read all the requested data in one go, however. You should loop until either you've read everything you need, or use readFully instead. For example:
public static byte[] readPortion(File file, int offset, int length)
throws IOException {
byte[] data = new byte[length];
try (RandomAccessFile raf = new RandomAccessFile(file)) {
raf.seek(offset);
raf.readFully(data);
}
return data;
}
EDIT: Your update talks about seeing text such as [B#6e0b... That suggests you're calling toString() on a byte[] at some point. Don't do that. Instead, you should use new String(data, StandardCharsets.UTF_8) or something similar - picking the appropriate encoding, of course.

Directly convert BufferedImage in a File object [duplicate]

This question already has answers here:
How to save a BufferedImage as a File
(6 answers)
Closed 8 years ago.
I saw many examples on the internet on how to convert a File into a BufferedImage, but I need to make a counter conversion.
I've tried some ways, but all are quite complicated.
I wonder if there is a direct way to accomplish this.
I have this in my code:
for (FileItem item : formItems) {
// processes only fields that are not form fields
if (!item.isFormField()) {
Date date = new Date();
String fileName = new File(item.getName()).getName();
String filePath = uploadPath + date.getTime() + fileName + ".tmp";
File storeFile = new File(filePath);
BufferedImage tempImg = ImageIO.read(storeFile);
//I make process in the tempImg
//I need save it
item.write(tempImg);
}
}
I don't need write a FileItem, but the BufferedImage that I have processed.

File outputfile = new File("image.jpg");
ImageIO.write(bufferedImage, "jpg", outputfile);
Is this what you are looking for?

Reading picture from byte array

I'm writing a Java applet that need to read some XML and image files inside a ZIP file. The Zip file will be download through HTTP and the applet is unsigned, so I need to use java.util.zip.ZipInputStream to manipulated the data. There's a problem when I'm trying to read a PNG file inside the Zip file.
The steps I handle the Zip file:
Download the Zip through Http
URL resourceURL = new URL(source);
HttpURLConnection httpConnection= (HttpURLConnection) resourceURL.openConnection();
httpConnection.connect();
InputStream resourceFileIn = httpConnection.getInputStream();
Use ZipInputStream to hold the data downloaded
ZipInputStream resourceZipIn = new ZipInputStream(resourceFileIn);
Iterate through every ZipEntry and extract the corresponding byte array
ArrayList<ExtractedEntry> extractedList = new ArrayList<ExtractedEntry>();
ZipEntry currentEntry;
while ((currentEntry = resourceZipIn.getNextEntry()) != null) {
byte[] byteHolder = new byte[(int) currentEntry.getSize()];
resourceZipIn.read(byteHolder, 0, byteHolder.length);
extractedList.add(new ExtractedEntry(currentEntry.getName(), byteHolder));
}
remarks: every ZipEntry extracted is hold by the following class
public class ExtractedEntry {
private String name;
private byte[] byteArray;
public ExtractedEntry(String name, byte[] byteArray) {
super();
this.name = name;
this.byteArray = byteArray;
}
public String getName() {
return name;
}
public byte[] getByteArray() {
return byteArray;
}
}
Find the file I want to read from in the extracted list
ExtractedEntry bgEntry = null;
for (int j = 0; j < extractedList.size() && bgEntry == null; j++) {
if (extractedList.get(j).getName().equals("background.png")) {
bgEntry = extractedList.get(j);
}
}
Perform specific action according to need
InputStream mapBgIn = new ByteArrayInputStream(bgEntry.getByteArray());
BufferedImage bgEntry = ImageIO.read(bgIn);
I list out only the action of reading the PNG file as that is where I encounter the problem. When I try to read the image in the way mentioned above, I always receive the following error at the last line of step 5.
javax.imageio.IIOException: Error reading PNG image data at com.sun.imageio.plugins.png.PNGImageReader.readImage(Unknown Source)
at com.sun.imageio.plugins.png.PNGImageReader.read(Unknown Source)
at javax.imageio.ImageIO.read(Unknown Source)
at javax.imageio.ImageIO.read(Unknown Source)
at com.quadd.soft.game.wtd.lib.resource.ResourceManager.loadHttpZipRes(ResourceManager.java:685)
Caused by: javax.imageio.IIOException: Unknown row filter type (= 7)!
at com.sun.imageio.plugins.png.PNGImageReader.decodePass(Unknown Source)
at com.sun.imageio.plugins.png.PNGImageReader.decodeImage(Unknown Source)
... 29 more
However, if I read the image in the following way starting from step 3, there's no problem.
ZipInputStream resourceZipIn = new ZipInputStream(resourceFileIn);
ZipEntry testEntry;
while ((testEntry = resourceZipIn.getNextEntry()) != null) {
if (testEntry.getName().equals("background.png")) {
BufferedImage bgEntry = ImageIO.read(resourceZipIn);
}
}
Therefore, I guess that there's some problem with my code at extracting the bytes from java.util.zip.ZipInputStream and put it back for reading the image. But I'm quite new to manipulating stream so I just couldn't figure out what exactly is the problem. I would like to ask if anyone could point out what mistake I've made in the code which cause the error.

The read method does not guarantee that it fills the byte array; it reads only a small block and returns the number of bytes that it read. You need a loop, or use a helper class that does fill the array.
For example
DataInputStream in = new DataInputStream(resourceZipIn);
in.readFully(byteHolder);
in.close();

Java - Trouble combining more than 2 .wav files

for a project I'm working on, I want to be able to concatenate multiple .wav files.
Through my research I was able to come up with this code:
File sample1 = new File("F:\\Programming\\Resources\\Java_Sound\\trumpet1.wav");
File sample2 = new File("F:\\Programming\\Resources\\Java_Sound\\trumpet2.wav");
File fileOut = new File("F:\\Programming\\Resources\\Java_Sound\\Test.wav");
AudioInputStream audio1 = AudioSystem.getAudioInputStream(sample1);
AudioInputStream audio2 = AudioSystem.getAudioInputStream(sample2);
AudioInputStream audioBuild = new AudioInputStream(new SequenceInputStream(audio1, audio2), audio1.getFormat(), audio1.getFrameLength() + audio2.getFrameLength());
//for(int i = 0; i < 5; i++){
// audioBuild = new AudioInputStream(new SequenceInputStream(audioBuild, audio2), audioBuild.getFormat(), audioBuild.getFrameLength() + audio2.getFrameLength());
//}
AudioSystem.write(audioBuild, AudioFileFormat.Type.WAVE, fileOut);
it works fine for combining two .wav files, however when I uncomment the for loop the produced .wav file only plays audio for the first concatenation. The audio track appears to end early, as wmp's duration bar only goes about 1\5 of the way across the screen.
I've assumed that the problem is with the header in the created .wav file. I've researched many different web pages discussing how a header in constructed, but all of them had slightly different definitions, but all said the header should be in hex. When converting the stream (not the audio stream, a standard FileInputStream) the headers I had were in decimal. Additionally, after the RIFF part, and before the WAVE part, is supposed to be the size of the whole file, not including the first 8 bytes. However some of mine included hyphens. To be honest I have no clue what those mean. Ignoring them however, the size of the test file after uncommenting the code above is still a larger number.
So after researching both how to concatenate multiple audio files, and how to create\manage .wav headers, I still have no clue why the rest of my audio isn't playing, if it even exists. Any help is greatly appreciated. Thanks in advance.

It might be because the input streams cannot be read more than once. Once you read an input stream, it will be at its end and attempt to read further will read no more bytes from that stream.
This should work with a slight modification, keep creating new audio input streams in your loop:
File sample1 = new File("f1.wav");
File sample2 = new File("f2.wav");
File fileOut = new File("combined.wav");
AudioInputStream audio1 = AudioSystem.getAudioInputStream(sample1);
AudioInputStream audio2 = AudioSystem.getAudioInputStream(sample2);
AudioInputStream audioBuild = new AudioInputStream(new SequenceInputStream(audio1, audio2), audio1.getFormat(), audio1.getFrameLength() + audio2.getFrameLength());
for(int i = 0; i < 5; i++)
{
audioBuild = new AudioInputStream(new SequenceInputStream(audioBuild, /* keep creating new input streams */ AudioSystem.getAudioInputStream(sample2)), audioBuild.getFormat(), audioBuild.getFrameLength() + audio2.getFrameLength());
}
AudioSystem.write(audioBuild, AudioFileFormat.Type.WAVE, fileOut);
Also, ensure your audio formats for the files are exactly the same. That is, same sample rate, same channel count, same bits per sample. Otherwise you'll need additional code to do sample conversion.

This is what I used to join any amount of wave files. I looped through a list of the string values for the wave file paths, and each time I join the previous resulting AudioInputStream with the next clip.
List<String> paths;
AudioInputStream clip1 = null;
for (String path : paths)
{
if(clip1 == null)
{
clip1 = AudioSystem.getAudioInputStream(new File(path));
continue;
}
AudioInputStream clip2 = AudioSystem.getAudioInputStream(new File(path));
AudioInputStream appendedFiles = new AudioInputStream(
new SequenceInputStream(clip1, clip2),
clip1.getFormat(),
clip1.getFrameLength() + clip2.getFrameLength());
clip1 = appendedFiles;
}
AudioSystem.write(clip1, AudioFileFormat.Type.WAVE, new File("exported.wav"));