Develop Reference

Copy raw data of block device using Java

I have 2 disks in the Linux system, say /dev/dsk1 and /dev/dsk2, and I'm trying to read the raw data from dsk1 in bytes and write them into dsk2, in order to make dsk2 an exact copy of dsk1. I tried to do that in the following way (executed with sudo):
import...
public class Main {
public static void main(String[] args) throws NoSuchAlgorithmException, IOException {
Path src = new File("/dev/dsk1").toPath();
Path dst = new File("/dev/dsk2").toPath();
FileChannel r = FileChannel.open(src, StandardOpenOption.READ, StandardOpenOption.WRITE);
FileChannel w = FileChannel.open(dst, StandardOpenOption.READ, StandardOpenOption.WRITE);
long size = r.size();
ByteBuffer byteBuffer = ByteBuffer.allocate(1024);
for (int offset = 0; offset < size; offset+=1024) {
r.position(offset);
w.position(offset);
r.read(byteBuffer);
byteBuffer.flip();
w.write(byteBuffer);
byteBuffer.clear();
}
r.close();
w.close();
}
}
but after writing all the bytes in dsk1 to dsk2, dsk2's filesystem seems to be corrupted. No files can be found in it and if I try to mkdir it will say "structure needs cleaning".
I've tested the above code on regular files, like a text1.txt containing a few characters as src and an empty text2.txt as dst, and it worked fine.
Did I miss something there when reading & writing raw data on block device?

You never check if read method read all 1024 bytes, or if write method wrote them all. Most likely you're leaving gaps in the copy.
There's no magic involved reading from and writing to devices. The first thing I would try is this:
try (FileInputStream src = new FileInputStream("/dev/dsk1");
FileOutputStream dst = new FileOutputStream("/dev/dsk2")) {
src.transferTo(dst);
}

FileOutputStream, directory

this is my directory structure
Inside the server I have the following code for saving a file that gets sent from the client
fileName = reader.readLine();
DataInputStream dis = null;
try {
dis = new DataInputStream(csocket.getInputStream());
FileOutputStream fos = new FileOutputStream(fileName);
buffer = new byte[4096];
int fileSize = 15123;
int read = 0;
int totalRead = 0;
int remaining = fileSize;
while((read = dis.read(buffer, 0, Math.min(buffer.length, remaining))) > 0) {
totalRead += read;
remaining -= read;
fos.write(buffer, 0, read);
}
fos.close();
dis.close();
} catch (IOException e) {
}
break;
I'm wondering how I would go about saving the file within the xml folder? I've tried using getClass().getResource and such but nothing seems to work.
fileName is just a simple string containing the name of the file, not a path or anything.
I get the correct path using this code:
File targetDir = new File(getClass().getResource("xml").getPath());
File targetFile = new File(targetDir, fileName);
targetFile.createNewFile();
System.out.println(targetFile.getAbsolutePath());
dis = new DataInputStream(csocket.getInputStream());
FileOutputStream fos = new FileOutputStream(targetFile.getAbsolutePath(), false);
But it still won't save it there...

The best way is to receive explicitly the target path for storing files, either through a .properties file or a command-line argument. In this way, you make your program flexible to be installed and adapted in different environments.
But if you wish your program to assume the target directory automatically, the best option is to set a relative path before creating the FileOutputStream, as long as you start your program always from the same path:
File targetDir=new File("xml");
File targetFile=new File(targetDir, fileName);
FileOutputStream fos = new FileOutputStream(targetFile);
This will work assuming the program is started from server as current directory.
Update
Other minor suggestions about your program:
Never base the exit condition of the loop on a hard-coded file size, because it is not possible to know it a priori. Instead, check explicitly if the value returned by read is less than 0 => that means End Of File reached.
Consequently, do not bother to calculate the exact amount of data to get through a call to read. Just enter the buffer size, because you are setting a maximum data size.
Never let exceptions catched without a proper treatment: If you know how to make your program recover, enter a proper code into the catch block. Otherwise, you'd better not catch them: Declare them in the throws clause and let them be propagated to the caller.
Always create stream resources through the try-with-resources instruction, to ensure they got closed at the end:
try (FileOutputStream fos = new FileOutputStream(...))
{
// ... use fos...
}
Save unnecessary instructions: If you don't care about if the file already exists on the filesystem or not, don't call createNewFile. But if you care, check the returned value and bifurcate consequently.

I try to create file and it does created but not at ProjectName\src\com\company\xml but in ProjectName\out\production\ProjectName\com\company\xml
my code:
File targetDir = new File(this.getClass().getResource("xml").getPath());
// get the parent of the file
String parentPath = targetDir.getParent( );
String fileName="xml/name.txt";
//do something
File targetFile = new File(parentPath, fileName);
targetFile.createNewFile();
Just pay attention that after compilation you will try to save it into a jar file and it a complicated thing to do.
usually you need to save file into file outside from your jar(separate in the root) like this:

How to make a copy of a file containing images and text using java

I have some word documents and excel sheets which has some images along with the file text content. I want to create a copy of that file and keep it at a specific location. I tried the following method which is creating file at specified location but the file is corrupted and cannot be read.
InputStream document = Thread.currentThread().getContextClassLoader().getResourceAsStream("upgradeworkbench/Resources/Upgrade_TD_Template.docx");
try {
OutputStream outStream = null;
Stage stage = new Stage();
stage.setTitle("Save");
byte[] buffer= new byte[document.available()];
document.read(buffer);
FileChooser fileChooser = new FileChooser();
fileChooser.setInitialFileName(initialFileName);
if (flag) {
fileChooser.getExtensionFilters().addAll(new FileChooser.ExtensionFilter("Microsoft Excel Worksheet", "*.xls"));
} else {
fileChooser.getExtensionFilters().addAll(new FileChooser.ExtensionFilter("Microsoft Word Document", "*.docx"));
}
fileChooser.setTitle("Save File");
File file = fileChooser.showSaveDialog(stage);
if (file != null) {
outStream = new FileOutputStream(file);
outStream.write(buffer);
// IOUtils.copy(document, outStream);
}
} catch (IOException ex) {
System.out.println(ex.getMessage());
}
Can anyone suggest me any different ways to get the proper file.
PS: I am reading the file using InputStream because it is inside the project jar.
PPS: I also tried Files.copy() but it didnt work.

I suggest you never trust on InputStream.available to know the real size of the input, because it just returns the number of bytes ready to be immediately read from the buffer. It might return a small number, but doesn't mean the file is small, but that the buffer is temporarily half-full.
The right algorithm to read an InputStream fully and write it over an OutputStream is this:
int n;
byte[] buffer=new byte[4096];
do
{
n=input.read(buffer);
if (n>0)
{
output.write(buffer, 0, n);
}
}
while (n>=0);

You can use the Files.copy() methods.
Copies all bytes from an input stream to a file. On return, the input stream will be at end of stream.

Use:
Files.copy(document, file.toPath(), StandardCopyOption.REPLACE_EXISTING);
As the class says, the second argument is a Path, not a File.
Generally, since this is 2015, use Path and drop File; if an API still uses File, make it so that it uses it at the last possible moment and use Path all the way.

Combining compressed Gzipped Text Files using Java

my question might not be entirely related to Java but I'm currently seeking a method to combine several compressed (gzipped) textfiles without the requirement to recompress them manually. Lets say I have 4 files, all text that is compressed using gzip and want to compress these into one single *.gz file without de + recompressing them. My current method is to open an InputStream and parse the file linewise, storing in a GZIPoutputstream, which works but isn't very fast.... I could of course also call
zcat file1 file2 file3 | gzip -c > output_all_four.gz
This would work, too but isn't really fast either.
My idea would be to copy the inputstream and write it to outputstream directly without "parsing" the stream, as I don't need to manipulate anything actually. Is something like this possible?

Find below a simple solution in Java (it does the same as my cat ... example). Any kind of buffering the input/output has been omitted to keep the code slim.
public class ConcatFiles {
public static void main(String[] args) throws IOException {
// concatenate the single gzip files to one gzip file
try (InputStream isOne = new FileInputStream("file1.gz");
InputStream isTwo = new FileInputStream("file2.gz");
InputStream isThree = new FileInputStream("file3.gz");
SequenceInputStream sis = new SequenceInputStream(new SequenceInputStream(isOne, isTwo), isThree);
OutputStream bos = new FileOutputStream("output_all_three.gz")) {
byte[] buffer = new byte[8192];
int intsRead;
while ((intsRead = sis.read(buffer)) != -1) {
bos.write(buffer, 0, intsRead);
}
bos.flush();
}
// ungezip the single gzip file, the output contains the
// concatenated input of the single uncompressed files
try (GZIPInputStream gzipis = new GZIPInputStream(new FileInputStream("output_all_three.gz"));
OutputStream bos = new FileOutputStream("output_all_three")) {
byte[] buffer = new byte[8192];
int intsRead;
while ((intsRead = gzipis.read(buffer)) != -1) {
bos.write(buffer, 0, intsRead);
}
bos.flush();
}
}
}

The above method works if you just require to gzip many zipped files. In my case I had made a web servlet and my response was in 20-30 KBs. So I was sending the zipped response.
I tried to zip all my individual JS files on server start only and then add dynamic code runtime using the above method. I could print the entire response in my log file but chrome was able to unzip the first file only. Rest output was coming in bytes.
After research I found out that this is not possible with chrome and they have closed the bug also without solving it.
https://bugs.chromium.org/p/chromium/issues/detail?id=20884

Size of file which we get through AssetManager function getAssets in Android

AssetManager mngr = getAssets();
test_file = mngr.open("sample.txt");
above test_file variable is of InputStream type. Any way to calculate the file size of sample.txt from it?

I have an alternative to get size of a file in assets using AssetFileDescriptor:
AssetFileDescriptor fd = getAssets().openFd("test.png");
Long size = fd.getLength();
Hope it helps.

test_file.available();
Is not a very reliable method to get the file length as is stated in the docs.
size = fd.getLength();
Using the FileDescriptor as shown by Ayublin is!
His answer should be promoted to the correct answer.

inputStream.available() might match the file size if the file is very small, but for larger files it isn't expected to match.
For a compressed asset, the only way to get the size reliably is to copy it to the filesystem, ex: context.getCacheDir() then read the length of the file from there. Here's some sample code that does this. It probably then also makes sense to use the file from the cache dir as opposed to the assets after this.
String filename = "sample.txt";
InputStream in = context.getAssets().open(filename);
File outFile = new File(context.getCacheDir(), filename);
OutputStream out = new FileOutputStream(outFile);
try {
int len;
byte[] buff = new byte[1024];
while ((len = in.read(buff)) > 0) {
out.write(buff, 0, len);
}
} finally {
// close in & out
}
long theRealFileSizeInBytes = outFile.length();
You should also delete the file from the cache dir when you are done with it (and the entire cache dir will also be deleted automatically when uninstalling the app).