Develop Reference

How do you append to the end of a gzipped file in Java?

Here is what I've tried
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.util.zip.GZIPInputStream;
import java.util.zip.GZIPOutputStream;
public class GZIPCompression {
public static void main(String[] args) throws IOException {
File file = new File("gziptest.zip");
try ( OutputStream os = new GZIPOutputStream(new FileOutputStream(file, true))) {
os.write("test".getBytes());
}
try ( GZIPInputStream inStream = new GZIPInputStream(new FileInputStream(file))) {
while (inStream.available() > 0) {
System.out.print((char) inStream.read());
}
}
}
}
Based on what I've read, this should append "test" to the end of gziptest.zip, but when I run the code, the file doesn't get modified at all. The strange thing is that if I change FileOutputStream(file, true) to FileOutputStream(file, false), the file does get modified, but its original contents are overriden which is of course not what I want.
I am using JDK 14.0.1.

A couple of things here.
Zip and GZip are different.. If you are doing a gzip test, your file should have the extension .gz, not .zip
To properly append "test" to the end of the gzip data, you should first use a GZIPInputStream to read in from the file, then tack "test" onto the uncompressed text, and then send it back out through GZipOutputStream

Referencing Files(Java) in an updater

I am writing an updater. I have this code:
package main;
import java.io.*;
import java.net.*;
import java.util.*;
import java.util.regex.Pattern;
import java.lang.*;
import static java.lang.System.out;
public class UpdaterCore
{
public static void main(String args[]) throws IOException
{
java.io.BufferedInputStream inv = new java.io.BufferedInputStream(new
java.net.URL("http://unicombox.tk/update/nv").openStream());
java.io.FileOutputStream fosv = new java.io.FileOutputStream("nv");
java.io.BufferedOutputStream boutv = new BufferedOutputStream(fosv,1024);
byte data[] = new byte[1024];
while(inv.read(data,0,1024)>=0)
{
boutv.write(data);
}
boutv.close();
inv.close();
//end version download
Scanner VersionReader= new Scanner(new File ("v")).useDelimiter(",");
int currentVersion= VersionReader.nextInt();
VersionReader.close();
Scanner NewVersionReader= new Scanner(new File ("nv")).useDelimiter(",");
int newVersion= NewVersionReader.nextInt();
NewVersionReader.close();
if (newVersion>currentVersion){
java.io.BufferedInputStream in = new java.io.BufferedInputStream(new
java.net.URL("http://unicombox.tk/update/update.zip").openStream());
java.io.FileOutputStream fos = new java.io.FileOutputStream("update.zip");
java.io.BufferedOutputStream bout = new BufferedOutputStream(fos,1024);
byte data1[] = new byte[1024];
while(in.read(data1,0,1024)>=0)
{
bout.write(data1);
}
bout.close();
in.close();
out.println("Update successfully downloaded!");
}
else{
out.println("You have the latest version!");
}
}
}
It gets the new version from a server, and then compares it to its current version. If the new version is greater than the current version, it downloads the update.
I am having one big problem. My program can never find the files "v" and "nv"!
"v" and "nv" are in the same folder as the compiled jar, yet I get a FileNotFound.
What am I doing wrong?

Get path to current directory (directory where the .jar file is placed) like this:
// import java.io.*;
// import java.net.URLDecoder;
// throws java.io.UnsupportedEncodingException
String path = UpdaterCore.class.getProtectionDomain().getCodeSource().getLocation().getPath();
String decodedPath = URLDecoder.decode(path, "UTF-8");
System.out.println(decodedPath);
and then create File instance like this
new File (decodedPath + File.separatorChar + "v")

You are probably running the program from a different directory, one level up?
You can use getAbsolutePath() on those java.io.File-s to find out what file path are you really trying to read from.
Or just use Marek Sebera's solution, it is fine.

Identifying file type in Java

Please help me to find out the type of the file which is being uploaded.
I wanted to distinguish between excel type and csv.
MIMEType returns same for both of these file. Please help.

I use Apache Tika which identifies the filetype using magic byte patterns and globbing hints (the file extension) to detect the MIME type. It also supports additional parsing of file contents (which I don't really use).
Here is a quick and dirty example on how Tika can be used to detect the file type without performing any additional parsing on the file:
import java.io.File;
import java.io.FileInputStream;
import java.io.InputStream;
import java.util.HashMap;
import org.apache.tika.metadata.HttpHeaders;
import org.apache.tika.metadata.Metadata;
import org.apache.tika.metadata.TikaMetadataKeys;
import org.apache.tika.mime.MediaType;
import org.apache.tika.parser.AutoDetectParser;
import org.apache.tika.parser.ParseContext;
import org.apache.tika.parser.Parser;
import org.xml.sax.helpers.DefaultHandler;
public class Detector {
public static void main(String[] args) throws Exception {
File file = new File("/pats/to/file.xls");
AutoDetectParser parser = new AutoDetectParser();
parser.setParsers(new HashMap<MediaType, Parser>());
Metadata metadata = new Metadata();
metadata.add(TikaMetadataKeys.RESOURCE_NAME_KEY, file.getName());
InputStream stream = new FileInputStream(file);
parser.parse(stream, new DefaultHandler(), metadata, new ParseContext());
stream.close();
String mimeType = metadata.get(HttpHeaders.CONTENT_TYPE);
System.out.println(mimeType);
}
}

I hope this will help. Taken from an example not from mine:
import javax.activation.MimetypesFileTypeMap;
import java.io.File;
class GetMimeType {
public static void main(String args[]) {
File f = new File("test.gif");
System.out.println("Mime Type of " + f.getName() + " is " +
new MimetypesFileTypeMap().getContentType(f));
// expected output :
// "Mime Type of test.gif is image/gif"
}
}
Same may be true for excel and csv types. Not tested.

I figured out a cheaper way of doing this with java.nio.file.Files
public String getContentType(File file) throws IOException {
return Files.probeContentType(file.toPath());
}
- or -
public String getContentType(Path filePath) throws IOException {
return Files.probeContentType(filePath);
}
Hope that helps.
Cheers.

A better way without using javax.activation.*:
URLConnection.guessContentTypeFromName(f.getAbsolutePath()));

If you are already using Spring this works for csv and excel:
import org.springframework.mail.javamail.ConfigurableMimeFileTypeMap;
import javax.activation.FileTypeMap;
import java.io.IOException;
public class ContentTypeResolver {
private FileTypeMap fileTypeMap;
public ContentTypeResolver() {
fileTypeMap = new ConfigurableMimeFileTypeMap();
}
public String getContentType(String fileName) throws IOException {
if (fileName == null) {
return null;
}
return fileTypeMap.getContentType(fileName.toLowerCase());
}
}
or with javax.activation you can update the mime.types file.

The CSV will start with text and the excel type is most likely binary.
However the simplest approach is to try to load the excel document using POI. If this fails try to load the file as a CSV, if that fails its possibly neither type.

How to convert InputStream to virtual File

I have a method which expects the one of the input variable to be of java.io.File type but what I get is only InputStream. Also, I cannot change the signature of the method.
How can I convert the InputStream into File type with out actually writing the file on to the filesystem?

Something like this should work. Note that for simplicity, I've used a Java 7 feature (try block with closeable resource), and IOUtils from Apache commons-io. If you can't use those it'll be a little longer, but the same idea.
import org.apache.commons.io.IOUtils;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
public class StreamUtil {
public static final String PREFIX = "stream2file";
public static final String SUFFIX = ".tmp";
public static File stream2file (InputStream in) throws IOException {
final File tempFile = File.createTempFile(PREFIX, SUFFIX);
tempFile.deleteOnExit();
try (FileOutputStream out = new FileOutputStream(tempFile)) {
IOUtils.copy(in, out);
}
return tempFile;
}
}

You can't. The input stream is just a generic stream of data and there is no guarantee that it actually originates from a File. If someone created an InputStream from reading a web service or just converted a String into an InputStream, there would be no way to link this to a file. So the only thing you can do is actually write data from the stream to a temporary file (e.g. using the File.createTempFile method) and feed this file into your method.

If you want to use the MultiPartFile in testing like getting a document from the resource folder -- you can use this.
import org.springframework.mock.web.MockMultipartFile;
ClassLoader classLoader = SomeClass.class.getClassLoader();
InputStream inputStream = classLoader.getResourceAsStream(BASE_DIR + fileName);
MockMultipartFile("file", "NameOfTheFile", "multipart/form-data", inputStream);

Why am I getting a NullPointerException when trying to read a file?

I use this test to convert txt to pdf :
package convert.pdf;
//getResourceAsStream(String name) : Returns an input stream for reading the specified resource.
//toByteArray : Get the contents of an InputStream as a byte[].
import java.io.FileOutputStream;
import java.io.IOException;
import org.apache.commons.io.IOUtils;
import convert.pdf.txt.TextConversion;
public class TestConversion {
private static byte[] readFilesInBytes(String file) throws IOException {
return IOUtils.toByteArray(TestConversion.class.getResourceAsStream(file));
}
private static void writeFilesInBytes(byte[] file, String name) throws IOException {
IOUtils.write(file, new FileOutputStream(name));
}
//just change the extensions and test conversions
public static void main(String args[]) throws IOException {
ConversionToPDF algorithm = new TextConversion();
byte[] file = readFilesInBytes("/convert/pdf/text.txt");
byte[] pdf = algorithm.convertDocument(file);
writeFilesInBytes(pdf, "text.pdf");
}
}
Problem:
Exception in thread "main" java.lang.NullPointerException
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1025)
at org.apache.commons.io.IOUtils.copy(IOUtils.java:999)
at org.apache.commons.io.IOUtils.toByteArray(IOUtils.java:218)
at convert.pdf.TestConversion.readFilesInBytes(TestConversion.java:17)
at convert.pdf.TestConversion.main(TestConversion.java:28)
I use the debugger, and the problem seems to be located here :
private static byte[] readFilesInBytes(String file) throws IOException {
return IOUtils.toByteArray(TestConversion.class.getResourceAsStream(file));
}
What is my problem?

Sounds like the resource probably doesn't exist with that name.
Are you aware that Class.getResourceAsStream() finds a resource relative to that class's package, whereas ClassLoader.getResourceAsStream() doesn't? You can use a leading forward slash in Class.getResourceAsStream() to mimic this, so
Foo.class.getResourceAsStream("/bar.png")
is roughly equivalent to
Foo.class.getClassLoader().getResourceAsStream("bar.png")
Is this actually a file (i.e. a specific file on the normal file system) that you're trying to load? If so, using FileInputStream would be a better bet. Use Class.getResourceAsStream() if it's a resource bundled in a jar file or in the classpath in some other way; use FileInputStream if it's an arbitrary file which could be anywhere in the file system.
EDIT: Another thing to be careful of, which has caused me problems before now - if this has worked on your dev box which happens to be Windows, and is now failing on a production server which happens to be Unix, check the case of the filename. The fact that different file systems handle case-sensitivity differently can be a pain...

Are you checking to see if the file exists before you pass it to readFilesInBytes()? Note that Class.getResourceAsStream() returns null if the file cannot be found. You probably want to do:
private static byte[] readFilesInBytes(String file) throws IOException {
File testFile = new File(file);
if (!testFile.exists()) {
throw new FileNotFoundException("File " + file + " does not exist");
}
return IOUtils.toByteArray(TestConversion.class.getResourceAsStream(file));
}
or better yet:
private static byte[] readFilesInBytes(String file) throws IOException {
InputStream stream = TestConversion.class.getResourceAsStream(file);
if (stream == null) {
throw new FileNotFoundException("readFilesInBytes: File " + file
+ " does not exist");
}
return IOUtils.toByteArray(stream);
}

This class reads a TXT file in the classpath and uses TextConversion to convert to PDF, then save the pdf in the file system.
Here TextConversion code :
package convert.pdf.txt;
//Conversion to PDF from text using iText.
import java.io.BufferedReader;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import convert.pdf.ConversionToPDF;
import convert.pdf.ConvertDocumentException;
import com.lowagie.text.Document;
import com.lowagie.text.DocumentException;
import com.lowagie.text.Font;
import com.lowagie.text.Paragraph;
import com.lowagie.text.pdf.PdfWriter;
public class TextConversion implements ConversionToPDF {
public byte[] convertDocument(byte[] documents) throws ConvertDocumentException {
try {
return this.convertInternal(documents);
} catch (DocumentException e) {
throw new ConvertDocumentException(e);
} catch (IOException e) {
throw new ConvertDocumentException(e);
}
}
private byte[] convertInternal(byte[] documents) throws DocumentException, IOException {
Document document = new Document();
ByteArrayOutputStream pdfResultBytes = new ByteArrayOutputStream();
PdfWriter.getInstance(document, pdfResultBytes);
document.open();
BufferedReader reader = new BufferedReader( new InputStreamReader( new ByteArrayInputStream(documents) ) );
String line = "";
while ((line = reader.readLine()) != null) {
if ("".equals(line.trim())) {
line = "\n"; //white line
}
Font fonteDefault = new Font(Font.COURIER, 10);
Paragraph paragraph = new Paragraph(line, fonteDefault);
document.add(paragraph);
}
reader.close();
document.close();
return pdfResultBytes.toByteArray();
}
}
And here the code to ConversionToPDF :
package convert.pdf;
// Interface implemented by the conversion algorithms.
public interface ConversionToPDF {
public byte[] convertDocument(byte[] documentToConvert) throws ConvertDocumentException;
}
I think the problem come from my file system (devbox on windows and server is Unix).
I will try to modify my classpath.

This problem may be caused by calling methods on test.txt, which can be a folder shortcut. In other words, you're calling a method on a file that doesn't exist, resulting in a NullPointerException.