How can I use the library to download a file and print out bytes saved? I tried using
import static org.apache.commons.io.FileUtils.copyURLToFile;
public static void Download() {
URL dl = null;
File fl = null;
try {
fl = new File(System.getProperty("user.home").replace("\\", "/") + "/Desktop/Screenshots.zip");
dl = new URL("http://ds-forums.com/kyle-tests/uploads/Screenshots.zip");
copyURLToFile(dl, fl);
} catch (Exception e) {
System.out.println(e);
}
}
but I cannot display bytes or a progress bar. Which method should I use?
public class download {
public static void Download() {
URL dl = null;
File fl = null;
String x = null;
try {
fl = new File(System.getProperty("user.home").replace("\\", "/") + "/Desktop/Screenshots.zip");
dl = new URL("http://ds-forums.com/kyle-tests/uploads/Screenshots.zip");
OutputStream os = new FileOutputStream(fl);
InputStream is = dl.openStream();
CountingOutputStream count = new CountingOutputStream(os);
dl.openConnection().getHeaderField("Content-Length");
IOUtils.copy(is, os);//begin transfer
os.close();//close streams
is.close();//^
} catch (Exception e) {
System.out.println(e);
}
}
If you are looking for a way to get the total number of bytes before downloading, you can obtain this value from the Content-Length header in http response.
If you just want the final number of bytes after the download, it is easiest to check the file size you just write to.
However if you want to display the current progress of how many bytes have been downloaded, you might want to extend apache CountingOutputStream to wrap the FileOutputStream so that everytime the write methods are called it counts the number of bytes passing through and update the progress bar.
Update
Here is a simple implementation of DownloadCountingOutputStream. I am not sure if you are familiar with using ActionListener or not but it is a useful class for implementing GUI.
public class DownloadCountingOutputStream extends CountingOutputStream {
private ActionListener listener = null;
public DownloadCountingOutputStream(OutputStream out) {
super(out);
}
public void setListener(ActionListener listener) {
this.listener = listener;
}
#Override
protected void afterWrite(int n) throws IOException {
super.afterWrite(n);
if (listener != null) {
listener.actionPerformed(new ActionEvent(this, 0, null));
}
}
}
This is the usage sample :
public class Downloader {
private static class ProgressListener implements ActionListener {
#Override
public void actionPerformed(ActionEvent e) {
// e.getSource() gives you the object of DownloadCountingOutputStream
// because you set it in the overriden method, afterWrite().
System.out.println("Downloaded bytes : " + ((DownloadCountingOutputStream) e.getSource()).getByteCount());
}
}
public static void main(String[] args) {
URL dl = null;
File fl = null;
String x = null;
OutputStream os = null;
InputStream is = null;
ProgressListener progressListener = new ProgressListener();
try {
fl = new File(System.getProperty("user.home").replace("\\", "/") + "/Desktop/Screenshots.zip");
dl = new URL("http://ds-forums.com/kyle-tests/uploads/Screenshots.zip");
os = new FileOutputStream(fl);
is = dl.openStream();
DownloadCountingOutputStream dcount = new DownloadCountingOutputStream(os);
dcount.setListener(progressListener);
// this line give you the total length of source stream as a String.
// you may want to convert to integer and store this value to
// calculate percentage of the progression.
dl.openConnection().getHeaderField("Content-Length");
// begin transfer by writing to dcount, not os.
IOUtils.copy(is, dcount);
} catch (Exception e) {
System.out.println(e);
} finally {
IOUtils.closeQuietly(os);
IOUtils.closeQuietly(is);
}
}
}
commons-io has IOUtils.copy(inputStream, outputStream). So:
OutputStream os = new FileOutputStream(fl);
InputStream is = dl.openStream();
IOUtils.copy(is, os);
And IOUtils.toByteArray(is) can be used to get the bytes.
Getting the total number of bytes is a different story. Streams don't give you any total - they can only give you what is currently available in the stream. But since it's a stream, it can have more coming.
That's why http has its special way of specifying the total number of bytes. It is in the response header Content-Length. So you'd have to call url.openConnection() and then call getHeaderField("Content-Length") on the URLConnection object. It will return the number of bytes as string. Then use Integer.parseInt(bytesString) and you'll get your total.
Related
I have implemented html to pdf conversion using openhtmltopdf and I use it in Struts 2 action and it works very well. However, in the case of very large data, e.g. the html data is > 3Mb (pdf file ~1.6Mb) when I test it with JMeter for 50 hits the application crashes with message java.lang.OutOfMemoryError: Java heap space.
If I increase the java limit with the -Xmx option I just get some extra hits
The code i use is like this:
First clean html
public class HtmlToXhtmlConverterHTMLCleaner2 extends AbstractHtmlToXhtmlConverter
implements IHtmlToXhtmlConverter {
public HtmlToXhtmlConverterHTMLCleaner2(String htmlData) {
super(htmlData);
}
#Override
public void convert() {
final HtmlCleaner cleaner = new HtmlCleaner();
CleanerProperties cleanerProperties = cleaner.getProperties();
cleanerProperties.setAdvancedXmlEscape(true);
cleanerProperties.setOmitXmlDeclaration(true);
cleanerProperties.setOmitDoctypeDeclaration(false);
cleanerProperties.setTranslateSpecialEntities(true);
cleanerProperties.setTransResCharsToNCR(true);
cleanerProperties.setRecognizeUnicodeChars(true);
cleanerProperties.setIgnoreQuestAndExclam(true);
cleanerProperties.setUseEmptyElementTags(false);
cleanerProperties.setPruneTags("script");
final XmlSerializer xmlSerializer = new PrettyXmlSerializer(cleanerProperties);
try {
final TagNode rootTagNode = cleaner.clean(htmlData);
this.xhtmlData = xmlSerializer.getAsString(rootTagNode);
} catch (Exception ex) {
ex.printStackTrace();
}
}
then convert cleaned html to pdf
public class PDFConverterHtmlToPdf extends AbstractPDFConverter implements IPDFConverter {
ByteArrayOutputStream psfData;
public PDFConverterHtmlToPdf(String xhtmlData, String cssFile) {
super();
this.xhtmlData = xhtmlData;
this.cssFile = cssFile;
}
#Override
public void convert() {
pdfData = new ByteArrayOutputStream();
try {
// There are more options on the builder than shown below.
PdfRendererBuilder builder = new PdfRendererBuilder();
if(cssFile != null && cssFile.length() > 0){
builder.withHtmlContent(xhtmlData, cssFile);
} else {
builder.withHtmlContent(xhtmlData, "");
}
builder.toStream(pdfData);
builder.run();
} catch (Exception e) {
e.printStackTrace();
}
}
}
then send data from strus2 action to request
private void buildPdfContent(String htmlContent) {
String pdfConverterCssFile = "http://localhost:8080/DocumentConverterApi/css/htmlToPdf.css";
PDFConverterHelp pdfConverterHelp = new PDFConverterHelp("demo.pdf",
htmlContent, pdfConverterCssFile);
pdfConverterHelp.build();
inputStream = new ByteArrayInputStream(pdfConverterHelp.getPDFFile().toByteArray());
pdfConverterHelp.closePdfData();
contentDisposition = "inline;filename=\"" + "demo.pdf\"";
}
I'm doing something wron?
Is there any other way to implement it without the risk of crashing the application?
I am new to Java and I have a project to do, so I have a java file and the user have to choose from a listing of files in a directory. The input from user is saved in a variable (fileName). I want to use that variable in another java file for doing some other work. I searched online but didn't find any solution that works for me. Probably I've done something wrong.
code of the first file:
public class Director {
private static void copyFileUsingStream(File source, File dest) throws IOException {
InputStream is = null;
OutputStream os = null;
try {
is = new FileInputStream(source);
os = new FileOutputStream(dest);
byte[] buffer = new byte[1024];
int length;
while ((length = is.read(buffer)) > 0) {
os.write(buffer, 0, length);
}
} finally {
is.close();
os.close();
}
}
public static void main(String[] args) throws IOException {
// Creates an array in which we will store the names of files and directories
String[] pathnames;
// Creates a new File instance by converting the given pathname string
// into an abstract pathname
File f = new File("C:\\Users\\miltos\\Desktop\\polimesa\\available_videos");
// Populates the array with names of files and directories
pathnames = f.list();
System.out.println("Files in the directory:");
// For each pathname in the pathnames array
for (String pathname : pathnames) {
// Print the names of files and directories
System.out.println(pathname);
}
Scanner myObj = new Scanner(System.in); // Create a Scanner object
System.out.println("Enter file name");
String fileName = myObj.nextLine();
File source = new File("C:\\Users\\miltos\\Desktop\\polimesa\\available_videos\\" + fileName);
File dest = new File("C:\\Users\\miltos\\Desktop\\polimesa\\raw_videos\\" + fileName);
copyFileUsingStream(source, dest);
}
}
code of the second file that i want to use the input:
public class TestFFMpeg {
static Logger log = LogManager.getLogger(TestFFMpeg.class);
public static void main(String[] args) {
FFmpeg ffmpeg = null;
FFprobe ffprobe = null;
try {
log.debug("Initialising FFMpegClient");
ffmpeg = new FFmpeg("C:\\Users\\miltos\\ffmpeg\\bin\\ffmpeg.exe");
ffprobe = new FFprobe("C:\\Users\\miltos\\ffmpeg\\bin\\ffprobe.exe");
} catch (IOException e) {
e.printStackTrace();
}
log.debug("Creating the transcoding");
FFmpegBuilder builder = new FFmpegBuilder()
.setInput("C:\\Users\\miltos\\Desktop\\polimesa\\raw_videos\\" + filename) //updated
.addOutput("C:\\Users\\miltos\\Desktop\\polimesa\\videos\\" + filename) //updated
.setVideoBitRate(200000)
.done();
log.debug("Creating the executor");
FFmpegExecutor executor = new FFmpegExecutor(ffmpeg, ffprobe);
log.debug("Starting the transcoding");
// Run a one-pass encode
executor.createJob(builder).run();
log.debug("Transcoding finished");
}
}
I created a variable names filename in class second also, which you will pass from the class one , while creating an object of class second like
TestFFMpeg obj = new TestFFMpeg();
obj.methodInSecondClass(filename);
Second Class :
public class TestFFMpeg {
static Logger log = LogManager.getLogger(TestFFMpeg.class);
public void methodInSecondClass(String filename){
FFmpeg ffmpeg = null;
FFprobe ffprobe = null;
try {
log.debug("Initialising FFMpegClient");
ffmpeg = new FFmpeg("C:\\Users\\miltos\\ffmpeg\\bin\\ffmpeg.exe");
ffprobe = new FFprobe("C:\\Users\\miltos\\ffmpeg\\bin\\ffprobe.exe");
} catch (IOException e) {
e.printStackTrace();
}
log.debug("Creating the transcoding");
FFmpegBuilder builder = new FFmpegBuilder()
.setInput("C:\\Users\\miltos\\Desktop\\polimesa\\available_videos\\"+filename) //this is where i want the same variable
.addOutput("C:\\Users\\miltos\\Desktop\\polimesa\\videos\\"+filename) //this is where i want the same variable
.setVideoBitRate(200000)
.done();
log.debug("Creating the executor");
FFmpegExecutor executor = new FFmpegExecutor(ffmpeg, ffprobe);
log.debug("Starting the transcoding");
// Run a one-pass encode
executor.createJob(builder).run();
log.debug("Transcoding finished");
}
}
I'm trying to implement a wrapped "move" function with Xodus, but something is not working out right:
#Override
public boolean move(String appId, String name, String targetName) {
final boolean[] success = new boolean[1];
final Environment env = manager.getEnvironment(xodusRoot, appId);
final VirtualFileSystem vfs = manager.getVirtualFileSystem(env);
env.executeInTransaction(
new TransactionalExecutable() {
#Override
public void execute(#NotNull final Transaction txn) {
File file = vfs.openFile(txn, name, false);
InputStream input = vfs.readFile(txn, file);
if(input != null) {
File targetFile = vfs.openFile(txn, targetName, true);
DataOutputStream output = new DataOutputStream(vfs.writeFile(txn, targetFile));
try {
output.write(ByteStreams.toByteArray(input));
} catch (IOException e) {
e.printStackTrace();
}
vfs.deleteFile(txn, name);
success[0] = true;
}
}
});
// vfs.shutdown();
// env.close();
return success[0];
}
The problem is the file gets moved but the byte array is not getting copied, not sure if the problem is because of multiple VFS operation in the same transaction. Can someone give me a hint of why the bytes from the source file are not getting copied properly?
Looks like you are trying to implement another version of VirtualFileSystem.renameFile(..).
I have an app in which I have to read a .txt file so that I can store some values and keep them. This is working pretty well, except for the fact that I want to make those values non-readable or "non-understandable" for external users.
My idea was to convert the file content into Hex or Binary and, in the reading process, change it back to Char. The thing is that I don't have access to methods such as String.Format due to my compiler.
Here's how I'm currently reading and keeping the values:
byte[] buffer = new byte[1024];
int len = myFile.read(buffer);
String data = null;
int i=0;
data = new String(buffer,0,len);
Class to open and manipulate the file:
public class File {
private boolean debug = false;
private FileConnection fc = null;
private OutputStream os = null;
private InputStream is = null;
private String fileName = "example.txt";
private String pathName = "logs/";
final String rootName = "file:///a:/";
public File(String fileName, String pathName) {
super();
this.fileName = fileName;
this.pathName = pathName;
if (!pathName.endsWith("/")) {
this.pathName += "/"; // add a slash
}
}
public boolean isDebug() {
return debug;
}
public void setDebug(boolean debug) {
this.debug = debug;
}
public void write(String text) throws IOException {
write(text.getBytes());
}
public void write(byte[] bytes) throws IOException {
if (debug)
System.out.println(new String(bytes));
os.write(bytes);
}
private FileConnection getFileConnection() throws IOException {
// check if subfolder exists
fc = (FileConnection) Connector.open(rootName + pathName);
if (!fc.exists() || !fc.isDirectory()) {
fc.mkdir();
if (debug)
System.out.println("Dir created");
}
// open file
fc = (FileConnection) Connector.open(rootName + pathName + fileName);
if (!fc.exists())
fc.create();
return fc;
}
/**
* release resources
*/
public void close() {
if (is != null)
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
is = null;
if (os != null)
try {
os.close();
} catch (IOException e) {
e.printStackTrace();
}
os = null;
if (fc != null)
try {
fc.close();
} catch (IOException e) {
e.printStackTrace();
}
fc = null;
}
public void open(boolean writeAppend) throws IOException {
fc = getFileConnection();
if (!writeAppend)
fc.truncate(0);
is = fc.openInputStream();
os = fc.openOutputStream(fc.fileSize());
}
public int read(byte[] buffer) throws IOException {
return is.read(buffer);
}
public void delete() throws IOException {
close();
fc = (FileConnection) Connector.open(rootName + pathName + fileName);
if (fc.exists())
fc.delete();
}
}
I would like to know a simple way on how to read this content. Binary or Hex, both would work for me.
So, with some understanding of the question, I believe you're really looking for a form of obfuscation? As mentioned in the comments, the easiest way to do this is likely a form of cipher.
Consider this example implementation of a shift cipher:
Common
int shift = 11;
Writing
// Get the data to be wrote to file.
String data = ...
// cipher the data.
char[] chars = data.toCharArray();
for (int i = 0; i < chars.length; ++i) {
chars[i] = (char)(chars[i] + shift);
}
String cipher = new String(chars);
// Write the data to the cipher file.
...
Reading
// Read the cipher file.
String data = ...
// Decipher the data.
char[] chars = data.toCharArray();
for (int i = 0; i < chars.length; ++i) {
chars[i] = (char)(chars[i] - shift);
}
String decipher = new String(chars);
// Use data as required.
...
Here's an example implementation on Ideone. The output:
Data : I can read this IP 192.168.0.1
Cipher : T+nly+}plo+st~+T[+<D=9<AC9;9<
Decipher: I can read this IP 192.168.0.1
I tried to keep this as low level as possible in order to satisfy the Java 3 requirement.
Note that this is NOT secure by any means. Shift ciphers (like most ciphers in a bubble) are trivial to break by malicious entities. Please do not use this if security is an actual concern.
Your solution is too complex. With java 8, you can try :
String fileName = "configFile.txt";
try (Stream<String> stream = Files.lines(Paths.get(fileName))) {
//TO-DO .Ex
stream.forEach(System.out::println);
} catch (IOException e) {
e.printStackTrace();
}
I have got this piece of code (I didn't write, just maintaining):
public class MyMultipartResolver extends CommonsMultipartResolver{
public List parseEmptyRequest(HttpServletRequest request) throws IOException, FileUploadException {
String contentType = request.getHeader(CONTENT_TYPE);
int boundaryIndex = contentType.indexOf("boundary=");
InputStream input = request.getInputStream();
byte[] boundary = contentType.substring(boundaryIndex + 9).getBytes();
MultipartStream multi = new MultipartStream(input, boundary);
multi.setHeaderEncoding(getHeaderEncoding());
ArrayList items = new ArrayList();
boolean nextPart = multi.skipPreamble();
while (nextPart) {
Map headers = parseHeaders(multi.readHeaders());
// String fieldName = getFieldName(headers);
String subContentType = getHeader(headers, CONTENT_TYPE);
if (subContentType == null) {
FileItem item = createItem(headers, true);
OutputStream os = item.getOutputStream();
try {
multi.readBodyData(os);
} finally {
os.close();
}
items.add(item);
} else {
multi.discardBodyData();
}
nextPart = multi.readBoundary();
}
return items;
}
}
I am using commons-fileupload.jar version 1.2.1 and obviously the code is using some deprecated methods...
Anyway, while trying to use this code to upload a very large file (780 MB) I get this:
org.apache.commons.fileupload.MultipartStream$MalformedStreamException: Stream ended unexpectedly
at org.apache.commons.fileupload.MultipartStream$ItemInputStream.makeAvailable(MultipartStream.java:983)
at org.apache.commons.fileupload.MultipartStream$ItemInputStream.read(MultipartStream.java:887)
at java.io.InputStream.read(InputStream.java:89)
at org.apache.commons.fileupload.util.Streams.copy(Streams.java:94)
at org.apache.commons.fileupload.util.Streams.copy(Streams.java:64)
at org.apache.commons.fileupload.MultipartStream.readBodyData(MultipartStream.java:593)
at org.apache.commons.fileupload.MultipartStream.discardBodyData(MultipartStream.java:619)
that is thrown from 'multi.discardBodyData();' line.
My question:
How can I avoid this error and be able to be able to succeed collecting the FileItems?
catch
(org.apache.commons.fileupload.MultipartStream.MalformedStreamException e)
{
e.printStackTrace();
return ERROR;
}
Catch the exception and handle it via ..either InputStream or Return Error use it in struts action tag