I have this code:
private static void flow(InputStream is, OutputStream os, byte[] buf)
throws IOException {
int numRead;
while ((numRead = is.read(buf)) >= 0) {
os.write(buf, 0, numRead);
}
}
Which basically streams from is to the OutputStream provided.
My goal is to cache the is when the flow has completed.
As such I have:
cacheService.cache(key, bytes);
The solution to this is to implement a Caching output stream:
public class CachingOutputStream extends OutputStream {
private final OutputStream os;
private final ByteArrayOutputStream baos = new ByteArrayOutputStream();
public CachingOutputStream(OutputStream os) {
this.os = os;
}
public void write(int b) throws IOException {
try {
os.write(b);
baos.write(b);
} catch (Exception e) {
if(e instanceof IOException) {
throw e;
} else {
e.printStackTrace();
}
}
}
public byte[] getCache() {
return baos.toByteArray();
}
public void close() throws IOException {
os.close();
}
public void flush() throws IOException {
os.flush();
}
}
And do this:
final CachingOutputStream cachingOutputStream = new CachingOutputStream(outputStream);
flow(inputStream, cachingOutputStream, buff);
cached = cachingOutputStream.getCache();
if(cached != null) {
cacheService.put(cacheKey, cached);
}
Using org.apache.poi.util.IOUtils,
IOUtils.toByteArray(inputStream);
Related
I have been trying to stream|transfer a huge file available in the local file-system over restapi using streamingoutput. I keep running into heapSpace error. Can anyone help me figure out what I am doing wrong? As per my understanding, streamingoutput shouldn't keep the file in memory.
Please find the code below:
public Response getBulkBillDownload(#QueryParam("requestID") String requestID,
#QueryParam("zipFileName") String zipFileName) throws RestException {
StreamingOutput stream = null;
try {
File file = null;
Optional<File> document = getCorporatePaymentManager().getBulkBillDownloadResponse(requestID, zipFileName);
if (document.isPresent()) {
file = document.get();
} else {
throw new RestException("File not found");
}
final FileInputStream fStream = new FileInputStream(file);
// register stream to Response and it will callback with server OutputStream
stream = new StreamingOutput() {
#Override
public void write(OutputStream output) throws IOException, WebApplicationException {
pipe(fStream, output);
}
};
} catch (Exception e) {
handleException(e);
}
return Response.status(200).entity(stream).header("Content-Disposition", "attachment; filename=" + zipFileName)
.build();
}
private void pipe(InputStream is, OutputStream os) throws IOException {
byte[] buf=new byte[1024];
int bytesread = 0, bytesBuffered = 0;
while( (bytesread = is.read( buf )) > -1 ) {
os.write( buf, 0, bytesread );
bytesBuffered += bytesread;
if (bytesBuffered > 1024 * 1024) { //flush after 1MB
bytesBuffered = 0;
os.flush();
}
}
os.close();
}
I tried using HttpRequestWrapper but it keeps giving me stream closed exception. Below is my HttpRequestWrapper code. I was trying to modify the request body in preHandle method. after modifying the request body I want to send it to the controller. It seems like HandlerInterceptorAdapter been called twice. In the second time it complains that the stream is closed. I've seen post related to this issue but I could not find a solution.
public class RequestWrapper extends HttpServletRequestWrapper {
private final String body;
public RequestWrapper(HttpServletRequest request) throws IOException {
super(request);
StringBuilder stringBuilder = new StringBuilder();
BufferedReader bufferedReader = null;
try {
InputStream inputStream = request.getInputStream();
if (inputStream != null) {
bufferedReader = new BufferedReader(new InputStreamReader(inputStream));
char[] charBuffer = new char[128];
int bytesRead = -1;
while ((bytesRead = bufferedReader.read(charBuffer)) > 0) {
stringBuilder.append(charBuffer, 0, bytesRead);
}
} else {
stringBuilder.append("");
}
} catch (IOException ex) {
throw ex;
} finally {
if (bufferedReader != null) {
try {
bufferedReader.close();
} catch (IOException ex) {
throw ex;
}
}
}
body = stringBuilder.toString();
}
#Override
public ServletInputStream getInputStream() throws IOException {
final ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(body.getBytes());
ServletInputStream servletInputStream = new ServletInputStream() {
#Override public boolean isFinished() {
return false;
}
#Override public boolean isReady() {
return false;
}
#Override public void setReadListener(ReadListener readListener) {
}
public int read() throws IOException {
return byteArrayInputStream.read();
}
};
return servletInputStream;
}
#Override
public BufferedReader getReader() throws IOException {
return new BufferedReader(new InputStreamReader(this.getInputStream()));
}
public String getBody() {
return this.body;
}
}
I am using JSCH with Shell to run multiple commands against a host. everything works, but my question is how can I get System.out and save that to file as well. I am looking to copy not re-direct. I am able to do one or the other but can't do both.
try (OutputStream logOutput = new BufferedOutputStream(new FileOutputStream(outputFilePath))) {
try (InputStream login = new BufferedInputStream(new FileInputStream(outputFilePath))) {
JSch jsch = new JSch();
Session session = jsch.getSession(user, host, 22);
session.setPassword(password);
session.setConfig(getProperties());
session.connect(10 * 1000);
Channel channel = session.openChannel("shell");
//channel.setOutputStream(System.out);// I want to activate it as well as the following command
channel.setOutputStream(logOutPut, true);// I am writing it to file
try (PipedInputStream commandSource = new PipedInputStream();
OutputStream commandSink = new PipedOutputStream(commandSource)) {
CommandSender sender = new CommandSender(commandSink);
Thread sendThread = new Thread(sender);
sendThread.start();
channel.setInputStream(commandSource);
channel.connect(15 * 1000);
sendThread.join();
if (sender.exception != null) {
throw sender.exception;
}
}
channel.disconnect();
session.disconnect();
You can make a subclass of FilterOutputStream which writes the same bytes to multiple OutputStreams:
public class MultiplexOutputStream
extends FilterOutputStream {
private final OutputStream[] streams;
public MultiplexOutputStream(OutputStream stream,
OutputStream... otherStreams) {
super(stream);
this.streams = otherStreams.clone();
for (OutputStream otherStream : otherStreams) {
Objects.requireNonNull(otherStream,
"Null OutputStream not permitted");
}
}
#Override
public void write(int b)
throws IOException {
super.write(b);
for (OutputStream stream : streams) {
stream.write(b);
}
}
#Override
public void write(byte[] bytes)
throws IOException {
super.write(bytes);
for (OutputStream stream : streams) {
stream.write(bytes);
}
}
#Override
public void write(byte[] bytes,
int offset,
int length)
throws IOException {
super.write(bytes, offset, length);
for (OutputStream stream : streams) {
stream.write(bytes, offset, length);
}
}
#Override
public void flush()
throws IOException {
super.flush();
for (OutputStream stream : streams) {
stream.flush();
}
}
#Override
public void close()
throws IOException {
super.close();
for (OutputStream stream : streams) {
stream.close();
}
}
}
To make use of it in your code:
channel.setOutputStream(new MultiplexOutputStream(logOutput, System.out), true);
I'm sending a http request to get binary files (here i'm trying an image)
public class Main {
public static void main(String[] args) throws UnknownHostException,
IOException {
new Main(args);
}
public Main(String[] args) throws UnknownHostException, IOException {
lance(args);
}
private void lance(String[] args) throws UnknownHostException, IOException {
Lanceur lan = new Lanceur("www.cril.univ-artois.fr", "/IMG/arton451.jpg");
lan.requete();
}
}
public class Lanceur {
Socket s;
InputStream readStream;
OutputStream writeStream;
String host;
String ressource;
public Lanceur(String host, String ressource) throws UnknownHostException,
IOException {
s = new Socket(InetAddress.getByName(host), 80);
readStream = s.getInputStream();
writeStream = s.getOutputStream();
this.host = host;
this.ressource = ressource;
}
public void requete() throws IOException {
// String[] length = null;
writeStream.write(new String("GET " + ressource + " HTTP/1.1\r\n"
+ "Host: www.google.com\r\n" + "\r\n").getBytes());
writeStream.flush();
AnswerReader as = new AnswerReader(readStream);
as.read();
as.writeFile(this.ressource);
s.close();
}
}
public class AnswerReader {
BufferedReader br;
DataInputStream dis;
String status;
Map<String, String> attrs;
byte[] content;
public AnswerReader(InputStream is) {
br = new BufferedReader(new InputStreamReader(is));
dis = new DataInputStream(new BufferedInputStream(is));
}
public void read() throws NumberFormatException {
readStatus();
try {
readAttrs();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
String contentL = attrs.get("Content-Length");
readContent(Integer.valueOf(contentL));
}
public void readStatus() {
try {
status = br.readLine();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
public void readAttrs() throws IOException {
attrs = new HashMap<String, String>();
String line;
for (line = br.readLine(); line.length() > 0; line = br.readLine()) {
int index = line.indexOf(':');
attrs.put(line.substring(0, index), line.substring(index + 2));
}
}
private void readContent(int size) {
this.content = new byte[size];
byte[] buff = new byte[1024];
int copied = 0;
int read = 0;
while (copied < size) {
try {
read = dis.read(buff);
if (read == -1)
break;
} catch (IOException e) {
e.printStackTrace();
}
// byte[] byteArray = new String(buff).getBytes();
System.arraycopy(buff, 0, content, copied, read);
copied += read;
}
System.out.println(copied + "///" + size);
}
public void writeFile(String name) throws IOException {
String tab[] = name.split("/");
String filename = tab[tab.length - 1];
FileOutputStream fos = new FileOutputStream("./" + filename);
fos.write(content);
fos.flush();
fos.close();
}
}
The problem comes from readContent(). The content-length is fine, but it doesn't read all the data. From this example it will reads that :
22325///38125
The BufferedReader is buffering some of the binary data. You'll have to find another way to read the header lines using the unbuffered input stream instead, e.g. DataInputStream.readLine(), deprecation or no.
But you should use HttpURLConnection. It's easier.
I have tried to use this question's answer to get a functioning implementation, but I get various errors and am now down to an EOFException and on debugging, it appears the file does not get written.
The goal is to download an image from a URL, save it to internal cache, then later fetch it from that cache for displaying. Where have I gone wrong? The EOFException is thrown in CachedImage.java on the line which reads byte[] data = (byte[]) ois.readObject();
CachedImage.java
package com.example.droid;
import java.io.IOException;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.io.Serializable;
import java.nio.BufferUnderflowException;
import java.nio.ByteBuffer;
import android.graphics.Bitmap;
public class CachedImage implements Serializable {
private static final long serialVersionUID = -12345678987654321L;
private transient Bitmap _bmp;
public CachedImage(Bitmap bmp) {
this._bmp = bmp;
}
public void writeObject(ObjectOutputStream oos) throws IOException {
oos.defaultWriteObject();
if (this._bmp != null) {
int bytes = this._bmp.getWidth() * this._bmp.getHeight() * 4;
ByteBuffer buffer = ByteBuffer.allocate(bytes);
this._bmp.copyPixelsToBuffer(buffer);
if (buffer.hasArray()) {
try {
String configName = this._bmp.getConfig().name();
byte[] array = buffer.array();
oos.writeObject(array);
oos.writeInt(this._bmp.getWidth());
oos.writeInt(this._bmp.getHeight());
oos.writeObject(configName);
} catch (BufferUnderflowException e) {
}
}
} else {
oos.writeObject(null);
}
}
private void readObject(ObjectInputStream ois) throws IOException, ClassNotFoundException {
ois.defaultReadObject();
byte[] data = (byte[]) ois.readObject();
if (data != null) {
int w = ois.readInt();
int h = ois.readInt();
String configName = (String) ois.readObject();
Bitmap.Config configBmp = Bitmap.Config.valueOf(configName);
Bitmap bitmap_tmp = Bitmap.createBitmap(w, h, configBmp);
ByteBuffer buffer = ByteBuffer.wrap(data);
bitmap_tmp.copyPixelsFromBuffer(buffer);
this._bmp = bitmap_tmp.copy(configBmp, true);
bitmap_tmp.recycle();
} else {
this._bmp = null;
}
}
public Bitmap getBitmap() {
return this._bmp;
}
}
And here are the code segments which trigger the calls:
Async callback function for when the image is fetched from the URL to write the image to internal:
#Override
protected void onPostExecute(Bitmap result) {
if (result != null) {
FileOutputStream output = null;
ObjectOutputStream oos = null;
try {
output = ICEApplication.getAppContext().openFileOutput(filename, Context.MODE_PRIVATE);
oos = new ObjectOutputStream(output);
CachedImage cachedImage = new CachedImage(result);
oos.writeObject(cachedImage);
} catch (Exception e) {
Log.d("DEBUG", "Exception: " + e.getMessage());
} finally {
if (output != null) {
try {
oos.close();
output.close();
} catch (IOException e) {
}
}
}
}
}
The code to read the image from disk after downloaded and saved:
Bitmap image = null;
FileInputStream input = null;
ObjectInputStream ois = null;
try {
input = ICEApplication.getAppContext().openFileInput(urldisplay);
ois = new ObjectInputStream(input);
CachedImage cachedImage = (CachedImage)ois.readObject();
image = cachedImage.getBitmap();
} catch (Exception e) {
Log.d("DEBUG", "Exception: " + e.getMessage());
return null;
} finally {
if (input != null) {
try {
ois.close();
input.close();
} catch (IOException e) {
}
}
}
I read from
http://www.javablogging.com/what-are-writeobject-and-readobject-customizing-the-serialization-process/
ObjectOutputStream uses reflection to find out if those methods are declared. It uses getPrivateMethod so those methods have to be declared private in order to be used by the ObjectOutputStream
So, change CachedImage's method writeObject to private(because you posted it as public).