Good night in my timezone.
I am building an http bot, and when i receive the response from the server i want to make two things.First is to print the body of the response and because i know that the body of the response is of the type TEXT/HTML the second thing that i make is to parse the response through a html parser(in this specific case NekoHtml).
Snippet of code :
//Print the first call
printResponse(urlConnection.getInputStream());
document = new InputSource(urlConnection.getInputStream());
parser.setDocument(document);
The problem is when i run the first line (printResponse) the second line will throw an exception.
Now the questions-> This happens because the InputStream can only be read one time ?every time that we read from the inputstream the bytes are cleaned?
How can we read more that one time the content from the inputstream ?
Thanks in advance
Best regards
In addition to what Ted Hopp said take a look at Apache Commons IO library. You will find:
IOUtils.toString(urlConnection.getInputStream(), "UTF-8") utility method that takes an input stream, fully reads it and returns a string in a given encoding
TeeInputStream is an InputStream decorator that takes will copy every read byte and copy it into a given output stream as well.
Should work:
InputStream is = new TeeInputStream(urlConnection.getInputStream(), System.out);
Read the response from the server into a byte array. You can then create a ByteArrayInputStream to repeatedly read the bytes.
As Ted Hopp said:
byte [] bytes = new byte[urlConnection.getInputStream().available()];
printResponse(new ByteArrayInputStream(bytes));
document = new InputSource(new ByteArrayInputStream(bytes));
parser.setDocument(document);
Related
Seriously.
I've been scratching around trying to find the answer to this conundrum for a while.
The request size is too large if the String is encoded, and the company won't take Base64 anyway. They actually want the binary code, but in JSON. Can anyone shed any light on how they think that other people might do this? Currently I'm processing it like this;
String addressProof = null;
if (proofRequired)
{
Part filePart = request.getPart("proof_of_address");
addressFileName = getSubmittedFileName(filePart);
InputStream fileContent = filePart.getInputStream();
final byte[] bytes = IOUtils.toByteArray(fileContent);
addressProof = new String(bytes);
//byte[] bytes64 = Base64.encodeBase64(fileBytes);
//addressProof = new String(fileBytes);
fileContent.close();
}
Am I being dim, or is this whole request, just a little bit flawed.
Many thanks.
You can send it (or receive) as a hex string. See how-to-convert-a-byte-array-to-a-hex-string-in-java.
Example output would be (if enclosed by a JSON object):
{
"content": "C5192E4190E54F5985ED09C6CD0D4BCC"
}
or just plain hex string: "C5192E4190E54F5985ED09C6CD0D4BCC"
You don't have to write it (or read) all at once. You can open two streams (in and out) and then stream the data. From file to response output stream or from request input stream to file.
Sorry but I am not sure if You want to send the bytes or receive them.
I have a web service capable of returning PDF files in two ways:
RAW: The file is simply included in the response body. For example:
HTTP/1.1 200 OK
Content-Type: application/pdf
<file_contents>
JSON: The file is encoded (Base 64) and served as a JSON with the following structure:
HTTP/1.1 200 OK
Content-Type: application/json
{
"base64": <file_contents_base64>
}
I want to be able to consume both services on Android / Java by using the following architecture:
// Get response body input stream (OUT OF THE SCOPE OF THIS QUESTION)
InputStream bodyStream = getResponseBodyInputStream();
// Get PDF file contents input stream from body stream
InputStream fileStream = getPDFFileContentsInputStream(bodyStream);
// Write stream to a local file (OUT OF THE SCOPE OF THIS QUESTION)
saveToFile(fileStream);
For the first case (RAW response), the response body will the file itself. This means that the getPDFFileContentsInputStream(InputStream) method implementation is trivial:
#NonNull InputStream getPDFFileContentsInputStream(#NonNull InputStream bodyStream) {
// Return the input
return bodyStream;
}
The question is: how to implement the getPDFFileContentsInputStream(InputStream) method for the second case (JSON response)?
You can use any json parser (like Jackson or Gson), and then use Base64InputStream from apache-commons codec.
EDIT: You can obtain an input stream from string using ByteArrayInputStream, i.e.
InputStream stream = new ByteArrayInputStream(exampleString.getBytes(StandardCharsets.UTF_8));
as stated here.
EDIT 2: This will cause 2 pass over the data, and if the file is big, you might have memory problems. To solve it, you can use Jackson and parse the content yourself like this example instead of obtaining the whole object through reflection. you can wrap original input stream in another one, say ExtractingInputStream, and this will skip the data in the underlying input stream until the encoded part. Then you can wrap this ExtractingInputStream instance in a Base64InputStream. A simple algorithm to skip unnecessary parts would be like this: In the constructor of ExtractingInputStream, skip until you have read three quotation marks. In read method, return what underlying stream returns except return -1 if the underlying stream returns quotation mark, which corresponds to the end of base 64 encoded data.
I have a file that I've been reading into a List via the following method:
List<String> doc = java.nio.file.Files.readAllLines(new File("/path/to/src/resources/citylist.csv").toPath(), StandardCharsets.UTF_8);
Is there any nice (single-line) Java 7/8/nio2 way to pull off the same feat with a file that's inside an executable Jar (and presumably, has to be read with an InputStream)? Perhaps a way to open an InputStream via the classloader, then somehow coerce/transform/wrap it into a Path object? Or some new subclass of InputStream or Reader that contains an equivalent to File.readAllLines(...)?
I know I could do it the traditional way in a half page of code, or via some external library... but before I do, I want to make sure that recent releases of Java can't already do it "out of the box".
An InputStream represents a stream of bytes. Those bytes don't necessarily form (text) content that can be read line by line.
If you know that the InputStream can be interpreted as text, you can wrap it in a InputStreamReader and use BufferedReader#lines() to consume it line by line.
try (InputStream resource = Example.class.getResourceAsStream("resource")) {
List<String> doc =
new BufferedReader(new InputStreamReader(resource,
StandardCharsets.UTF_8)).lines().collect(Collectors.toList());
}
You can use Apache Commons IOUtils#readLines:
List<String> doc = IOUtils.readLines(inputStream, StandardCharsets.UTF_8);
Calling Simple toBytes() does produce the bytes but exel throws Warning.
Lost Document information
Googling around gave me this link and looking at Javadocs for worksheet and POI HOW-TO say similar things . Basically I can not get Bytes without loosing some information and should use the write method instead.
While write does work fine I really need to send the bytes over . Is there any way I can do that ? That is get the bytes with out getting any warning .
As that mailing list post said
Invoking HSSFWorkbook.getBytes() does not return all of the data necessary to re-
construct a complete Excel file.
You can use the write method with a ByteArrayOutputStream to get at the byte array.
ByteArrayOutputStream bos = new ByteArrayOutputStream();
try {
workbook.write(bos);
} finally {
bos.close();
}
byte[] bytes = bos.toByteArray();
(The close call is not really needed for a ByteArrayOutputStream, but imho it is good style to include anyway in case its later changed to a different kind of stream.)
How about:
ByteArrayOutputStream baos = new ByteArrayOutputStream();
workbook.write(baos);
byte[] xls = baos.toByteArray();
In order to get a full excel file out, you must call the write(OutputStream) method. If you want bytes from that, just give a ByteArrayOutputStream
Im reading from a file (data.bin) using the following approach -
fis1 = new FileInputStream(file1);
String data;
dis1 = new DataInputStream(fis);
buffread1=new BufferedReader(new InputStreamReader(dis1));
while( (data= buffread1.readLine())!=null){
}
Now im getting the io exception of read error. Now im guessing that I probably am not able read the data in the file as they are stored in the following format.
#SP,IN-1009579,13:00:33,20/01/2010, $Bœ™šAe%N
B\VÈ–7$B™šAciC B]|XçF [s + ýŒ 01210B3âEªP6#·B.
the above is just one line of the file and i want to read every line of that file and carry out operation on the data that is read.
Any pointers on how the above can be accomplished would be of great help.
Cheers
That look like part of binary data. You don't want to read it entirely as character data. Rather use an InputStream instead of Reader to read binary data. To learn more about the IO essentials, consult Sun's own IO tutorial.
I guess what you want is just: (DataInputStream expects some objects that have been serialized as an array of bytes...)
buffread1=new BufferedReader(new FileReader(file1));
while( (data= buffread1.readLine())!=null){
}