Can anyone help me why the java code is having issue and printing all data in one go instead of prinitng each chunk as javascript code
Java Code :
import org.glassfish.jersey.client.ChunkedInput;
import javax.ws.rs.client.Client;
import javax.ws.rs.client.ClientBuilder;
import javax.ws.rs.core.GenericType;
import javax.ws.rs.core.Response;
public class RunClient {
public static void main(String args[]) throws InterruptedException {
Client client = ClientBuilder.newClient();
//2 is to increase amount of data and 3(seconds) is for time b/w chunked output ,can be changed
final Response response = client.target("http://jerseyexample-ravikant.rhcloud.com/rest/jws/streaming/2/3").request()
.get();
final ChunkedInput<String> chunkedInput = response.readEntity(new GenericType<ChunkedInput<String>>() {
});
String chunk;
while ((chunk = chunkedInput.read()) != null) {
System.err.println("Next chunk received: " );
System.out.println(chunk);
}
}
}
JavaScript : (Open Page http://jerseyexample-ravikant.rhcloud.com/rest/jws and then press F12 and run below in console as javascript call not allowed from other domain)
//2 is to increase amount of data and 3(seconds) is for time b/w chunked output ,can be changed
var xhr = new XMLHttpRequest()
xhr.open("GET", "http://jerseyexample-ravikant.rhcloud.com/rest/jws/streaming/2/3", true)
xhr.onprogress = function () {
console.log("PROGRESS:", xhr.responseText) ;console.log("\n");
}
xhr.send()
EDIT : Just for help it also works will normal java connection
String uri = "http://jerseyexample-ravikant.rhcloud.com/rest/jws/streaming/3/1";
URL url = new URL(uri);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("GET");
connection.setDoOutput(true);
BufferedReader in = new BufferedReader(new InputStreamReader(connection.getInputStream()));
String line;
while ((line = in.readLine()) != null) {
System.out.println(line);
}
in.close();
My WebService Code
#Path("streaming/{param}/{sleepTime}")
#GET
#Produces(MediaType.TEXT_PLAIN)
public ChunkedOutput<String> getChunkedStream(#PathParam("param") String loopcount,#PathParam("sleepTime") String sleepTime) throws Exception {
final ChunkedOutput<String> output = new ChunkedOutput<>(String.class);
final Integer val=Integer.parseInt(loopcount);
final Integer isleepTime=Integer.parseInt(sleepTime)*1000;
new Thread(new Runnable() {
#Override
public void run() {
try {
StringBuffer chunk = null;
for (int i = 0; i < 10; i++) {
chunk = new StringBuffer();
for (int j = 0; j < val; j++) {
chunk.append(" Message #" + i+ "#"+j);
}
output.write(chunk.toString()+"\n");
System.out.println("write");
Thread.sleep(isleepTime);
}
} catch (Exception e) {
e.printStackTrace();
} finally {
try {
System.out.println("output.close();");
output.close();
} catch (IOException ex) {
ex.printStackTrace();
}
}
}
}).start();
return output;
}
Form jersey docs:
Writing chunks with ChunkedOutput is simple, you only call method write() which writes exactly one chunk to the output. With the input reading it is slightly more complicated. The ChunkedInput does not know how to distinguish chunks in the byte stream unless being told by the developer. In order to define custom chunks boundaries, the ChunkedInput offers possibility to register a ChunkParser which reads chunks from the input stream and separates them. Jersey provides several chunk parser implementation and you can implement your own parser to separate your chunks if you need. In our example above the default parser provided by Jersey is used that separates chunks based on presence of a \r\n delimiting character sequence.
So your server has to separate chunks with \r\n, or you have to register a ChunkParser.
Assuming you have a constant finalising each chunk you could try:
Client client = ClientBuilder.newClient();
//2 is to increase amount of data and 3(seconds) is for time b/w chunked output ,can be changed
final Response response = client.target("http://jerseyexample-ravikant.rhcloud.com/rest/jws/streaming/2/3").request()
.get();
final ChunkedInput<String> chunkedInput = response.readEntity(new GenericType<ChunkedInput<String>>() {
});
chunkedInput.setParser(ChunkedInput.createParser(BOUNDARY));
String chunk;
while ((chunk = chunkedInput.read()) != null) {
System.err.println("Next chunk received: " );
System.out.println(chunk);
}
While BOUNDARY is a finalizing string for each chunk.
The in.readLine solution in your edit will break down "chunks" by every newline, even if one chunk consist a \n, it will be interpreted as 2 chunks.
Related
Please explain to me, what is the cause of distortion of JSON-String sent from JavaFX application to Tomcat server. Part of symbols is replaced by strange square symbols:
Conversation to JSON pass correctly – I checked it by printing JSON-String to console in JavaFX app after serializing. But after transfer to Tomcat-server by DoPost method JSON-String is distorted. And I tried to transfer XML too – it distorts by the same way.
After small investigation, I got some result: String sent by DoPost stays correct if it's length is 7782 symbols or less. Plus one symbol – and squares appears:
Both the JavaFX app and Tomcat server starts on local machine under Intellij IDEA, so it's not a network problem.
Very thanks to user Nick on ru.stackoverflow.com:
https://ru.stackoverflow.com/users/216863/nick
He gave exact cause – length of inputStream. GZIP gives not correct value of inputStream length to Servlet.
And when I replaced this code:
`public String getInputString(HttpServletRequest req) {
String receivedString = "";
int len = req.getContentLength();
byte[] input = new byte[len];
try {
ServletInputStream sin = req.getInputStream();
int c = 0;
int count = 0;
while ((c = sin.read(input, count, (input.length - count))) > 0) {
count += 1;
}
sin.close();
} catch (IOException e) {
}
receivedString = new String(input);
return receivedString;
}`[PasteBin-1][1]
by this:
`public String getInputString(HttpServletRequest req) {
String receivedString = "";
try (BufferedReader reader = new BufferedReader(new InputStreamReader(req.getInputStream()))) {
StringBuilder sb = new StringBuilder("");
while (reader.ready()) {
sb.append(reader.readLine());
}
if (sb.length() > 0) {
receivedString = sb.toString();
}
} catch (IOException e) {
}
return receivedString;
}`[PasteBin-2][2]
everything started to work correctly.
This is same question thread on russian:
answer on ru.stackoverflow.com
I'm using SocketChannel for single connection like this:
int sampleBufferSize = 50;
StringBuilder data = new StringBuilder();
ByteBuffer bf = ByteBuffer.allocate(sampleBufferSize);
SocketChannel sc = new SocketChannel();
while(true)
if(sc.read(bf) > 0){
bf.flip();
while(bf.hasRemaining())
data.append((char) bf.get());
bf.clear();
}else{
fireDataReceived(data.toString());
data.delete(0, data.length());
}
This code is not very efficient, but it reads HTTP POST request 130 KB from the same PC in 0.05 seconds. Now I'm trying to write a class with similar functionality but using Socket. Here is the code:
private static final int TIMEOUT_MILLIS = 50;
private boolean reading = false;
private long readBeginTime = 0;
private StringBuilder buffer = new StringBuilder();
private Thread thread = new Thread(){
public void run(){
while(!isInterrupted()){
try {
int b = getInputStream().read();
if(b == -1){
if(reading)
fireDataReceived();
close();
}else{
if(!reading){
reading = true;
readBeginTime = System.currentTimeMillis();
setSoTimeout(TIMEOUT_MILLIS);
}
buffer.append((char) b);
}
} catch (SocketTimeoutException e){
fireDataReceived();
} catch (IOException e) {
e.printStackTrace();
if(reading)
fireDataReceived();
close();
}
}
}
};
private void fireDataReceived(){
BufferedSocketEvent e = new BufferedSocketEvent(this, System.currentTimeMillis() - readBeginTime, buffer.toString());
buffer.setLength(0);
reading = false;
try {
setSoTimeout(0);
} catch (SocketException e1) {
e1.printStackTrace();
}
for(BufferedSocketListener listener: listeners)
listener.dataReceived(e);
}
And the problem is that it takes 0.4 seconds for the same request and I have no idea why does it take so long. Please, explain what is wrong with my code.
The problem with your streams code is that you're reading a byte at a time, which is slow, and also appending to the StringBuilder one at a time, ditto. Try this:
BufferedReader in = new BufferedReader(new InputStreamReader(socket.getInputStream());
char[] chars = new char[8192];
int count;
while ((count = in.read(chars)) > 0)
{
buffer.append(chars, 0, count);
fireDataReceived();
}
Note that it isn't correct to use read timeouts as a means to separate requests. You need to parse the data to do that. TCP is a streaming protocol without message boundaries, and there are no guarantees about separate sends being received separately.
You have to use readLine() or similar method. End of HTTP header is marked by empty line. If you don't detect lines you cannot know when to stop reading data. Relying on timeouts or TCP fragments is not a correct solution. If request doesn't contain new line, you need to wait until one appears or the connection is terminated/timed out. You should wait at least a couple of seconds.
I would also get rid of those listeners. Being able to add multiple listeners for single socket might seem like a good idea but the only sensible thing to do with HTTP header is to parse it. The parser also needs to inform the reader when to stop reading.
I would start with something like this:
ServerSocket serverSocket = new ServerSocket(8888);
Socket clientSocket = serverSocket.accept();
BufferedReader reader = new BufferedReader(new InputStreamReader(clientSocket.getInputStream(), "ASCII"));
String[] r = reader.readLine().split(" ");
String type = r[0];
String resource = r[1];
String version = r[2];
Map<String, String> headers = new HashMap<String,String>();
while(true) {
String line = reader.readLine();
if(line.isEmpty()) {
break;
}
String headerLine[] = line.split(":",2);
headers.put(headerLine[0],headerLine[1].trim());
}
processHeader(type, resource, version, headers);
I'm a building a basic program to query Target's API with a store ID and Product ID which returns the aisle location. I think I'm using the URL constructor incorrectly, however (I've had trouble with it in the past and still don't fully understand them). Below is the code I have, redacted the API Key for obvious reasons. The URL I create is valid when put into a browser and no exceptions are thrown but at the the end when I print out the contents of the page it is null. What am I missing? Any help is really appreciated!
package productVerf;
import java.net.*;
import java.io.*;
public class Verify {
public static void main(String args[]) {
// first input is store id second input is product id
String productID = args[0];
String storeID = args[1];
String file = "/v2/products/storeLocations?productId=" + productID
+ "&storeId=" + storeID
+ "&storeId=694&key=REDACTED";
URL locQuery;
URLConnection lqConection = null;
try {
locQuery = new URL("http", "api.target.com", file);
lqConection = locQuery.openConnection();
} catch (IOException e) {
e.printStackTrace();
}
BufferedReader response;
String responseString = "";
try {
response = new BufferedReader(new InputStreamReader(
lqConection.getInputStream()));
while (response.readLine() != null) {
responseString += response.readLine();
}
} catch (IOException e) {
e.printStackTrace();
}
System.out.println(responseString);
}
}
Maybe you are reading only even lines
you are reading a line twice? (in while statement...), it looks you reads the first line which is dropped in while condition test. If your response contains only one line, nothing will be readed
use this:
String line;
while ((line=response.readLine()) != null) {
responseString += line;
}
I'm trying to implement an Android application to connect to the open source software Motion. The goal is to be able to check the status of the application and get the last image captured.
I do not program in Java very much, my background is principally in C and Python. I've not had any real issues with understanding the UI part of Android, but I've found it to be incredibly painful to work with any sort of byte buffer. The Motion software has an HTTP API that is very simple. Opening the URL connection is easy in Java. The response from the default page looks like this
Motion 3.2.12 Running [4] Threads
0
1
2
3
For my purposes the first thing the application needs to do it parse out the number of threads. At some point I can also retrieve the version number from the first line, but that's not really important presently.
Here's my code
package com.hydrogen18.motionsurveillanceviewer;
import java.io.BufferedInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.HttpURLConnection;
import java.net.URL;
import java.util.List;
public class MotionHttpApi {
String host;
int port = 80;
boolean secure = false;
int numberOfThreads = -1;
String getBaseUrl()
{
StringBuilder sb = new StringBuilder();
sb.append(secure ? "https://" : "http://");
sb.append(host);
sb.append(':');
sb.append(port);
return sb.toString();
}
public int getNumberOfCameras() throws IOException
{
if(numberOfThreads == -1)
{
retrieveSplash();
}
if(numberOfThreads == 1)
{
return 1;
}
return numberOfThreads - 1;
}
void retrieveSplash () throws IOException
{
URL url = new URL(getBaseUrl());
HttpURLConnection conn = (HttpURLConnection)url.openConnection();
if(conn.getResponseCode()!=HttpURLConnection.HTTP_OK)
{
throw new IOException("Got response code" + conn.getResponseCode());
}
try{
Byte[] buffer = new Byte[512];
byte[] sbuf = new byte[128];
int offset = 0;
InputStream in = new BufferedInputStream(conn.getInputStream());
boolean foundInfoString= false;
while( ! foundInfoString)
{
//Check to make sure we have not run out of space
if(offset == buffer.length)
{
throw new IOException("Response too large");
}
//Read into the smaller buffer since InputStream
//can't write to a Byte[]
final int result = in.read(sbuf,0,sbuf.length);
//Copy the data into the larger buffer
for(int i = 0; i < result;++i)
{
buffer[offset+i] = sbuf[i];
}
//Add to the offset
offset+=result;
//Wrap the array as a list
List<Byte> list = java.util.Arrays.asList(buffer);
//Find newline character
final int index = list.indexOf((byte) '\n');
//If the newline is present, extract the number of threads
if (index != -1)
{
//Find the number of threads
//Thread number is in the first lin like "[X]"
final int start = list.indexOf((byte)'[');
final int end = list.indexOf((byte)']');
//Sanity check the bounds
if(! (end > start))
{
throw new IOException("Couldn't locate number of threads");
}
//Create a string from the Byte[] array subset
StringBuilder sb = new StringBuilder();
for(int i = start+1; i != end; ++i)
{
final char c = (char) buffer[i].byteValue();
sb.append(c);
}
String numThreadsStr = sb.toString();
//Try and parse the string into a number
try
{
this.numberOfThreads = Integer.valueOf(numThreadsStr);
}catch(NumberFormatException e)
{
throw new IOException("Number of threads is NaN",e);
}
//No more values to extract
foundInfoString = true;
}
//If the InputStream got EOF and the into string has not been found
//Then an error has occurred.
if(result == -1 && ! foundInfoString )
{
throw new IOException("Never got info string");
}
}
}finally
{
//Close the connection
conn.disconnect();
}
}
public MotionHttpApi(String host,int port)
{
this.host = host;
this.port = port;
}
}
The code works just fine when you call getNumberOfCameras(). But I think I must not be really understandings omething in terms of java, because the retrieveSplash method is far too complex. I could do the same thing in just 10 or so lines of C or 1 line of Python. Surely there must be a saner way to manipulate bytes in java?
I think there are some style issues, like I probably should not be throwing IOException whenever the integer fails to parse. But that's a separate issue.
Read the first line as Gautam Tandon suggested and then use a regex.
You can then check if the regex matches and even easily extract the number.
Regex' can be created at http://txt2re.com. I've already done that for you.
The page even creates Java, Pyhton, C, etc. files for you to work with.
// URL that generated this code:
// http://txt2re.com/index-java.php3?s=Motion%203.2.12%20Running%20[4]%20Threads&-7&-19&-5&-20&-1&2&-22&-21&-62&-63&15
import java.util.regex.*;
class Main
{
public static void main(String[] args)
{
String txt="Motion 3.2.12 Running [4] Threads";
String re1="(Motion)"; // Word 1
String re2="( )"; // White Space 1
String re3="(3\\.2\\.12)"; // MMDDYY 1
String re4="( )"; // White Space 2
String re5="(Running)"; // Word 2
String re6="( )"; // White Space 3
String re7="(\\[)"; // Any Single Character 1
String re8="(\\d+)"; // Integer Number 1
String re9="(\\])"; // Any Single Character 2
String re10="( )"; // White Space 4
String re11="((?:[a-z][a-z]+))"; // Word 3
Pattern p = Pattern.compile(re1+re2+re3+re4+re5+re6+re7+re8+re9+re10+re11,Pattern.CASE_INSENSITIVE | Pattern.DOTALL);
Matcher m = p.matcher(txt);
if (m.find())
{
String word1=m.group(1);
String ws1=m.group(2);
String mmddyy1=m.group(3);
String ws2=m.group(4);
String word2=m.group(5);
String ws3=m.group(6);
String c1=m.group(7);
String int1=m.group(8);
String c2=m.group(9);
String ws4=m.group(10);
String word3=m.group(11);
System.out.print("("+word1.toString()+")"+"("+ws1.toString()+")"+"("+mmddyy1.toString()+")"+"("+ws2.toString()+")"+"("+word2.toString()+")"+"("+ws3.toString()+")"+"("+c1.toString()+")"+"("+int1.toString()+")"+"("+c2.toString()+")"+"("+ws4.toString()+")"+"("+word3.toString()+")"+"\n");
}
}
}
//-----
// This code is for use with Sun's Java VM - see http://java.sun.com/ for downloads.
//
// Paste the code into a new java application or a file called 'Main.java'
//
// Compile and run in Unix using:
// # javac Main.java
// # java Main
//
String int1=m.group(8); gives you the desired integer. Of course you can simplify the above code. It's way to verbose right now.
You can simplify the retrieveSplash method considerably by using BufferedReader. Here's a simpler version of your function:
void retrieveSplash_simpler() throws IOException {
URL url = new URL(getBaseUrl());
HttpURLConnection conn = (HttpURLConnection)url.openConnection();
// open the connection
conn.connect();
// create a buffered reader to read the input stream line by line
BufferedReader reader = new BufferedReader(new InputStreamReader(conn.getInputStream()));
// find number of threads
String firstLine = reader.readLine();
int x = firstLine.indexOf("[");
int y = firstLine.indexOf("]");
if (x > 0 && y > 0 && x < y) {
try {
numberOfThreads = Integer.parseInt(firstLine.substring(x+1, y));
} catch (NumberFormatException nfe) {
// disconnect and throw exception
conn.disconnect();
throw new IOException("Couldn't locate number of threads");
}
} else {
// disconnect and throw exception
conn.disconnect();
throw new IOException("Couldn't locate number of threads");
}
// disconnect
conn.disconnect();
}
I'd further clean up the above method by using try/catch/finally blocks at the appropriate places so that I don't have to duplicate that "conn.disconnect()". But I didn't do that here to keep it simple (try/catch/finally do become tricky sometimes...).
Apologies for the long code post but am wondering if someone can help with a multithreading question (I am quite new to multi-threading). I am trying to design a facade class to a RESTFUL web services API that can be shared with multiple threads. I am using HttpURLConnection to do the connection and Google GSON to convert to and from JSON data.
The below class is what I have so far. In this example it has one public method to make an API call (authenticateCustomer()) and the private methods are used to facilitate the API call (i.e to build the POST data string, make a POST request etc).
I make one instance of this class and share it with 1000 threads. The threads call the authenticateCustomer() method. Most of the threads work but there is some threads that get a null pointer exception which is because I haven't implemented any synchronization. If I make the authenticateCustomer() method 'synchronized' it works. The problem is this results in poor concurrency (say, for example, the POST request suddenly takes a long time to complete, this will then hold up all the other threads).
Now to my question. Is the below class not stateless and therefore thread-safe? The very few fields that are in the class are declared final and assigned in the constructor. All of the methods use local variables. The Gson object is stateless (according to their web site) and created as a local variable in the API method anyway.
public final class QuizSyncAPIFacade
{
// API Connection Details
private final String m_apiDomain;
private final String m_apiContentType;
private final int m_bufferSize;
// Constructors
public QuizSyncAPIFacade()
{
m_apiDomain = "http://*****************************";
m_apiContentType = ".json";
m_bufferSize = 8192; // 8k
}
private String readInputStream(InputStream stream) throws IOException
{
// Create a buffer for the input stream
byte[] buffer = new byte[m_bufferSize];
int readCount;
StringBuilder builder = new StringBuilder();
while ((readCount = stream.read(buffer)) > -1) {
builder.append(new String(buffer, 0, readCount));
}
return builder.toString();
}
private String buildPostData(HashMap<String,String> postData) throws UnsupportedEncodingException
{
String data = "";
for (Map.Entry<String,String> entry : postData.entrySet())
{
data += (URLEncoder.encode(entry.getKey(), "UTF-8") + "=" + URLEncoder.encode(entry.getValue(), "UTF-8") + "&");
}
// Trim the last character (a trailing ampersand)
int length = data.length();
if (length > 0) {
data = data.substring(0, (length - 1));
}
return data;
}
private String buildJSONError(String message, String name, String at)
{
String error = "{\"errors\":[{\"message\":\"" + message + "\",\"name\":\"" + name + "\",\"at\":\"" + at + "\"}]}";
return error;
}
private String callPost(String url, HashMap<String,String> postData) throws IOException
{
// Set up the URL for the API call
URL apiUrl = new URL(url);
// Build the post data
String data = buildPostData(postData);
// Call the API action
HttpURLConnection conn;
try {
conn = (HttpURLConnection)apiUrl.openConnection();
} catch (IOException e) {
throw new IOException(buildJSONError("Failed to open a connection.", "CONNECTION_FAILURE", ""));
}
// Set connection parameters for posting data
conn.setRequestMethod("POST");
conn.setUseCaches(false);
conn.setDoInput(true);
conn.setDoOutput(true);
// Write post data
try {
DataOutputStream wr = new DataOutputStream(conn.getOutputStream());
wr.writeBytes(data);
wr.flush();
wr.close();
} catch (IOException e) {
throw new IOException(buildJSONError("Failed to post data in output stream (Connection OK?).", "POST_DATA_FAILURE", ""));
}
// Read the response from the server
InputStream is;
try {
is = conn.getInputStream();
} catch (IOException e) {
InputStream errStr = conn.getErrorStream();
if (errStr != null)
{
String errResponse = readInputStream(errStr);
throw new IOException(errResponse);
}
else
{
throw new IOException(buildJSONError("Failed to read error stream (Connection OK?).", "ERROR_STREAM_FAILURE", ""));
}
}
// Read and return response from the server
return readInputStream(is);
}
/* -------------------------------------
*
* Synchronous API calls
*
------------------------------------- */
public APIResponse<CustomerAuthentication> authenticateCustomer(HashMap<String,String> postData)
{
// Set the URL for this API call
String apiURL = m_apiDomain + "/customer/authenticate" + m_apiContentType;
Gson jsonConv = new Gson();
String apiResponse = "";
try
{
// Call the API action
apiResponse = callPost(apiURL, postData);
// Convert JSON response to the required object type
CustomerAuthentication customerAuth = jsonConv.fromJson(apiResponse, CustomerAuthentication.class);
// Build and return the API response object
APIResponse<CustomerAuthentication> result = new APIResponse<CustomerAuthentication>(true, customerAuth, null);
return result;
}
catch (IOException e)
{
// Build and return the API response object for a failure with error list
APIErrorList errorList = jsonConv.fromJson(e.getMessage(), APIErrorList.class);
APIResponse<CustomerAuthentication> result = new APIResponse<CustomerAuthentication>(false, null, errorList);
return result;
}
}
}
If you are getting an error it could be because you are overloading the authentication service (something which doesn't happen if you do this one at a time) Perhaps it returning a error like 500, 503 or 504 which you could be ignoring and getting nothing you expect back, you return null http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html
I would use less threads assuming you don't have 1000 cpus, its possible having this many threads will be slower rather than more efficeint.
I would also check that your service is returning correctly every time and investigate why you get a null value.
If your service can only handle say 20 requests at once, you can try using a Semaphore as a last resort. This can be using to limit the numebr of concurrent requests.
Any stateless class is inherently threadsafe, provided that the objects it accesses are either private to the thread, or threadsafe themselves.