Develop Reference

Force a fresh HTTP Response

I have a method which sends a Request and gets a Response back from a Web Site.
The response changes every 4 hours.
I have the same Java Code on two Computers.
On the first one the Response is still the same as 24 hours ago, even now.
Edit : The second one have also an old (cached) response but a day later (when I first started the Programm).
If I try to do the same Request/Response over this website (https://apitester.com/) it gives me each time the expected result. However my program does not.
I'm using the same settings everywhere such as the website url and User Agent.
So how can I force a new, fresh Version of the Response?
(Cloudflare maybe caching the site for a once used ip adress ?)
Edited Version (still do not work):
String url = "WEBSITE_PLACEHOLDER";
final String USER_AGENT = "AGENT_PLACEHOLDER";
URL obj = new URL(url);
HttpURLConnection con = (HttpURLConnection) obj.openConnection();
// add reuqest header
con.setRequestProperty("Cache-Control", "no-cache");
con.setRequestProperty("Cache-Control", "must-revalidate");
con.setUseCaches(false);
con.setRequestMethod("POST");
con.setRequestProperty("User-Agent", USER_AGENT);
con.setRequestProperty("Accept-Language", "en-US,en;q=0.5");
// Send post request
int responseCode = con.getResponseCode();
if(responseCode == 200) {
BufferedReader in = new BufferedReader(new
InputStreamReader(con.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
//System.out.println(response.toString());
return response.toString();
}
else {
throw new Exception();
}
}
#Mike
- Runned Header and Body Response on Both Systems
1st Computer (getting even now only the old response Body
I will marked the needed token with "This is what i get and This is what i need"
Printing Response Header...
Transfer-Encoding ,Value : [chunked]
null ,Value : [HTTP/1.1 200 OK]
Strict-Transport-Security ,Value : [max-age=5184000]
CF-RAY ,Value : [46bb5ec486df978c-FRA]
Server ,Value : [cloudflare]
Connection ,Value : [keep-alive]
Vary ,Value : [Accept-Encoding]
Set-Cookie ,Value : [__cfduid=d77d33fbc1ca290cd37b93c3eacff74c31539869538;
expires=Fri, 18-Oct-19 13:32:18 GMT; path=/; domain=."website"; HttpOnly]
Date ,Value : [Thu, 18 Oct 2018 13:32:18 GMT]
Content-Type ,Value : [application/json; charset=utf-8]
Server - cloudflare
Get Response Body:
{"response":{"token":"26557695w66YornObzClheLDDBTd","ruleset":"guest","nextPing":562312,
//This is what i get
"signed":"**eyJkYXRhIjoie1widGltZVwiOjE1Mzk4Njk1MzgwNTAsXCJ2YWxpZFVudGlsXCI6MTUzOTg3MDEzODA1MCxcImlwc1wiOltcIjkwLjE4Ny4zNS45M1wiXSxcInJ1bGVzZXRcIjpcImd1ZXN0XCIsXCJ2ZXJpZmllZFwiOmZhbHNlLFwiZXJyb3JcIjpudWxsLFwiYXBwXCI6e1wicGxhdGZvcm1cIjpudWxsLFwidmVyc2lvblwiOm51bGwsXCJzZXJpdmNlXCI6bnVsbH19Iiwic2lnbmF0dXJlIjoicEhsWllHcHNOQnZFVTZ1N1BWQk5uZmo4RDlpZlV5L2RneWx3TDcrdjE3NDNMQndPSGpRZEFWRkdLNmNqSCt2amtBTVd6eWZtZUh4TXN1ZWttdXJEQ3NPNXVtN0FjM0RKSFNneDFzRmZNN2ZORFJRSkY5TFpHSmJBNnEvSWtyQ0NEM01sdlpjTlNic3lhT3FGbDlZNWVaZXpHTHd1REZoUjVkZ0QyRFFaK3RRPSJ9**"}}
2nd Computer (same Programm, new Response)
Printing Response Header...
Transfer-Encoding ,Value : [chunked]
null ,Value : [HTTP/1.1 200 OK]
Strict-Transport-Security ,Value : [max-age=5184000]
CF-RAY ,Value : [46bb5a3f1427234e-FRA]
Server ,Value : [cloudflare]
Connection ,Value : [keep-alive]
Vary ,Value : [Accept-Encoding]
Set-Cookie ,Value : [__cfduid=d0ba210989762b597b0149f82d524ee371539869352; expires=Fri, 18-Oct-19 13:29:12 GMT; path=/; domain="website"; HttpOnly]
Date ,Value : [Thu, 18 Oct 2018 13:29:12 GMT]
Content-Type ,Value : [application/json; charset=utf-8]
Server - cloudflare
Get Response Body:
{"response":{"token":"265576292ukGhcUmhtl8rDBrDJtV","ruleset":"guest","nextPing":834688,
//This is what i need
"signed":"eyJkYXRhIjoie1widGltZVwiOjE1Mzk4NjkzNTI4ODAsXCJ2YWxpZFVudGlsXCI6MTUzOTg2OTk1Mjg4MCxcImlwc1wiOltcIjg3LjEyMy4xMi4yNDNcIl0sXCJydWxlc2V0XCI6XCJndWVzdFwiLFwidmVyaWZpZWRcIjpmYWxzZSxcImVycm9yXCI6bnVsbCxcImFwcFwiOntcInBsYXRmb3JtXCI6bnVsbCxcInZlcnNpb25cIjpudWxsLFwic2VyaXZjZVwiOm51bGx9fSIsInNpZ25hdHVyZSI6IkllRSsrN01GblNheE9VWjZDdWtsU2lUb1RCcFI3VTZQZWJyNlE5UU1jR3ZzSExoeDdZWGFmSytreGJqVzNMSnVCb0hMM3hGMWpHQjVuVWtiRzh2Yys3dkMrR3hBckRETnkvMmlpbXhaa3I4Yk1OZE5QQThIamxFR0FON2V5UlQ4VWNGR05ES1QwaUVtQ1dJVXZiZjhJbW5xaStDR0QwUC9LR3RDYnFIVGFVYz0ifQ=="}}

The responses are coming from CloudFlare, which is likely caching the response from the origin server. You need to make sure your request instructs the server, and any proxy servers along the way, to not use caching. You should be able to accomplish this using the Cache-Control header.
https://stackoverflow.com/a/14544664/657224

The Solution was this two commands
con.setDefaultUseCaches(false);
con.setUseCaches(false);

Why is Jersey Client response giving garbage data?

I cannot figure out why my code is giving me garbage data when I read the Http Response entity. This is only happening when I issue a request to one specific URL with data that causes a 400 response. My code attempts to read the response entity, but as you can see below it is garbage.
Here is a simplified test case:
import org.junit.Test;
import javax.ws.rs.client.Client;
import javax.ws.rs.client.ClientBuilder;
import javax.ws.rs.client.Invocation;
import javax.ws.rs.client.WebTarget;
import javax.ws.rs.core.Response;
import java.util.List;
import java.util.Map;
public class Sandbox {
#Test
public void jaguarTestCase() {
Client client = ClientBuilder.newClient();
WebTarget target = client.target("http://www.jaguarusa.com/owners/vin-recall.html?view=vinRecallQuery&vin=xxxxxxxxxxxxxxxxx");
Invocation.Builder builder = target.request();
Response response = builder.get(Response.class);
System.out.println("Response Code:");
System.out.println("\t" + response.getStatus() + " - " + response.getStatusInfo().getReasonPhrase());
System.out.println("\nResponse Headers:");
for (Map.Entry<String, List<String>> entry : response.getStringHeaders().entrySet()) {
System.out.print("\t" + entry.getKey() + ": ");
for (String value : entry.getValue()) {
System.out.println(value);
}
}
String responseEntity = response.readEntity(String.class);
System.out.println("\nResponse Entity: ");
System.out.println(responseEntity);
}
}
And the output from that testcase:
Response Code:
400 - Bad Request
Response Headers:
X-Frame-Options: SAMEORIGIN
Cache-Control: no-cache, no-store, must-revalidate
Server: Apache-Coyote/1.1
Connection: keep-alive
Content-Encoding: gzip
Vary: Accept-Encoding
Content-Length: 135
Date: Tue, 08 Nov 2016 00:03:23 GMT
Content-Language: en-US
Content-Type: application/json;charset=UTF-8
Response Entity:
� \�1
�#E��:H
���B��C2����X�wנ��z�{%1�vw��:ga�����4$ ������k�Q�-i�����y��T��!f��� c� ��iK-��?z� ���dW��
This is what the entity body is supposed to be (paste the URL in any browser and see for yourself):
{
"errorMessage" : "Please check your details and try again.",
"error" : 400,
"errorTitle" : "Sorry, that is not a valid VIN.",
"vin" : "XXXXXXXXXXXXXXXXX"
}
I am using the JDK version 1.8.0_102. I think the problem is happening when the response entity is parsed because the reported content-length of 135 is the correct value, confirmed by running this request in a Chrome browser debug window.
The Content-Type response header shows charset=UTF-8, which is what my JVM is running as. What gives? I'm completely stumped after working on this all afternoon.

If you inspect your response headers, you will notice
Content-Encoding: gzip
The garbled text is actually zipped.

You can uncompress that response.
WebTarget target = ...
target.register(GZipEncoder.class);
After this change, the readEntity thing you are doing should work fine.
This works in Jersey 2.26.
For earlier versions, the solutions are slightly different. Refer to https://stackoverflow.com/a/7574663/2695332

Android notification push empty using GCM and java with json message

I'm struggling to push notifications to Android devices.
Here is a small piece of code I wrote:
String API_KEY = "AIzaSy....";
String url = "https://android.googleapis.com/gcm/send";
String to = "ce0kUrW...";
String data = "{\"to\": \"" + to + "\"}";
RequestBody body = RequestBody.create(MediaType.parse("application/json"), data);
OkHttpClient client = new OkHttpClient();
Request request = new Request.Builder().url(url).post(body).addHeader("Authorization", "key=" + API_KEY).build();
Response response = client.newCall(request).execute();
System.out.println(response.headers());
System.out.println(response.body().string());
Looking at https://developers.google.com/cloud-messaging/http#send-to-sync, it works fine when I try the send-to-sync option. I get a notification sound on my phone, but there is no actual notification, since there is no message or data linked to the push.
Now when I replace my data String with the following line, I still get the same result:
String data = "{ \"notification\": {\"title\": \"Test title\", \"text\": \"Hi, this is a test\"},\"to\" : \""
+ to + "\"}";
I'm not sure what I'm missing. My guess is that my format of the notification is wrong, but I've tried every combination that I've found so far during my research.
They writes to the log looks like this:
Content-Type: application/json; charset=UTF-8
Date: Thu, 21 Apr 2016 10:51:23 GMT
Expires: Thu, 21 Apr 2016 10:51:23 GMT
Cache-Control: private, max-age=0
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Server: GSE
Alternate-Protocol: 443:quic
Alt-Svc: quic=":443"; ma=2592000; v="32,31,30,29,28,27,26,25"
Transfer-Encoding: chunked
OkHttp-Selected-Protocol: http/1.1
OkHttp-Sent-Millis: 1461235877551
OkHttp-Received-Millis: 1461235877855
{"multicast_id":6596853786874127657,"success":1,"failure":0,"canonical_ids":0,"results":[{"message_id":"0:1461235883968556%6ff215a7f9fd7ecd"}]}

The log entry indicates that your message was processed successfully. However for the notification with payload option you should send a JSON message like this:
{
"to" : "<your-recipient-id>",
"notification" : {
"body" : "Hi, this is a test",
"title" : "My test"
}
}
Please change as suggested and let us know if this helps.
You can check the Messaging Concepts and Options on Developer Guides - the guide provides some useful examples.
Maybe as an example - this is how you can handle the notifications.
In your app's onMessageReceived() you process it by retrieving the notification payload using notification as a key. For example:
public void onMessageReceived(String from, Bundle data) {
String notificationJSONString = data.getString("notification");
//then you can parse the notificationJSONString into a JSON object
JSONObject notificationJSON = new JSONObject(notificationJSONString );
String body = notificationJSON.getString("body");
Log.d(TAG, "Notification Message is : " + body);
}

How to add headers of HTTP request to response

Sorry if the question is possibly repeated. I'm not familiar with Java and I'm stuck with a Cordova plugin which returns headers in a non-JSON structure which I think is Map.soString() presentation of request.headers()
//These parts works fine returning response body
HttpRequest request = HttpRequest.post(this.getUrlString());
this.setupSecurity(request);
request.headers(this.getHeaders());
request.acceptJson();
request.contentType(HttpRequest.CONTENT_TYPE_JSON);
request.send(getJsonObject().toString());
int code = request.code();
String body = request.body(CHARSET);
JSONObject response = new JSONObject();
response.put("status", code);
// in this line I must put JSON converted headers instead of request.headers()
response.put("headers", request.headers());
I've tried
String headers = request.headers().toString();
and
JSONObject headers = new JSONObject(request.headers());
to change the aforementioned line to
response.put("headers", headers);
but none of them worked.
How should I send headers as JSON in response?
More context:
Currently the response headers are:
{
null=[HTTP/1.0 200 OK],
Content-Type=[application/json],
Date=[Mon, 25 Jan 2016 07:47:31 GMT],
Server=[WSGIServer/0.1 Python/2.7.6],
Set-Cookie=[csrftoken=tehrIvP7gXzfY3F9CWrjbLXb2uGdwACn; expires=Mon, 23-Jan-2017 07:47:31 GMT; Max-Age=31449600; Path=/, sessionid=iuza9r2wm3zbn07aa2mltbv247ipwfbs; expires=Mon, 08-Feb-2016 07:47:31 GMT; httponly; Max-Age=1209600; Path=/],
Vary=[Accept, Cookie],
X-Android-Received-Millis=[1453708294595],
X-Android-Sent-Millis=[1453708294184], X-Frame-Options=[SAMEORIGIN]
}
and are sent in body of response. so I need to parse them, but I can't do.

This should be the way to do it:
JSONObject headers = new JSONObject(request.headers());
However, the "toString()" display of the headers seem to be showing a map entry with a null key. That won't work in JSON: an JSON object attribute name cannot be null. My guess is that the null key caused the crash.
So I think you need to filter out the "bad" entry; i.e. code it something like this:
JSONObject headers = new JSONObject()
for (Map.Entry entry: request.headers().entries()) {
if (entry.getKey() != null) {
headers.put(entry.getKey(), entry.getValue());
}
}

Using Jersey to get a CSRF token through REST and use it in a login

Using Jersey 2.19, How do I get a CSRF token from a server which uses Spring Security 3 and make a successful login? I have two projects, a client which uses REST, and a server which was created using JHipster.
First, I'm making a get request to http://localhost:8080 and I'm getting this response headers:
Cache-Control:no-cache, no-store, max-age=0, must-revalidate
Content-Language:en
Content-Length:17229
Content-Type:text/html;charset=utf-8
Date:Tue, 21 Jul 2015 19:24:40 GMT
Expires:0
Last-Modified:Thu, 02 Jul 2015 17:07:31 GMT
Pragma:no-cache
Server:Apache-Coyote/1.1
Set-Cookie:CSRF-TOKEN=0902449b-bac7-43e8-bf24-9ec2c1faa48b; Path=/
X-Application-Context:application:dev:8081
X-Content-Type-Options:nosniff
X-XSS-Protection:1; mode=block
I extract the Set-Cookie header and I get the CSRF token from there. Then I'm making a post request this way:
http://localhost:8080/api/authentication?j_username=user&j_password=user&submit=Login
With this request headers:
Content-Type: application/x-www-form-urlencoded
X-CSRF-TOKEN: <extracted token>
Using Chrome's plugin postman, I can make a correct post request for login, but with Jersey, I'm unable to send correctly the CSRF token (I get 403 response).
This is the response:
{"timestamp":1437507680089,"status":403,"error":"Forbidden","message":"Expected CSRF token not found. Has your session expired?","path":"/api/authentication"}
This is the jersey code:
WebTarget hostTarget = getClient().target("http://localhost:8080");
Response r = hostTarget.request().get();
String header = r.getHeaderString("Set-Cookie");
String csrf = null;
List<HttpCookie> cookies = HttpCookie.parse(header);
for (HttpCookie c : cookies) {
if("CSRF-TOKEN".equals(c.getName())){
csrf = c.getValue();
break;
}
}
WebTarget loginTarget = hostTarget.path("/api/authentication");
loginTarget = loginTarget.queryParam("j_username", username)
.queryParam("j_password", password)
.queryParam("submit", "Login");
Builder req = loginTarget.request(MediaType.APPLICATION_JSON_TYPE);
if (csrf != null) {
req = req.header("X-CSRF-TOKEN", csrf);
}
Response cr = req.post(Entity.entity(null,
MediaType.APPLICATION_FORM_URLENCODED_TYPE));
System.out.println("Response: " + cr.readEntity(String.class));
Thanks for your time.

After much trial and error, I found the solution. Is important to take in count cookies (as indicated by Roman Vottner) for REST configuration to communicate with spring security. The important cookie that must be present is JSESSIONID and the header X-CSRF-TOKEN (or whatever header name is configured in the server), so capture them in a initial request and send them again.
I've decided to send all the cookies to the server in this way.
WebTarget hostTarget = getClient().target("http://localhost:8080");
Response r = hostTarget.request().get();
String headerCookies = r.getHeaderString("Set-Cookie");
Map<String, NewCookie> cookies = r.getCookies();
String csrf = cookies.get("CSRF-TOKEN").getValue();
WebTarget loginTarget = hostTarget.path("/api/authentication");
loginTarget = loginTarget.queryParam("j_username", username)
.queryParam("j_password", password)
.queryParam("submit", "Login");
Builder req = loginTarget.request(MediaType.APPLICATION_JSON_TYPE);
req = req.header("Cookie", headerCookies);
if (csrf != null) {
req = req.header("X-CSRF-TOKEN", csrf);
}
Response cr = req.post(Entity.entity(null,
MediaType.APPLICATION_FORM_URLENCODED_TYPE));
//The response is empty (in my case) with status code 200
System.out.println("Response: " + cr.readEntity(String.class));