Develop Reference

Jenkins giving 403 in EC2 Server after passing the crumb in the request headers

i am using the CSRF Jenkins crumbs in the API call to create a new job in Jenkins from Java.
I tried the following
Called the API to get the crumb data
http://admin:11542c80972c3a2b863453d234de68b1d#10.139.163.33/crumbIssuer/api/json
I also tried with the below URL
http://10.139.163.33/crumbIssuer/api/json
The below is the JSON response obtained from the server
{"_class":"hudson.security.csrf.DefaultCrumbIssuer","crumb":"b272a09b604e7b7cc8ee1431f0a0143fa1422db2fb5f92955b0356a31da37463","crumbRequestField":"Jenkins-Crumb"}
In the next step, I am making a call to the Jenkins to create a new job with the header as
Jenkins-Crumb:b272a09b604e7b7cc8ee1431f0a0143fa1422db2fb5f92955b0356a31da37463
Jenkins is giving me 403, I am using HttpGet to get the token and using HttpPost with the header as above and sending to jenkins.
When i try with postman, it is not giving this error. I am running the Java application in 1 ec2 server and jenkins on another ec2 server.
There are no proxies, I also tried to use the various options like the enable proxy compatibilty, restarting jenkins etc, but not working.
Please give any pointers.
Java code used is
HttpPost postRequest = new HttpPost(url);
JenkinsCrumb crumb = jenkinsHelper.getCrumb();
String encodedPassword = Base64.getEncoder().encodeToString((user + ":" + pwd).getBytes());
postRequest.addHeader(HttpHeaders.AUTHORIZATION, "Basic " + encodedPassword);
postRequest.addHeader(new BasicHeader(crumb.getCrumbRequestField(), crumb.getCrumb()));
return postRequest;
The code to get the crumb is
String urlWithToken = "http://" + (user + ":" + pwd) + "#";
HttpGet request = new HttpGet(jenkinsBaseUrl.replace("http://", urlWithToken) + "crumbIssuer/api/json");
request.addHeader(HttpHeaders.AUTHORIZATION, "Basic " + encodedPassword);
CloseableHttpResponse httpResponse = httpClient.execute(request);
I have also tried with the CURL command and still getting the same response

I was able to fix this issue by using the same HttpClient (CloseableHttpClient) for both the CRUMB and the POST Request. Earlier, I was using 2 separate clients one for getting the crumb and new one for the posting of the data. Using a shared httpclient for both of these resulted in success state.
Hope this helps any other developer facing similar issue.

Getting Target host is not specified

I am calling prometheus server via Grafana I am able to make below request using postman but when I am trying same uri with java code getting below exception
Caused by: org.apache.http.ProtocolException: Target host is not specified
at org.apache.http.impl.conn.DefaultRoutePlanner.determineRoute(DefaultRoutePlanner.java:71)
at org.apache.http.impl.client.InternalHttpClient.determineRoute(InternalHttpClient.java:125)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
... 31 common frames omitted
My piece of code is as given below.
CloseableHttpClient httpClient = HttpClientBuilder.create().build();
String PM_UI_SERVER_URI = "http://" + PM_SERVER_HOST + ":" + getPMUiServerPort();
String uriStr= PM_UI_SERVER_URI + PM_SERVER_BASE_URI + queryString +"&start="+String.valueOf(startTime)+"&end="+String.valueOf(endTime)+"&step=150";
//logger.info("Execute Query uri string: "+uriStr);
String str="http://10.61.244.58:31000/api/datasources/proxy/1/api/v1/query_range?query=em_core_used_heap_mem_mb{job=\"eric-em-om-server\"}&start=1592981880&end=1592982180&step=15";
String encodedurl = URLEncoder.encode(str,"UTF-8");
//URI uri = new URI(encodedurl);
//HttpGet httpget = new HttpGet("http://10.61.244.58:31000/api/datasources/1");
HttpGet httpget = new HttpGet(encodedurl);
httpget.addHeader("Authorization", token);
httpget.addHeader("Content-Type", "application/json");
CloseableHttpResponse response = httpClient.execute(httpget);
Can someone please help as I am stucked here.

Try to build up the URL piece by piece rather than going straight for the completed URL. You can do this in a debug session using the expression builder in intelliJ IDE.
set breakpoint at the line ... = httpClient.execute(httpget); and exercise this code from a test/running the application in debug mode.
highlight httpClient.execute(httpget)
either right click this selection and click "Expression builder" OR use Alt+F8
now try to perform the execute the GET (HttpGet) request for http://10.61.244.58:31000/
ensure you're getting a 200 response status or equivalent.
next you should be able to add the next bit like http://10.61.244.58:31000/api/datasources/proxy/1/api/v1/query_range (Note I'd also try without the proxy as the URL may be problematic because your requests are being proxied according to the docs here: https://grafana.com/docs/grafana/latest/http_api/data_source/#data-source-proxy-calls
I'd try then adding the query params individually and combined. This isn't going to necessarily resolve your problem... but if you can at the same time tail any logs on the grafana server/proxy server you may get some more detailed information that will help lead your investigation.

Post with HttpClient returns Bad Request

I am using Java, Spring boot and Apache HttpClient to try send a post request. The documentation of the resource I am trying to reach can be found here:
https://docs.enotasgw.com.br/v2/reference#incluiralterar-empresa
Below is my code:
CloseableHttpClient httpClient = HttpClients.createDefault();
HttpPost post = new HttpPost(incluirEmpresa);
post.setHeader("Content-Type", "application/json");
post.setHeader("Accept", "application/json");
post.setHeader("Authorization", "Basic " + apiKey);
try {
StringEntity entity = new StringEntity(json);
//tried to add these two lines to see if they would fix the error, but it is the same
entity.setContentEncoding("application/json");
entity.setContentType("application/json");
post.setEntity(entity);
System.out.println(json);
System.out.println("======================");
CloseableHttpResponse response = httpClient.execute(post);
System.out.println(response.getStatusLine().getReasonPhrase() + " - " + response.getStatusLine().getReasonPhrase());
idEmpresa = response.getEntity().getContent().toString();
}
My response is 400 - Bad Request. On the interactive documentation link above, when I post my Json, I receive the error of duplicate entry, which is what I expect since the information I am sending is already on the database.
Since the interactive documentation returns the error of duplicate, I know the problem is not within my json format, but on my post request. The documentation have samples on C#, but not on Java, which is what I am using.
By the way, the json is variable is a string in case this is relevant.
Could someone try to point to me what is wrong with my post code?

Found out what I was missing.
After reviewing what was being sent to the API, i noticed the json was not in the expected format. So I did some research and found that, at least for my case, setting the headers with the content type was not enough, I also had to set the Entity that was being set to the HttpPost, to do that, i had to change this line of the code:
StringEntity entity = new StringEntity(json);
to this:
StringEntity entity = new StringEntity(json, ContentType.APPLICATION_JSON);
After that change, the requests started to work as expected.

Capture the URI, resource name,as seen in Developer Tools, from GET POST request method in Java

I am doing this as part of enhancing a Selenium Webdriver script.
I have tried using httpclient with Java and a lot of other things but I am not able to get anywhere.
Please help!
Ths is the scenario:
After a certain action is performed in a webpage like a button click,
GET/POST methods can be seen in the Developer Tools in Chrome.
I have taken the example of Google here.
What I need here is to collect all the resource names until a certain resource appears (If you open the developer tools in Chrome and navigate to google.com , under the Network tab on the leftmost column you will see tia.png , just an example).
There are two things that should be achieved:
ensure that a certain resource was loaded
make sure the page is completely loaded (all GET / POST methods have been transferred) before any other action is taken.
The httpclient, httpurlconnection only capture one request, but a page sends a lot of requests. How do we capture all of them?

By Using http apache client , you can use something like this :-
To solve your problem, you can get the response code "response.getStatusLine().getStatusCode()" and check for the expected response code and can also play with the outcome of the response.
URI url = new URI("String URI");
DefaultHttpClient client = new DefaultHttpClient();
HttpPost post = new HttpPost(url);
HttpResponse response = HttpClientBuilder.create().build().execute(post);
System.out.println("Response Code : " + response.getStatusLine().getStatusCode());
BufferedReader rd = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
StringBuffer result = new StringBuffer();
String line = "";
while ((line = rd.readLine()) != null) {
result.append(line);
}
System.out.println("Ressponse 1 >>>>"+result.toString());
And , if you need to send parameters in your post request like :-
List<NameValuePair> urlParameters = new ArrayList<NameValuePair>();
urlParameters.add(new BasicNameValuePair("username", "yourusername"));
urlParameters.add(new BasicNameValuePair("password", "yourPassword"));
post.setEntity(new UrlEncodedFormEntity(urlParameters));

This is the answer to my question :
https://kenneth.io/blog/2014/12/28/taking-chrome-devtools-outside-the-browser/
https://github.com/auchenberg/chrome-devtools-app
From the blog :
It’s a standalone app that runs Chrome DevTools in its own process.
It’s powered by node-webkit, and it’s able to run on Windows, Mac and
Linux, completely independently of Chrome.
I wanted to capture all the requests (as seen in the screenshot) between the application and the server so that I could ensure all the critical resources, requests have been transferred to and fro and a page has been completely loaded.
I was unable to do this, perform programmatically what we see in the Chrome Dev tool under the Network tab in the image here.
Kenneth's work let's one achieve this to a degree.

HTTP Bad Request response to Java POST call

I'm attempting to query a REST api using POST requests in a java application. I think I've set everything correctly, but I keep getting a Bad Request response.
HttpPost request = new HttpPost(requestURI);
request.addHeader("accept", "application/json");
request.addHeader(HttpHeaders.CONTENT_TYPE, "application/json");
HttpEntity entity = new StringEntity(requestBody + new Integer(PatientId).toString() + "}");
request.setEntity(entity);
The requestBody, accompanied by the number and curly brace, are valid JSON, and the requestURI is copy and pasted straight out of the API documentation, so I know I shouldn't be getting a Bad Request due to those.
Am I missing something in the setup?

The Content-Length header is missing. Some servers don't report the correct 4xx error (411 Length Required) and just issue a generic Bad Request error.

It ended up being a random slash that wasn't included in my URI.