Develop Reference

How to send HTTP request from Java

i'm creating java module to parse JSON file.
To receive file i need to send HTTP request. When I use curl my request looks like this:
curl -X GET "https://***" -H "accept: application/json" -H "apikey: ***"
How can I send the equivalent HTTP request from Java

Java has a lot of options to work with HTTP.
Option 1
Since Java 9, there is a built-in HTTP client. So You can use it to create a request without any third-party libraries.
A simple example is something like this:
HttpRequest request2 = HttpRequest.newBuilder()
.uri(new URI("some url"))
.header("someHeader", "value1")
.header("anotherHeader", "value2")
.GET()
.build();
For more examples see here
Option 2
Use third party libraries, there are many: OkHttpClient, More "old-school" Apache Http Client (HttpComponents
Option 3
If you're using spring, you might consider using Spring's WebClient. There are also wrappers in spring like RestTemplate that can come handy, but it really depends on what would you like to work with.
Many clients are coming with http connection pools that should be properly set up.
In addition, in your example, I see that you work with https - all these clients support it but it should be properly set up.

If you are using Spring then try WebClient - it is a bit harder to understand in the begging (at least harder than RestTemplate) but it pays of since RestTemplate will be discontinued.
You can find an example here
https://www.baeldung.com/spring-webclient-resttemplate
#GetMapping(value = "/tweets-non-blocking",
produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<Tweet> getTweetsNonBlocking() {
log.info("Starting NON-BLOCKING Controller!");
Flux<Tweet> tweetFlux = WebClient.create()
.get()
.uri(getSlowServiceUri())
.retrieve()
.bodyToFlux(Tweet.class);
tweetFlux.subscribe(tweet -> log.info(tweet.toString()));
log.info("Exiting NON-BLOCKING Controller!");
return tweetFlux;
}
Just be aware that this is non-blocking (e.g. asynchronous) solution so you won't get the response right away, but you subscribe to the request and then process the response when it is available. There are also blocking options in WebClient

Java has its own classes that allow you to send HTTP request. See class HttpURLConnection. However, I recommend using 3d party libraries that significantly simplify this task. Good libraries would be Apache Http client or OK Http client. I also can offer you to use another Open source library that has an HTTP client as well. It is called MgntUtils library and it is written by me. In this case your code would look something like this:
HttpClient workingClient = new HttpClient();
workingClient.setRequestProperty("accept", "application/json;charset=UTF-8");
workingClient.setRequestProperty("apikey", "***");
workingClient.setConnectionUrl("https://***");
ByteBuffer buffer =
workingClient.sendHttpRequestForBinaryResponse(HttpMethod.GET);
//or of your API returns contents of file as a string
String jsonStr = workingClient.sendHttpRequest(HttpMethod.GET);
After that, your ByteBuffer buffer or String jsonStr will hold the content of your JSON file. And now you can do whatever you need with it. Here is Javadoc for HttpClient class. The MgntUtils library can be obtained as maven artifacts here or on Github (including source code and Javadoc)

use existing http client for SOAP call

I've got a working Dropwizard project, which has several ways of getting data it needs. One of those ways is the JAX-RS client that Dropwizard provides, the JerseyClient. This client is configured so that it suits my needs (uses the proper proxy, timeouts etc...)
Now my project has a new requirement for which I need to do a SOAP call. I've got that functionally working using the following code:
// not the actual structure, edited to make a minimal example
// SERVICE_QNAME and PORT_QNAME are hardcoded strings, config.url comes
// from the configuration
import javax.xml.ws.*;
import javax.xml.ws.soap.*;
import javax.xml.namespace.QName;
Service service = Service.create(SERVICE_QNAME);
service.addPort(PORT_QNAME, SOAPBinding.SOAP11HTTP_BINDING, config.url);
Dispatch dispatch = service.createDispatch(PORT_QNAME, SOAPMessage.class, Service.Mode.MESSAGE);
dispatch.getRequestContext().put(BindingProvider.ENDPOINT_ADDRESS_PROPERTY, config.url);
Message message = MessageFactory.newInstance(SOAPConstants.SOAP_1_2_PROTOCOL).createMessage();
// do stuff to fill the message
response = dispatch.invoke(message);
This is all out-of-the-box behaviour, anything happening here is provided either by java (8) or Dropwizard.
This code however uses it's own http connectors, bypassing anything I've set up in my JAX-RS client. I would like to re-use the JerseyClient's http capabilities in the JAX-WS client in a non-copy-paste kinda way.
Is there a way I can set up the Dispatch so that it will use the existing http connectors? Or some other SOAP client to achieve the same?

Thank you #zloster for the research and suggestions. I decided to take another route however.
I found the SAAJ standard and am using that now. I created a subclass for javax.xml.soap.SOAPConnection and based that on com.sun.xml.internal.messaging.saaj.client.p2p.HttpSOAPConnection. That part wasn't all that hard and leaves me with a relatively small class.
Now in my code I replaced the code above with something along these lines:
SOAPConnection soapConnection = new JerseySOAPConnection(httpClient, soapProtocol);
Message message = MessageFactory.newInstance(soapProtocol).createMessage();
// do stuff to fill the message
response = soapConnection.call(message, config.url);
Due to my implementation not all that portable, but I don't really need it to be. Again, thanks for those who helped me get to this!

Can JCIFS be used with Jersey?

I have trouble adding NTLM authentication to my existing application that uses Jersey and Apache HttpClient. I was only able to authenticate using JCIFS, the default NTLM authentication from HttpClient does not work (I get 401).
The example from Apache HttpClient page shows how to use CloseableHttpClient:
https://hc.apache.org/httpcomponents-client-4.5.x/ntlm.html
Registry<AuthSchemeProvider> authSchemeRegistry = RegistryBuilder.<AuthSchemeProvider>create()
.register(AuthSchemes.NTLM, new JCIFSNTLMSchemeFactory())
.register(AuthSchemes.BASIC, new BasicSchemeFactory())
.register(AuthSchemes.DIGEST, new DigestSchemeFactory())
.register(AuthSchemes.SPNEGO, new SPNegoSchemeFactory())
.register(AuthSchemes.KERBEROS, new KerberosSchemeFactory())
.build();
CloseableHttpClient httpClient = HttpClients.custom()
.setDefaultAuthSchemeRegistry(authSchemeRegistry)
.build();
But with CloseableHttpClient I cannot use methods like target:
WebTarget target = client.target(this.my Address).path(elementPath)
.resolveTemplate(P_ID, myId);
There is only execute.
I'm not sure if I should rewrite my whole application and use only basic HttpClient calls like:
HttpGet httpGet = new HttpGet(repositoryAddress + "/" + "element/70032_1498404600000(,,arm)");
CloseableHttpResponse response = httpClient.execute(httpGet);
or there is some other way to set AuthSchemes in javax.ws.rs.client.Client, which can be used in Jersey?

I faced similar issues and my approach was below :
1) If you are using ApacheConnectorProvider as Connector you can Override ApacheConnector code (found here https://github.com/jersey/jersey/tree/master/connectors/apache-connector/src/main/java/org/glassfish/jersey/apache/connector). In my case I had to create custom ConnectorProvider and Connector.
2) Create a custom property or use HttpClientContext.AUTHSCHEME_REGISTRY and put it in the ClientConfig ( This is how we set properties for client in Jersey Client).
3) The Custom connector gets called when you call builder.get( or post or any other method). In the custom connector you can check for the property set in the above step. If it is set, you can set the DefaultAuthSchemeRegistry just like it is specified for ClosableHttpClient(ApacheConnector uses ClosableHttpClient in its implementation).
This may be kind of a hack but works fine for me. Hope this helps :)

How to force the http client to not handle the authentication challenges automatically in httpClient 4.3.x?

I am migrating my httpclient 3.x to httpclient 4.3.x.
With httpClient 3.x I used setDoAuthentication(false) which ensured that the method would not try to handle the authentication challenges automatically. How can I set the same functionality with httpClient 4.3.x?
I already went through the javadocs of httpmethods and httpclient and there is no way to do it. The closest I have been is the use of disableAuthCaching() in httpClientBuilder, but not sure whether it would serve the purpose.
Can anyone please provide some pointers?
I have read somewhere that there is a parameter through which we can achieve this behavior but I am not able to find it anywhere. Sorry if it looks like a pretty basic question.

I think I got it. I guess it can be done by setting the setDoAuthentication() method to false in the requestConfig for httpClient.

#spock8190 thanks for your post this gave me the correct hint.
But the new method name is setAuthenticationEnabled (at least for 4.5)
Determines whether authentication should be handled automatically.
Example:
HttpGet httpGet = new HttpGet("test.de");
RequestConfig requestConfig = RequestConfig.custom().setAuthenticationEnabled(true).build();
httpGet.setConfig(requestConfig);

How can I monitor an external request in New Relic's external-services page?

I'm using the WSRequest class to make requests to an external web-service in my Play web-application.
Here's a small snippet of the code I'm using to make SOAP calls:
WSRequest req = WS.url("http://mydomain.com/soap");
req.setHeader("Content-Type", "text/xml;charset=UTF-8");
Map<String, Object> args = new HashedMap();
args.put("countryCode", countryCode);
req.body(requestTemplate.render(args));
HttpResponse res = req.post();
Document document=null;
try{
document = res.getXml();
}catch(RuntimeException e){
System.out.println(res.getString());
throw e;
}
I'd like SOAP calls made to this endpoint to show up on the "External Services" page of New Relic. How can I instrument my code in order to accomplish this? I haven't been able to find much information about this in the documentation except for a Ruby example.

At this time, the New Relic Java agent only notices external calls made with the following methods:
HttpURLConnection
CommonsHttp
Jets3t
S3
AsyncHttpClient
The Play WS API is not among those and, at this time, there is no way to for the user to do anything to modify the behavior.
I would recommend contacting New Relic tech support with a feature request to add external call support for Play's WS API.

You can now implement custom network tracking:
https://docs.newrelic.com/docs/mobile-sdk-api/android-api#track-custom