I've a small Vert.x implementation of a Reverse proxy that is functionally working fine.
Downstream requests are done with tryUseCompression(true) (gziped), which our servers can answer accordingly.
To avoid additional decompression / compression we would like that when we receive a request asking for compression, and the downstream answers with the same compression (all in gzip for example) then the HTTP Proxy should not try to decompress the downstream result but instead return as is to the upstream response.
As far as I could follow Vertx code, Netty is being allways called to decode the messages that it receives and we only get the result
See Http1xClientHandler::handleMessage:
HttpObject obj = (HttpObject) msg;
DecoderResult result = obj.decoderResult();
Also found at the HttpChannelConnector::applyHttp1xConnectionOtions
if (client.getOptions().isTryUseCompression()) {
pipeline.addLast("inflater", new HttpContentDecompressor(true));
}
Is it possible to tell Vertx / Netty not to decompress the result of an individual call?
Thanks for the help
No it's not possible. Your best option would be to handle compression/decompression in your code.
Related
I try to send message to channel
GatewayDiscordClient gatewayDiscordClient = DiscordClient
.builder("TOKEN")
.build()
.login()
.block();
gatewayDiscordClient.rest().getChannelById(Snowflake.of("ChannelId")).createMessage("p");
But channel doesn't displaythe message in history. How can I fix that?
Discord4J utilizes the Reactor framework which is "lazily executed".
You must .subscribe() or .block() on publishers (usually a Mono<T> or Flux<T>).
So, to get your code to create the message:
gatewayDiscordClient.rest().getChannelById(Snowflake.of("ChannelId")).createMessage("p").subscribe();
I would advise however, to not use the RestX classes unless you are not using the gateway (which, in your question, is being used). Rest classes generally provide less of the helpful abstractions D4J offers and is meant to be used internally by the library or for a webserver, not a bot connected to the gateway.
To send a message without using rest classes your code would look like this:
client.getChannelById(channelIdHere)
.ofType(MessageChannel.class)
.flatMap(channel -> channel.createMessage("Your content here"))
.subscribe();
i'm creating java module to parse JSON file.
To receive file i need to send HTTP request. When I use curl my request looks like this:
curl -X GET "https://***" -H "accept: application/json" -H "apikey: ***"
How can I send the equivalent HTTP request from Java
Java has a lot of options to work with HTTP.
Option 1
Since Java 9, there is a built-in HTTP client. So You can use it to create a request without any third-party libraries.
A simple example is something like this:
HttpRequest request2 = HttpRequest.newBuilder()
.uri(new URI("some url"))
.header("someHeader", "value1")
.header("anotherHeader", "value2")
.GET()
.build();
For more examples see here
Option 2
Use third party libraries, there are many: OkHttpClient, More "old-school" Apache Http Client (HttpComponents
Option 3
If you're using spring, you might consider using Spring's WebClient. There are also wrappers in spring like RestTemplate that can come handy, but it really depends on what would you like to work with.
Many clients are coming with http connection pools that should be properly set up.
In addition, in your example, I see that you work with https - all these clients support it but it should be properly set up.
If you are using Spring then try WebClient - it is a bit harder to understand in the begging (at least harder than RestTemplate) but it pays of since RestTemplate will be discontinued.
You can find an example here
https://www.baeldung.com/spring-webclient-resttemplate
#GetMapping(value = "/tweets-non-blocking",
produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<Tweet> getTweetsNonBlocking() {
log.info("Starting NON-BLOCKING Controller!");
Flux<Tweet> tweetFlux = WebClient.create()
.get()
.uri(getSlowServiceUri())
.retrieve()
.bodyToFlux(Tweet.class);
tweetFlux.subscribe(tweet -> log.info(tweet.toString()));
log.info("Exiting NON-BLOCKING Controller!");
return tweetFlux;
}
Just be aware that this is non-blocking (e.g. asynchronous) solution so you won't get the response right away, but you subscribe to the request and then process the response when it is available. There are also blocking options in WebClient
Java has its own classes that allow you to send HTTP request. See class HttpURLConnection. However, I recommend using 3d party libraries that significantly simplify this task. Good libraries would be Apache Http client or OK Http client. I also can offer you to use another Open source library that has an HTTP client as well. It is called MgntUtils library and it is written by me. In this case your code would look something like this:
HttpClient workingClient = new HttpClient();
workingClient.setRequestProperty("accept", "application/json;charset=UTF-8");
workingClient.setRequestProperty("apikey", "***");
workingClient.setConnectionUrl("https://***");
ByteBuffer buffer =
workingClient.sendHttpRequestForBinaryResponse(HttpMethod.GET);
//or of your API returns contents of file as a string
String jsonStr = workingClient.sendHttpRequest(HttpMethod.GET);
After that, your ByteBuffer buffer or String jsonStr will hold the content of your JSON file. And now you can do whatever you need with it. Here is Javadoc for HttpClient class. The MgntUtils library can be obtained as maven artifacts here or on Github (including source code and Javadoc)
I am writing a REST API in JAX-RS 2.0, JDK 8 for the below requirement
POST API /server/fileUpload/ (Multipart Form data) where I need to send a Big .AI (Adobe Illustrator) File in this.
The Server, takes the file and return Status 202 (Accepted), Acknowledging that file transfer happened Successfully. (From endpoint to Server)
Now at the Server, I am using Java + Imagemagik to convert .AI File (20-25 MB File) to small JPG Thumbnail, place on a Apache HTTP Server and share the location (like http://happyplace/thumbnail0987.jpg)
Now the Second Response should come from Server with Status 200 OK and Thumbnail URL
is it feasible with one REST API? (Async/similar)
or should I split it to 2 API calls, Please suggest
No. In http, one request gets one response. The client must send a second request to get a second response.
You can use WebSockets for that.
If you are calling from script the call will be async you can handle the Thumbnail URL when you get a response. When you are calling from java program i suggest to run it on a different thread, If the execution is not sequential i.e ( Remaining lines can be executed without getting URL). If url is needed for the remaining section of code you can make one call and wait for the response then execute remaining code.
You need to make different APIs for both scenarios. One for showing file upload status and another for all file conversion and manipulation.
On the client side second request must be callback of first request.
The best way to handle these kind of scenario is to use Java Reactive (Project Reactor, WebFlux).
You can return two response using custom middlewares in asp.net (however not recommended).
Return response from one middleware and subsequently you can invoke next middleware and return second response from second middleware
I have a few questions about a specific REST call I'm making in JAVA. I'm quite the novice, so I've cobbled this together from several sources. The call itself looks like this:
String src = AaRestCall.subTrackingNum(trackingNum);
The Rest call class looks like this:
public class AaRestCall {
public static String subTrackingNum (Sting trackingNum) throws IOException {
URL url = new URL("https://.../rest/" + trackingNum);
String query = "{'TRACKINGNUM': trackingNum}";
//make connection
URLConnection urlc = url.openConnection();
//use post mode
urlc.setDoOutput(true);
urlc.setAllowUserInteraction(false);
//send query
PrintStream ps = new PrintStream(urlc.getOutputStream());
ps.print(query);
ps.close();
//get result
BufferedReader br = new BufferedReader(new InputStreamReader(urlc
.getInputStream()));
StringBuilder sb = new StringBuilder();
String line = null;
while ((line=br.readLine())!=null) {
sb.append(line);
}
br.close();
return sb.toString();
}
}
Now, I have a few questions on top of the what is wrong with this in general.
1) If this rest call is returning a JSON object, is that going to get screwed up by going to a String?
2) What's the best way to parse out the JSON that is returning?
3) I'm not really certain how to format the query field. I assume that's supposed to be documented in the REST API?
Thanks in advance.
REST is a pattern applied on top of HTTP. From your questions, it seems to me that you first need to understand how HTTP (and chatty socket protocols in general) works and what the Java API offers for deal with it.
You can use whatever Json library out there to parse the HTTP response body (provided it's a 200 OK, that you need to check for, and also watch out for HTTP redirects!), but it's not how things are usually built.
If the service exposes a real RESTful interface (opposed to a simpler HTTP+JSON) you'll need to use four HTTP verbs, and URLConnection doesn't let you do so. Plus, you'll likely want to add headers for authentication, or maybe cookies (which in fact are just HTTP headers, but are still worth to be considered separately). So my suggestion is building the client-side part of the service with the HttpClient from Apache commons, or maybe some JAX-RS library with client support (for example Apache CXF). In that way you'll have full control of the communication while also getting nicer abstractions to work with, instead of consuming the InputStream provided by your URLConnection and manually serializing/deserializing parameters/responses.
Regarding the bit about how to format the query field, again you first need to grasp the basics of HTTP. Anyway, the definite answer depends on the remote service implementation, but you'll face four options:
The query string in the service URL
A form-encoded body of your HTTP request
A multipart body of your HTTP request (similar to the former, but the different MIME type is enough to give some headache) - this is often used in HTTP+JSON services that also have a website, and the same URL can be used for uploading a form that contains a file input
A service-defined (for example application/json, or application/xml) encoding for your HTTP body (again, it's really the same as the previous two points, but the different MIME encoding means that you'll have to use a different API)
Oh my. There are a couple of areas where you can improve on this code. I'm not even going to point out the errors since I'd like you to replace the HTTP calls with a HTTP client library. I'm also unaware of the spec required by your API so getting you to use the POST or GET methods properly at this level of abstraction will take more work.
1) If this rest call is returning a JSON object, is that going to get
screwed up by going to a String?
No, but marshalling that json into an obect is your job. A library like google gson can help.
2) What's the best way to parse out the JSON that is returning?
I like to use gson like I mentioned above, but you can use another marshal/unmarhal library.
3) I'm not really certain how to format the query field. I assume
that's supposed to be documented in the REST API?
Yes. Take a look at the documentation and come up with java objects that mirror the json structure. You can then parse them with the following code.
gson.fromJson(json, MyStructure.class);
Http client
Please take a look at writing your HTTP client using a library like apache HTTP client which will make your job much easier.
Testing
Since you seem to be new to this, I'd also suggest you take a look at a tool like Postman which can help you test your API calls if you suspect that the code you've written is faulty.
I think that you should use a REST client library instead of writing your own, unless it is for educational purposes - then by all means go nuts!
The REST service will respond to your call with a HTTP response, the payload may and may not be formatted as a JSON string. If it is, I suggest that you use a JSON parsing library to convert that String into a Java representation.
And yes, you will have to resort to the particular REST API:s documentation for details.
P.S. The java URL class is broken, use URI instead.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
What is the best Java library to use for HTTP POST, GET etc. in terms of performance, stability, maturity etc.? Is there one particular library that is used more than others?
My requirements are submitting HTTPS POST requests to a remote server. I have used the java.net.* package in the past as well as org.apache.commons.httpclient.* package. Both have got the job done, but I would like some of your opinions/recommendations.
imho: Apache HTTP Client
usage example:
import org.apache.commons.httpclient.*;
import org.apache.commons.httpclient.methods.*;
import org.apache.commons.httpclient.params.HttpMethodParams;
import java.io.*;
public class HttpClientTutorial {
private static String url = "http://www.apache.org/";
public static void main(String[] args) {
// Create an instance of HttpClient.
HttpClient client = new HttpClient();
// Create a method instance.
GetMethod method = new GetMethod(url);
// Provide custom retry handler is necessary
method.getParams().setParameter(HttpMethodParams.RETRY_HANDLER,
new DefaultHttpMethodRetryHandler(3, false));
try {
// Execute the method.
int statusCode = client.executeMethod(method);
if (statusCode != HttpStatus.SC_OK) {
System.err.println("Method failed: " + method.getStatusLine());
}
// Read the response body.
byte[] responseBody = method.getResponseBody();
// Deal with the response.
// Use caution: ensure correct character encoding and is not binary data
System.out.println(new String(responseBody));
} catch (HttpException e) {
System.err.println("Fatal protocol violation: " + e.getMessage());
e.printStackTrace();
} catch (IOException e) {
System.err.println("Fatal transport error: " + e.getMessage());
e.printStackTrace();
} finally {
// Release the connection.
method.releaseConnection();
}
}
}
some highlight features:
Standards based, pure Java, implementation of HTTP versions 1.0
and 1.1
Full implementation of all HTTP methods (GET, POST, PUT, DELETE,
HEAD, OPTIONS, and TRACE) in an
extensible OO framework.
Supports encryption with HTTPS (HTTP over SSL) protocol.
Granular non-standards configuration and tracking.
Transparent connections through HTTP proxies.
Tunneled HTTPS connections through HTTP proxies, via the CONNECT
method.
Transparent connections through SOCKS proxies (version 4 & 5) using
native Java socket support.
Authentication using Basic, Digest and the encrypting NTLM (NT
Lan Manager) methods.
Plug-in mechanism for custom authentication methods.
Multi-Part form POST for uploading large files.
Pluggable secure sockets implementations, making it easier to
use third party solutions
Connection management support for use in multi-threaded
applications. Supports setting the
maximum total connections as well as
the maximum connections per host.
Detects and closes stale connections.
Automatic Cookie handling for reading Set-Cookie: headers from the
server and sending them back out in a
Cookie: header when appropriate.
Plug-in mechanism for custom cookie policies.
Request output streams to avoid buffering any content body by
streaming directly to the socket to
the server.
Response input streams to efficiently read the response body by
streaming directly from the socket to
the server.
Persistent connections using KeepAlive in HTTP/1.0 and persistance
in HTTP/1.1
Direct access to the response code and headers sent by the server.
The ability to set connection timeouts.
HttpMethods implement the Command Pattern to allow for parallel
requests and efficient re-use of
connections.
Source code is freely available under the Apache Software License.
I would recommend Apache HttpComponents HttpClient, a successor of Commons HttpClient
I would also recommend to take a look at HtmlUnit.
HtmlUnit is a "GUI-Less browser for Java programs".
http://htmlunit.sourceforge.net/
I'm somewhat partial to Jersey. We use 1.10 in all our projects and haven't run into an issue we couldn't solve with it.
Some reasons why I like it:
Providers - created soap 1.1/1.2 providers in Jersey and have eliminated the need to use the bulky AXIS for our JAX-WS calls
Filters - created database logging filters to log the entire request (including the request/response headers) while preventing logging of sensitive information.
JAXB - supports marshaling to/from objects straight from the request/response
API is easy to use
In truth, HTTPClient and Jersey are very similar in implementation and API. There is also an extension for Jersey that allows it to support HTTPClient.
Some code samples with Jersey 1.x:
https://blogs.oracle.com/enterprisetechtips/entry/consuming_restful_web_services_with
http://www.mkyong.com/webservices/jax-rs/restful-java-client-with-jersey-client/
HTTPClient with Jersey Client:
https://blogs.oracle.com/PavelBucek/entry/jersey_client_apache_http_client
I agree httpclient is something of a standard - but I guess you are looking for options so...
Restlet provides a http client specially designed for interactong with Restful web services.
Example code:
Client client = new Client(Protocol.HTTP);
Request r = new Request();
r.setResourceRef("http://127.0.0.1:8182/sample");
r.setMethod(Method.GET);
r.getClientInfo().getAcceptedMediaTypes().add(new Preference<MediaType>(MediaType.TEXT_XML));
client.handle(r).getEntity().write(System.out);
See https://restlet.talend.com/ for more details
I want to mention the Ning Async Http Client Library. I've never used it but my colleague raves about it as compared to the Apache Http Client, which I've always used in the past. I was particularly interested to learn it is based on Netty, the high-performance asynchronous i/o framework, with which I am more familiar and hold in high esteem.
May I recommend you corn-httpclient. It's simple,fast and enough for most cases.
HttpForm form = new HttpForm(new URI("http://localhost:8080/test/formtest.jsp"));
//Authentication form.setCredentials("user1", "password");
form.putFieldValue("input1", "your value");
HttpResponse response = form.doPost();
assertFalse(response.hasError());
assertNotNull(response.getData());
assertTrue(response.getData().contains("received " + val));
maven dependency
<dependency>
<groupId>net.sf.corn</groupId>
<artifactId>corn-httpclient</artifactId>
<version>1.0.0</version>
</dependency>
Google HTTP Java Client looks good to me because it can run on Android and App Engine as well.