Issue with LoggingFeature not logging the complete response body - java

I'm currently working building an endpoint that will make a couple of requests to an external API. In this endpoint I'm logging all of the request bodies to the terminal. But I'm having an issue where jerseys LoggingFeature doesn't log the complete response body for a request that is done. The log gets cut in the middle of the body. I've tried messing around with the logger level, verbosity but that did nothing. I tried changing maxEntitySize to be a large value but that doesn't change anything either, I'm not really sure what the issue is and why it won't log the complete response body. I've also tried logging to a file instead of the terminal but unfortunately I got the same result.
Here's what the log looks like (as it's proprietary software I had to change some information)
INFO [2022-10-06 13:54:04,261] se.bla.bla.bla.clients.BlaClientImpl: 1 * Sending client request on thread dw-18 - GET /api/bla/log/98014384
1 > GET https://url.com/2574339575
1 > Accept: application/json
1 > Authorization: Bearer tokenstuff
INFO [2022-10-06 13:54:04,475] se.bla.bla.bla.clients.BlaClientImpl: 1 * Client response received on thread dw-18 - GET /api/bla/log/98014384
1 < 200
1 < Cache-Control: no-cache
1 < Content-Type: application/json; charset=utf-8
1 < Date: Thu, 06 Oct 2022 13:54:04 GMT
1 < Etag: Yee
1 < Server-Response-Time: 169.44395400000002
1 < Transfer-Encoding: chunked
1 < Vary: Origin
1 < X-Request-Id: Woo
1 < X-Runtime: 0.169337
{example1: "2134123123",example2: "2134123123",example3: "2134123123",example4: "2134123123",example5: "2134123123",example6: "2134123123",example7: "2134123123",example8: "2134123123",example9: "2134123123",example10: "2134123123",example11: "2134123123",example12: "2134123123",example13: "2134123123",example14: "2134123123",example15: "2134123123",example16: "2134123123",example17: "2134123123",example18: "21341
And I've registered the logger like so.
val restlog = getLogger(BlaClientImpl::class.java.name)
restlog.addHandler(FileHandler("rest.log", 83886080, 1))
val requestBuilder =
client
.register(
LoggingFeature(
restlog,
Level.INFO,
LoggingFeature.Verbosity.PAYLOAD_ANY,
83886080
)
)
I'm quite new to the Kotlin/Java world so maybe I'm missing something fundamental.

The issue was a settings with Dropwizard/Jersey. Apparently I had to set gzipEnabled = true

Related

Spring Boot returns "200 OK" (Empty Response) for resource directory, not file

I am building a basic Spring Boot app, using the built is resource handler.
registry.addResourceHandler("/**")
.addResourceLocations("classpath:/public/")
.setCachePeriod(60 * 60 * 24 * 365)
.resourceChain(true)
.addResolver(versionResourceResolver);
The file structure is as follows:
src
main
java
..
resources
public
js
app.js
Making a request to localhost:8080/js/app.js works as expected, returning the contents of the file with the correct MIME type.
However, making a request to the container directory also returns 200 OK and an empty response!
$ curl -v http://localhost:8080/js/
* Trying ::1...
* Connected to localhost (::1) port 8080 (#0)
> GET /js/ HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.49.1
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Mon, 28 Nov 2016 21:13:26 GMT
< X-Application-Context: application
< Last-Modified: Mon, 28 Nov 2016 21:06:28 GMT
< Cache-Control: max-age=31536000
< Accept-Ranges: bytes
< Content-Type: application/octet-stream
< Content-Length: 0
<
* Connection #0 to host localhost left intact
How can I get this to return 404 Not Found instead? It's not a huge issue, but definitely not what I'd expected.

okhttp content-length is -1 with big files

I am downloading a file with okhttp and things work fine - now I want to show the progress and hit a road-bump. The returned content-length is -1.
It comes back correctly from the server:
⋊> ~ curl -i http://ipfs.io/ipfs/QmRMHb4Vhv8LtYqw8RkDgkdZYxJHfrfFeQaHbNUqJYmdF2 13:38:11
HTTP/1.1 200 OK
Date: Tue, 14 Jun 2016 11:38:16 GMT
Content-Type: application/octet-stream
Content-Length: 27865948
I traced the problem down to OkHeaders.java here:
public static long contentLength(Headers headers) {
return stringToLong(headers.get("Content-Length"));
}
I see all the other headers here in headers - but not Content-Length - so headers.get("Content-Length") returns null. Anyone has a clue how this can get lost?
Interestingly if I change the url to "http://google.com" I get a content-length from okhttp - but with curl both look same Content-Length wise - this really confuses me
Update: it seems to correlate with he size of the file. If I use smaller content from the same server I get a Content-Length with okhttp. The problem only happens when the file is big
It looks like above a certain size the server uses chunked encoding and you won't get a content length.
HTTP/1.1 200 OK
Date: Tue, 14 Jun 2016 14:30:07 GMT
Content-Type: application/octet-stream
Transfer-Encoding: chunked

Downloading from dropbox url ignores range

If i want to download a file from a dropbox url my http header range is ignored:
httpRequest = new HttpGet(url.toURI());
httpRequest.addHeader("Range", "bytes=" + startPos + "-" + dwnInfo.getStopRange());
httpRequest.addHeader("Accept-Encoding", "");
So instead of making my file download in x chunks of 5mb for ex, the connection ignores the specified range and it downloads x chunks of Y mb, where y is the full size of the file.
Downloading from an amazon storange link i don't have any problems.
Anyone else encountered this situation ? This only happens from some days ago. This wasn't a issue until now.
I tried to look on dropbox dev page but didn't see anything that specifies if they removed the accepted range on urls
The link you gave is to an HTML page (total size ~46KB), so even if range retrieval worked there, it wouldn't be very useful.
Per https://www.dropbox.com/help/201/en, you can turn a share link into a direct link to the file by changing the domain to dl.dropboxusercontent.com, so your link becomes https://dl.dropboxusercontent.com/s/5c7atlfmacjf3qn/02%20Armin%20Van%20Buuren%20-%20A%20State%20Of%20Trance%20Year%20Mix%202013%20%28Cd%202%29.mp3, and range retrieval works for that URL.
(Here I'm using httpie.)
$ http get https://dl.dropboxusercontent.com/s/5c7atlfmacjf3qn/02%20Armin%20Van%20Buuren%20-%20A%20State%20Of%20Trance%20Year%20Mix%202013%20%28Cd%202%29.mp3 range:bytes=0-0
HTTP/1.1 206 PARTIAL CONTENT
Connection: keep-alive
Content-Length: 1
Content-Type: audio/mpeg
Date: Wed, 18 Jun 2014 14:53:32 GMT
Server: nginx
accept-ranges: bytes
cache-control: max-age=0
content-range: bytes 0-0/146014047
etag: 346n
pragma: public
set-cookie: uc_session=2cqmevWxG8lmGt743KMXebc23dRC5iuZEfm8Etx6V2VShWk60jmnUJajFnH1wRG4; Domain=dropboxusercontent.com; Path=/; secure; httponly
x-dropbox-request-id: 2f0c5986a62cf2f0b06af1704ece5bd7
x-server-response-time: 535
I

App-Engine directing endpoint to previous class name, now 404 not found

I've been using App-Engine as the backend for an Android and iOS application. It's been working without problem with both the local development server (over http) and actual app-engine (over https).
Then I noticed that, while renaming endpoints, I accidentally duplicated a word in the class name of an endpoint: RegionRegionIconsEndpoint instead of simply RegionIconsEndpoint. It was a 1-line fix.
public class RegionRegionIconsEndpoint {
#ApiMethod(name = "getRegionIcons", path="regionIcons", httpMethod = HttpMethod.POST)
public RegionInfoVersion.RegionIcons getRegionIcons(User user, #Named("id") String id)
throws OAuthRequestException {
...
}
}
became
public class RegionIconsEndpoint {
#ApiMethod(name = "getRegionIcons", path="regionIcons", httpMethod = HttpMethod.POST)
public RegionInfoVersion.RegionIcons getRegionIcons(User user, #Named("id") String id)
throws OAuthRequestException {
...
}
}
I generated new cloud-endpoint libraries and continued development using the local development server. All good.
When I deployed it to the real App-Engine service, however, a problem arose. When my app starts, there are a series of calls to other endpoints defined just as the one shown above; these always work fine. Then there are calls to this endpoint. A typical call looks like this:
POST https://my-app.appspot.com/_ah/api/client/v1/regionIcons?id=foo
Authorization is also provided and the expected result comes back most of the time... say 80%. The AE logs look like this:
2014-05-02 21:36:30.551 /_ah/spi/com.example.app.endpoints.RegionIconsEndpoint.getRegionIcons 200 48ms 0kb Google-HTTP-Java-Client/1.16.0-rc (gzip) module=default version=1
70.80.59.221 - - [02/May/2014:18:36:30 -0700] "POST /_ah/spi/com.example.app.endpoints.RegionIconsEndpoint.getRegionIcons HTTP/1.1" 200 149 - "Google-HTTP-Java-Client/1.16.0-rc (gzip)" "my-app.appspot.com" ms=49 cpu_ms=41 cpm_usd=0.000017 app_engine_release=1.9.4 instance=006c1b117c1b2d35341e0f407ae5785a825b65e5
The remaining times, I get a 404 Not Found response and the AE logs have this:
2014-05-02 21:36:30.852 /_ah/spi/BackendService.logMessages 204 16ms 0kb module=default version=1
10.1.0.41 - - [02/May/2014:18:36:30 -0700] "POST /_ah/spi/BackendService.logMessages HTTP/1.1" 204 0 - - "my-app.appspot.com" ms=16 cpu_ms=0 app_engine_release=1.9.4 instance=006c1b117c1b2d35341e0f407ae5785a825b65e5
E 2014-05-02 21:36:30.851
Request URL: https://my-app.appspot.com/_ah/api/client/v1/regionIcons?id=foo
Method: client.getRegionIcons
Error Code: 404
Reason: notFound
Message: service 'com.example.app.endpoints.RegionRegionIconsEndpoint' not found
2014-05-02 21:36:30.802 /_ah/spi/com.example.app.endpoints.RegionRegionIconsEndpoint.getRegionIcons 404 16ms 0kb Google-HTTP-Java-Client/1.16.0-rc (gzip) module=default version=1
70.80.59.221 - - [02/May/2014:18:36:30 -0700] "POST /_ah/spi/com.example.app.endpoints.RegionRegionIconsEndpoint.getRegionIcons HTTP/1.1" 404 166 - "Google-HTTP-Java-Client/1.16.0-rc (gzip)" "my-app.appspot.com" ms=16 cpu_ms=0 cpm_usd=0.000019 app_engine_release=1.9.4 instance=006c1b117c1b2d35341e0f407ae5785a825b65e5
You can see on the Message line that, sometimes, AE is still trying to process the call using the old class name with the duplicated word! I've done searches over my entire code-base and the generated files and I cannot find the string "RegionRegion" anywhere. I've checked the web.xml file a dozen times and it has only the new "RegionIconsEndpoint" class name.
Wondering if somehow Google's servers were keeping old information around, I deployed the new version of my app as 2-dot-my-app.appspot.com. The behavior remains exactly the same except that there are no AE log messages for the requests that fail with 404 on this version. Successful request logs are as before.
Both my Android and iPad apps are experiencing this. In addition, I've managed to reproduce it using the web and Google's API explorer on my-app.appspot.com. In this last case, a successful request shows this:
200 OK
cache-control: no-cache, no-store, max-age=0, must-revalidate
content-encoding: gzip
content-length: 171
content-type: application/json; charset=UTF-8
date: Sat, 03 May 2014 03:07:05 GMT
etag: "G170GGjYGsLnxTffzUEJmTttHzU/LUWzmydK3mjH7IeRbEc_n9J6cDQ"
expires: Fri, 01 Jan 1990 00:00:00 GMT
pragma: no-cache
server: GSE
{
"iconsVid": "foo",
"iconsVersion": 3,
"kind": "client#resourcesItem",
"etag": "\"G170GGjYGsLnxTffzUEJmTttHzU/LUWzmydK3mjH7IeRbEc_n9J6cDQ\""
}
and a failed request shows this:
404 Not Found
cache-control: private, max-age=0
content-encoding: gzip
content-length: 169
content-type: application/json; charset=UTF-8
date: Sat, 03 May 2014 03:08:34 GMT
expires: Sat, 03 May 2014 03:08:34 GMT
server: GSE
{
"error": {
"errors": [
{
"domain": "global",
"reason": "notFound",
"message": "service 'com.example.app.endpoints.RegionRegionIconsEndpoint' not found"
}
],
"code": 404,
"message": "service 'com.example.app.endpoints.RegionRegionIconsEndpoint' not found"
}
}
again clearly showing an access to the old class name. When trying to do the same to the v2 version that I deployed (2-dot-my-app.appspot.com), it's different. A success request ends like this:
200 OK
cache-control: no-cache, no-store, max-age=0, must-revalidate
content-encoding: gzip
content-length: 171
content-type: application/json; charset=UTF-8
date: Sat, 03 May 2014 03:12:08 GMT
etag: "EP5CWx59se1v4KdDnkfEx7cTkis/LUWzmydK3mjH7IeRbEc_n9J6cDQ"
expires: Fri, 01 Jan 1990 00:00:00 GMT
pragma: no-cache
server: GSE
{
"iconsVid": "foo",
"iconsVersion": 3,
"kind": "client#resourcesItem",
"etag": "\"EP5CWx59se1v4KdDnkfEx7cTkis/LUWzmydK3mjH7IeRbEc_n9J6cDQ\""
}
and a failed request ends like this:
404 Not Found
cache-control: no-cache, no-store, max-age=0, must-revalidate
content-encoding: gzip
content-length: 29
content-type: text/html; charset=UTF-8
date: Sat, 03 May 2014 03:06:10 GMT
expires: Fri, 01 Jan 1990 00:00:00 GMT
pragma: no-cache
server: GSE
Not Found
I don't know what else to try. To me, it looks like a bug in App-Engine.
So... any ideas what is going on here and how to fix or work around it?
2014-05-04: I tried changing the method from POST to GET: exact same behavior. I tried changing the path from regionIcons to regionIconsFoo: exact same behavior. I tried changing the #API version from v1 to v2: exact same behavior.
Finally, I tried changing the name of the class back to the previous (with the duplicated word): I get fewer failures (maybe 5% instead of 20%) but they still occur with the failing requests trying to access the now non-existent class name without the duplicated word.
Restoring the correct name resumes the originally described behavior with the original failure rate.
I've been struggling with similar problem. Check logs on appengine.com project site. You should see log about updating, if it has additional info about error - check it.
Sometimes AE works on local machine well but deployment process reveals some bugs.
Edit:
1. Rename the class back to old "double" name, upload it to AE and check if all requests are working without a bug, if yes, rename the class again. (if it's appengine bug it should fix it).
2. Create as simple as possible api and substitute it with your project. Update it to AE and check with api explorer is everything ok, without methods from your main project. If it's ok, once again swap "test" project with your true one and upload to AE.
This isn't an answer because it doesn't address the cause, but it is my solution.
I duplicated the working class back into the old class name.
public class RegionRegionIconsEndpoint {
#ApiMethod(name = "getRegionIconsOld", path="regionIconsOld", httpMethod = HttpMethod.POST)
public RegionInfoVersion.RegionIcons getRegionIcons(User user, #Named("id") String id)
throws OAuthRequestException {
...
}
}
Now in the log, even though I'm only ever calling getRegionIcons, I see indications that both classes are being called with the "old" version handling about 20% of the requests. It's a hack and I don't like it, but it works and the clients are happy with it.
If you can't beat 'em, join 'em.

Can't receive an HTTP response from Mule

I am creating a basic service component that takes a URL as input at http inbound endpoint. Code snippet from the mule-config file is as follows:
<service name="follow">
<inbound>
<http:inbound-endpoint address="http://localhost:8765/follow/" synchronous="true"/>
</inbound>
<component class="org.mule.application.mytwitter.Follow" />
</service>
and the function that is called from the Java component class is:
public Object onCall(MuleEventContext eventContext) throws Exception {
MuleMessage msg = eventContext.getMessage();
String str = msg.getStringProperty("http.request", null);
msg.setPayload(str);
msg.setStringProperty("http.status","200");
msg.setStringProperty("Content-Type","text/html");
System.out.println("Reached here:" + str);
return msg;
}
I wish to receive an HTTP response(payload) by hitting the service through CURL as:
curl -vv "http://localhost:8765/follow/"
but I'm not receiving any payload:
> * About to connect() to localhost port 8765 (#0)
* Trying ::1... connected
* Connected to localhost (::1) port 8765 (#0)
> GET /follow/ HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-pc-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8k zlib/1.2.3.3 libidn/1.15
> Host: localhost:8765
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: text/plain
< Date: Tue, 27 Dec 2011 03:14:00 IST
< Server: Mule Core/2.1.2
< Expires: Tue, 27 Dec 2011 03:14:00 IST
< Content-Length: 0
< Connection: close
<
* Closing connection #0
Am I missing something? The function in component class is being called and output is printed in console.
Are you intentionally using such an old version of mule? There's a much newer 3.2.1 version available. And I suggest you to move to flow-style messaging instead of services.
But to answer your problem, if you want to have the response payload to print out, then you should add a string transformer to you configuration. I don't remember exactly how to configure this to the service element, but if you use flow's then you can add a response block to the end of the flow.
<http:http-response-to-string-transformer />
I hope this helps.
never mind guys. i figured out something and it works fine now. there's an attribute called syncResponse for inbound-endpoint. Setting it to true makes it work synchronously. I think some problem to do with Mule 2.1.2 or maybe some system settings.

Categories

Resources