Okhttp big file upload fails - java

I am experiencing an issue with Okhttp + Node.js Formidable serverside big file upload.
Currently the upload works for < 100Mb files but fails for bigger files.
For bigger files, the onprogress event serverside is fired until 99% progress, regardless of the file size, then it stops, reports request abort, and the onfile event is not fired.
Already tried timeouts workarounds, even defined a custom SocketFactory to manually set the socket keepalive and sotimeout.
So, Im stuck here. Any help would be appreciated.
CustomSocketFactory MySocketFactory = new CustomSocketFactory();
OkHttpClient client = new OkHttpClient.Builder()
.socketFactory(MySocketFactory)
.readTimeout(largenumber, TimeUnit.SECONDS)
.writeTimeout(largenumber, TimeUnit.SECONDS)
.connectTimeout(largenumber, TimeUnit.SECONDS)
.build();
RequestBody requestBody = new MultipartBody.Builder()
.setType(MultipartBody.FORM)
.addFormDataPart("nombre_file", file_pais.getName())
.addFormDataPart("nombre_pais", pais.getName())
.addFormDataPart("file", file_pais.getName(),
RequestBody.create(MediaType.parse("application/octet-stream"),
new File(filepath)
.build();
Request request = new Request.Builder()
.url(server_url)
.post(requestBody)
.build();
Call call = client.newCall(request);
Response response = call.execute();
response.body().close();

Its way too easy using snoopy api: one line of code if you exclude identifiers definition :)
URI uri = ...;
Path fileToUpload = ...;
Snoopy.builder()
.config(SnoopyConfig.defaults())
.build()
.post(uri)
.followRedirects(true)
.failIfNotSuccessfulResponse(true)
.body(fileToUpload)
.consumeAsString();
https://bitbucket.org/abuwandi/snoopy
Tested on large files and it worked like a charm

Related

how to fetch metrics data from prometheus and use it within java springboot App?

I want to create an Alarm when CPU memory/Hard Disk Memory is full. I am using spring-boot f/w along with Prometheus. I can view all metrics details on HTTP requests:- http://localhost:9090/actuator/prometheus. but IDK how can I fetch the data from Prometheus in my project to raise an alarm or to do any action.
your suggestion/links will be more valuable. Thank you.
You can scrap metrics from your actuator endpiont in the same way as Prometheus do it.
I use OkHttpClient for that.
Example of my client:
OkHttpClient client = new OkHttpClient.Builder()
.retryOnConnectionFailure(false)
.followRedirects(false)
.protocols(Collections.singletonList(Protocol.H2_PRIOR_KNOWLEDGE))
.build();
All settings are optional. But pay attention to the protocol - it should be the same as on your application's server.
After that you need to build url:
String url = "http://localhost:9090/actuator/prometheus?includedNames=<nameOfThePropertyThatYouNeed>";
You can include here more than 1 property:
String url = "http://localhost:9090/actuator/prometheus?includedNames=<propertyNameOne>,<propertyNameTwo>,<propertyNameThree>";
After that you make request:
Request request =new Request.Builder()
.url(url)
.get()
.build();
Response response = client.newCall(request).execute();
String responseBody = response.body().string();
Now you need to parse responseBody. Do it in the way as Prometheus does it and use classes of Prometheus:
InputStream inputStream = new ByteArrayInputStream(responseBody.getBytes())
CollectorPrometheusMetricsWalker walker = new CollectorPrometheusMetricsWalker();
PrometheusMetricsProcessor<MetricFamily> processor = new TextPrometheusMetricsProcessor(inputStream, walker);
processor.walk();
List<MetricFamily> metricList = walker.getAllMetricFamilies();
MetricFamily object stores all metrics with the same name but with different tags.
Use metricFamily.getMetrics() to get List<Metric>
Use metric.getValue() to get the value of metric.
You should use AlertManager that is part of Prometheus suit.

Slow responsetime from Amazon Ec2 instance

Im running Java/Spring application with a Angular6 frontend. Within the Spring backend I calling another API thru http. but its around 6x slower than expected to get at response...
Isolating the API-call I get:
Locally running the same setup (Spring, Tomcat 8.5) from my dev-machine: 10-12 sec
Locally thru Postman: 10-12 sec
locally thru cUrl: 10-12 sec
Using cUrl from the console on the AWS EC2 instance 10-12sec.
Calling as intended (Spring running in Tomcat) on the AWS EC2 instance: 60-75sec.
Tried to change the Native Spring http (ResponseEntity postForEntity) call to OkHttp with no change in the result. The Api request and response is a tiny Json-string but the APi is slow so response times around 10 sec is normal.
As it seems to be something wrong with the Tomcat or java on the EC2. Can it be some parameters that I missed? or what else can i try to isolate and solve the problem?
The amount of data sent and revived is trivial (sending ~350 bytes, receiving less than that)
Its just the call to the API that takes a long time.
logger.debug("CallApi start");
start = System.currentTimeMillis();
try {
ObjectMapper mapper = new ObjectMapper();
String s = mapper.writeValueAsString(apiCalcDTO.getInput());
OkHttpClient client = new OkHttpClient.Builder()
.connectTimeout(120, TimeUnit.SECONDS)
.writeTimeout(120, TimeUnit.SECONDS)
.readTimeout(120, TimeUnit.SECONDS)
.build();
MediaType mediaType = MediaType.parse("application/x-www-form-urlencoded");
RequestBody body = RequestBody.create(mediaType, body);
Request request = new Request.Builder()
.url("http://www.example.com/api/1.0/xxx")
.post(body)
.addHeader("Content-Type", "application/x-www-form-urlencoded")
.addHeader("cache-control", "no-cache")
.build();
Response response = client.newCall(request).execute();
elapsed = System.currentTimeMillis() - start;
logger.debug("CallApi time = " + elapsed);

Okhttp: Adding unencrypted custom headers in for Proxy server on https requests

We are using okhttp v3.8.0 in our project. We have to add custom header specifically for proxy server on https requests. The issues is that when i set ".header("Something", "FRR")", header would be encrypted on https requests as well, so it would not be identified by Proxy server. How can I achieve that? I want to send the header unencrypted in Initial method.
That's how I send my request to proxy server right now:
OK_HTTP_CLIENT = builder
.connectTimeout(60, TimeUnit.SECONDS)
.writeTimeout(60, TimeUnit.SECONDS)
.readTimeout(60, TimeUnit.SECONDS)
.cookieJar(cookieJar)
.retryOnConnectionFailure(true)
.proxy(proxy)
.proxyAuthenticator(proxyAuthenticatorMainAccount)
.build();
Request request = new Request.Builder()
.url(url)
.header("Something", "FRR")
.build();
Response response = OK_HTTP_CLIENT.newCall(request).execute();
There is screenshot here, explain what i want to achieve in more details
This isn't possibly in OkHttp currently. It's being tracked here. If it's important to you, please explain your situation and we’ll respond accordingly.

OPTIONS/HEAD REST API request with Okhttp3

I`m writing some Rest client on Android and I met a problem - I have no idea how to make HEAD and OPTIONS requests.
There are no problems with GET/POST/PUT/DELETE/PATCH requests in OkHttp3, basically they looks like:
request = new Request.Builder()
.url(url)
.headers(headerBuilder.build())
.post(bodyBuilder.build())
.build();
And OkHttp3 doesnt provide additional methods like head() or option().
So how can I make HEAD and OPTIONS requests using OkHttp3?
Found answer, may be it will be useful for someone else
OkHttp3 still has method
Builder method(String method, RequestBody body)
So OPTIONS requests looks like
Request request = new Request.Builder()
.url(url)
.headers(headerBuilder.build())
.method("OPTIONS",requestBody)
.build();
same for HEAD
It appears (at least in the current implementation, API 3.12.0), HEAD request can be made just like GET and others:
Request request = new Request.Builder()
.url(url)
.head()
.build();
OPTION still has to be implemented using .method()

BlobstoreService.getUploads always returns NULL

I'm trying to store an image to the Google Appengine Blobstore from an android device. What I've done so far:
Created an Enpoint (Google Cloud Endpoints) that returns an upload URL (Working)
Created a POST request with OKHTTP3 that sends the image file in a multipartform (Working? Maybe not?)
Created a Servlet that is passed to the upload URL to handle getting the keys. (It gets called, but getUpload always returns null.)
I'm thinking maybe it has to do with how I'm sending my POST request?
OkHttpClient client = new OkHttpClient();
RequestBody body = new MultipartBody.Builder()
.setType(MultipartBody.FORM)
.addPart(
Headers.of("Content-Disposition", "form-data; name=\"photo\""),
RequestBody.create(MediaType.parse("image/jpeg"), file)
)
.build();
Request request = new Request.Builder()
.url(url)
.post(body)
.build();
In my Servlet I can see a param named "photo" but calling:
List<BlobKey> blobs = blobstoreService.getUploads(req).get("photo");
returns null. Zero BlobKeys...
I'm sure I'm missing something dumb... Any help would be incredibly appreciated!
So in the end it WAS the POST request.
RequestBody body = new MultipartBody.Builder()
.setType(MultipartBody.FORM)
.addFormData("photo", "photoname")
.addFormData("photo", "photo.jpg", RequestBody.create(MediaType.parse("image/jpeg"), file)
.build();

Categories

Resources