For some reason zip files produced on nodeJS gets rejected on a Java server where you can only use "binary" data upload.
If I post the file with Postman using binary it works fine, but when sending through nodeJS (request/request-promise/http ...etc) it does not work.
It gives:
java.util.zip.ZipException: only DEFLATED entries can have EXT descriptor
at java.util.zip.ZipInputStream.readLOC(ZipInputStream.java:310)
at java.util.zip.ZipInputStream.getNextEntry(ZipInputStream.java:122)
at com.ws...
The files are valid (it accepts via postman!)
Spent nearly two days on this issue. Read through tons on stack overflow post (most of them did not have proper answer in this specific topic) and googled around, but all the efforts in vain.
I kept experimenting and the only solution which seemed to work just fine is using child process and curl...
In case someone else falls into this really annoying problem:
var exec = require('child_process').exec;
var args = "-X POST \
https://mywebsite.com/zip-upload \
-H 'Authorization: Bearer gFChWxzeCIZVLM2q...WlvDB6zq2uOHfUcdX' \
-H 'Cache-Control: no-cache' \
-H 'Content-Type: application/zip' \
--data-binary #./data.zip";
exec('curl ' + args, function (error, stdout, stderr) {
console.log('stdout: ' + stdout);
console.log('stderr: ' + stderr);
if (error !== null) {
console.log('exec error: ' + error);
}
});
In my opinion, child_process is the last choice. If you have tried request or request-promise and failed, how about trying htp:
const htp = require('htp');
htp.post(
"https://mywebsite.com/zip-upload",
{ "Authorization" : "earer gFChWxzeCIZVLM2q...WlvDB6zq2uOHfUcdX"
, "Cache-Control" : "no-cache"
, "Content-Type" : "application/zip"
},
fs.createReadStream("./data.zip")
)
.then(function(response) { /* ... */ })
.catch(function(error) { /* ... */ })
;
I was facing similar problem. In my case, I was using custom value for Content-Type (e.g. - "application/abc+json") to upload a zip file via POST request in fetch(). It was working in Postman, but not in nodejs server.
I was able to get it working by adding Content-Length as header before making request via fetch().
Request headers I used - Content-Type, Content-Length, Authorization, and bunch of other API specific headers.
Hopefully this helps anyone facing issue.
Related
I have a rest controller in my Java backend to download files as a streams: end points return a StreamingResponseBody stream, when I try to download a file with curl it works fine:
curl -H "Authorization: Bearer <MyToken>" http://localhost:9001/rest/api/download --output test.zip
But not with angular, I'm using HttpModule get method:
this.http.get(url, headers).subscribe ...
However I can see that the file is being downloaded in the network panel of chrome developper tools, but I have no idea where the file is being saved, as if it was streamed instead of downloaded:
Can any body explain this behaviour, and how to download the stream intead of broadcasting it.
Maybe you can try something like...
this.http.get(`your_url`, { headers: headers, responseType: 'blob', observe: 'response'})
.map( res => (
{
content: res.body,
fileName: res.headers.get('content-filename')
}
));
hope this helps!
I developed a Business Network Definition with Hyperledger Composer, deployed it on a Hyperledger Fabric example-chain (running locally on a VirtualBox-installation of Ubuntu with Docker-containers) and started Composer's REST-server with composer-rest-server (and options -c [cardname] -n always -w true). Then I played a bit in the browser and all is working fine.
Now comes the problematic bit: I want to write a Java-program interacting with this REST API and somehow the API seems unable to parse my Java-sent requests. I copied the JSON-string that Java produces to the browser, ran it there and it worked fine. But if I send it programmatically I get the error (see at the end).
This is what I send through the browser
{"cId":"C_ID7","cDomain":"example.com"}
The browser tells me with CURL it should look like this
curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{"cId":"C_ID7","cDomain":"example.com"}' 'http://localhost:3000/api/com.example.Company'
Now I want to send the same message through Java+JSOUP with the following code:
Response resp = Jsoup.connect(baseURL + namespace + "Company").ignoreContentType(true).method(Method.POST)
.ignoreHttpErrors(true)
.requestBody("{\"cId\":\"C_ID7\",\"cDomain\":\"example.com\"}").execute();
The same (without the requestBody and Method.GET) works for GET-requests. ignoreContentType(true) is necessary, because JSOUP will not handle 'application/json', see here. ignoreHttpErrors(true) is necessary, to get through the Http 500 error, which conceals the true error, which I need for trouble-shooting.
So then finally what I see, when I execute the above code, is the following error:
{"error":{"statusCode":500,"name":"Error","message":"Property names containing dot(s) are not supported. Model: com_example_Company, dynamic property: {\"cId\":\"C_ID7\",\"cDomain\":\"example.com\"}","stack":"Error: Property names containing dot(s) are not supported. Model: com_example_Company, dynamic property: {\"cId\":\"C_ID7\",\"cDomain\":\"example.com\"}\n at com_example_Company.ModelBaseClass._initProperties
(/home/[user]
]/.nvm/versions/node/v8.9.4/lib/node_modules/composer-rest-server/node_modules/loopback-datasource-juggler/lib/model.js:249:17)\n
at com_example_Company.ModelBaseClass (/home/[user]/.nvm/versions/node/v8.9.4/lib/node_modules/composer-rest-server/node_modules/loopback-datasource-juggler/lib/model.js:60:8)\n
at com_example_Company.Model (eval at createModelClassCtor
(/home/[user]/.nvm/versions/node/v8.9.4/lib/node_modules/composer-rest-server/node_modules/loopback-datasource-juggler/lib/model-builder.js:671:21), <anonymous>:12:24)\n
at com_example_Company.PersistedModel (eval at createModelClassCtor
(/home/[user]/.nvm/versions/node/v8.9.4/lib/node_modules/composer-rest-server/node_modules/loopback-datasource-juggler/lib/model-builder.js:671:21), <anonymous>:12:24)\n
at new com_example_Company (eval at createModelClassCtor
(/home/[user]/.nvm/versions/node/v8.9.4/lib/node_modules/composer-rest-server/node_modules/loopback-datasource-juggler/lib/model-builder.js:671:21), <anonymous>:12:24)\n
at Function.DataAccessObject.create (/home/[user]/.nvm/versions/node/v8.9.4/lib/node_modules/composer-rest-server/node_modules/loopback-datasource-juggler/lib/dao.js:359:13)\n
at SharedMethod.invoke
(/home/[user]/.nvm/versions/node/v8.9.4/lib/node_modules/composer-rest-server/node_modules/strong-remoting/lib/shared-method.js:270:25)\n
at HttpContext.invoke (/home/[user]/.nvm/versions/node/v8.9.4/lib/node_modules/composer-rest-server/node_modules/strong-remoting/lib/http-context.js:297:12)\n
at phaseInvoke (/home/[user]/.nvm/versions/node/v8.9.4/lib/node_modules/composer-rest-server/node_modules/strong-remoting/lib/remote-objects.js:676:9)\n at runHandler
(/home/[user]/.nvm/versions/node/v8.9.4/lib/node_modules/composer-rest-server/node_modules/strong-remoting/node_modules/loopback-phase/lib/phase.js:135:5)\n
at iterate (/home/[user]/.nvm/versions/node/v8.9.4/lib/node_modules/composer-rest-server/node_modules/strong-remoting/node_modules/loopback-phase/node_modules/async/lib/async.js:146:13)\n
at Object.async.eachSeries
(/home/[user]/.nvm/versions/node/v8.9.4/lib/node_modules/composer-rest-server/node_modules/strong-remoting/node_modules/loopback-phase/node_modules/async/lib/async.js:162:9)\n
at runHandlers (/home/[user]/.nvm/versions/node/v8.9.4/lib/node_modules/composer-rest-server/node_modules/strong-remoting/node_modules/loopback-phase/lib/phase.js:144:13)\n
at iterate
(/home/[user]/.nvm/versions/node/v8.9.4/lib/node_modules/composer-rest-server/node_modules/strong-remoting/node_modules/loopback-phase/node_modules/async/lib/async.js:146:13)\n
at /home/[user]/.nvm/versions/node/v8.9.4/lib/node_modules/composer-rest-server/node_modules/strong-remoting/node_modules/loopback-phase/node_modules/async/lib/async.js:157:25\n
at /home/[user]/.nvm/versions/node/v8.9.4/lib/node_modules/composer-rest-server/node_modules/strong-remoting/node_modules/loopback-phase/node_modules/async/lib/async.js:154:25"}}
where obviously [user] is my username. So I had a look at the loopback-datasource-juggler sources and found that in model-builder.js (source on Github) instead of parsing the different properties of the string, it is given the complete string ("{\"cId\":\"C_ID7\",\"cDomain\":\"example.com\"). In line 269 the builder checks whether there are dot-characters in the property-name. Since in our case the property name is the complete JSON-string, there are dots and the program fails.
Unfortunately XML does not seem to be supported at this point.
Now: How can this happen? My only guess is an encoding thing done by Java, because the same JSON works in the browser. Any ideas what could have caused this?
And one smaller issue: The error message in the code is ''Property names containing dot(s) are not supported. ' + 'Model: %s, property: %s'', but in the error I receive it says something about "dynamic property". Am I at the wrong point for searching what causes the error?
Since you have already specified 'Content-Type: application/json', you needn't have double quotes in the requestBody. The following should work:
Response resp = Jsoup.connect(baseURL + namespace + "Company").ignoreContentType(true).method(Method.POST)
.ignoreHttpErrors(true)
.requestBody({\"cId\":\"C_ID7\",\"cDomain\":\"example.com\"}).execute();
the curl example is what the REST APIs (rather than merely the browser) know about the business network REST API endpoints.
for your requestBody, you will need the fully qualified class (ie meaning asset/participant/transaction class with leading namespace) - an example is :
curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{ \
"$class": "org.example.biznet.SampleAsset", \
"assetId": "1", \
"value": "103300" \
}' 'http://localhost:3000/api/SampleAsset'
So yours will be something like
Response resp = Jsoup.connect(baseURL + namespace + "Company").ignoreContentType(true).method(Method.POST)
.ignoreHttpErrors(true)
.requestBody("{\"\$class\": \"org.example.biznet.SampleAsset\", \"cId\":\"C_ID7\", \"cDomain\":\"example.com\"}").execute();
I believe you also needed at least one other double-quotes (total above now : 14) and a closing curly bracket in addition.
I found this example of a API request. Unfortunately I didn't find any other example how to upload an image to the API.
As I'm not familiar with Python I'm trying to understand how to do the same in a cURL command.
import requests
auth_headers = {
'app_id': 'your_app_id',
'app_key': 'your_app_key'
}
url = 'https://XXXXXXX'
files = {
'source': open('media/test.jpg')
}
data = {
'timeout': 60
}
response = requests.post(url, files=files, data=data, headers=auth_headers)
I tried to convert it by trying out a cURL to python converter, but I don't know how to build it with the files.
In the end I want to do the request in JAVA, but I think if I would know the request in cURL I can figure it out.
Hope anyone can help me with that.
This will do it:
#!/bin/bash
args=(
-H 'app_id: your_app_id'
-H 'app_key: your_app_key'
-F 'source=#/path/to/file'
-F 'timeout=60'
'http://httpbin.org/post'
)
curl "${args[#]}"
or, as a one-liner:
curl -H 'app_id: your_app_id' -H 'app_key: your_app_key' -F 'source=#/path/to/file' -F 'timeout=60' 'http://httpbin.org/post'
Use -H to specify header fields (repeat for every field) and -F to specify form fields - either as key=value pairs, or filename=#path pairs. When -F is used, POST method is the default, and Content-Type is multipart/form-data (but that too can be overridden).
I'm migrating a project from PHP to Java.
I had no problem doing this, but the project contains a HTTP request which I couldn't set up yet. It has to grab a JSON from an API (Authorisation required) and then save it as a variable (which gets passed on to my parser class).
This is the code I've used in php:
$result = file_get_contents($url, null, stream_context_create(array(
'http' => array(
'method' => 'GET',
'header' => 'Accept: application/json' . "\r\n"
. 'Authorization: Bearer ' . $token . "\r\n",
),
)));
$url is the request url and $token the authorisation for the API.
I've already tried a couple of solutions and libraries from StackOverflow, but couldn't get any of these to work in the Java Version. Also I haven't found code which includes the response code, which would actually allow the program to notify the user what went wrong.
Update: I've tried to use the libery linked in the comments, but encoding the url just doesn't work, I get a java.io.UnsupportedEncodingException:
String url = URLEncoder.encode(requestURL,"UTF-8");
I also tried:
Url url = URLEncoder.encode(requestURL,"UTF-8");
which resulted in the same error.
Documentation doesn't help to fix this. Can someone explain what I'm doing wrong or give me some example code?
I tried to submit data to an endpoint but it said the data size was too large, so I changed the method to POST and received the error:
This API does not support parsing form-encoded input.
Next I changed the type to application/json, still with post and now I am getting:
{
"error": {
"errors": [
{
"domain": "global",
"reason": "parseError",
"message": "Parse Error"
}
],
"code": 400,
"message": "Parse Error"
}
}
What is the best way to post a large amount of data, i.e. 2730 bytes to an endpoint and have it handle it properly? In my case the field in question is of type Text as I am over the 500 character limit for app engine to hold in a String.
Also, as with many things, this works great on my local machine, it only gives this error on the live app engine instance.
Thanks!
Not sure if your problem is related, but I received the "This API does not support parsing form-encoded input." error when I was attempting to use curl to send a POST message like this:
curl -X POST -d '{"name": "Foo"}' http://foo.appspot.com/_ah/api/foo/1/endpoint
The problem was that I was not setting the content type. curl POSTs with Content-Type: application/x-www-form-urlencoded if it's not specified on the command line. Google cloud endpoints don't accept this content type.
When I changed the curl invocation to include the content type, it worked:
curl -X POST -d '{"name": "Foo"}' --header "Content-Type: application/json" http://foo.appspot.com/_ah/api/foo/1/endpoint