I am using the StreamLambdaHandler in aws-serverless-java-container to create a REST Api as AWS Lambda function. It is set up to run behind an Api Gateway api which uses Lambda Proxy Integration (/{proxy+} - ANY - Method Execution).
Everything works fine but I have one #Controller route that is supposed to return a zip file (binary data).
Whatever I am trying, all I get back is a Base64 encoded response body. Pasting logs from the Test call in Api Gateway below. When running the URL in a browser, it downloads a file foo.zip but it's actually not a zip file but:
> file foo.zip
> foo.zip: ASCII text, with CRLF line terminators
In API Gateway, I have added */* under Settings > Binary Media Types as described. Update: after this step deploy the API Resources > Actions > Deploy API
What am I missing? Here is some code:
Pojo (simplified)
public final class LicenseEntity {
public String name;
public byte[] content;
}
Controller code
defaultHeaders = new HttpHeaders();
defaultHeaders.add("X-Requested-With", "*");
defaultHeaders.add("Access-Control-Allow-Methods", "GET, POST, DELETE, PUT, PATCH, OPTIONS");
defaultHeaders.add("Access-Control-Allow-Headers", "Content-Type,X-Amz-Date,Authorization,X-Api-Key,x-requested-with");
#RequestMapping(value = "/download/licenses/{licenseId}", produces="application/zip")
public ResponseEntity<StreamingResponseBody> download(#PathVariable("licenseId") String licenseId) {
// left out
return createResponseEntity(..);
}
private ResponseEntity<StreamingResponseBody> createResponseEntity(List<LicenseEntity> entities) {
var zipFileHeaders = new HttpHeaders();
for (String keyName: defaultHeaders.keySet()) {
zipFileHeaders.add(keyName, defaultHeaders.getFirst(keyName));
}
zipFileHeaders.add(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=foo.zip");
zipFileHeaders.add(HttpHeaders.CONTENT_TYPE, "application/octet-stream");
return ResponseEntity
.ok()
.headers(zipFileHeaders)
.body(out -> {
try (var zipOutputStream = new ZipOutputStream(out);) {
for (LicenseEntity entity : entities) {
try (InputStream inputStream = new ByteArrayInputStream(entity.content);) {
zipOutputStream.putNextEntry(new ZipEntry(entity.name));
IOUtils.copy(inputStream, zipOutputStream);
}
zipOutputStream.closeEntry();
}
}
});
}
Api Gateway Logs
Execution log for request 478b0d95-a8db-41e9-8382-0de77ec54cc0
Thu Nov 19 16:31:11 UTC 2020 : Starting execution for request: 478b0d95-a8db-41e9-8382-0de77ec54cc0
Thu Nov 19 16:31:11 UTC 2020 : HTTP Method: GET, Resource Path: /v1/download/licenses/9d53f58e-ec14-4d3e-b93f-a014a0a01b5a
Thu Nov 19 16:31:11 UTC 2020 : Method request path: {proxy=v1/download/licenses/9d53f58e-ec14-4d3e-b93f-a014a0a01b5a}
Thu Nov 19 16:31:11 UTC 2020 : Method request query string: {}
Thu Nov 19 16:31:11 UTC 2020 : Method request headers: {}
Thu Nov 19 16:31:11 UTC 2020 : Method request body before transformations:
Thu Nov 19 16:31:11 UTC 2020 : Endpoint request URI: https://lambda.eu-west-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:eu-west-1:ACCOUNTID:function:FUNCTIONNAME/invocations
Thu Nov 19 16:31:11 UTC 2020 : Endpoint request headers: {x-amzn-lambda-integration-tag=478b0d95-a8db-41e9-8382-0de77ec54cc0, Authorization=************************************************************************************************************************************************************************************************************************************************************************************************************************f22b2b, X-Amz-Date=20201119T163111Z, x-amzn-apigateway-api-id=HASHCODE, X-Amz-Source-Arn=arn:aws:execute-api:eu-west-1:ACCOUNTID:HASHCODE/test-invoke-stage/GET/{proxy+}, Accept=application/json, User-Agent=AmazonAPIGateway_HASHCODE, X-Amz-Security-Token=IQoJb3JpZ2luX2VjEOj//////////wEaCWV1LXdlc3QtMSJGMEQCIDBqxIr89XxH+0xD/MjXhenc3o4h18uZk9dmaifikAkaAiBLInRUvlJrH8Jp7esPn2NO7CzGkNow05ysMS5RaDSy/iq0AwhhEAEaDDYzMTE0NDAwMjA5OSIMl1iChMw1yeXx/oyCKpED/P/jygty9Mud/QVn5b2o3mLBDppbiy9Ns5X2LLTEckvy/azM6HL+25vBvnB3zJVBahBP369P44UYmPBAa1UniayEX0Kk3oa6sJgDHY9X4O6Dt09NYsUykIBfPxJS0F5uhdjGGNKEUNdKV8zTf0GZ [TRUNCATED]
Thu Nov 19 16:31:11 UTC 2020 : Endpoint request body after transformations: {"resource":"/{proxy+}","path":"/v1/download/licenses/9d53f58e-ec14-4d3e-b93f-a014a0a01b5a","httpMethod":"GET","headers":null,"multiValueHeaders":null,"queryStringParameters":null,"multiValueQueryStringParameters":null,"pathParameters":{"proxy":"v1/download/licenses/9d53f58e-ec14-4d3e-b93f-a014a0a01b5a"},"stageVariables":null,"requestContext":{"resourceId":"md5zfv","resourcePath":"/{proxy+}","httpMethod":"GET","extendedRequestId":"WQ2YgFwcjoEFVvQ=","requestTime":"19/Nov/2020:16:31:11 +0000","path":"/{proxy+}","accountId":"ACCOUNTID","protocol":"HTTP/1.1","stage":"test-invoke-stage","domainPrefix":"testPrefix","requestTimeEpoch":1605803471936,"requestId":"478b0d95-a8db-41e9-8382-0de77ec54cc0","identity":{"cognitoIdentityPoolId":null,"cognitoIdentityId":null,"apiKey":"test-invoke-api-key","principalOrgId":null,"cognitoAuthenticationType":null,"userArn":"arn:aws:iam::ACCOUNTID:user/NAME","apiKeyId":"test-invoke-api-key-id","userAgent":"aws-internal/3 aws- [TRUNCATED]
Thu Nov 19 16:31:11 UTC 2020 : Sending request to https://lambda.eu-west-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:eu-west-1:ACCOUNTID:function:FUNCTIONNAME/invocations
Thu Nov 19 16:31:12 UTC 2020 : Received response. Status: 200, Integration latency: 177 ms
Thu Nov 19 16:31:12 UTC 2020 : Endpoint response headers: {Date=Thu, 19 Nov 2020 16:31:12 GMT, Content-Type=application/json, Content-Length=946, Connection=keep-alive, x-amzn-RequestId=1f12b2ae-2bc6-4a57-a1a3-cbf15fb48a2b, x-amzn-Remapped-Content-Length=0, X-Amz-Executed-Version=$LATEST, X-Amzn-Trace-Id=root=1-5fb69dcf-d3d8068ef93ecbbd68717aef;sampled=0}
Thu Nov 19 16:31:12 UTC 2020 : Endpoint response body before transformations: {"statusCode":200,"multiValueHeaders":{"Access-Control-Allow-Headers":["Content-Type,X-Amz-Date,Authorization,X-Api-Key,x-requested-with"],"Access-Control-Allow-Methods":["GET, POST, DELETE, PUT, PATCH, OPTIONS"],"Content-Disposition":["attachment; filename=foo.zip"],"Content-Type":["application/octet-stream"],"Vary":["Origin","Access-Control-Request-Method","Access-Control-Request-Headers"],"X-Requested-With":["*"]},"body":"UEsDBBQACAgIAOaDc1EAAAAAAAAAAAAAAAAFAAAAMC50eHTLSM3JyQcAUEsHCIamEDYHAAAABQAA\r\nAFBLAwQUAAgICADmg3NRAAAAAAAAAAAAAAAABQAAADEudHh0y0jNyckHAFBLBwiGphA2BwAAAAUA\r\nAABQSwMEFAAICAgA5oNzUQAAAAAAAAAAAAAAAAUAAAAyLnR4dMtIzcnJBwBQSwcIhqYQNgcAAAAF\r\nAAAAUEsBAhQAFAAICAgA5oNzUYamEDYHAAAABQAAAAUAAAAAAAAAAAAAAAAAAAAAADAudHh0UEsB\r\nAhQAFAAICAgA5oNzUYamEDYHAAAABQAAAAUAAAAAAAAAAAAAAAAAOgAAADEudHh0UEsBAhQAFAAI\r\nCAgA5oNzUYamEDYHAAAABQAAAAUAAAAAAAAAAAAAAAAAdAAAADIudHh0UEsFBgAAAAADAAMAmQAA\r\nAK4AAAAAAA==","isBase64Encoded":true}
Thu Nov 19 16:31:12 UTC 2020 : Method response body after transformations: UEsDBBQACAgIAOaDc1EAAAAAAAAAAAAAAAAFAAAAMC50eHTLSM3JyQcAUEsHCIamEDYHAAAABQAA
AFBLAwQUAAgICADmg3NRAAAAAAAAAAAAAAAABQAAADEudHh0y0jNyckHAFBLBwiGphA2BwAAAAUA
AABQSwMEFAAICAgA5oNzUQAAAAAAAAAAAAAAAAUAAAAyLnR4dMtIzcnJBwBQSwcIhqYQNgcAAAAF
AAAAUEsBAhQAFAAICAgA5oNzUYamEDYHAAAABQAAAAUAAAAAAAAAAAAAAAAAAAAAADAudHh0UEsB
AhQAFAAICAgA5oNzUYamEDYHAAAABQAAAAUAAAAAAAAAAAAAAAAAOgAAADEudHh0UEsBAhQAFAAI
CAgA5oNzUYamEDYHAAAABQAAAAUAAAAAAAAAAAAAAAAAdAAAADIudHh0UEsFBgAAAAADAAMAmQAA
AK4AAAAAAA==
Thu Nov 19 16:31:12 UTC 2020 : Method response headers: {Access-Control-Allow-Headers=Content-Type,X-Amz-Date,Authorization,X-Api-Key,x-requested-with, Access-Control-Allow-Methods=GET, POST, DELETE, PUT, PATCH, OPTIONS, Content-Disposition=attachment; filename=foo.zip, Content-Type=application/octet-stream, Vary=Origin,Access-Control-Request-Method,Access-Control-Request-Headers, X-Requested-With=*, X-Amzn-Trace-Id=Root=1-5fb69dcf-d3d8068ef93ecbbd68717aef;Sampled=0}
Thu Nov 19 16:31:12 UTC 2020 : Successfully completed execution
Thu Nov 19 16:31:12 UTC 2020 : Method completed with status: 200
Update
I never redeployed my API Gateway API after adding the binary media type. Apparently, this needs to be done, in order to get the changes out.
I'm trying to select a record from particular view in Airtable using the Java library
So far i have set break points to check if everything is initialised and it seems be okay. My api key is correct and the Airtable instance is setup correctly.
Airtable airtable = new Airtable().configure(AIRTABLE_API_KEY);
Base basebase = airtable.base("my-airtable-base");
This is my error:
Nov 12, 2020 5:53:08 PM org.apache.http.client.protocol.ResponseProcessCookies processCookies
WARNING: Invalid cookie header: "Set-Cookie: brw=brwkel6HWNoWVEl49; path=/; expires=Fri, 12 Nov 2021 17:53:08 GMT; domain=.airtable.com; samesite=none; secure; httponly". Invalid 'expires' attribute: Fri, 12 Nov 2021 17:53:08 GMT
Exception in thread "main" com.sybit.airtable.exception.AirtableException: {"error":"NOT_FOUND"} (UNDEFINED_ERROR) [Http code 404]
at com.sybit.airtable.exception.HttpResponseExceptionHandler.onResponse(HttpResponseExceptionHandler.java:29)
at com.sybit.airtable.Table.select(Table.java:206)
at com.sybit.airtable.Table.select(Table.java:327)
at com.hived.AirtableInstance.selectTableView(AirtableInstance.java:43)
at com.hived.Main.main(Main.java:25)
This is the function that is causing the error:
public void selectTableView() throws AirtableException, HttpResponseException {
List<Bus> stops = base.table("Bus").select("Stops");
}
I was expecting it to pass all stops from the bus table into the stops list.
I'm not sure what I'm doing wrong, so any help would be much appreciated.
Turns out you need to do a few things.
Add log4j to your project
Add slf4j-simple to your project
Once those dependencies are added. You should now see there is no warning anymore.
Now to remove the error. It turns out instead of referencing the base name, you need to call the base id instead.
base = airtable.base("applJilugnJCtDRdh");
Thats it! Hope this helps anyone else.
Below is the sample code . i want to call a rest api using gluon.
RestClient restClient = RestClient.create()
.host("https://abc.api.in")
.path("v1/signup")
.formParam("name",name)
.formParam("email",email)
.formParam("password",password)
.formParam("repeatPassword",repeatPassword)
.formParam("refID",refID)
.method("POST").contentType("text/plain");
GluonObservableObject<SignUpDAO> sample = DataProvider.retrieveObject(restClient.createObjectDataReader(SignUpDAO.class));
But it gives warning like and it is not showing any further progress.I am using Task to call rest client from my controller.
INFO: Created Rest Connection:
Method: POST
Request URL: urlname
Form Params: {password=[test123], repeatPassword=[test123], name=[pqr], refID=[dsfdfkdlfk], email=[test#gmail.com]}
ContentType: text/plain
Consumer Credentials: null / null
Aug 19, 2016 12:17:20 PM com.gluonhq.connect.converter.JsonConverter readFromJson
INFO: Property password not defined on json object for class class com.gluonapplication.com.coinsecureapp.controller.SignUpDAO.
Aug 19, 2016 12:17:20 PM com.gluonhq.connect.converter.JsonConverter readFromJson
INFO: Property repeatPassword not defined on json object for class class com.gluonapplication.com.coinsecureapp.controller.SignUpDAO.
Aug 19, 2016 12:17:20 PM com.gluonhq.connect.converter.JsonConverter readFromJson
INFO: Property name not defined on json object for class class com.gluonapplication.com.coinsecureapp.controller.SignUpDAO.
Aug 19, 2016 12:17:20 PM com.gluonhq.connect.converter.JsonConverter readFromJson
INFO: Property refID not defined on json object for class class com.gluonapplication.com.coinsecureapp.controller.SignUpDAO.
Aug 19, 2016 12:17:20 PM com.gluonhq.connect.converter.JsonConverter readFromJson
INFO: Property email not defined on json object for class class com.gluonapplication.com.coinsecureapp.controller.SignUpDAO.
I'm downloading a JAR file, and would like to utilize If-Modified-Since so I don't get the whole file if I don't need it, but for some reason my vanilla Apache (afaik) isn't returning the 304 correctly.
This is from wireshark:
GET /whatever.jar HTTP/1.1
If-Modified-Since: Sat, 04 Jan 2014 21:46:26 GMT
User-Agent: Jakarta Commons-HttpClient/3.1
Host: example.com
HTTP/1.1 200 OK
Date: Sat, 04 Jan 2014 20:32:31 GMT
Server: Apache/2.2.4 (Unix) mod_ssl/2.2.4 OpenSSL/0.9.8e DAV/2 mod_jk/1.2.26 PHP/5.3.6 SVN/1.4.4
Last-Modified: Sat, 04 Jan 2014 19:13:14 GMT
ETag: "b6c037-1ddad9f-d17a6680"
Accept-Ranges: bytes
Content-Length: 31305119
Vary: User-Agent
Content-Type: text/plain
... [bunch of bytes] ...
There aren't other headers I need to specify, is there? Am I missing a module that Apache needs in order to read this header correctly?
Any other thoughts or suggestions?
Here is my Java code, for reference:
File jarFile = new File(filePath);
GetMethod get = new GetMethod(downloadUrl);
Date lastModified = new Date(jarFile.lastModified());
get.setRequestHeader("If-Modified-Since", DateUtil.formatDate(lastModified));
HttpClient client = new HttpClient();
int code = client.executeMethod(get);
UPDATE: Solution
The If-Modified-Date needed to exactly match the server, and I achieved this by explicitly setting the lastModifiedDate on the downloaded file:
String serverModified = get.getResponseHeader("Last-Modified").getValue();
jarFile.setLastModified(DateUtil.parseDate(serverModified).getTime());
After doing this, subsequent calls would not download the file.
In order to use the "If-Modified-Since" header, you must send an identical header value as the "Last-Modified" header, that is Sat, 04 Jan 2014 19:13:14 GMT != Sat, 04 Jan 2014 21:46:26 GMT. Apache cannot guarantee the file wasn't modified and given a past time on purpose (perhaps through a version control roll-back).
If you want, you may check the "Last-Modified" header on the client side, by using a HeadMethod first to avoid "getting" the resource if it hasn't been modified. Then you would use a "GetMethod" if it has been modified.
See RFC2616 - Section 9, "HTTP/1.1: Method Definitions" for more.
In Java, the following regular expression
To: a#b\.com.*Subject: Please verify your email address
somehow doesn't find the match in this text:
Dez 21, 2012 10:29:58 AM com.google.appengine.api.datastore.dev.LocalDatastoreService init
INFO: Local Datastore initialized:
Type: High Replication
Storage: In-memory
Dez 21, 2012 10:29:58 AM com.google.appengine.api.mail.dev.LocalMailService log
INFO: MailService.send
Dez 21, 2012 10:29:58 AM com.google.appengine.api.mail.dev.LocalMailService log
INFO: From:
Dez 21, 2012 10:29:58 AM com.google.appengine.api.mail.dev.LocalMailService log
INFO: To: a#b.com
Dez 21, 2012 10:29:58 AM com.google.appengine.api.mail.dev.LocalMailService log
INFO: Subject: Please verify your email address
Dez 21, 2012 10:29:58 AM com.google.appengine.api.mail.dev.LocalMailService log
INFO: Body:
Dez 21, 2012 10:29:58 AM com.google.appengine.api.mail.dev.LocalMailService log
INFO: Content-type: text/plain
Dez 21, 2012 10:29:58 AM com.google.appengine.api.mail.dev.LocalMailService log
INFO: Data length: 4
My Java code looks like this:
Matcher matcher = Pattern.compile(regex, Pattern.MULTILINE).matcher(input);
if (matcher.find()) {
...
}
This is a bit strange, since the pattern seems to work when I test it online with this tool: http://regexpal.com
So, Java must be interpreting the pattern a bit differently. Is there any way to get error messages of the Matcher?
Update It should find:
To: a#b.com
Dez 21, 2012 10:29:58 AM com.google.appengine.api.mail.dev.LocalMailService log
INFO: Subject: Please verify your email address
You'll want to use Pattern.DOTALL instead of Pattern.MULTILINE.
DOTALL makes the . match newlines. (Which is what you want)
MULTILINE makes ^ and $ work on a per-line basis.