My current AWS setup is a lambda function that is being triggered whenever I put an object into a S3 bucket. I implemented the lambda's handler function in Java. What I want to do is simply accessing the file that was uploaded and triggered the execution of the lambda function. E.g., if I upload sample.json to the bucket, I want to access the contents of this file in my handler function.
I know I can do something like this:
public Void handleRequest(S3Event input, Context context) {
for (S3EventNotificationRecord record : input.getRecords()) {
String key = record.getS3().getObject().getKey();
String bucket = record.getS3().getBucket().getName();
AmazonS3 s3Client = new AmazonS3Client(credentials);
try {
S3Object s3Object = s3Client.getObject(new GetObjectRequest(bucket, key));
InputStream input = s3Object.getObjectContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(input));
while (true) {
String line = reader.readLine();
if (line == null) break;
// Do something with line...
}
// ...
The problem is that I am not allowed to use access keys. Thus, I cannot create an s3Client to download the file with. In other words, I have to get the object from the argument that my handler method takes, i.e., S3Event input. How would I do that?
If your Lambda function is configured with an appropriate IAM role (that allows s3:GetObject of the relevant S3 object), then you don't need to explicitly provide credentials in your code.
Here's sample Java code to get a object in response to an object uploaded Lambda event:
public class S3GetTextBody implements RequestHandler<S3Event, String> {
public String handleRequest(S3Event s3event, Context context) {
try {
S3EventNotificationRecord record = s3event.getRecords().get(0);
// Retrieve the bucket & key for the uploaded S3 object that
// caused this Lambda function to be triggered
String bkt = record.getS3().getBucket().getName();
String key = record.getS3().getObject().getKey().replace('+', ' ');
key = URLDecoder.decode(key, "UTF-8");
// Read the source file as text
AmazonS3 s3Client = AmazonS3ClientBuilder.defaultClient();
String body = s3Client.getObjectAsString(bkt, key);
System.out.println("Body: " + body);
return "ok";
} catch (Exception e) {
System.err.println("Exception: " + e);
return "error";
}
}
}
Imports:
import java.net.URLDecoder;
import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.RequestHandler;
import com.amazonaws.services.lambda.runtime.events.S3Event;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.s3.event.S3EventNotification.S3EventNotificationRecord;
Related
I'm getting the following exception when trying to read a file to AWS S3 Error:
Unable to load AWS credentials from any provider in the chain.
I have generated a public bucket and also generated an AWS IAM role with full S3 bucket access and textract access.
I am trying to read an image from S3 bucket and run AWS Textract service.
Java code:
package com.textract;
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3Client;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.s3.model.S3Object;
import com.amazonaws.services.s3.model.S3ObjectInputStream;
import software.amazon.awssdk.core.SdkBytes;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.textract.TextractClient;
import software.amazon.awssdk.services.textract.model.*;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
public class App {
public static Map<String, String> getRelationships(Map<String, Block> blockMap, Map<String, Block> keyMap,Map<String, Block> valueMap) {
Map<String, String> result = new LinkedHashMap<>();
for(Map.Entry<String, Block> itr : keyMap.entrySet()) {
Block valueBlock = findValue(itr.getValue(), valueMap);
String key = getText(itr.getValue(), blockMap);
String value = getText(valueBlock, blockMap);
result.put(key, value);
}
return result;
}
public static Block findValue(Block keyBlock, Map<String, Block> valueMap) {
Block b = null;
for(Relationship relationship : keyBlock.relationships()) {
if(relationship.type().toString().equals("VALUE")) {
for(String id : relationship.ids()) {
b = valueMap.get(id);
}
}
}
return b;
}
public static String getText(Block result, Map<String, Block> blockMap) {
StringBuilder stringBuilder = new StringBuilder();
for(Relationship relationship : result.relationships()) {
if(relationship.type().toString().equals("CHILD")) {
for(String id : relationship.ids()) {
Block b = blockMap.get(id);
if(b.blockTypeAsString().equals("WORD")) {
stringBuilder.append(b.text()).append(" ");
}
}
}
}
return stringBuilder.toString();
}
public static void main(String[] args) {
BasicAWSCredentials creds = new BasicAWSCredentials("<Secret Key>", "<Access Key>");
AmazonS3 s3client = AmazonS3Client.builder()
.withRegion("ap-southeast-1")
.withCredentials(new AWSStaticCredentialsProvider(creds))
.build();
// AmazonS3 s3client = AmazonS3ClientBuilder.standard().build();
S3Object s3Object = s3client.getObject("<S3 Bucket>", "image.jpg");
S3ObjectInputStream s3ObjectInputStream = s3Object.getObjectContent();
SdkBytes bytes = SdkBytes.fromInputStream(s3ObjectInputStream);
Document doc = Document.builder().bytes(bytes).build();
List<FeatureType> list = new ArrayList<>();
list.add(FeatureType.FORMS);
AnalyzeDocumentRequest request = AnalyzeDocumentRequest.builder().featureTypes(list).document(doc).build();
TextractClient textractClient = TextractClient.builder().region(Region.US_WEST_2).build();
AnalyzeDocumentResponse response = textractClient.analyzeDocument(request);
List<Block> blocks = response.blocks();
Map<String, Block> blockMap = new LinkedHashMap<>();
Map<String, Block> keyMap = new LinkedHashMap<>();
Map<String, Block> valueMap = new LinkedHashMap<>();
for (Block b : blocks) {
String block_id = b.id();
blockMap.put(block_id, b);
if(b.blockTypeAsString().equals("KEY_VALUE_SET")) {
for(EntityType entityType : b.entityTypes()) {
if(entityType.toString().equals("KEY")) {
keyMap.put(block_id, b);
} else {
valueMap.put(block_id, b);
}
}
}
}
System.out.println(getRelationships(blockMap, keyMap, valueMap));
textractClient.close();
}
}
Console error:
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Exception in thread "main" software.amazon.awssdk.core.exception.SdkClientException: Unable to load credentials from any of the providers in the chain AwsCredentialsProviderChain(credentialsProviders=[SystemPropertyCredentialsProvider(), EnvironmentVariableCredentialsProvider(), WebIdentityTokenCredentialsProvider(), ProfileCredentialsProvider(), ContainerCredentialsProvider(), InstanceProfileCredentialsProvider()]) : [SystemPropertyCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., EnvironmentVariableCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., WebIdentityTokenCredentialsProvider(): Either the environment variable AWS_WEB_IDENTITY_TOKEN_FILE or the javaproperty aws.webIdentityTokenFile must be set., ProfileCredentialsProvider(): Profile file contained no credentials for profile 'default': ProfileFile(profiles=[]), ContainerCredentialsProvider(): Cannot fetch credentials from container - neither AWS_CONTAINER_CREDENTIALS_FULL_URI or AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment variables are set., InstanceProfileCredentialsProvider(): Unable to load credentials from service endpoint.]
at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:98)
at software.amazon.awssdk.auth.credentials.AwsCredentialsProviderChain.resolveCredentials(AwsCredentialsProviderChain.java:112)
at software.amazon.awssdk.auth.credentials.internal.LazyAwsCredentialsProvider.resolveCredentials(LazyAwsCredentialsProvider.java:45)
at software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider.resolveCredentials(DefaultCredentialsProvider.java:104)
at software.amazon.awssdk.awscore.client.handler.AwsClientHandlerUtils.createExecutionContext(AwsClientHandlerUtils.java:76)
at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.createExecutionContext(AwsSyncClientHandler.java:68)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$1(BaseSyncClientHandler.java:97)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:167)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:94)
at software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45)
at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:55)
at software.amazon.awssdk.services.textract.DefaultTextractClient.analyzeDocument(DefaultTextractClient.java:215)
at com.textract.App.main(App.java:83)
You are only passing the credentials to the S3 client. You aren't passing them to the Textract client. If you are creating the credentials provider manually like that, you will need to pass it to each client instance that is created.
I highly suggest using the environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY to provide credentials, instead of setting them directly in your code like that, as it will be much more flexible when you run this code in other environments, and will allow the AWS SDK to pick up the credentials automatically without the need for you to create a credentials provider at all.
I finally got it working with the below code with the below article:
Calling Amazon Textract Synchronous Operations
String document="input.png";
String bucket="bucket";
AmazonTextract client = AmazonTextractClientBuilder.defaultClient();
DetectDocumentTextRequest request = new DetectDocumentTextRequest()
.withDocument(new Document()
.withS3Object(new S3Object()
.withName(document)
.withBucket(bucket)));
DetectDocumentTextResult result = client.detectDocumentText(request);
Working Code
package aws.cloud.work;
import java.io.IOException;
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.textract.AmazonTextract;
import com.amazonaws.services.textract.AmazonTextractClientBuilder;
import com.amazonaws.services.textract.model.DetectDocumentTextRequest;
import com.amazonaws.services.textract.model.DetectDocumentTextResult;
import com.amazonaws.services.textract.model.Document;
import com.amazonaws.services.textract.model.S3Object;
public class TextractOriginalMaster3 {
static AmazonTextractClientBuilder clientBuilder = AmazonTextractClientBuilder.standard().withRegion(Regions.AP_SOUTHEAST_1);
public static void main(String[] args) throws IOException {
//Set AWS Credentials to use Textract
clientBuilder.setCredentials(new AWSStaticCredentialsProvider(new
BasicAWSCredentials("Accesskey", "Secretkey")));
//**Getting document from S3 Bucket Path
String document = "image.png";
String bucket = "Bucket Name";
//Calling AWS Textract Client
AmazonTextract client = clientBuilder.build();
DetectDocumentTextRequest request = new DetectDocumentTextRequest()
.withDocument(new Document()
.withS3Object(new S3Object()
.withName(document)
.withBucket(bucket)));
DetectDocumentTextResult result = client.detectDocumentText(request);
System.out.println(result);
result.getBlocks().forEach(block -> {
if (block.getBlockType().equals("LINE"))
System.out.println("text is " + block.getText() + " confidence is " + block.getConfidence());
});
}
}
To answer this, we can load the credentials using StaticCredentialsProvider
Please refer the below code.
TextractClient textractClient = TextractClient.builder().region(Region.of(region)).credentialsProvider(StaticCredentialsProvider.create(AwsBasicCredentials
.create(accessKey,secretKey))).build();
I am able to call AWS Textract to read an image from my local path. How can I integrate this textract code to read the image uploaded onto a created S3 bucket with the S3 bucket codes.
Working Textract Code to textract images from local path
package aws.cloud.work;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileWriter;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.io.InputStream;
import org.json.simple.JSONArray;
import org.json.simple.JSONObject;
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.textract.AmazonTextract;
import com.amazonaws.services.textract.AmazonTextractClientBuilder;
import com.amazonaws.services.textract.model.DetectDocumentTextRequest;
import com.amazonaws.services.textract.model.DetectDocumentTextResult;
import com.amazonaws.services.textract.model.Document;
import com.amazonaws.util.IOUtils;
public class TextractDemo {
static AmazonTextractClientBuilder clientBuilder = AmazonTextractClientBuilder.standard()
.withRegion(Regions.US_EAST_1);
private static FileWriter file;
public static void main(String[] args) throws IOException {
//AWS Credentials to access AWS Textract services
clientBuilder.setCredentials(new AWSStaticCredentialsProvider(
new BasicAWSCredentials("Access Key", "Secret key")));
//Set the path of the image to be textract. Can be configured to use from S3
String document="C:\\Users\\image-local-path\\sampleTT.jpg";
ByteBuffer imageBytes;
//Code to use AWS Textract services
try (InputStream inputStream = new FileInputStream(new File(document))) {
imageBytes = ByteBuffer.wrap(IOUtils.toByteArray(inputStream));
}
AmazonTextract client = clientBuilder.build();
DetectDocumentTextRequest request = new DetectDocumentTextRequest()
.withDocument(new Document().withBytes(imageBytes));
/*
* DetectDocumentTextResult result = client.detectDocumentText(request);
* System.out.println(result); result.getBlocks().forEach(block ->{
* if(block.getBlockType().equals("LINE")) System.out.println("text is "+
* block.getText() + " confidence is "+ block.getConfidence());
*/
//
DetectDocumentTextResult result = client.detectDocumentText(request);
System.out.println(result);
JSONObject obj = new JSONObject();
result.getBlocks().forEach(block -> {
if (block.getBlockType().equals("LINE"))
System.out.println("text is " + block.getText() + " confidence is " + block.getConfidence());
JSONArray fields = new JSONArray();
fields.add(block.getText() + " , " + block.getConfidence());
obj.put(block.getText(), fields);
});
//To import the results into JSON file and output the console output as sample.txt
try {
file = new FileWriter("/Users/output-path/sample.txt");
file.write(obj.toJSONString());
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
file.flush();
file.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
This is an example of the console out where the "text" and corresponding "confidence scores" are returned
S3 bucket code integration I managed to find from the docs:
String document = "sampleTT.jpg";
String bucket = "textract-images";
AmazonS3 s3client = AmazonS3ClientBuilder.standard()
.withEndpointConfiguration(
new EndpointConfiguration("https://s3.amazonaws.com","us-east-1"))
.build();
// Get the document from S3
com.amazonaws.services.s3.model.S3Object s3object = s3client.getObject(bucket, document);
S3ObjectInputStream inputStream = s3object.getObjectContent();
BufferedImage image = ImageIO.read(inputStream);
(Edited) - Thanks #smac2020, I currently have a working Rekognition Code that reads from my AWS console S3 bucket and runs the Rekognition services that I am referencing to. However, I am unable to modify and merge it with
the Textract source code
package com.amazonaws.samples;
import com.amazonaws.auth.AWSCredentialsProvider;
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.services.rekognition.AmazonRekognition;
import com.amazonaws.services.rekognition.AmazonRekognitionClientBuilder;
import com.amazonaws.services.rekognition.model.AmazonRekognitionException;
import com.amazonaws.services.rekognition.model.DetectLabelsRequest;
import com.amazonaws.services.rekognition.model.DetectLabelsResult;
import com.amazonaws.services.rekognition.model.Image;
import com.amazonaws.services.rekognition.model.Label;
import com.amazonaws.services.rekognition.model.S3Object;
import java.util.List;
public class DetectLabels {
public static void main(String[] args) throws Exception {
String photo = "sampleTT.jpg";
String bucket = "Textract-bucket";
// AmazonRekognition rekognitionClient = AmazonRekognitionClientBuilder.standard().withRegion("ap-southeast-1").build();
AWSCredentialsProvider credentialsProvider = new AWSStaticCredentialsProvider (new BasicAWSCredentials("Access Key", "Secret Key"));
AmazonRekognition rekognitionClient = AmazonRekognitionClientBuilder.standard().withCredentials(credentialsProvider).withRegion("ap-southeast-1").build();
DetectLabelsRequest request = new DetectLabelsRequest()
.withImage(new Image()
.withS3Object(new S3Object()
.withName(photo).withBucket(bucket)))
.withMaxLabels(10)
.withMinConfidence(75F);
try {
DetectLabelsResult result = rekognitionClient.detectLabels(request);
List <Label> labels = result.getLabels();
System.out.println("Detected labels for " + photo);
for (Label label: labels) {
System.out.println(label.getName() + ": " + label.getConfidence().toString());
}
} catch(AmazonRekognitionException e) {
e.printStackTrace();
}
}
}
Looks like you are trying to read an Amazon S3 object from a Spring boot app and then pass that byte array to DetectDocumentTextRequest.
There is a tutorial that shows a very similar use case where a Spring BOOT app reads the bytes from an Amazon S3 object and passes it to the Amazon Rekognition service (instead of Textract).
The Java code is:
// Get the byte[] from this AWS S3 object.
public byte[] getObjectBytes (String bucketName, String keyName) {
s3 = getClient();
try {
GetObjectRequest objectRequest = GetObjectRequest
.builder()
.key(keyName)
.bucket(bucketName)
.build();
ResponseBytes<GetObjectResponse> objectBytes = s3.getObjectAsBytes(objectRequest);
byte[] data = objectBytes.asByteArray();
return data;
} catch (S3Exception e) {
System.err.println(e.awsErrorDetails().errorMessage());
System.exit(1);
}
return null;
}
See this AWS development article to see how to build a Spring BOOT app that has this functionality.
Creating an example AWS photo analyzer application using the AWS SDK for Java
This example uses the AWS SDK For Java V2. If you are not familiar with working with the latest SDK version, I recommend that you start here:
Get started with the AWS SDK for Java 2.x
I want to move the file inside the S3 folder to another folder which is in the same s3 Bucket. I tried the below code
CopyObjectRequest copyObjRequest = new CopyObjectRequest(bucketName,
srcFolder+"/"+Filename, bucketName,
targetFolder+"/"+Filename);
s3Client.copyObject(copyObjRequest);
DeleteObjectRequest deleteObjRequest = new DeleteObjectRequest(bucketName,
srcFolder+"/"+Filename);
s3Client.deleteObject(deleteObjRequest);
The folder may contain multiple file, i want to move only the selected file. Above code is not showing any error, but nothing happens. Can anyone please suggest me the right solution for it.
It would be a good initial stab just to run the following code and check what the output is, without and deletions.
Also worth checking the ACL and bucket policy on the object.
This is the format expected
CopyObjectRequest(java.lang.String sourceBucketName, java.lang.String sourceKey, java.lang.String destinationBucketName, java.lang.String destinationKey)
If you want a copy of the object in the same bucket
CopyObjectRequest copyObjRequest = new CopyObjectRequest("myBucket", "myObject.txt", "myBucket", "myNewObject.txt");
s3Client.copyObject(copyObjRequest);
If you want a copy of the object in a different bucket
CopyObjectRequest copyObjRequest = new CopyObjectRequest("myBucket", "myObject.txt", "myOtherBucket", "myNewObject.txt");
s3Client.copyObject(copyObjRequest);
Sample code for testing
import java.io.IOException;
import com.amazonaws.AmazonServiceException;
import com.amazonaws.SdkClientException;
import com.amazonaws.auth.profile.ProfileCredentialsProvider;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.s3.model.CopyObjectRequest;
public class CopyObjectSingleOperation {
public static void main(String[] args) throws IOException {
String clientRegion = "*** Client region ***";
String bucketName = "*** Bucket name ***";
String sourceKey = "*** Source object key *** ";
String destinationKey = "*** Destination object key ***";
try {
AmazonS3 s3Client = AmazonS3ClientBuilder.standard()
.withCredentials(new ProfileCredentialsProvider())
.withRegion(clientRegion)
.build();
// Copy the object into a new object in the same bucket.
CopyObjectRequest copyObjRequest = new CopyObjectRequest(bucketName, sourceKey, bucketName, destinationKey);
s3Client.copyObject(copyObjRequest);
}
catch(AmazonServiceException e) {
// The call was transmitted successfully, but Amazon S3 couldn't process
// it, so it returned an error response.
e.printStackTrace();
}
catch(SdkClientException e) {
// Amazon S3 couldn't be contacted for a response, or the client
// couldn't parse the response from Amazon S3.
e.printStackTrace();
}
}
}
I am trying to use TransferManager class in my java program to download a file from a Requester Pays Buckets.
I am getting "Status Code: 403" exception from amazon AWS.
I wrote a simple program to test this feature and compare it single connection way of downloading a file. Here is my code:
import java.io.*;
import com.amazonaws.auth.*;
import com.amazonaws.services.s3.*;
import com.amazonaws.services.s3.model.*;
import com.amazonaws.services.s3.transfer.*;
import com.amazonaws.util.IOUtils;
public class RequesterPaysTest {
final static AWSCredentials awsCredentials = new BasicAWSCredentials(MY IAMAccessKey,MY IAMSecretKey);
final static String bucketName = "7268982505fe.mixnode.com";
final static String fileName = "5379-7268982505fe-0-1496081968663.warc.gz";
final static AWSCredentialsProvider awsCredentialsProvider = new AWSStaticCredentialsProvider(awsCredentials);
final static AmazonS3 s3client = AmazonS3ClientBuilder.standard().withCredentials(awsCredentialsProvider).withRegion(MY bucketRegion).build();
final static GetObjectRequest getRequest = new GetObjectRequest(bucketName, fileName, true);
static void testSimpleRequesterPays() {
try {
S3Object object = s3client.getObject(getRequest);
InputStream objectData = object.getObjectContent();
FileOutputStream out = new FileOutputStream (new File(fileName));
IOUtils.copy(objectData, out);
out.close();
System.out.println(" Simple RequesterPays successful");
} catch (Exception e) {
System.out.println(" Simple RequesterPays unsuccessful: " + e.getMessage());
}
}
static void testTransferManagerRequesterPays() {
try {
TransferManager tx = TransferManagerBuilder.standard().withS3Client(s3client).build();
Download download = tx.download(getRequest, new File(fileName));
while (download.isDone() == false)
Thread.sleep(10);
System.out.println(" TransferManager RequesterPays successful");
} catch (Exception e) {
System.out.println(" TransferManager RequesterPays unsuccessful: " + e.getMessage());
}
}
public static void main(String[] args) throws IOException {
testSimpleRequesterPays();
testTransferManagerRequesterPays();
}
}
And here is the output:
Simple RequesterPays successful
TransferManager RequesterPays unsuccessful: Forbidden (Service: Amazon S3; Status Code: 403; Error Code: 403 Forbidden; Request ID: 77D5EBF5EE195A7A)
As you can see, the same file can be downloaded using simple method but not with TransferManager. I tried to create my own bucket and played around with permissions but it did not work.
I was wondering if I miss anything in my code? Or whether AWS s3 does not have support for using TransferManager on a requester pays bucket?
TransferManager internally using http HEAD method instead of GET for downloading objects(file(s)). You can find it by debugging TransferManager SDK code. If you don't have HEAD method configuration in your aws api gateway, you have to create HEAD method configuration to match with TransferManager SDK code. By the way if resource not found it should give 404 but you are getting 403. reason is here https://forums.aws.amazon.com/thread.jspa?threadID=216684
I found this answer and I think it applies, but with a little twist.
How to return multiple objects from a Java method?
I have two JSON formatted files using the YML subset, one of which is a list of devices, and the other is a file that lists the attributes of a particular type of device.
The choice of dividing up the list of Device instances into one file and the attributes in another file is to allow the Device manufacturer to change the attributes without having to go back and rewrite/recompile hard coded attributes.
In any case, I could use two different calls to the JSONParser and then add the list of attributes to the Device object in the end, but that solution seems like a waste of code since, except for the inner part of the while loop where the values are set, they do exactly the same thing.
I thought something like a Ruby-ish Yield might do the trick in the inner loop, but not sure if this exists in Java.
So, without much further ado, here is the code:
// Import the json simple parser, used to read configuration files
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import java.util.Iterator;
import java.util.LinkedHashMap;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import org.json.simple.JSONArray;
import org.json.simple.JSONObject;
import org.json.simple.JSONValue;
import org.json.simple.parser.*;
public class LoadJson {
private String path = "";
private String fileType = "";
//public LoadJson(String path, String fileType){
//this.path = path;
//this.fileType = fileType;
//}
public static Device openJSON(String fileType, String deviceName) {
JSONParser parser = new JSONParser();
Device myDevice = new Device();
ContainerFactory containerFactory = new ContainerFactory(){
public List creatArrayContainer() {
return new LinkedList();
}
public Map createObjectContainer() {
return new LinkedHashMap();
}
};
try {
File appBase = new File("."); //current directory
String path = appBase.getAbsolutePath();
System.out.println(path);
Map obj = (Map)parser.parse(new FileReader(path+fileType),containerFactory);
Iterator iter = obj.entrySet().iterator();
//Iterator iterInner = new Iterator();
while(iter.hasNext()){
//LinkedList entry = (LinkedList)iter.next();
LinkedList myList = new LinkedList();
Map.Entry entry = (Map.Entry)iter.next();
myList = (LinkedList) (entry.getValue());
Iterator iterate = myList.iterator();
while (iterate.hasNext())
{
LinkedHashMap entry2 = (LinkedHashMap)iterate.next();
if(fileType=="mav2opc")
{
String deviceName1 = entry2.get("DeviceName").toString();
String userName = entry2.get("UserName").toString();
String password = entry2.get("Password").toString();
String deviceType = entry2.get("DeviceType").toString();
String ipAddress = entry2.get("IPAddress").toString();
myDevice = new Device(deviceName1, userName, password, deviceType,ipAddress);
openJSON(deviceType,deviceName1);
System.out.println(myDevice);
} else
{
//Add a tag
String tagName = entry2.get("tagName").toString();
String tagType = entry2.get("tagType").toString();
String tagXPath = entry2.get("tagXPath").toString();
String tagWritable = entry2.get("tagWritable").toString();
}
}
}
//System.out.println("==toJSONString()==");
//System.out.println(JSONValue.toJSONString(json));
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} catch(ParseException pe){
System.out.println(pe);
}
return myDevice;
}
}
Ok, so you have two files, one with list of devices and the other, per-device file, which has attributes of the device. The structures of the two files is exactly same, I am guessing something like
The devices file:
{{"DeviceName:"d1","UserName":"u1","Password":"p1","DeviceType":"t1","IPAddress":"i1"},
{"DeviceName:"d2","UserName":"u2","Password":"p2","DeviceType":"t2","IPAddress":"ir"}}
And the per-device file
{{"tagName:"n1","tagType":"t1","tagXPath":"X1","tagWritable":true}}
In the per-device file there is only one map, though it is inside a list. The processing logic is to open the file, create the parser, read json and for each entry in the list process the map.
The logic for processing the map is the only difference between the two. Note that right now you are discarding what you read from the per-device file, which you will have to store somewhere in the myDevice
One way to do this is using a callback: create an interface MapHandler that has a method process. openJSON takes a parameter of this type and calls process on it for each method.
The device-level handler can be constructed with the myDevice being processed and set the fields.