I am working in a functionality where i need to upload an Image/File to firebase storage using java and expose it as an API. I have already achieved this functionality in angular 4 typescript. But now i need this method as an Java Rest API, so that my peer can also consume the same method instead of writing a new one. So is there any API or methods to write the image to firebase storage ?
Try this:
FirebaseOptions options = FirebaseOptions.builder()
.setCredentials(credential)
.setDatabaseUrl(projectUrl)
.setStorageBucket("YOUR BUCKET LINK")
.build();
FirebaseApp fireApp = FirebaseApp.initializeApp(options);
StorageClient storageClient = StorageClient.getInstance(fireApp);
InputStream testFile = new FileInputStream("YOUR FILE PATH");
String blobString = "NEW_FOLDER/" + "FILE_NAME.EXT";
storageClient.bucket().create(blobString, testFile , Bucket.BlobWriteOption.userProject("YOUR PROJECT ID"));
If the Java project is running in a trusted environment (such as their development machine, a server you control, or Cloud Functions), they can use the Firebase Admin SDK to access Cloud Storage.
See the Firebase Admin SDK documentation on how to get started, and then the Google Cloud Storage documentation for Java clients for more. Specifically have a look at the sample of uploading a file in Java:
BlobId blobId = BlobId.of("bucket", "blob_name");
BlobInfo blobInfo = BlobInfo.newBuilder(blobId).setContentType("text/plain").build();
Blob blob = storage.create(blobInfo, "Hello, Cloud Storage!".getBytes(UTF_8));
If you are using Spring Boot, you can try this:
Create a class to expose it as a web service in your API:
import com.yourcompany.yourproject.services.FirebaseFileService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.multipart.MultipartFile;
#RestController
public class ResourceController {
#Autowired
private FirebaseFileService firebaseFileService;
#PostMapping("/api/v1/test")
public ResponseEntity create(#RequestParam(name = "file") MultipartFile file) {
try {
String fileName = firebaseFileService.saveTest(file);
// do whatever you want with that
} catch (Exception e) {
// throw internal error;
}
return ResponseEntity.ok().build();
}
}
Create a service to upload the image to firebase storage.
#Service
public class FirebaseFileService {
private Storage storage;
#EventListener
public void init(ApplicationReadyEvent event) {
try {
ClassPathResource serviceAccount = new ClassPathResource("firebase.json");
storage = StorageOptions.newBuilder().
setCredentials(GoogleCredentials.fromStream(serviceAccount.getInputStream())).
setProjectId("YOUR_PROJECT_ID").build().getService();
} catch (Exception ex) {
ex.printStackTrace();
}
}
public String saveTest(MultipartFile file) throws IOException{
String imageName = generateFileName(file.getOriginalFilename());
Map<String, String> map = new HashMap<>();
map.put("firebaseStorageDownloadTokens", imageName);
BlobId blobId = BlobId.of("YOUR_BUCKET_NAME", imageName);
BlobInfo blobInfo = BlobInfo.newBuilder(blobId)
.setMetadata(map)
.setContentType(file.getContentType())
.build();
storage.create(blobInfo, file.getInputStream());
return imageName;
}
private String generateFileName(String originalFileName) {
return UUID.randomUUID().toString() + "." + getExtension(originalFileName);
}
private String getExtension(String originalFileName) {
return StringUtils.getFilenameExtension(originalFileName);
}
}
Note you need to download Firebase config file and store it as "firebase.json" under the src/main/resources folder.
https://support.google.com/firebase/answer/7015592?hl=en
Also you need to add the Maven dependency:
<dependency>
<groupId>com.google.firebase</groupId>
<artifactId>firebase-admin</artifactId>
<version>6.14.0</version>
</dependency>
String blobString = DIR + fileIdWithExtension;
Blob blob = storageClient.bucket().create(blobString, file.getInputStream(), Bucket.BlobWriteOption.userProject(PROJECT_ID));
But keep in mind that you should initialize firebase. For example
private FirebaseApp initFirebase() {
FileInputStream serviceAccount;
try {
serviceAccount = new FileInputStream(fileUploadPath);
} catch (FileNotFoundException e) {
throw new FileStorageException(ErrorMessage.FILE_NOT_FOUND + "firebaseConfig.json");
}
FirebaseOptions options;
try {
options = new FirebaseOptions.Builder()
.setCredentials(GoogleCredentials.fromStream(serviceAccount))
.setStorageBucket(BUCKET_NAME)
.build();
} catch (IOException e) {
e.printStackTrace();
throw new FailedToSetCredentialsException(ErrorMessage.COULD_NOT_SET_CREDENTIALS);
}
return FirebaseApp.initializeApp(options);
}
Related
i have set on my properties like this
gcp.storage.json-path=${GOOGLE_APPLICATION_CREDENTIALS}
and have code like this
public CloudStorageService(
#Value("${gcp.storage.bucket-name-file}") String fileBucketName,
#Value("${gcp.storage.bucket-name}") String bucketName,
#Value("${gcp.storage.json-path}") String jsonPath
) {
this.fileBucketName = fileBucketName;
this.bucketName = bucketName;
this.jsonPath = jsonPath;
log.info("fileBuckerName: {}", fileBucketName);
log.info("bucketName: {}", bucketName);
log.info("jsonPath: {}", jsonPath);
}
#PostConstruct
void init() {
try {
GoogleCredentials credentials = GoogleCredentials.fromStream(new FileInputStream(jsonPath))
.createScoped(Lists.newArrayList("https://www.googleapis.com/auth/cloud-platform", "https://www.googleapis.com/auth/devstorage.read_write"));
// storage = StorageOptions.getDefaultInstance().getService();
but when i deploy my code on kubernetes i cannot upload my file to GCS. my question is how to read data from my kubernetes workload identity using java ?
I'm creating a spring boot application with the AWS SDK. I am having some troubles with my S3 configuration class. I originally had all of the access keys and secrets keys stored in the code, and am transitioning to having an IAM user credentials in application.properties that has access to secrets manager, and getting all of the credentials for services through that. For some reason, the values are not properly populating. I believe it's possible that the constructor is executing before the value injection. Not sure on the best solution to fix this, thanks in advance for any help.
Here's the code for the S3 config class –
#Configuration
public class S3Config {
private String accessKeyId;
private String secretAccessKey;
#Bean
public AmazonS3 amazonS3Client(#Value("${aws.access-key-id}") String accessKeyId, #Value("${aws.secret-access-key") String secretAccessKey) {
this.accessKeyId = accessKeyId;
this.secretAccessKey = secretAccessKey;
BasicAWSCredentials awsCredentials = getBasicAWSCredentials();
return AmazonS3ClientBuilder
.standard()
.withRegion("xx")
.withCredentials(new AWSStaticCredentialsProvider(awsCredentials))
.build();
}
private BasicAWSCredentials getBasicAWSCredentials() {
String secretName = "xx";
String region = "xx";
AWSSecretsManager client = AWSSecretsManagerClientBuilder.standard()
.withRegion(region)
.withCredentials(
new AWSStaticCredentialsProvider(
new BasicAWSCredentials(
accessKeyId,
secretAccessKey
)
)
)
.build();
String secret = "";
String decodedBinarySecret;
GetSecretValueRequest getSecretValueRequest = new GetSecretValueRequest()
.withSecretId(secretName);
GetSecretValueResult getSecretValueResult = null;
try {
getSecretValueResult = client.getSecretValue(getSecretValueRequest);
} catch (DecryptionFailureException e) {
// Secrets Manager can't decrypt the protected secret text using the provided KMS key.
// Deal with the exception here, and/or rethrow at your discretion.
throw e;
} catch (InternalServiceErrorException e) {
// An error occurred on the server side.
// Deal with the exception here, and/or rethrow at your discretion.
throw e;
} catch (InvalidParameterException e) {
// You provided an invalid value for a parameter.
// Deal with the exception here, and/or rethrow at your discretion.
throw e;
} catch (InvalidRequestException e) {
// You provided a parameter value that is not valid for the current state of the resource.
// Deal with the exception here, and/or rethrow at your discretion.
throw e;
} catch (ResourceNotFoundException e) {
// We can't find the resource that you asked for.
// Deal with the exception here, and/or rethrow at your discretion.
throw e;
}
if (getSecretValueResult.getSecretString() != null) {
secret = getSecretValueResult.getSecretString();
} else {
decodedBinarySecret = new String(Base64.getDecoder().decode(getSecretValueResult.getSecretBinary()).array());
}
JsonObject secrets = new Gson().fromJson(secret, JsonObject.class);
return new BasicAWSCredentials(secrets.get("accessKey").getAsString(), secrets.get("secretKey").getAsString());
}
And here's an example of me using the S3 config in a service class. This was all working before I introduced the application.properties file and #Value annotations.
#Service
public class FileService {
private AmazonS3 amazonS3Client;
private String bucketName;
private FileRepository fileRepository;
public FileService(AmazonS3 amazonS3Client, FileRepository fileRepository) {
this.amazonS3Client = amazonS3Client;
bucketName = "xx";
this.fileRepository = fileRepository;
}
I'm trying to pass a file to my springboot backend. (which will then be uploaded to an s3 bucket), but I'm receiving this error that I can't figure out.
The file itself will contain an array, of an array of strings
Error -
java.lang.IllegalStateException: No primary or single public constructor found for class java.io.File - and no default constructor found either
Data Source -
// Data saved to S3 bucket / downloadableData function
if (this.lng !== "0.000000" && this.lng !== "") {
this.locationData.push([`Longitude: ${this.lng}, Latitude: ${this.lat}, Uncertainty Radius: ${this.uncertainty_radius} meters, Address: ${this.place_name}, Source: TEXT`])
this.locationData = JSON.parse(JSON.stringify(this.locationData))
}
Axios Post -
downloadableData() {
const blob = new Blob([this.locationData], {type: 'application/json'});
const data = new FormData();
data.append("document", blob);
axios.post("http://localhost:8080/api/v1/targetLocation/uploadStreamToS3Bucket", blob)
},
Springboot method -
public void uploadStreamToS3Bucket(File locations) {
try {
AmazonS3 s3Client = AmazonS3ClientBuilder.standard()
.withRegion(String.valueOf(awsRegion))
.build();
String bucketName = "downloadable-cases";
String fileName = connectionRequestRepository.findStream() +".json";
s3Client.putObject(new PutObjectRequest(bucketName, fileName, locations));
} catch (AmazonServiceException ex) {
System.out.println("Error: " + ex.getMessage());
}
}
Data example
I see you want to upload a file that contain JSON data in it. This can be done in a Spring BOOT app via logic like this.
<p>Upload images to an S3 Bucket. Each image will be analyzed!</p>
<form method="POST" onsubmit="myFunction()" action="/upload" enctype="multipart/form-data">
<input type="file" name="file" /><br/><br/>
<input type="submit" value="Submit" />
</form>
To handle this upload in a Spring Controller, you can use this logic:
// Upload a file to place into an Amazon S3 bucket.
#RequestMapping(value = "/upload", method = RequestMethod.POST)
#ResponseBody
public ModelAndView singleFileUpload(#RequestParam("file") MultipartFile file) {
try {
byte[] bytes = file.getBytes();
String name = file.getOriginalFilename() ;
// Put the file into the bucket.
s3Client.putObject(bytes, bucketName, name);
} catch (IOException e) {
e.printStackTrace();
}
return new ModelAndView(new RedirectView("photo"));
}
Now you have the byte array and file name. You can place this into an Amazon S3 bucket by using the AWS SDK for Java V2.
private S3Client getClient() {
Region region = Region.US_WEST_2;
S3Client s3 = S3Client.builder()
.credentialsProvider(EnvironmentVariableCredentialsProvider.create())
.region(region)
.build();
return s3;
}
// Places an image into a S3 bucket.
public String putObject(byte[] data, String bucketName, String objectKey) {
s3 = getClient();
try {
PutObjectResponse response = s3.putObject(PutObjectRequest.builder()
.bucket(bucketName)
.key(objectKey)
.build(),
RequestBody.fromBytes(data));
return response.eTag();
} catch (S3Exception e) {
System.err.println(e.getMessage());
System.exit(1);
}
return "";
}
Here is a complete document that shows this use case. This use case actually uses the Amazon Rekognition service to analyze photos in an Amazon S3 bucket; however, it still demonstrates how to successfully upload a file from your desktop to an Amazon S3 bucket Also - it's implemented using the AWS SDK For Java V2 - which is the version that Amazon Recommends.
Creating a dynamic web application that analyzes photos using the AWS SDK for Java
So I have problem when deleting file from GCS bucket, I create my file using java, the code is like:
public void upload(String projectId, String bucketName, String filePath, String fileName)
throws IOException, URISyntaxException {
File f = new File(gcsCredDirectory+gcsCredFileName);
if (!f.exists()) {
f.mkdirs();
}
try(InputStream is = new FileInputStream(f)) {
StorageOptions storageOptions = StorageOptions.newBuilder()
.setProjectId(projectId).setCredentials(fromStream(is)).build();
Storage storage = storageOptions.getService();
BlobId blobId = BlobId.of(bucketName, fileName);
BlobInfo blobInfo = BlobInfo.newBuilder(blobId).build();
Blob result = storage.create(blobInfo, Files.readAllBytes(Paths.get(filePath)));
URL url = storage.signUrl(blobInfo, MAX_EXPIRED_DATE, TimeUnit.DAYS, SignUrlOption.withV4Signature());
} catch (Exception e) {
LOGGER.error("ERROR at GoogleCloudStorageServiceImpl.upload cause : ", e);
throw e;
}
}
The code to create went well, I get the Url to download the file I uploaded and actually can download the file, but after I deleting the file through this code:
public boolean delete(String projectId, String bucketName, String fileName) {
File f = new File(gcsCredDir+gcsCredFileName);
if (!f.exists()) {
f.mkdirs();
}
try(InputStream is = new FileInputStream(f)) {
StorageOptions storageOptions = StorageOptions.newBuilder()
.setProjectId(projectId)
.setCredentials(fromStream(is))
.build();
boolean result = storageOptions.getService().delete(bucketName, fileName);
LOGGER.info("Object " + fileName + " was deleted from " + bucketName);
return result;
} catch (Exception e) {
return false;
}
}
I was able to see the log Object + fileName + was deleted from + bucketName, but when I access the Url to download the file, I can still download it. I expect the download should failed because the file was deleted.
Any advice?
Thank you
Google has its own caches which will store what you upload for some time after you delete it. You need to override the settings using Headers on upload. Set Cache-Control: max-age=0, no-cache. You can also specify public or private.
public means intermediate servers may cache the result (for faster response times).
private means only the requesting client may cache the response, but not intermediate servers. This is usually set to enable a client to get a fresh copy each time the request is made.
To try and force the cache to drop the data, some servers accept PURGE requests. These can be issued via curl -vv -X PURGE http(s)://example.com/path/to/resource
Edit:
You can set the cache control headers using gsutil: https://cloud.google.com/storage/docs/gsutil/addlhelp/WorkingWithObjectMetadata
i have used following java code to detect a text of a image, but here problem is i couldn't able to authenticate correctly
public static void detectText() throws Exception, IOException {
GoogleCredential credential = GoogleCredential.getApplicationDefault();
List<AnnotateImageRequest> requests = new ArrayList<>();
ByteString imgBytes = ByteString.readFrom(new FileInputStream("/home/buddika/Desktop/car_number_pate_16.jpeg"));
Image img = Image.newBuilder().setContent(imgBytes).build();
Feature feat = Feature.newBuilder().setType(Type.TEXT_DETECTION).build();
AnnotateImageRequest request =
AnnotateImageRequest.newBuilder()
.addFeatures(feat).setImage(img).build();
requests.add(request);
try (ImageAnnotatorClient client = ImageAnnotatorClient.create()) {
BatchAnnotateImagesResponse response = client.batchAnnotateImages(requests);
List<AnnotateImageResponse> responses = response.getResponsesList();
for (AnnotateImageResponse res : responses) {
if (res.hasError()) {
System.out.println("Error: %s\n"+ res.getError().getMessage());
return;
}
// For full list of available annotations, see http://g.co/cloud/vision/docs
for (EntityAnnotation annotation : res.getTextAnnotationsList()) {
System.out.println("Text: %s\n" + annotation.getDescription());
System.out.println("Position : %s\n" + annotation.getBoundingPoly());
}
}
}
}
i am having below error message once execute this code lines
Exception in thread "main" java.io.IOException: The Application Default Credentials are not available. They are available if running on Google App Engine, Google Compute Engine, or Google Cloud Shell. Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
at com.google.api.client.googleapis.auth.oauth2.DefaultCredentialProvider.getDefaultCredential(DefaultCredentialProvider.java:98)
at com.google.api.client.googleapis.auth.oauth2.GoogleCredential.getApplicationDefault(GoogleCredential.java:213)
at com.google.api.client.googleapis.auth.oauth2.GoogleCredential.getApplicationDefault(GoogleCredential.java:191)
at com.security.management.system.api.google_cloud_api.TextDetect.detectText(TextDetect.java:157)
at com.security.management.system.api.google_cloud_api.TextDetect.main(TextDetect.java:48)
following libs were used
import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.cloud.vision.spi.v1.ImageAnnotatorClient;
import com.google.cloud.vision.v1.*;
import com.google.cloud.vision.v1.Feature.Type;
import com.google.protobuf.ByteString;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.PrintStream;
import java.util.ArrayList;
import java.util.List;
note i have used the .json file also which downloaded accring to the instruction given from this link
note :- i have set the GOOGLE_APPLICATION_CREDENTIALS variable in .bashrc file
could you please help me on this issue
I used google cloud vision api with spring boot. And able to extract text from images.
for credentials added below properties.
application.properties file:
spring.cloud.gcp.project-id=mycloud-123456
spring.cloud.gcp.credentials.location=file:src/main/resources/service-account.json.
added dependency in pom.xml
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-gcp-starter-vision</artifactId>
<version>1.2.3.RELEASE</version>
</dependency>
Your above function is working in the project. commented line which were defining GoogleCredential.
#RequestMapping("/gettext")
public String detectText() throws Exception, IOException {
//GoogleCredential credential = GoogleCredential.getApplicationDefault();
String finalText = "Ta Da! No value";
List<AnnotateImageRequest> requests = new ArrayList<>();
ByteString imgBytes = ByteString.readFrom(new FileInputStream("G:\\files\\type1.jpeg"));
Image img = Image.newBuilder().setContent(imgBytes).build();
Feature feat = Feature.newBuilder().setType(Type.TEXT_DETECTION).build();
AnnotateImageRequest request =
AnnotateImageRequest.newBuilder()
.addFeatures(feat).setImage(img).build();
requests.add(request);
try (ImageAnnotatorClient client = ImageAnnotatorClient.create()) {
BatchAnnotateImagesResponse response = client.batchAnnotateImages(requests);
List<AnnotateImageResponse> responses = response.getResponsesList();
for (AnnotateImageResponse res : responses) {
if (res.hasError()) {
System.out.println("Error: %s\n"+ res.getError().getMessage());
return "";
}
// For full list of available annotations, see http://g.co/cloud/vision/docs
for (EntityAnnotation annotation : res.getTextAnnotationsList()) {
finalText += annotation.getDescription();
System.out.println("Text: %s\n" + annotation.getDescription());
System.out.println("Position : %s\n" + annotation.getBoundingPoly());
}
}
}
return finalText;
}
Note: Created service account and downloaded json, steps Create a service account
Note: I haven't used the GOOGLE_APPLICATION_CREDENTIALS variable. so deleted library com.google.api.client.googleapis.auth.oauth2.GoogleCredential.