I am developing a Java Azure function that needs to download a file from Azure Datalake Gen2.
When the function tries to read the file, it freezes and no exception is thrown, and nothing is written to the console.
I am using the azure-storage-file-datalake SDK for Java dependency and this is my code:
import com.azure.storage.common.StorageSharedKeyCredential;
import com.azure.storage.file.datalake.DataLakeDirectoryClient;
import com.azure.storage.file.datalake.DataLakeFileClient;
import com.azure.storage.file.datalake.DataLakeFileSystemClient;
import com.azure.storage.file.datalake.DataLakeServiceClient;
import com.azure.storage.file.datalake.DataLakeServiceClientBuilder;
public DataLakeServiceClient GetDataLakeServiceClient(String accountName, String accountKey)
{
StorageSharedKeyCredential sharedKeyCredential = new StorageSharedKeyCredential(accountName, accountKey);
DataLakeServiceClientBuilder builder = new DataLakeServiceClientBuilder();
builder.endpoint("https://" + accountName + ".dfs.core.windows.net");
builder.credential(sharedKeyCredential);
return builder.buildClient();
}
public void DownloadFile(DataLakeFileSystemClient fileSystemClient, String fileName) throws Exception{
DataLakeDirectoryClient directoryClient = fileSystemClient.getDirectoryClient("DIR");
DataLakeDirectoryClient subdirClient= directoryClient.getSubdirectoryClient("SUBDIR");
DataLakeFileClient fileClient = subdirClient.getFileClient(fileName);
File file = new File("downloadedFile.txt");
OutputStream targetStream = new FileOutputStream(file);
fileClient.read(targetStream);
targetStream.close();
}
#FunctionName("func")
public HttpResponseMessage run(
#HttpTrigger(name = "req", methods = {HttpMethod.GET}, authLevel = AuthorizationLevel.ANONYMOUS)
HttpRequestMessage<Optional<String>> request,
final ExecutionContext context
)
{
String fileName= request.getQueryParameters().get("file");
DataLakeServiceClient datalakeClient= GetDataLakeServiceClient("datalake", "<the shared key>");
DataLakeFileSystemClient datalakeFsClient= datalakeClient.getFileSystemClient("fs");
DownloadFile(datalakeFsClient, fileName);
}
The app freezes when it hits fileClient.read(targetStream);
I've tried with really small files, I've checked the credentials and the file paths, the access rights to datalake, I've switched to SAS token - the result is the same: no error at all, but the app freezes.
I am using these Maven dependencies:
<dependency>
<groupId>com.microsoft.azure.functions</groupId>
<artifactId>azure-functions-java-library</artifactId>
</dependency>
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-storage-file-datalake</artifactId>
<version>12.2.0</version>
</dependency>
so i was facing the same problem.Then i came across this.
https://github.com/Azure/azure-functions-java-library/issues/113
This worked for me on java 8,azure function v3.
Set FUNCTIONS_WORKER_JAVA_LOAD_APP_LIBS to True
in the function app Application settings.Then save and restart the function app.It will work.
Please check and do update if it worked for you as well.
Related
I want to access a database via the data-api which is AWS providing since the start of 2020.
This is my Maven code (only aws dependency shown):
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk</artifactId>
<version>1.11.790</version>
</dependency>
<dependency>
<groupId>software.amazon.rdsdata</groupId>
<artifactId>rds-data-api-client-library-java</artifactId>
<version>1.0.4</version>
</dependency>
This is my Java code
public class Opstarten {
public static final String RESOURCE_ARN = "arn:aws:rds:eu-central <number - name >";
public static final String SECRET_ARN = "arn:aws:secretsmanager:eu-central-1:<secret>";
public static final String DATABASE = "dbmulesoft";
public static void main(String[] args) {
// TODO Auto-generated method stub
new Opstarten().testme();
}
public void testme( ) {
var account1 = new Account(1, "John"); //plain POJO conform AWS manual hello world example
var account2 = new Account(2, "Mary");
RdsDataClient client = RdsDataClient.builder().database(DATABASE)
.resourceArn(RESOURCE_ARN)
.secretArn(SECRET_ARN).build();
client.forSql("INSERT INTO accounts(accountId, name) VALUES(:accountId, :name)").
withParameter(account1).withParameter(account2).execute();
}
}
Error I am having:
Exception in thread "main" java.lang.NullPointerException
at com.amazon.rdsdata.client.RdsDataClient.executeStatement(RdsDataClient.java:134)
at com.amazon.rdsdata.client.Executor.executeAsSingle(Executor.java:92)
at com.amazon.rdsdata.client.Executor.execute(Executor.java:77)
at nl.bpittens.aws.rds.worker.Opstarten.testme(Opstarten.java:47)
at nl.bpittens.aws.rds.worker.Opstarten.main(Opstarten.java:29)
When I debug it I see that the client object is nog null but the rdsDataService is null as a method or parameter of the client object.
I have checked AWS side for Java RDS Data API but nothing is mentioned there.
Any idea whats wrong ?
Looks like you aren't passing RDS data service, you need to do as follows:
AWSRDSData awsrdsData = AWSRDSDataClient.builder().build();
RdsDataClient client = RdsDataClient.builder()
.rdsDataService(awsrdsData)
.database(DATABASE)
.resourceArn(RESOURCE_ARN)
.secretArn(SECRET_ARN)
.build();
You can also configure mapping options as follows:
MappingOptions mappingOptions = MappingOptions.builder()
.ignoreMissingSetters(true)
.useLabelForMapping(true)
.build();
AWSRDSData awsrdsData = AWSRDSDataClient.builder().build();
RdsDataClient client = RdsDataClient.builder()
.rdsDataService(awsrdsData)
.database(DATABASE)
.resourceArn(RESOURCE_ARN)
.secretArn(SECRET_ARN)
.mappingOptions(mappingOptions)
.build();
I have a requirement of uploading images to cloudinary using cloudinary's java API.I have created cloudinary account through which i got api Cloud name,API Key,API Secret. Using these things i am able to upload my images to cloudinary like bellow:
#PostMapping("/uploadPic")
public ResponseEntity<Object> upload(#RequestParam("file") MultipartFile multipartFile){
String cloudinaryImgURL=null;
try {
File fileDir = new File("rowFiles");
if (! fileDir.exists()){
fileDir.mkdir();
}
String fileName=multipartFile.getOriginalFilename();
File physicalFile=new File(multipartFile.getOriginalFilename());
FileOutputStream fout=new FileOutputStream(fileDir.getName()+"/"+physicalFile);
fout.write(multipartFile.getBytes());
fout.close();
//For stack-overflow question using dummy values for credientials.
Cloudinary cloudinary = new Cloudinary(ObjectUtils.asMap(
"cloud_name", "your_cloud_name",
"api_key", "your_api_key",
"api_secret", "your_secret_key"));
File toUpload = new File("rowFiles/"+fileName);
Map params = ObjectUtils.asMap("public_id", "SRWRestImageBase/"+fileName);
Map uploadResult =Singleton.getCloudinary().uploader().upload(toUpload, params);
toUpload.delete();
System.out.println("==============>>"+uploadResult.get("url"));
cloudinaryImgURL=uploadResult.get("url").toString();
} catch (Exception e) {
System.out.println("upload:"+e.getMessage());
// TODO: handle exception
}
return new ResponseEntity<Object>("File uploaded successfully:"+cloudinaryImgURL,HttpStatus.OK);
}
But now my problem is that i keep this code in public git repo, and from there i push this code to Heroku. But using this method will expose my cloudinary details like Cloud name,API Key,API Secret to everyone which i don't want.
Looking at cloudinary's documentation for getting started i found the way of using environment variable to store these values and access it from there, but document doesn't guide me properly.
I have used
In a Java EE environment you can set an environment variable available to your Java EE container:
CLOUDINARY_URL=cloudinary://{api_key}:{api_secret}#{cloud_name}
This will enable you to receive a Cloudinary instance:
Cloudinary cloudinary = Singleton.getCloudinary();
But i am getting compile-time error for class Singleton that it is not able to find in any jar. Looking at some other article, this class should com.cloudinary.Singleton, but my cloudinary jar does not have this class.
I am using cloudinary gradle dependancy as:
// https://mvnrepository.com/artifact/com.cloudinary/cloudinary-http43
compile group: 'com.cloudinary', name: 'cloudinary-http43', version: '1.2.2'
It would be appreciated if someone can guide me in right direction.
Thanks in Advance.
I have found my own solution.
We have to set environment variable CLOUDINARY_URL=cloudinary://{api_key}:{api_secret}#{cloud_name} and restart eclipse to reflect the environment variable changes and following changes. This way it wont expose my account details to public repo.
#PostMapping("/uploadPic")
public ResponseEntity<Object> upload(#RequestParam("file") MultipartFile multipartFile){
String cloudinaryImgURL=null;
try {
File fileDir = new File("rowFiles");
if (! fileDir.exists()){
fileDir.mkdir();
}
String fileName=multipartFile.getOriginalFilename();
File physicalFile=new File(multipartFile.getOriginalFilename());
FileOutputStream fout=new FileOutputStream(fileDir.getName()+"/"+physicalFile);
fout.write(multipartFile.getBytes());
fout.close();
File toUpload = new File("rowFiles/"+fileName);
Cloudinary cloudinary = new Cloudinary();
System.out.println("API Key:"+cloudinary.config.apiKey);
Map params = ObjectUtils.asMap("public_id", "SRWRestImageBase/"+fileName);
Map uploadResult = cloudinary.uploader().upload(toUpload, params);
toUpload.delete();
System.out.println("==============>>"+uploadResult.get("url"));
cloudinaryImgURL=uploadResult.get("url").toString();
} catch (Exception e) {
System.out.println("upload:"+e.getMessage());
}
return new ResponseEntity<Object>("File uploaded successfully:"+cloudinaryImgURL,HttpStatus.OK);
}
I was creating a Java web service server, using eclipse IDE. that server is the following.
Note: I am working in UBUNTU
package com.tesis.service;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.RejectedExecutionException;
import com.mathworks.engine.*;
/**
* #author root
*
*/
public class CNNPredict
{
public String cNNPredict(int[] Image, int Height, int Width) throws Exception
{
String FilePath = "/home/user/Documents/MATLAB/Project";
char[] CharFilePath = FilePath.toCharArray();
MatlabEngine eng = MatlabEngine.startMatlab();
eng.feval("cd", CharFilePath);
String result = eng.feval("CNNPredict",Image,Height,Width);
return result;
}
}
As you can see I am using MATLAB engine.
Matlab engine documentation. I checked that cNNPredict method is working properly by copying it into a new Java project and It worked perfectly.
I added the .jar files required to run java engine to the Dynamic web project where the web service is located.
Apparently this web service runs without problems Web Service working in local host
If I click on "CnnPredict" link I get the wsdl direction of the class , this direction is what I use to link the client with the server.
this is the client code:
public static void main(String[] args) throws IOException, CNNPredictExceptionException
{
CNNPredictStub stub = new CNNPredictStub();
CNNPredict cnn = new CNNPredict();
BufferedImage img = null;
System.out.println("Reading image ...");
img = ImageIO.read(new File("/home/riosgamarra/Documents/MATLAB/TesisGamarrarios/101_ObjectCategories/laptop/image_0009.jpg"));
int[] UnrolledImage = convertToGray(img);
cnn.setImage(UnrolledImage);
cnn.setWidth(img.getWidth());
cnn.setHeight(img.getHeight());
System.out.println(stub.cNNPredict(cnn).get_return());
}
It has no errors, but when I run it this error message shows up:
Exception in thread "main" org.apache.axis2.AxisFault: <faultstring>com/mathworks/engine/MatlabEngine</faultstring>
at org.apache.axis2.util.Utils.getInboundFaultFromMessageContext(Utils.java:513)
at org.apache.axis2.description.OutInAxisOperationClient.handleResponse(OutInAxisOperation.java:368)
at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:414)
at org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
at org.apache.axis2.client.OperationClient.execute(OperationClient.java:150)
at com.tesis.service.CNNPredictStub.cNNPredict(CNNPredictStub.java:197)
at com.tesis.client.CallWS.main(CallWS.java:40)
what I am missing ? do I need to add any special permissions to the server project ? What Am I missing ?
Note: I run the client clicking on the class and selecting Run as > Java application.
at com.tesis.service.CNNPredictStub.cNNPredict(CNNPredictStub.java:197)
is where the exception is but
public class CNNPredict
{
public String cNNPredict(int[] Image, int Height, int Width) throws Exception
{
String FilePath = "/home/user/Documents/MATLAB/Project";
char[] CharFilePath = FilePath.toCharArray();
MatlabEngine eng = MatlabEngine.startMatlab();
eng.feval("cd", CharFilePath);
String result = eng.feval("CNNPredict",Image,Height,Width);
return result;
}
}
is not the stub. First we need to the right code to look at. The matlab api is straight forward. My guess is that the stub is making the wrong call
I am developing a code for xcel generation and download using apache poi. LocalHost server and app server is jboss. When i run the code on localhost, a temp folder is generated in jboss's deployment folder and in that the xcel is generated and then downloaded through frontend. I am using java spring angularjs and html. This runs fine on localhost but after deploying on app server the xcel is not downloaded and it gives 500:internal server error.
angularjs controller code:
$scope.generateExcel=function(sDate,eDate,doc,search)
{
console.log("hello");
var sDate = document.getElementById('sD').value
var eDate = document.getElementById('eD').value
$scope.obj.sDate = sDate;
$scope.obj.eDate = eDate;
$scope.obj.iou = doc;
$scope.obj.du = search;
console.log($scope.obj);
$http.post('abc/generateExcel',$scope.obj).then(function()
{
//console.log(path);
$window.location.href="/ProjectName/file_name.xls";
})
.error(function()
{
console.log("Error!!");
});
};
java code:
//Method
public HttpServletResponse generateExcel ( HttpServletRequest request , HttpServletResponse response, String sD, String eD, String doc, String search)
{
//EXCEL GENERATION HERE
response.setContentType("application/vnd.ms-excel");
response.setHeader("Content-Disposition", "attachment;filename=filename.xls");
//Path Specification
String path = request.getRealPath("/file_name.xls");
//System.out.println("Here...");
System.out.println(path);
FileOutputStream fileOut2 = new FileOutputStream(path);
workbook.write(fileOut2);
/*returning response*/
}
It's difficult to answer this unless one knows what's the error you are getting on the server side. Put your server code in a try-catch block. Rerun the code, and check the server logs. Paste them here.
try{
String path = request.getRealPath("/file_name.xls");
//System.out.println("Here...");
System.out.println(path);
FileOutputStream fileOut2 = new FileOutputStream(path);
workbook.write(fileOut2);
} catch(Exception e){
e.printStackTrace(); // this should print some error in server logs
}
I have a problem with decrypting a PDF document with Apache PdfBox (v1.8.2) lib. Encryption works, but decryption with the same password throws an exception. (Java 1.6)
package com.test;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.encryption.AccessPermission;
import org.apache.pdfbox.pdmodel.encryption.StandardDecryptionMaterial;
import org.apache.pdfbox.pdmodel.encryption.StandardProtectionPolicy;
public class PdfEncDecTest {
static String pdfPath = "G:\\files\\filed5b3.pdf";
public final static String PDF_OWNER_PASSWORD = "cd1j";
public final static String PDF_USER_PASSWORD = "";
public static void main(String[] args) throws Exception {
PDDocument document = PDDocument.load(pdfPath);
AccessPermission ap = new AccessPermission();
ap.setCanPrint(true);
ap.setCanExtractContent(false);
ap.setCanExtractForAccessibility(false);
StandardProtectionPolicy spp = new StandardProtectionPolicy(PDF_OWNER_PASSWORD, PDF_USER_PASSWORD, ap);
document.protect(spp);
document.save(pdfPath+".pdf");
document.close();
PDDocument doc = PDDocument.load(pdfPath+".pdf");
if(doc.isEncrypted()) {
StandardDecryptionMaterial sdm = new StandardDecryptionMaterial(PDF_OWNER_PASSWORD);
doc.openProtection(sdm); // org.apache.pdfbox.exceptions.CryptographyException: Error: The supplied password does not match either the owner or user password in the document.
doc.decrypt(PDF_OWNER_PASSWORD); // the same like above
}
doc.close();
}
}
I don't know what is wrong. With version 1.8.7 I get the same exception. I've posted the full code above.
Exception in thread "main" org.apache.pdfbox.exceptions.CryptographyException: Error: The supplied password does not match either the owner or user password in the document.
at org.apache.pdfbox.pdmodel.encryption.StandardSecurityHandler.prepareForDecryption(StandardSecurityHandler.java:265)
at org.apache.pdfbox.pdmodel.encryption.StandardSecurityHandler.decryptDocument(StandardSecurityHandler.java:156)
at org.apache.pdfbox.pdmodel.PDDocument.openProtection(PDDocument.java:1595)
at org.apache.pdfbox.pdmodel.PDDocument.decrypt(PDDocument.java:942)
at com.test.PdfEncDecTest.main(PdfEncDecTest.java:29)
I've put sample project to github: https://github.com/marioosh-net/pdfbox
You need the user password.
if (doc.isEncrypted())
{
StandardDecryptionMaterial sdm = new StandardDecryptionMaterial(PDF_USER_PASSWORD);
doc.openProtection(sdm);
// don't call decrypt() here
}
this works even if the user password is not null. The user password is for what the ordinary human thinks encryption is, the owner password is an encryption for the security rights.
edit: sorry, my answer is wrong, although it was helpful. You can open a PDF with the user password (you'll possibly get restricted rights) or with the owner password (you'll get full rights). What may have happened is that there is a bug with matching the owner password with 40bit keys (which is the
default). This bug is currently being investigated, see PDFBOX-2456 and search for "MD5".
I have tested your code and it work's fine for me.
i am using
<dependency>
<groupId>org.apache.pdfbox</groupId>
<artifactId>pdfbox</artifactId>
<version>1.8.7</version>
</dependency>