I am using the AWS Java SDK and trying to run some tests; getting:
Unable to load AWS credentials from the /AwsCredentials.properties file on the classpath
The credentials file # ~/.aws/ is correct per AWS specs; 777'd it to ensure no access issues.
I am not using any IDE plug-ins; per AWS docs, having a credentials file # ~/.aws/ should suffice. Anyone have this working with just the SDK installed? If I hard-code the file path into the ClasspathPropertiesFileCredentialsProvider() request it spits the error back with the path instead of the AwsCredentials.properties string, which doesn't exist anywhere (yes, tried making one of those in ~/.aws/ as well).
Thanks much for any insights, code is below straight from Amazon:
import com.amazonaws.auth.ClasspathPropertiesFileCredentialsProvider;
import com.amazonaws.regions.Region;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.sns.AmazonSNSClient;
import com.amazonaws.services.sns.model.PublishRequest;
import com.amazonaws.services.sns.model.PublishResult;
public class SNS {
public static void main(String[] args) {
AmazonSNSClient snsClient = new AmazonSNSClient(new ClasspathPropertiesFileCredentialsProvider());
snsClient.setRegion(Region.getRegion(Regions.US_EAST_1));
String msg = "this is a test";
PublishRequest publishRequest = new PublishRequest("my arn", msg);
PublishResult publishResult = snsClient.publish(publishRequest);
System.out.println("MessageId - " + publishResult.getMessageId());
}
}
If you use DefaultAWSCredentialsProviderChain instead of ClasspathPropertiesFileCredentialsProvider, it will automatically check various default locations for AWS credentials. (Documentation)
Have you verified that your $HOME environment variable is set for the process you are running? The AWS SDK relies on $HOME to determine the proper location of your .aws folder.
Well that didn't work the way I'd planned it; couldn't get the .aws path as a classpath (tried adding as an external class folder).
Ran the below to find the actual classpaths in my project:
public static void main (String args[]) {
ClassLoader cl = ClassLoader.getSystemClassLoader();
URL[] urls = ((URLClassLoader)cl).getURLs();
for(URL url: urls){
System.out.println(url.getFile());
}
}
and then dropped my AWS credentials into a new AwsCredentials.properties file in one of the dirs from above (I had one; the rest were jar files).
Changed the tag values in the file to "accessKey" and "secretKey" from what was there (aws_access_key, aws_secret_access_key) and it worked.
Thanks to everyone for their inputs.
Related
How to set repository name and tag on DockerImageAsset of CDK?
This is my code: (groovy)
private uploadImage(String name, File directory, File jarFile) {
def asset = DockerImageAsset.Builder.create(scope, cdkId("$name-image"))
.directory(directory.toString())
.buildArgs([
"jarFile" : jarFile.name,
"environment": config.environment
])
.build()
println "ImageURL: $asset.imageUri"
}
This is the image url printed:
ImageURL: 9999999999999.dkr.ecr.us-west-2.${Token[AWS.URLSuffix.1]}/aws-cdk/assets:89ae89e0b3f7652a4ac70a4e18d6b2acec2abff96920e81e6487da58c8b820f3
I guess this is meant to be this way for CI/CD, where it doesn't matter the repository name/tag.
But if you need to use this image outside of CI/CD environment, it turns into a mess as we have many different projects and versions in the same repository (aws-cdk/assets).
I see a method called "repositoryName" but it is deprecated and I couldn't find an alternative or an explanation of why it is deprecated.
This isnt a direct answer to your question, as this solution uses NodeJS rather than Groovy, but it may help others that end up here.
You could check out the cdk-ecr-deployment library in the cdklabs repository, which gives a construct allowing you to deploy docker images to ECR.
This is done in three steps:
Create the repo
const repo = new ecr.Repository(this, 'NginxRepo', {
repositoryName: 'nginx',
removalPolicy: RemovalPolicy.DESTROY,
});
Create the image
const image = new DockerImageAsset(this, 'CDKDockerImage', {
directory: path.join(__dirname, 'docker'),
});
Tag and deploy the image to the repo
new ecrDeploy.ECRDeployment(this, 'DeployDockerImage', {
src: new ecrDeploy.DockerImageName(image.imageUri),
dest: new ecrDeploy.DockerImageName(`${repo.repositoryUri}:latest`),
});
See the example for more context on how this is used.
I couldn't find a solution create a polyglot source out of multiple files in GraalVM.
What exactly I want to achieve:
I have a python project:
my-project:
.venv/
...libs
__main__.py
src/
__init__.py
Service.py
Example sourcecode:
# __main__.py
from src.Service import Service
lambda url: Service(url)
# src/Service.py
import requests
class Service:
def __init__(self, url):
self.url = url
def invoke(self):
return requests.get(self.url)
This is very simple example, where we've got an entry-point script, project is structured in packages and there is one external library (requests).
It works, when I run it from command-line with python3 __main__.py, but I can't get it work, when embedding it in Java (it can't resolve imports).
Example usage in java:
import org.graalvm.polyglot.Context;
import org.graalvm.polyglot.Source;
import org.graalvm.polyglot.Value;
import java.io.File;
import java.io.IOException;
public class Runner {
public static void main(String[] args) throws IOException {
Context context = Context.newBuilder("python")
.allowExperimentalOptions(true)
.allowAllAccess(true)
.allowIO(true)
.build();
try (context) {
// load lambda reference:
Value reference = context.eval(Source.newBuilder("python", new File("/path/to/my-project/__main__.py")).build());
// invoke lambda with `url` argument (returns `Service` object)
Value service = reference.execute("http://google.com");
// invoke `invoke` method of `Service` object and print response
System.out.println("Response: " + service.getMember("invoke").execute());
}
}
}
It fails with Exception in thread "main" ModuleNotFoundError: No module named 'src'.
The solution works for javascript project (having similar index.js to __main__.py, its able to resolve imports - GraalVM "sees" other project's files, but somehow it doesn't, when using python.
I found out, that python is able to run zip package with project inside, but this also doesn't work with GraalVM.
Is there any chance to accomplish it? If not, maybe there is a similar tool to webpack for python (if I could create a single-file bundle, it should also work).
Btw, I don't know python at all, so I may missing something.
Thanks for any help!
I am building a java program that has the option to play YouTube videos in an embedded player.
The problem is that most of the music videos won't play and I get the following error:
"This video contains content from (Media Corporation Name). It is restricted from playback on certain sites."
I tried loading the same URL in Chrome and got the same results.
https://www.youtube.com/embed/TMZi25Pq3T8
However, after some research, I quickly got it fixed by installing a Chrome Extension that allows me to add HTTP Request Headers and added a Referer header that follows this structure "https://www..com" and got it working.
So I thought that must be it.
I added the following code in order to add request headers to my JavaFX WebView / WebEngine:
URI uri = URI.create("https://www.youtube.com/embed/TMZi25Pq3T8");
List<String> cookies = new ArrayList<>();
cookies.add("User-Agent=BDM/v0.92");
cookies.add("Referer=https://www.youtube.com");
Map<String, List<String>> headers = new LinkedHashMap<String, List<String>>();
headers.put("Set-Cookie", cookies);
try {
CookieHandler.getDefault().put(uri, headers);
} catch (IOException ex) {
ex.printStackTrace();
}
System.out.println(webView.getEngine().getUserAgent());
webView.getEngine().load(uri.toString());
Still, no success, the same error message.
The website that I'm using to extract data about releases through their API, Discogs, is able to play "restricted" videos as well. What am I missing here?
LATER EDIT:
Further clarifications:
I would like to apologize for the mistakes I made:
The line System.out.println(webView.getEngine().getUserAgent()); doesn't print "BDM/v0.92" as I first stated, it prints the default JavaFX user agent, "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/538.19 (KHTML, like Gecko) JavaFX/8.0 Safari/538.19". And this leads to number 2
As Roman Nazarenko pointed out, I was confusing cookies with request headers.
This leads to the real question, how can I send HTTP Request headers for JavaFX WebEngine? The only option is to set the user agent by calling webView.getEngine().setUserAgent("myUserAgent");
I found a hack here but this didin't work for me: https://twitter.com/codingfabian/status/524942996748652544
Thanks!
I managed to solve the issue by using javassist and this tutorial on how to instrument Java code.
As I stated in my question, the YouTube player needs a Referer header to play some videos (like music videos owned by VEVO, Sony Music Enternatinment, etc.).
What I did is I intercepted prepareConnection method from the URLLoader class that is used by JavaFX's WebEngine and inserted my instruction at the top of the method body:
c.setRequestProperty("Referer", "https://www.discogs.com");
(Again, please follow the tutorial for all the instructions)
(Note: Even though the tutorial above is explains very well the concepts, it doesn't really touch much on the role and structure of a MANIFEST.MF file, so please check this link for more info about this aspect)
These are my two classes:
MyJavaAgent.java
package com.busytrack.discographymanager.headerfixagent;
import java.lang.instrument.Instrumentation;
public class MyJavaAgent {
public static void premain(String agentArgument, Instrumentation instrumentation) {
ClassTransformer transformer = new ClassTransformer();
instrumentation.addTransformer(transformer);
}
}
ClassTransformer.java
package com.busytrack.discographymanager.headerfixagent;
import java.io.ByteArrayInputStream;
import java.lang.instrument.ClassFileTransformer;
import java.lang.instrument.IllegalClassFormatException;
import java.security.ProtectionDomain;
import javassist.ClassPool;
import javassist.CtClass;
import javassist.CtMethod;
public class ClassTransformer implements ClassFileTransformer {
public byte[] transform(ClassLoader loader, String className, Class<?> classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) throws IllegalClassFormatException {
byte[] byteCode = classfileBuffer;
if (className.equals("com/sun/webkit/network/URLLoader")) {
try {
ClassPool classPool = new ClassPool(true);
CtClass ctClass = classPool.makeClass(new ByteArrayInputStream(classfileBuffer));
CtMethod method = ctClass.getDeclaredMethod("prepareConnection");
String src = "$1.setRequestProperty(\"Referer\", \"https://www.discogs.com\");"; // Confused about there being "$1" instead of "c"? Please read below
method.insertBefore(src);
byteCode = ctClass.toBytecode();
ctClass.detach();
} catch (Exception e) {
e.printStackTrace();
}
}
return byteCode;
}
}
This is why I used "$1" to access the method parameter, instead of "c":
The statement and the block can refer to fields and methods. They can also refer to the parameters to the method that they are inserted into if that method was compiled with the -g option (to include a local variable attribute in the class file). Otherwise, they must access the method parameters through the special variables $0, $1, $2, ... described below. Accessing local variables declared in the method is not allowed although declaring a new local variable in the block is allowed.
The entire javassist tutorial can be found here.
After packing the two classes and the MANIFEST.MF file in a separate JAR, import it in your IDE (I used Eclipse) and add the following VM argument:
-javaagent:./(your-jar-name).jar
In Eclipse, you can add VM arguments like this:
right click on your project -> Run As -> Run Configurations... -> open the Arguments tab -> insert your VM argument -> Apply
I hope this helps someone out there. I know I spent a few days on this issue.
I don't know if this is the best approach but it does the job for me.
Still, it makes me wonder why isn't there a straightforward way of setting Request Headers for JavaFX's WebEngine...
Later edit:
I found a much cleaner and easier approach for loading Java Agents, dynamically, without the need to create a separate JAR, manifest file, importing them, passing the -javaagent VM parameter at startup, etc.
I used the ea-agent-loader (JAR download link).
Import the JAR in your IDE and change the MyJavaAgent class (the one that had the premain method) to this:
package com.busytrack.discographymanager.headerfixagent;
import java.lang.instrument.Instrumentation;
public class MyJavaAgent {
public static void agentmain(String agentArgument, Instrumentation instrumentation) {
ClassTransformer transformer = new ClassTransformer();
instrumentation.addTransformer(transformer);
}
}
My main method from the MainClass looks like this:
public static void main(String[] args) {
AgentLoader.loadAgentClass(MyJavaAgent.class.getName(), null); // Load the MyJavaAgent class
launch(args); // Start the JavaFX application
}
I wanted to be able to load the Agent dynamically because, using the static method required me to create separate launchers for all platforms and pass the -javaagent parameter on startup. Now, I can export a runnable JAR from eclipse like I usually do and the agent will load automatically (no VM parameters required). Thanks, BioWare for this tool! :D
I am learning magnolia cms. I am trying to use the resources module. I have actually 2 problems.
Cannot upload a bunch of files. I have a few files, but in some time I will have to upload some more. Modules import feature wants me to upload an xml file. But I don't know how to generate it properly. Tried to import through JCR, but after that I can't see those files in resources app. Tried to configure the module to search files in file system: I set fileSystemLoader to class info.magnolia.module.resources.loaders.FileSystemResourceLoader and set some path. It did not work for me too. Maybe I just don't understand at what time should be activated files upload feature. At the application start up time it did not work.
How to properly use these resources in my template? What ftl tag should I use?
I don't use STK module.
Thanks for your patience if you decide to help me.
Magnolia version: 5.2 CE
JDK iced tea: 1.7.0_51
OS: Linux/OpenSUSE 12.3
I've used previously (on 4.5.x) script below to perform the task via groovy module. It should work on 5.2 as well.
import static groovy.io.FileType.FILES
import info.magnolia.jcr.util.NodeUtil
import org.apache.commons.lang.StringUtils
import info.magnolia.cms.util.ContentUtil
class Globals {
static def folderName = '//some/folder/in/filesystem/on/server'
}
def loadImageFolder() {
session = ctx.getJCRSession("resources")
parentFolder = session.getNode("/templating-kit/jelinek-image/obrazky-produkty")
new File(Globals.folderName).eachFileRecurse(FILES) {
name = it.name
// set file name
extension = StringUtils.substringAfterLast(name, '.')
name = StringUtils.substringBeforeLast(name, '.')
// persist
resource = NodeUtil.createPath(parentFolder,name , "mgnl:content")
// persistResource
resource.setProperty("mgnl:template", "resources:binary")
resource.setProperty("extension", extension)
binary = resource.addNode("binary", "mgnl:resource")
binary.setProperty("jcr:data", new FileInputStream(it.absolutePath))
binary.setProperty("extension", extension)
binary.setProperty("fileName", name)
binary.setProperty("jcr:mimeType", "image/"+extension)
binary.setProperty("size", it.length())
}
session.save()
}
loadImageFolder()
return 'done'
I followed these steps in the hopes of getting storage emulator on localhost working.
I am using Windows 8 RTM.
Downloaded Eclipse and copied it to Program Files.
Installed Java JDK 7.
Installed Azure SDK.
Installed Azure plugin for Eclipse.
Launched storage emulator from the "Start" screen.
Created a Java project.
Added External jars in the build path for Azure to this project.
Wrote this simple sample code:
import com.microsoft.windowsazure.services.blob.client.CloudBlobClient;
import com.microsoft.windowsazure.services.blob.client.CloudBlobContainer;
import com.microsoft.windowsazure.services.core.storage.CloudStorageAccount;
public class AzureStore {
public static final String storageConnectionString = "DefaultEndpointsProtocol=http;"
+ "UseDevelopmentStorage=true;"
+ "AccountName=devstoreaccount1;"
+ "BlobEndpoint=http://127.0.0.1:10000;"
+ "AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==";
public static void main(String[] args) throws Exception {
// Retrieve storage account from connection-string
CloudStorageAccount storageAccount = CloudStorageAccount
.parse(storageConnectionString);
// Create the blob client
CloudBlobClient blobClient = storageAccount.createCloudBlobClient();
// Get a reference to a container
// The container name must be lower case
CloudBlobContainer container = blobClient
.getContainerReference("tweet");
try {
// Create the container if it does not exist
System.out.println(container.createIfNotExist());
} catch (Exception e) {
e.printStackTrace();
}
}
}
It gives the following exception:
com.microsoft.windowsazure.services.core.storage.StorageException: The value for one of the HTTP headers is not in the correct format.
at com.microsoft.windowsazure.services.core.storage.StorageException.translateException(StorageException.java:104)
at com.microsoft.windowsazure.services.blob.client.CloudBlobContainer$2.execute(CloudBlobContainer.java:334)
at com.microsoft.windowsazure.services.blob.client.CloudBlobContainer$2.execute(CloudBlobContainer.java:291)
at com.microsoft.windowsazure.services.core.storage.utils.implementation.ExecutionEngine.executeWithRetry(ExecutionEngine.java:110)
at com.microsoft.windowsazure.services.blob.client.CloudBlobContainer.createIfNotExist(CloudBlobContainer.java:339)
at com.microsoft.windowsazure.services.blob.client.CloudBlobContainer.createIfNotExist(CloudBlobContainer.java:257)
at AzureStore.main(AzureStore.java:26)
I am confused at this point, as what might be wrong. Can someone help me?
I think the error is happening because of incorrect storage service version in the API. In your code you're trying to create a blob container in development storage. The "x-ms-version" request header value is sent as "2012-02-12" which though is the latest one but still not supported by the development storage. Development storage still supports "2011-08-18".
If you try your code against cloud storage, you should be able to create that blob container.
If you're only doing your development against development storage, one thing you could do is download the source code from GitHub (https://github.com/WindowsAzure/azure-sdk-for-java/downloads) and modify the following line of code in Constants.java
public static final String TARGET_STORAGE_VERSION = "2012-02-12";
to
public static final String TARGET_STORAGE_VERSION = "2011-08-18";
and compile the source code again. This may break some new functionality introduced in the latest service release (like asynchronous copy blobs etc.)
Other alternative is to wait out for the new SDK to come out and hope that the emulator in that version support the latest storage service version.
More about URI class
See if below works for you.
URI BlobEndPoint = new URI("http://127.0.0.1:10000/devstoreaccount1");
CloudBlobClient bClient = new CloudBlobClient(BlobEndPoint, new StorageCredentialsAccountAndKey(AccountName,
AccountSecurityKey));