Develop Reference

Switch between AWS, Azure. Design pattern

We have developed some lambda function and deployed on AWS which are working fine,
Anyhow, client is now planning for AZURE.
They may even switch back to AWS or any other vendor in future.
We have a separate maven project for AWS related stuff.
Hence, our business logic and classes remains same.
What I have done is created a maven project and added individual lambda functions to this project as dependencies.
Then made a factory class which will get impl based on property AZURE or AWS(using class.forName and reflection).
SO, I can switch to Azure by just removing maven dependency and adding AZURE dependency.
According to picture my plan was to create new AzureUtils and AzureWrapper project and Directly use Azure Cloud, by switching cloud in cloudFactory which is present in Generic utils and that would even work hopefully (Not tested) AWS is working anyhow like that.
Now the problem is client does not want everything packed up in 1 jar, i.e no no to all lambdas in a single jar. He want some layer where the switching should take place.
Now Which design patter would be useful, what would be the approach.
Currently my Lambda function looks like below
public class Hello implements RequestHandler<S3Event, Context > {
public String handleRequest(S3Event s3event, Context context) {
.................
call to business processor as in diag
}
}
And azure function looks somewhat like a simple class with annotations
public class Function {
#FunctionName("hello")
public HttpResponseMessage run(
#HttpTrigger(name = "req", methods = { HttpMethod.GET, HttpMethod.POST }, authLevel = AuthorizationLevel.ANONYMOUS) HttpRequestMessage<Optional<String>> request,
final ExecutionContext context) {
context.getLogger().info("Java HTTP trigger processed a request.");
// Parse query parameter
String query = request.getQueryParameters().get("name");
String name = request.getBody().orElse(query);
if (name != null) {
call to business processor as in diagram
}
}
}
After all this I have only 2 questions
I would like to know first if the design in diagram is right thing to do.
And what my client is asking for a wrapper something magical which should handle both type of cloud implementations. is this even possible?
if possible guide me in right direction
Any help is greatly appreciated.

about you secound question how to handle both type of cloud, please check this 3rd part solution serverless.com. It's a company that create own serverless wrapper, so that you can be free of vendor lock

How do I hide OSGi services so that other bundles can't find them?

I am developing an application that is build on top of Apache Felix and JavaFX. The application can be extended by 3rd party bundles that implement a specific Interface and make it available to the OSGi Runtime Service Registry.
The problem is that those bundles (or plugins) should not be able to retrieve any of the services that are just used internally by my application. An example would be a PersistenceService that is used to save the processed data. Plugins are (in my application) by definition not allowed to store any data through my service but are allowed to save them through a specific service designed for the plugins only.
I had the idea of using the FindHook Interface offered by OSGi to filter out those requests but that didn't work good. Obviously, to make it work, the bundle needs to me loaded at the very start, eve before my core application gets loaded. I ensured this happens by specifying the start level for this bundle using the felix.auto.deploy.install.1 = "file\:bundles/de/zerotask/voices-findhook/0.1-SNAPSHOT/voices-findhook-0.1-SNAPSHOT.jar"
As far as I understood, the start level of the system bundle will be 1 which means that my bundle should always be loaded right after the system bundle.
Here is my implementation of the FindHook interface:
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.Arrays;
import java.util.Collection;
import java.util.HashSet;
import java.util.Set;
import org.osgi.framework.BundleContext;
import org.osgi.framework.ServiceReference;
import org.osgi.framework.hooks.service.FindHook;
/**
*
* #author PositiveDown
*/
public class VoicesFindHook implements FindHook {
private static Logger log = LoggerFactory.getLogger(VoicesFindHook.class);
private static final String[] INTERNAL_BUNDLE_TABLE = new String[]{
"de.zerotask.voices-core-actions",
"de.zerotask.voices-findhook",
"de.zerotask.voices-interfaces-persistable",
"de.zerotask.voices-models",
"de.zerotask.voices-models-actions",
"de.zerotask.voices-services-configuration-internal",
"de.zerotask.voices-services-input-internal",
"de.zerotask.voices-services-licenses-internal",
"de.zerotask.voices-services-modelsmanager-internal",
"de.zerotask.voices-services-persistence-internal",
"de.zerotask.voices-services-window-internal",
"de.zerotask.voices-ui-dialogs-about",
"de.zerotask.voices-ui-dialogs-newprofile",
"de.zerotask.voices-ui-dockable-listview",
"de.zerotask.voices-ui-dockable-properties",
"de.zerotask.voices-ui-layout",
"de.zerotask.voices-utils-io",
"de.zerotask.voices-utils-services",
"de.zerotask.voices-utils-ui"
};
private static final String[] INTERNAL_SERVICES_TABLE = new String[]{
// model services
// configuration service
"de.zerotask.voices.services.configuration.IConfiguration",
// window service
"de.zerotask.voices.services.window.IWindowService",
// persistence services
"de.zerotask.voices.services.persistence.IPathResolver",
"de.zerotask.voices.services.persistence.IPersistenceService"
};
private static final Set<String> INTERNAL_BUNDLES = new HashSet<>(Arrays.asList(INTERNAL_BUNDLE_TABLE));
private static final Set<String> INTERNAL_SERVICES = new HashSet<>(Arrays.asList(INTERNAL_SERVICES_TABLE));
#Override
public void find(BundleContext context, String name, String filter, boolean allServices, Collection<ServiceReference<?>> references) {
// only allow the usage of internal interfaces from internal packages
String symbolicName = context.getBundle().getSymbolicName();
// debug
log.debug("Processing Bundle {} and service {}", symbolicName, name);
// if the service is one of the internal ones, proceed
if (INTERNAL_SERVICES.contains(name)) {
// retrieve the bundle id
log.debug("Service {} is in internal table", name);
// if the name is not in the internal bundle table, remove all service references
if (!INTERNAL_BUNDLES.contains(symbolicName)) {
log.debug("Bundle {} not in internal table => removing service references...", symbolicName);
// remove them
references.clear();
}
}
}
}
The idea is to have a table of internal bundles``` andinternal services```. Each time a service is looked up, the hook will check if it is an internal service. If this is the case, it will also check if the caller bundle is an internal bundle. If that's not true, the hook will remove all services found from the collection.
I am by far no OSGi expert but this method should work because it is based on the SymbolicNames which are unique in each container.
I have tested the above code with two small test bundles. One providing the interface + implementation and the other one consuming it. I changed the hook so it will not return any services for the consumer bundle (to just simply check if it works).
No my problem is, the consumer bundle gets somehow loaded first. I have no idea why. By doing this it basically breaks my loading property set in the properties file.
I am not sure if this helps but the provider bundle's name starts with an 'y', the consumer one with an 't' and the hook one with an 'v'.
The funny thing is, Felix is loading them in alphabetically order.
I would really appreciate any help here.

Services are implicitly available to every bundle – that is the purpose of services after all.
You can work around this with various hacks like FindHooks etc, but as you have already discovered you are constantly fighting against the true nature of the OSGi Framework and services.
It sounds more like you are creating an isolation system between a kernel and a user space, so that you cannot accidentally pollute the user area with kernel services and vice versa. The proper way (IMHO) to achieve this is with a separate OSGi Framework instance for the two areas. It's quite simple to run up a new Framework using the FrameworkFactory API. Then you can expose select packages and services from the kernel using the BundleContext of the system bundle of the user-area Framework.
However as BJ points out in comments, you may be over-engineering this. What's the worst that can happen if the plugins can see your system services? If those services are well designed then the answer should be "not a lot".

I see two options:
ServicePermission, this is the standard way;
or
ServiceFactory, you decide what bundle can get the real service. Others receive a fake implementation.

Configuring DropWizard Programmatically

I have essentially the same question as here but am hoping to get a less vague, more informative answer.
I'm looking for a way to configure DropWizard programmatically, or at the very least, to be able to tweak configs at runtime. Specifically I have a use case where I'd like to configure metrics in the YAML file to be published with a frequency of, say, 2 minutes. This would be the "normal" default. However, under certain circumstances, I may want to speed that up to, say, every 10 seconds, and then throttle it back to the normal/default.
How can I do this, and not just for the metrics.frequency property, but for any config that might be present inside the YAML config file?

Dropwizard reads the YAML config file and configures all the components only once on startup. Neither the YAML file nor the Configuration object is used ever again. That means there is no direct way to configure on run-time.
It also doesn't provide special interfaces/delegates where you can manipulate the components. However, you can access the objects of the components (usually; if not you can always send a pull request) and configure them manually as you see fit. You may need to read the source code a bit but it's usually easy to navigate.
In the case of metrics.frequency you can see that MetricsFactory class creates ScheduledReporterManager objects per metric type using the frequency setting and doesn't look like you can change them on runtime. But you can probably work around it somehow or even better, modify the code and send a Pull Request to dropwizard community.

Although this feature isn't supported out of the box by dropwizard, you're able to accomplish this fairly easy with the tools they give you. Note that the below solution definitely works on config values you've provided, but it may not work for built in configuration values.
Also note that this doesn't persist the updated config values to the config.yml. However, this would be easy enough to implement yourself simply by writing to the config file from the application. If anyone would like to write this implementation feel free to open a PR on the example project I've linked below.
Code
Start off with a minimal config:
config.yml
myConfigValue: "hello"
And it's corresponding configuration file:
ExampleConfiguration.java
public class ExampleConfiguration extends Configuration {
private String myConfigValue;
public String getMyConfigValue() {
return myConfigValue;
}
public void setMyConfigValue(String value) {
myConfigValue = value;
}
}
Then create a task which updates the config:
UpdateConfigTask.java
public class UpdateConfigTask extends Task {
ExampleConfiguration config;
public UpdateConfigTask(ExampleConfiguration config) {
super("updateconfig");
this.config = config;
}
#Override
public void execute(Map<String, List<String>> parameters, PrintWriter output) {
config.setMyConfigValue("goodbye");
}
}
Also for demonstration purposes, create a resource which allows you to get the config value:
ConfigResource.java
#Path("/config")
public class ConfigResource {
private final ExampleConfiguration config;
public ConfigResource(ExampleConfiguration config) {
this.config = config;
}
#GET
public Response handleGet() {
return Response.ok().entity(config.getMyConfigValue()).build();
}
}
Finally wire everything up in your application:
ExampleApplication.java (exerpt)
environment.jersey().register(new ConfigResource(configuration));
environment.admin().addTask(new UpdateConfigTask(configuration));
Usage
Start up the application then run:
$ curl 'http://localhost:8080/config'
hello
$ curl -X POST 'http://localhost:8081/tasks/updateconfig'
$ curl 'http://localhost:8080/config'
goodbye
How it works
This works simply by passing the same reference to the constructor of ConfigResource.java and UpdateConfigTask.java. If you aren't familiar with the concept see here:
Is Java "pass-by-reference" or "pass-by-value"?
The linked classes above are to a project I've created which demonstrates this as a complete solution. Here's a link to the project:
scottg489/dropwizard-runtime-config-example
Footnote: I haven't verified this works with the built in configuration. However, the dropwizard Configuration class which you need to extend for your own configuration does have various "setters" for internal configuration, but it may not be safe to update those outside of run().
Disclaimer: The project I've linked here was created by me.

I solved this with bytecode manipulation via Javassist
In my case, I wanted to change the "influx" reporter
and modifyInfluxDbReporterFactory should be ran BEFORE dropwizard starts
private static void modifyInfluxDbReporterFactory() throws Exception {
ClassPool cp = ClassPool.getDefault();
CtClass cc = cp.get("com.izettle.metrics.dw.InfluxDbReporterFactory"); // do NOT use InfluxDbReporterFactory.class.getName() as this will force the class into the classloader
CtMethod m = cc.getDeclaredMethod("setTags");
m.insertAfter(
"if (tags.get(\"cloud\") != null) tags.put(\"cloud_host\", tags.get(\"cloud\") + \"_\" + host);tags.put(\"app\", \"sam\");");
cc.toClass();
}

How to set up a configured embedder for use of meta filters (-skip) with Serenity, JBehave and Selenium

While creating new scenarios I only want to test the scenario I am currently working with. For this purpose I want to use the Meta: #skip tag before my scenarios. As I found out I have to use the embedder to configure the used meta tags, so I tried:
configuredEmbedder().useMetaFilters(Arrays.asList("-skip"));
but actually this still has no effect on my test scenarios. I used it in the constructor of my SerenityStories test suite definition. Here is the complete code of this class:
public class AcceptanceTestSuite extends SerenityStories {
#Managed
WebDriver driver;
public AcceptanceTestSuite() {
System.setProperty("webdriver.chrome.driver", "D:/files/chromedriver/chromedriver.exe");
System.setProperty("chrome.switches", "--lang=en");
System.setProperty("restart.browser.each.scenario", "true");
configuredEmbedder().useMetaFilters(Arrays.asList("-skip"));
runSerenity().withDriver("chrome");
}
#Override
public Configuration configuration() {
Configuration configuration = super.configuration();
Keywords keywords = new LocalizedKeywords(DEFAULTSTORYLANGUAGE);
Properties properties = configuration.storyReporterBuilder().viewResources();
properties.setProperty("encoding", "UTF-8");
configuration.useKeywords(keywords)
.useStoryParser(new RegexStoryParser(keywords, new ExamplesTableFactory(new LoadFromClasspath(this.getClass()))))
.useStoryLoader(new UTF8StoryLoader()).useStepCollector(new MarkUnmatchedStepsAsPending(keywords))
.useDefaultStoryReporter(new ConsoleOutput(keywords)).storyReporterBuilder().withKeywords(keywords).withViewResources(properties);
return configuration;
}
}
Is this the wrong place or have I missed something? Still all scenarios are executed.
EDIT:
I changed following classes and now I think that it "works"
public AcceptanceTestSuite() {
System.setProperty("webdriver.chrome.driver", "D:/files/chromedriver/chromedriver.exe");
System.setProperty("chrome.switches", "--lang=de");
System.setProperty("restart.browser.each.scenario", "true");
this.useEmbedder(configuredEmbedder());
runSerenity().withDriver("chrome");
}
#Override
public Embedder configuredEmbedder() {
final Embedder embedder = new Embedder();
embedder.embedderControls()
.useThreads(1)
.doGenerateViewAfterStories(true)
.doIgnoreFailureInStories(false)
.doIgnoreFailureInView(false)
.doVerboseFailures(true);
final Configuration configuration = configuration();
embedder.useConfiguration(configuration);
embedder.useStepsFactory(stepsFactory());
embedder.useMetaFilters(Arrays.asList("-skip"));
return embedder;
}
But now I get the message [pool-1-thread-1] INFO net.serenitybdd.core.Serenity - TEST IGNORED but the scenario is still executed. Only in the result page I get the info that this scenario is ignored (but still executed). Is there a way to SKIP the scenario so it won't run?

I could not make it run with using configuredEmbedder() but by adding -Dmetafilter="+working -finished" as goals in my mvn run configurations and using the tags #working for scenarios I'm working with and which I want to run and #finsihed for scenarios I don't want to execute. Still I have to change the run configuration if I want to change the meta tags so it is not very comfortable but still I get what I was looking for.

As long as you document it well (some doc in https://github.com/serenity-bdd/the-serenity-book would be brilliant), I think as a JBehave/Serenity user you are well enough placed to decide which option makes the most sense.

Investigation
I debugged the serenity-jbehave classes, trying to understand why setting
configuredEmbedder().useMetaFilters(Collections.singletonList("-skip"))
is not working in all the possible places I put it within my class extending the SerenityStories, I found the strategic code place where metaFilters in ExtendedEmbedder#embedder are overwritten with what we define in our class into settings from serenity-jbehave.
This method is SerenityReportingRunner#createPerformableTree:
private PerformableTree createPerformableTree(List<CandidateSteps> candidateSteps, List<String> storyPaths) {
ExtendedEmbedder configuredEmbedder = this.getConfiguredEmbedder();
configuredEmbedder.useMetaFilters(getMetaFilters());
BatchFailures failures = new BatchFailures(configuredEmbedder.embedderControls().verboseFailures());
PerformableTree performableTree = configuredEmbedder.performableTree();
RunContext context = performableTree.newRunContext(getConfiguration(), candidateSteps,
configuredEmbedder.embedderMonitor(), configuredEmbedder.metaFilter(), failures);
performableTree.addStories(context, configuredEmbedder.storyManager().storiesOfPaths(storyPaths));
return performableTree;
}
This line changes the set metaFilters:
configuredEmbedder.useMetaFilters(getMetaFilters());
It overrides the current metaFilters value.
Going further the call chain, we get to the logic that defines from where it gets metaFilters, i.e. where we can actually set it.
SerenityReportingRunner#createPerformableTree
↓
SerenityReportingRunner#getMetaFilters
↓
SerenityReportingRunner#getMetafilterSetting
This is the method we need!
private String getMetafilterSetting() {
Optional<String> environmentMetafilters = getEnvironmentMetafilters();
Optional<String> annotatedMetafilters = getAnnotatedMetafilters(testClass);
Optional<String> thucAnnotatedMetafilters = getThucAnnotatedMetafilters(testClass);
return environmentMetafilters.orElse(annotatedMetafilters.orElse(thucAnnotatedMetafilters.orElse("")));
}
As we see here, the metaFilters can be defined in three places, and they override each other. In the priority lowering order, they are:
Value of metafilter (exactly all lowercase!) VM property.
Value of on net.serenitybdd.jbehave.annotations.Metafilter annotation on our SerenityStories class.
Value of on net.thucydides.jbehave.annotations.Metafilter annotation on our SerenityStories class. This annotation is deprecated, but left in place for backwards-compatibility.
Solution that is working with the current serenity-jbehave version
I've tried/debugged all these three options, they work and override each other as described above.
1. Use environment metafilter property
Added this to my JVM run arguments:
-Dmetafilter=skip
2. Use the modern #Metafilter annotation
import net.serenitybdd.jbehave.SerenityStories;
import net.serenitybdd.jbehave.annotations.Metafilter;
#Metafilter("-skip")
public class Acceptance extends SerenityStories {
3. Use the deprecated #Metafilter annotation
import net.serenitybdd.jbehave.SerenityStories;
import net.thucydides.jbehave.annotations.Metafilter;
#Metafilter("-skip") // warned as deprecated
public class Acceptance extends SerenityStories {
Solution for my current project is to use the current #Metafilter("-skip") annotation on my test class, to not depend on/have to change VM properties of the particular Jenkins/local dev execution.
Possible pull request to make
https://github.com/serenity-bdd/serenity-core/issues/95 — here Serenity guys have suggested me to do a PR with this fix, since they are not concentrated on Serenity + JBehave now.
I understand where to make the changes (in the code chain described above), but I don't know what overriding logic should be:
— MetaFilters from configuredEmbedder override any of ENV/annotation MetaFilters.
OR
— Any ENV/annotation MetaFilters override Metafilters from configuredEmbedder
OR
— MetaFilters from configuredEmbedder are merged with ENV/annotation MetaFilters. This option required merging priority.
Any suggestions?
In any type of fix, I would prefer add the explicit logs about how the overriding is now working into SerenityReportingRunner#getMetafilterSetting, since the current behaviour is really non-obvious and took lots of time to investigate.

OSGI classcast exception on felix

i'm fairly new to osgi and am trying to get a functional proof of concept together.
The setup is that my common api is created in a bundle creatively named common-api.jar with no bundle activator, but it exports all it's interfaces. the one of interest in this situation is DatabaseService.java.
I then have a Second bundle called systemx-database-service. That implements the database service interface. this works fine as in the activator of the implementation bundle i test the connection to the database and select some arbitraty values. I also register the service i want to be available to the other bundle's like so:
context.registerService(DatabaseService.class.getName(), new SystemDatabaseServiceImpl(context), new Properties());
The basic idea being when you look for a service reference for a Database service you'll get back the SystemDatabaseService implementation.
When i do a inspect service the output it this:
-> inspect s c 69
System Database Service (69) provides services:
----------------------------------------------
objectClass = za.co.xxx.xxx.common.api.DatabaseService
service.id = 39
which would lead me to believe that if i do this in a test bundle:
context.getService(context.getServiceReference(DatabaseService.class));
i should get back an instance of DatabaseService.class, but alas no such luck. it simply seems like it cannot find the service. stick with me here my story gets stranger.
figuring there is no where to go but up i wrote this monstrosity:
for (Bundle bundle : bundles) {
if (bundle.getSymbolicName().equals("za.co.xxx.xxx.database-service")) {
ServiceReference[] registeredServices = bundle.getRegisteredServices();
for (ServiceReference ref : registeredServices) {
DatabaseService service = (DatabaseService) context.getService(ref);
// use service here.
}
}
}
}
now i can actually see the service reference, but i get this error
java.lang.ClassCastException: za.co.xxx.xxx.database.service.impl.SystemDatabaseServiceImpl cannot be cast to za.co.xxx.xx.common.api.DatabaseService
which is crazy since the implementation clearly implements the interface!
Any help would be appreciated. Please keep in mind i'm very new at the osgi way of thinking so my whole approach here might be flawed.
oh. if anyone wants the manifests i can post them. and i'm using the maven-bnd-plugin to build and executing on felix.
thanks
Nico

The test bundle must resolve to the same import of the DatabaseService interface as the SystemDatabaseServiceImpl. If this does not occur, then getServiceReference documents that it will return null even if a service is found. By locating the bundle manually and attempting to locate the service and cast, you're showing why getServiceReference behaves in this way: if it returned arbitrary services, Java casts would fail.
I would recommend printing DatabaseService.class.getClassLoader() in both the impl bundle and test bundle to prove if they're the same bundle. If they're not, then you need to adjust your OSGi MANIFEST.MF metadata to ensure that they have a consistent view of the interface class.
For example, is the DatabaseService interface included in both the test and impl bundles? If yes, you need to move that interface to either the impl bundle (and Export-Package) or to a third interface bundle and Export-Package. Then, adjust the other bundles to Import-Package.