Develop Reference

SnapStart function priming - Doesn't seem to be working

I’m somewhat new to Kotlin/Java, but I have been using AWS Lambda for several years now (all Python and Node). I’ve been trying to “successfully” enable SnapStart on a SpringBoot Lambda using Kotlin running on java11 corretto (the only runtime supported currently), but it doesn’t seem to be working as I would have expected.
I have hooked into the CRaC lifecycle methods beforeCheckpoint and afterRestore. In beforeCheckpoint I’ve initialized the SpringBoot application and I can see it in the deployment logs (AWS creates log streams for the deployment phase with SnapStart lambdas).
However, the concerning thing is I’m also seeing the SpringBoot app get initialized in the function invocation logs too. I would have expected that to only happen during the deployment/initialization phase when the snapshot is being created. As a result I’m not really seeing a tremendous improvement on latency or overall.
Any ideas why this is happening?

I ran into essentially the same issue (with Java instead of Kotlin) and the solution was to switch the runtime->handler from
org.springframework.cloud.function.adapter.aws.SpringBootStreamHandler
to
org.springframework.cloud.function.adapter.aws.FunctionInvoker::handleRequest

It would probably be worth mentioning that as of 2023-02-20 SnapStart isn't engaged for $LATEST version of an AWS Lambda function, i.e. make sure you are invoking a particular published version. Otherwise, Best practices for working with Lambda SnapStart article says that the main performance killers are dynamically loaded classes, and network connections that need to be re-established from time to time.
From Snapstart Integration issue raised for Spring Cloud Function on GitHub I tend to think that switching to org.springframework.cloud.function.adapter.aws.FunctionInvoker probably somewhat helps, but doesn't address the performance challenges mentioned above. I'm not sure if I'm interpreting olegz's advice correctly, but what worked best so far for my AWS lambda function built with Spring Boot/Spring Cloud Function is a "warm-up" config. It hooks into the CRaC lifecycle via beforeCheckpoint() and issues dummy requests to S3 and DynamoDB before the VM snapshot is made. This way most dynamically-loaded classes are pre-loaded, and network connections are pre-established, before any subsequent function invocation takes place.
package eu.mycompany.mysamplesystem.attachmentstore.configuration;
import com.amazonaws.services.lambda.runtime.events.S3Event;
import eu.mycompany.mysamplesystem.attachmentstore.handlers.MainEventHandler;
import lombok.extern.slf4j.Slf4j;
import org.crac.Core;
import org.crac.Resource;
import org.springframework.context.annotation.Configuration;
import software.amazon.awssdk.services.s3.model.NoSuchKeyException;
import java.util.ArrayList;
import java.util.List;
#Configuration
#Slf4j
public class WarmUpConfig implements Resource {
private final MainEventHandler mainEventHandler;
public WarmUpConfig(final MainEventHandler mainEventHandler) {
Core.getGlobalContext().register(this);
this.mainEventHandler = mainEventHandler;
}
#Override
public void beforeCheckpoint(org.crac.Context<? extends Resource> context) {
log.debug("Warm-up MainEventHandler by issuing dummy requests");
dummyS3Invocation();
dummyDynamoDbInvocation();
}
#Override
public void afterRestore(org.crac.Context<? extends Resource> context) {
}
public void dummyS3Invocation() {
S3Event s3Event = generateWarmUpEvent("ObjectCreated:Put");
try {
mainEventHandler.handleRequest(s3Event, null);
throw new IllegalStateException("Warm-up event processing should have reached S3 and failed with S3Exception");
} catch (NoSuchKeyException e) {
log.debug("S3Exception is expected, since it is a warm-up");
}
}
public void dummyDynamoDbInvocation() {
S3Event s3Event = generateWarmUpEvent("ObjectRemoved:Delete");
mainEventHandler.handleRequest(s3Event, null);
}
private S3Event generateWarmUpEvent(String eventName) {
S3Event.S3BucketEntity s3BucketEntity = new S3Event.S3BucketEntity("hopefully_non_existing_bucket", null, null);
S3Event.S3ObjectEntity s3ObjectEntity = new S3Event.S3ObjectEntity("hopefully/non/existing.key", 0L, null, null, null);
S3Event.S3Entity s3Entity = new S3Event.S3Entity(null, s3BucketEntity, s3ObjectEntity, null);
List<S3Event.S3EventNotificationRecord> records = new ArrayList<>();
records.add(new S3Event.S3EventNotificationRecord(null, eventName, null, null, null, null, null, s3Entity, null));
return new S3Event(records);
}
}
P.S.: The MainEventHandler is basically the entry point to all the business logic exposed by the Function.
#SpringBootApplication
#RequiredArgsConstructor
public class Lambda {
private final MainEventHandler mainEventHandler;
public static void main(String... args) {
SpringApplication.run(Lambda.class, args);
}
#Bean
public Function<Message<S3Event>, String> defaultFunctionLambda() {
return message -> {
Context context = message.getHeaders().get("aws-context", Context.class);
return mainEventHandler.handleRequest(message.getPayload(), context);
};
}
}

Why does the glassfish server only load a bean after it has been restarted?

When the rtu.smallview.xhtml action event is triggered it requests info from the java bean, from the database select and hands it back to the xhtml.
The xhtml was not displaying the data from the database, so I added breakpoints in the java bean to figure out what was going wrong, but when the program loaded it never hit the breakpoint in the bean.
The server output is saying this when the program is loaded:
Info: WELD-000119: Not generating any bean definitions from Beans.RTU.RTU_SmallView_Bean because of underlying class loading error: Type pojo.rtu.RTU_unit not found. If this is unexpected, enable DEBUG logging to see the full error.
So I stopped the server, clean and built the project again, and when it runs for the first time it loads the bean, the information is retrieved and displayed. Though if I clean and build the project again, when it runs the second time it displays the same WELD-000119 error.
I copy and pasted just the code to make the RTU section run to a new project and the server doesn't ever throw this error, and it works every time the bean is requested and every time the server is started.
Edit 1:
When I restart NetBeans and Clean and Build the project after it starts it says this:
Note: C:\Users\Administrator\Documents\NetBeansProjects\OIUSA_1\src\java\Beans\RTU\RTU_SmallView_Bean.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
This is the only java class it says this about, so here is the code I used for that class:
package Beans.RTU;
import Database.RTU.RTU_SmallView_Select;
import java.util.ArrayList;
import java.util.List;
import javax.inject.Named;
import javax.enterprise.context.RequestScoped;
import pojo.rtu.RTU_unit;
/**
*
* #author Administrator
*/
#Named(value = "rtu_SmallView_Bean")
#RequestScoped
public class RTU_SmallView_Bean {
public RTU_SmallView_Bean() {
try {
RTU_SmallView_Select selectData;
selectData = new RTU_SmallView_Select();
this.smallViewList = selectData.getData();
} catch (Exception ex) {
System.out.println(ex.getMessage());
}
}
List<RTU_unit> smallViewList = new ArrayList();
String unit_type;
int unit_number;
String rig_name;
String location_name;
public List<RTU_unit> getSmallViewList() {
return smallViewList;
}
public void setSmallViewList(List<RTU_unit> smallViewList) {
this.smallViewList = smallViewList;
}
public String getUnit_type() {
return unit_type;
}
public void setUnit_type(String unit_type) {
this.unit_type = unit_type;
}
public int getUnit_number() {
return unit_number;
}
public void setUnit_number(int unit_number) {
this.unit_number = unit_number;
}
public String getRig_name() {
return rig_name;
}
public void setRig_name(String rig_name) {
this.rig_name = rig_name;
}
public String getLocation_name() {
return location_name;
}
public void setLocation_name(String location_name) {
this.location_name = location_name;
}
}
My project structure is as follows:
Sources:
Beans.RTU.RTU_SmallView_Bean.java
Database.RTU.RTU_SmallView_Select.java
pojo.rtu.RTU_unit.java
Webpages:
rtu.rtu_smallview.xhtml
I am thinking it has something to do with the actual server, but I'm not sure where to start looking for this error. If you would like to see the actual code for the beans and what not, let me know and I'll edit the question with all the code. Thanks

Problem has been solved, the file RTU_Unit.java was in a folder called pojo.rtu. I deleted the folder, made it again with a new name pojo.rtus, refactored the file RTU_Unit.java for the new folder and the problem has gone away.

How to install or register a Saxon HE 10.3 Configuration? Configuration is not being used

I'm trying to use a custom Configuration for saxon HE 10.3.
The Configuration is not being used. Presumably the config needs to be registered or installed? But how?
Here's my code:
final Configuration config = new net.sf.saxon.Configuration();
/**/ config.setLocalizerFactory(new LocalizerFactory() {
public Numberer getNumberer(final String language, final String country) {
if (language.equals("de")) {
return Numberer_de.getInstance();
} else {
return null;
}
}
});
net.sf.saxon.Transform.main(new String[] {
"-s:source.xml",
"-xsl:stylesheet.xslt",
"-o:result.txt"
});

You really don't want to be running net.sf.saxon.Transform.main from a Java application: use either the s9api or JAXP transformation APIs. The net.sf.saxon.Transform.main interface is designed for use from the command line, and it can therefore only modify the configuration through command line switches. It also has drawbacks like shutting down the Java VM if the transformation fails.
There is a workaround, which is to use the -init option on the command line to trigger user-supplied initialisation code (which has access to the Configuration object), but that's only really digging yourself deeper into your hole. I'd recommend switching to the s9api API.
Documentation: https://saxonica.com/documentation/index.html#!using-xsl/embedding

If you want to change the configuration when running Saxon from the command line, as Michael said, there is the -init option to pass in the name of a class implementing the Initializer interface https://saxonica.com/documentation/index.html#!javadoc/net.sf.saxon.lib/Initializer so you would use roughly e.g.
package com.example;
import net.sf.saxon.option.local.Numberer_de;
import net.sf.saxon.lib.Initializer;
import net.sf.saxon.Configuration;
import net.sf.saxon.lib.LocalizerFactory;
import net.sf.saxon.lib.Numberer;
import javax.xml.transform.TransformerException;
public class MyInitializer implements Initializer {
public override void initialize(Configuration config) throws TransformerException {
config.setLocalizerFactory(new LocalizerFactory() {
public Numberer getNumberer(final String language, final String country) {
if (language.equals("de")) {
return Numberer_de.getInstance();
} else {
return null;
}
}
});
}
}
compile that, put it on the classpath and then run e.g. java -cp saxon-he-10.3.jar;com/example/MyInitializer;net/sf/saxon/option/local/Numberer_de net.sf.saxon.Transform -init:com.example.MyInitializer -s:source.xml -xsl:stylesheet.xslt -o:result.txt.
Or you can subclass net.sf.saxon.Transform.
On the other hand, if you don't want to run Saxon from the command line but from the JAXP API then I think one approach is to create the Configuration e.g.
Configuration config = new Configuration();
config.setLocalizerFactory(new LocalizerFactory() {
public Numberer getNumberer(final String language, final String country) {
if (language.equals("de")) {
return Numberer_de.getInstance();
} else {
return null;
}
}
});
TransformerFactory transformerFactory = new TransformerFactoryImpl(config);
Templates templates = transformerFactory.newTemplates(xsltSource);

What was missing, was how to inject the Config. This worked for me:
import net.sf.saxon.Configuration;
import net.sf.saxon.TransformerFactoryImpl;
import net.sf.saxon.lib.Feature;
final TransformerFactoryImpl factory = (TransformerFactoryImpl) TransformerFactory.newInstance();
factory.getProcessor().setConfigurationProperty(Feature.CONFIGURATION, config);

How to log the active configuration in a Spring Boot application?

I would really like to use YAML config for Spring Boot, as I find it quite readable and useful to have a single file showing what properties are active in my different profiles. Unfortunately, I'm finding that setting properties in application.yml can be rather fragile.
Things like using a tab instead of spaces will cause properties to not exist (without warnings as far as I can see), and all too often I find that my active profiles are not being set, due to some unknown issue with my YAML.
So I was wondering whether there are any hooks that would enable me to get hold of the currently active profiles and properties, so that I could log them.
Similarly, is there a way to cause start-up to fail if the application.yml contains errors? Either that or a means for me to validate the YAML myself, so that I could kill the start-up process.

In addition to other answers: logging active properties on context refreshed event.
Java 8
package mypackage;
import lombok.extern.slf4j.Slf4j;
import org.springframework.context.event.ContextRefreshedEvent;
import org.springframework.context.event.EventListener;
import org.springframework.core.env.ConfigurableEnvironment;
import org.springframework.core.env.MapPropertySource;
import org.springframework.stereotype.Component;
import java.util.ArrayList;
import java.util.Collection;
import java.util.List;
#Slf4j
#Component
public class AppContextEventListener {
#EventListener
public void handleContextRefreshed(ContextRefreshedEvent event) {
printActiveProperties((ConfigurableEnvironment) event.getApplicationContext().getEnvironment());
}
private void printActiveProperties(ConfigurableEnvironment env) {
System.out.println("************************* ACTIVE APP PROPERTIES ******************************");
List<MapPropertySource> propertySources = new ArrayList<>();
env.getPropertySources().forEach(it -> {
if (it instanceof MapPropertySource && it.getName().contains("applicationConfig")) {
propertySources.add((MapPropertySource) it);
}
});
propertySources.stream()
.map(propertySource -> propertySource.getSource().keySet())
.flatMap(Collection::stream)
.distinct()
.sorted()
.forEach(key -> {
try {
System.out.println(key + "=" + env.getProperty(key));
} catch (Exception e) {
log.warn("{} -> {}", key, e.getMessage());
}
});
System.out.println("******************************************************************************");
}
}
Kotlin
package mypackage
import mu.KLogging
import org.springframework.context.event.ContextRefreshedEvent
import org.springframework.context.event.EventListener
import org.springframework.core.env.ConfigurableEnvironment
import org.springframework.core.env.MapPropertySource
import org.springframework.stereotype.Component
#Component
class AppContextEventListener {
companion object : KLogging()
#EventListener
fun handleContextRefreshed(event: ContextRefreshedEvent) {
printActiveProperties(event.applicationContext.environment as ConfigurableEnvironment)
}
fun printActiveProperties(env: ConfigurableEnvironment) {
println("************************* ACTIVE APP PROPERTIES ******************************")
env.propertySources
.filter { it.name.contains("applicationConfig") }
.map { it as EnumerablePropertySource<*> }
.map { it -> it.propertyNames.toList() }
.flatMap { it }
.distinctBy { it }
.sortedBy { it }
.forEach { it ->
try {
println("$it=${env.getProperty(it)}")
} catch (e: Exception) {
logger.warn("$it -> ${e.message}")
}
}
println("******************************************************************************")
}
}
Output like:
************************* ACTIVE APP PROPERTIES ******************************
server.port=3000
spring.application.name=my-app
...
2017-12-29 13:13:32.843 WARN 36252 --- [ main] m.AppContextEventListener : spring.boot.admin.client.service-url -> Could not resolve placeholder 'management.address' in value "http://${management.address}:${server.port}"
...
spring.datasource.password=
spring.datasource.url=jdbc:postgresql://localhost/my_db?currentSchema=public
spring.datasource.username=db_user
...
******************************************************************************

I had the same problem, and wish there was a debug flag that would tell the profile processing system to spit out some useful logging. One possible way of doing it would be to register an event listener for your application context, and print out the profiles from the environment. I haven't tried doing it this way myself, so your mileage may vary. I think maybe something like what's outlined here:
How to add a hook to the application context initialization event?
Then you'd do something like this in your listener:
System.out.println("Active profiles: " + Arrays.toString(ctxt.getEnvironment().getActiveProfiles()));
Might be worth a try. Another way you could probably do it would be to declare the Environment to be injected in the code where you need to print the profiles. I.e.:
#Component
public class SomeClass {
#Autowired
private Environment env;
...
private void dumpProfiles() {
// Print whatever needed from env here
}
}

Actuator /env service displays properties, but it doesn't displays which property value is actually active. Very often you may want to override your application properties with
profile-specific application properties
command line arguments
OS environment variables
Thus you will have the same property and different values in several sources.
Snippet bellow prints active application properties values on startup:
#Configuration
public class PropertiesLogger {
private static final Logger log = LoggerFactory.getLogger(PropertiesLogger.class);
#Autowired
private AbstractEnvironment environment;
#PostConstruct
public void printProperties() {
log.info("**** APPLICATION PROPERTIES SOURCES ****");
Set<String> properties = new TreeSet<>();
for (PropertiesPropertySource p : findPropertiesPropertySources()) {
log.info(p.toString());
properties.addAll(Arrays.asList(p.getPropertyNames()));
}
log.info("**** APPLICATION PROPERTIES VALUES ****");
print(properties);
}
private List<PropertiesPropertySource> findPropertiesPropertySources() {
List<PropertiesPropertySource> propertiesPropertySources = new LinkedList<>();
for (PropertySource<?> propertySource : environment.getPropertySources()) {
if (propertySource instanceof PropertiesPropertySource) {
propertiesPropertySources.add((PropertiesPropertySource) propertySource);
}
}
return propertiesPropertySources;
}
private void print(Set<String> properties) {
for (String propertyName : properties) {
log.info("{}={}", propertyName, environment.getProperty(propertyName));
}
}
}

If application.yml contains errors it will cause a failure on startup. I guess it depends what you mean by "error" though. Certainly it will fail if the YAML is not well formed. Also if you are setting #ConfigurationProperties that are marked as ignoreInvalidFields=true for instance, or if you set a value that cannot be converted. That's a pretty wide range of errors.
The active profiles will probably be logged on startup by the Environment implementation (but in any case it's easy for you to grab that and log it in your launcher code - the toString() of teh Environment will list the active profiles I think). Active profiles (and more) are also available in the /env endpoint if you add the Actuator.

In case you want to get the active profiles before initializing the beans/application, the only way I found is registering a custom Banner in your SpringBootServletInitializer/SpringApplication (i.e. ApplicationWebXml in a JHipster application).
e.g.
#Override
protected SpringApplicationBuilder configure(SpringApplicationBuilder builder)
{
// set a default to use when no profile is configured.
DefaultProfileUtil.addDefaultProfile(builder.application());
return builder.sources(MyApp.class).banner(this::printBanner);
}
/** Custom 'banner' to obtain early access to the Spring configuration to validate and debug it. */
private void printBanner(Environment env, Class<?> sourceClass, PrintStream out)
{
if (env.getProperty("spring.datasource.url") == null)
{
throw new RuntimeException(
"'spring.datasource.url' is not configured! Check your configuration files and the value of 'spring.profiles.active' in your launcher.");
}
...
}

How do you implement a ManagedServiceFactory in OSGi?

Im currently trying to setup my own implementation of a ManagedServiceFactory. Here is what I'm trying to do: I need multiple instances of some service on a per-configuration base. With DS the components worked perfectly but now I found out that these services should handle there own lifecycle (i.e. (de)registration at the service registry) depending on the availability of some external resource, which is impossible with DS.
Thus my idea was to create a ManagedServiceFactory, which then would receive configs from the ConfigurationAdmin and create instances of my class. These again would try to connect to the resource in a seperate thread and register themselves as service when they're ready to operate.
Since I had no luck implementing this yet, I tried to break everything down to the most basic parts, not even dealing with the dynamic (de)registration, just trying to get the ManagedServiceFacotry to work:
package my.project.factory;
import java.util.Dictionary;
import java.util.HashMap;
import java.util.Hashtable;
import java.util.Map;
import org.osgi.framework.BundleActivator;
import org.osgi.framework.BundleContext;
import org.osgi.framework.Constants;
import org.osgi.framework.ServiceRegistration;
import org.osgi.service.cm.ConfigurationException;
import org.osgi.service.cm.ManagedServiceFactory;
public class Factory implements BundleActivator, ManagedServiceFactory {
private ServiceRegistration myReg;
private BundleContext ctx;
private Map<String, ServiceRegistration> services;
#Override
public void start(BundleContext context) throws Exception {
System.out.println("starting factory...");
this.ctx = context;
java.util.Dictionary properties = new Hashtable<String, Object>();
properties.put(Constants.SERVICE_PID, "my.project.servicefactory");
myReg = context.registerService(ManagedServiceFactory.class, this,
properties);
System.out.println("registered as ManagedServiceFactory");
services = new HashMap<String, ServiceRegistration>();
}
#Override
public void stop(BundleContext context) throws Exception {
for(ServiceRegistration reg : services.values()) {
System.out.println("deregister " + reg);
reg.unregister();
}
if(myReg != null) {
myReg.unregister();
} else {
System.out.println("my service registration as already null " +
"(although it shouldn't)!");
}
}
#Override
public String getName() {
System.out.println("returning facotry name");
return "ServiceFactory";
}
#Override
public void updated(String pid, Dictionary properties)
throws ConfigurationException {
System.out.println("retrieved update for pid " + pid);
ServiceRegistration reg = services.get(pid);
if (reg == null) {
services.put(pid, ctx.registerService(ServiceInterface.class,
new Service(), properties));
} else {
// i should do some update here
}
}
#Override
public void deleted(String pid) {
ServiceRegistration reg = services.get(pid);
if (reg != null) {
reg.unregister();
}
}
}
Now, it should receive configurations from the ConfigurationAdmin for PID my.project.servicefactory, shouldn't it?
But it does not receive any configurations from the ConfigurationAdmin. The bundle is started, the service is registered and in the web console, I can see the config admin holds a reference to my ManagedServiceFactory. Is there a certain property which should be set? The interface specification does not suggest that. Actually my implementation is more or less the same as the example there. I've no idea what I'm doing wrong here, any pointers to the solutions are very welcome.
Also, I orginally thought to implement the ManagedServiceFactory itself as DS, which also should be possible, but I failed at the same point: no configurations are handed over by the ConfigAdmin.
update
To clarify the question: I think that this is mainly an configuration problem. As I see it, I should be able to specify two PIDs for the factory, one which identifies a configuration for the factory itself (if any), and one which would produce services trough this factory, which I thought should be the factory.pid. But the framework constants do not hold anything like this.
update 2
After searching a bit the Felix Fileinstall source code, I found out that it treats configuration files differently when there is a - in the filename or not. Having the configuration file named my.project.servicefactory.cfg it did not work, but the configs named my.project.servicefactory-foo.cfg and my.project.servicefactory-bar.cfg were properly handed over to my ManagedServiceFactory as expected, and multiple services with ServiceInterface were registered. Hurray!
update 3
As proposed by Neil, I put the declarative service part in a new question to bound the scope of this one.

I think that the problem is you have a singleton configuration record rather than a factory record. You need to call Config Admin with the createFactoryConfiguration method using my.project.servicefactory as the factoryPid.
If you are using Apache Felix FileInstall (which is a nice easy way to create config records without writing code) then you need to create a file called my.project.servicefactory-1.cfg in the load directory. You can create further configurations with the same factoryPID by calling them my.project.servicefactory-2.cfg etc.