For my Spring Boot application, I am trying to use an environment variable that holds the list of properties.topics in application.yml (see configuration below).
properties:
topics:
- topic-01
- topic-02
- topic-03
I use the configuration file to populate properties bean (see this spring documentation), as shown below
import java.util.ArrayList;
import java.util.List;
import org.springframework.boot.context.properties.ConfigurationProperties;
#ConfigurationProperties("properties")
public class ApplicationProperties {
private List<String> topics = new ArrayList<>();
public void setTopics(List<String> topics) {
this.topics = topics;
}
public List<String> getTopics() {
return this.topics;
}
}
With the use of environment variable, I can have the list's content change without changing the application.yml. However, all examples that I could find so far only for cases where an environment variable holding only single value, not a collection of values in my case.
Edit:
To make it clear after #vancleff's comment, I do not need the values of the environment variable to be saved to application.yml.
Another edit:
I think by oversimplifying my question, I shoot myself in the foot. #LppEdd answer works well with the example given in my question. However, what happens if instead of a collection of simple string topic names, I need a bit more complex structure. For example, something like
properties:
topics:
-
name: topic-01
id: id-1
-
name: topic-02
id: id-2
-
name: topic-03
id: id-3
a bit late for the show but, I was facing the same problem and this solves it
https://github.com/spring-projects/spring-boot/wiki/Relaxed-Binding-2.0#lists-1
MY_FOO_1_ = my.foo[1]
MY_FOO_1_BAR = my.foo[1].bar
MY_FOO_1_2_ = my.foo[1][2]`
So, for the example in the question:
properties:
topics:
-
name: topic-01
id: id-1
-
name: topic-02
id: id-2
-
name: topic-03
id: id-3
The environment variables should look like this:
PROPERTIES_TOPICS_0_NAME=topic-01
PROPERTIES_TOPICS_0_ID=id-01
PROPERTIES_TOPICS_1_NAME=topic-02
PROPERTIES_TOPICS_1_ID=id-02
PROPERTIES_TOPICS_2_NAME=topic-03
PROPERTIES_TOPICS_2_ID=id-03
Suggestion, don't overcomplicate.
Say you want that list as an Environment variable. You'd set it using
-Dtopics=topic-01,topic-02,topic-03
You then can recover it using the injected Environment Bean, and create a new List<String> Bean
#Bean
#Qualifier("topics")
List<String> topics(final Environment environment) {
final var topics = environment.getProperty("topics", "");
return Arrays.asList(topics.split(","));
}
From now on, that List can be #Autowired.
You can also consider creating your custom qualifier annotation, maybe #Topics.
Then
#Service
class TopicService {
#Topics
#Autowired
private List<String> topics;
...
}
Or even
#Service
class TopicService {
private final List<String> topics;
TopicService(#Topics final List<String> topics) {
this.topics = topics;
}
...
}
What you could do is use an externalized file.
Pass to the environment parameters the path to that file.
-DtopicsPath=C:/whatever/path/file.json
Than use the Environment Bean to recover that path. Read the file content and ask Jackson to deserialize it
You'd also need to create a simple Topic class
public class Topic {
public String name;
public String id;
}
Which represents an element of this JSON array
[
{
"name": "topic-1",
"id": "id-1"
},
{
"name": "topic-2",
"id": "id-2"
}
]
#Bean
List<Topic> topics(
final Environment environment,
final ObjectMapper objectMapper) throws IOException {
// Get the file path
final var topicsPath = environment.getProperty("topicsPath");
if (topicsPath == null) {
return Collections.emptyList();
}
// Read the file content
final var json = Files.readString(Paths.get(topicsPath));
// Convert the JSON to Java objects
final var topics = objectMapper.readValue(json, Topic[].class);
return Arrays.asList(topics);
}
Also facing the same issue , fixed with having a array in deployment.yaml from values.yml replacing the default values of application.yml
example as :
deployment.yml -
----------------
env:
- name : SUBSCRIBTION_SITES_0_DATAPROVIDER
value: {{ (index .Values.subscription.sites 0).dataprovider | quote }}
- name: SUBSCRIBTION_SITES_0_NAME
value: {{ (index .Values.subscription.sites 0).name | quote }}
values.yml -
---------------
subscription:
sites:
- dataprovider: abc
name: static
application.yml -
------------------
subscription:
sites:
- dataprovider: ${SUBSCRIBTION_SITES_0_DATAPROVIDER:abcd}
name: ${SUBSCRIBTION_SITES_0_NAME:static}
Java Code :
#Getter
#Setter
#Configuration
#ConfigurationProperties(prefix = "subscription")
public class DetailsProperties {
private List<DetailsDto> sites;
DetailsProperties() {
this.sites = new ArrayList<>();
}
}
Pojo mapped :
#Getter
#Setter
#Data
#Builder
#AllArgsConstructor
#NoArgsConstructor
public class DetailsDto {
private String dataprovider;
private String name;
}
I built a quick little utility to do this.
import java.util.LinkedList;
import java.util.List;
import org.springframework.core.env.Environment;
/**
* Convenience methods for dealing with properties.
*/
public final class PropertyUtils {
private PropertyUtils() {
}
public static List<String> getPropertyArray(Environment environment, String propertyName) {
final List<String> arrayPropertyAsList = new LinkedList<>();
int i = 0;
String value;
do {
value = environment.getProperty(propertyName + "[" + i++ + "]");
if (value != null) {
arrayPropertyAsList.add(value);
}
} while (value != null);
return arrayPropertyAsList;
}
}
You could modify this without too many changes to support multiple fields as well. I've seen similar things done to load an array of database configurations from properties.
Related
In a Spring Boot application, Spring Boot is used to build a Properties object from a YAML file as follows:
YamlPropertiesFactoryBean yamlFactory = new YamlPropertiesFactoryBean();
yamlFactory.setResources(new DefaultResourceLoader().getResource("application.yml"));
Properties properties = yamlFactory.getObject();
The reason why Spring Boot's own parser is used is that it not only reads YAML-compliant settings, but also dot-notated properties like e.g:
artist.elvis.name: "Elvis"
artist.elvis.message: "Aloha from Hawaii"
Now that the Properties object is built, I want to map it into an object like the following for example:
#JsonIgnoreProperties(ignoreUnknown = true)
private record Artist(Elvis elvis) {
private record Elvis(String name, String message) { }
}
My question is:
How can this be done with Jackson? Or is there another/better solution for this?
Many thanks for any help
I saw functionality like that in Ratpack framework.
e.g.:
var propsFileUrl =
Thread.currentThread()
.getContextClassLoader()
.getResource("application.properties");
ApplicationProperties applicationProperties =
ConfigData.builder()
.props(propsFileUrl)
.build()
.get(ApplicationProperties.class);
under the hood it is indeed done by using jackson's object mapper, but the logic is not as trivial to post it here.
here's the library:
https://mvnrepository.com/artifact/io.ratpack/ratpack-core/2.0.0-rc-1
application.yml is the default yml file, so no custom configuration is required. Value annotation should be able to read the properties.
#Value("${artist.elvis.name}")
private String name;
Next part I am not sure about your requirements, but hope this is what you are looking for.
To bind to this object 'constructor' can be a good option.
Class for elvis
#Bean
public class Elvis {
private String name;
private String message;
public Elvis(#Value("${artist.elvis.name"}) final String name, #Value("${artist.elvis.message"}) final String message) {
this.name=name;
this.message=message
}
// getter setter for name and message
}
Now Autowire the created bean to Artist bean
#Bean("artists")
public class Artists {
#Autowired
private Elvis elvis
pubic Elvis getElvis() {
return elvis;
}
}
I am trying to get application property object by value, i already did this in Java, but from some reason using Kotlin i can not manage to do it.
So basically what i have is list of application properties that looks like this:
ee.car.config.audi.serial=1
ee.car.config.audi.base=platform1
ee.car.config.bmw.serial=2
ee.car.config.bmw.base=platform2
so as you can see car identifiers (audi,bmw,peugeot,etc..) are dynamic, and i need simply by serial value to get object that represents the specific car and by car key(audi, bmw) to get all other properties.
And what i did is simple, i created configuration properties class like this:
#Configuration
#ConfigurationProperties(prefix = "ee.car")
data class FesEngineKeys(
val config: HashMap<String, EeCarConfigParam> = HashMap()
) {
fun getOrDefaultEEConfig(engineKey: String): EeCarConfigParam? {
return config.getOrDefault(engineKey, config["audi"])
}
And then object to map keys after dynamic value:
data class EeCarConfigParam {
var serial: String,
var base: String
}
But problem here is, in FesEngineKeys class, config property is empty, it looks like EeCarConfigParam can not be mapped.
Also interesting part is when i change:
val config: HashMap<String, EeCarConfigParam> = HashMap() to
val config: HashMap<String, String> = HashMap()
then i can see that config param is populated with all the values.
This code already works in Java and it looks like this:
#Configuration
#Getter
#Setter
#ConfigurationProperties(prefix = "ee.car")
public class FESEngineKeys {
private Map<String, EeCarConfigParam> config = new HashMap<>();
public EeCarConfigParam getOrDefaultEEConfig(String engineKey) {
return config.getOrDefault(engineKey, config.get("audi"));
}
public EeCarConfigParam findBySerial(String serial) {
return config.values().stream().filter(cfg -> cfg.getSerial().equalsIgnoreCase(serial)).findFirst().orElse(null);
}
}
#Data
public class EeCarConfigParam {
private String serial;
private String base;
}
I really don't know why in the Kotlin case it is not working, i probably made some very basic mistake, and i would really appreciate if anyone can explain what is happening here
Okay i got it.
According to that: https://docs.spring.io/spring-boot/docs/2.0.x/reference/html/boot-features-kotlin.html the support for what you want is very limited.
I made it working like that - not pretty nice :-( :
#ConfigurationProperties(prefix = "ee.car")
class FesEngineKeyProperties() {
var config: MutableMap<String, EeCarConfigParam?>? = mutableMapOf()
fun getBase(serial: String): String {
if(config == null) return ""
return config!!["audi"]?.base ?: ""
}
}
class EeCarConfigParam() {
lateinit var serial: String
lateinit var base: String
}
#SpringBootApplication
#EnableConfigurationProperties(FesEngineKeyProperties::class)
class SandboxApplication
fun main(args: Array<String>) {
runApplication<SandboxApplication>(*args)
}
I was able to handle this issue, it is somehow related to kotlin, because once when i instead of this:
data class EeCarConfigParam {
var serial: String,
var base: String
}
used "norma" Java class, everything started working, so all code from my question stays the same, only difference is this: instead of Kotlin EeCardConfigParam i created Java class like this:
public class EeCarConfigParam {
private String publicUrl;
private String base;
}
Note: with all default getters, setters, equals, hash and toString methods.
I have the following enum:
public enum MyEnum {
NAME("Name", "Good", 100),
FAME("Fame", "Bad", 200);
private String lowerCase;
private String atitude;
private long someNumber;
MyEnum(String lowerCase, String atitude, long someNumber) {
this.lowerCase = lowerCase;
this.atitude = atitude;
this.someNumber = someNumber;
}
}
I want to setup the someNumber variable different for both instances of the enum using application.properties file.
Is this possible and if not, should i split it into two classes using an abstract class/interface for the abstraction?
You can't/shouldn't change values of a enum in Java. Try using a class instead:
public class MyCustomProperty {
// can't change this in application.properties
private final String lowerCase;
// can change this in application.properties
private String atitude;
private long someNumber;
public MyCustomProperty (String lowerCase, String atitude, long someNumber) {
this.lowerCase = lowerCase;
this.atitude = atitude;
this.someNumber = someNumber;
}
// getter and Setters
}
Than create a custom ConfigurationProperties:
#ConfigurationProperties(prefix="my.config")
public class MyConfigConfigurationProperties {
MyCustomProperty name = new MyCustomProperty("name", "good", 100);
MyCustomProperty fame = new MyCustomProperty("fame", "good", 100);
// getter and Setters
// You can also embed the class MyCustomProperty here as a static class.
// For details/example look at the linked SpringBoot Documentation
}
Now you can change the values of my.config.name.someNumber and my.config.fame.someNumber in the application.properties file. If you want to disallow the change of lowercase/atitude make them final.
Before you can use it you have to annotate a #Configuration class with #EnableConfigurationProperties(MyConfigConfigurationProperties.class). Also add the org.springframework.boot:spring-boot-configuration-processor as an optional dependency for a better IDE Support.
If you want to access the values:
#Autowired
MyConfigConfigurationProperties config;
...
config.getName().getSumeNumber();
Well what you can do is the following:
Create a new class: MyEnumProperties
#ConfigurationProperties(prefix = "enumProperties")
#Getter
public class MyEnumProperties {
private Map<String, Long> enumMapping;
}
Enable ConfigurationProperties to your SpringBootApplication/ any Spring Config via
#EnableConfigurationProperties(value = MyEnumProperties.class)
Now add your numbers in application.properties file like this:
enumProperties.enumMapping.NAME=123
enumProperties.enumMapping.FAME=456
In your application code autowire your properties like this:
#Autowired
private MyEnumProperties properties;
Now here is one way to fetch the ids:
properties.getEnumMapping().get(MyEnum.NAME.name()); //should return 123
You can fetch this way for each Enum value the values defined in your application.properties
I have entity Market and Event. Market entity has a column:
#ManyToOne(fetch = FetchType.EAGER)
private Event event;
Next I have a repository:
public interface MarketRepository extends PagingAndSortingRepository<Market, Long> {
}
and a projection:
#Projection(name="expanded", types={Market.class})
public interface ExpandedMarket {
public String getName();
public Event getEvent();
}
using REST query /api/markets?projection=expanded&sort=name,asc I get successfully the list of markets with nested event properties ordered by market's name:
{
"_embedded" : {
"markets" : [ {
"name" : "Match Odds",
"event" : {
"id" : 1,
"name" : "Watford vs Crystal Palace"
},
...
}, {
"name" : "Match Odds",
"event" : {
"id" : 2,
"name" : "Arsenal vs West Brom",
},
...
},
...
}
}
But what I need is to get list of markets ordered by event's name, I tried the query /api/markets?projection=expanded&sort=event.name,asc but it didn't work. What should I do to make it work?
Based on the Spring Data JPA documentation 4.4.3. Property Expressions
... you can use _ inside your method name to manually define traversal points...
You can put the underscore in your REST query as follows:
/api/markets?projection=expanded&sort=event_name,asc
Just downgrade spring.data.rest.webmvc to Hopper release
<spring.data.jpa.version>1.10.10.RELEASE</spring.data.jpa.version>
<spring.data.rest.webmvc.version>2.5.10.RELEASE</spring.data.rest.webmvc.version>
projection=expanded&sort=event.name,asc // works
projection=expanded&sort=event_name,asc // this works too
Thanks #Alan Hay comment on this question
Ordering by nested properties works fine for me in the Hopper release but I did experience the following bug in an RC version of the Ingalls release.bug in an RC version of the Ingalls release. This is reported as being fixed,
jira issue - Sorting by an embedded property no longer works in Ingalls RC1
BTW, I tried v3.0.0.M3 that reported that fixed but not working with me.
We had a case when we wanted to sort by fields which were in linked entity (it was one-to-one relationship). Initially, we used example based on https://stackoverflow.com/a/54517551 to search by linked fields.
So the workaround/hack in our case was to supply custom sort and pageable parameters.
Below is the example:
#org.springframework.data.rest.webmvc.RepositoryRestController
public class FilteringController {
private final EntityRepository repository;
#RequestMapping(value = "/entities",
method = RequestMethod.GET)
public ResponseEntity<?> filter(
Entity entity,
org.springframework.data.domain.Pageable page,
org.springframework.data.web.PagedResourcesAssembler assembler,
org.springframework.data.rest.webmvc.PersistentEntityResourceAssembler entityAssembler,
org.springframework.web.context.request.ServletWebRequest webRequest
) {
Method enclosingMethod = new Object() {}.getClass().getEnclosingMethod();
Sort sort = new org.springframework.data.web.SortHandlerMethodArgumentResolver().resolveArgument(
new org.springframework.core.MethodParameter(enclosingMethod, 0), null, webRequest, null
);
ExampleMatcher matcher = ExampleMatcher.matching()
.withIgnoreCase()
.withStringMatcher(ExampleMatcher.StringMatcher.CONTAINING);
Example example = Example.of(entity, matcher);
Page<?> result = this.repository.findAll(example, PageRequest.of(
page.getPageNumber(),
page.getPageSize(),
sort
));
PagedModel search = assembler.toModel(result, entityAssembler);
search.add(linkTo(FilteringController.class)
.slash("entities/search")
.withRel("search"));
return ResponseEntity.ok(search);
}
}
Used version of Spring boot: 2.3.8.RELEASE
We had also the repository for Entity and used projection:
#RepositoryRestResource
public interface JpaEntityRepository extends JpaRepository<Entity, Long> {
}
Your MarketRepository could have a named query like :
public interface MarketRepository exten PagingAndSortingRepository<Market, Long> {
Page<Market> findAllByEventByName(String name, Page pageable);
}
You can get your name param from the url with #RequestParam
This page has an idea that works. The idea is to use a controller on top of the repository, and apply the projection separately.
Here's a piece of code that works (SpringBoot 2.2.4)
import ro.vdinulescu.AssignmentsOverviewProjection;
import ro.vdinulescu.repository.AssignmentRepository;
import org.apache.commons.lang3.StringUtils;
import org.springframework.data.domain.Page;
import org.springframework.data.domain.PageRequest;
import org.springframework.data.domain.Pageable;
import org.springframework.data.domain.Sort;
import org.springframework.data.projection.ProjectionFactory;
import org.springframework.data.web.PagedResourcesAssembler;
import org.springframework.hateoas.EntityModel;
import org.springframework.hateoas.PagedModel;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
#RepositoryRestController
public class AssignmentController {
#Autowired
private AssignmentRepository assignmentRepository;
#Autowired
private ProjectionFactory projectionFactory;
#Autowired
private PagedResourcesAssembler<AssignmentsOverviewProjection> resourceAssembler;
#GetMapping("/assignments")
public PagedModel<EntityModel<AssignmentsOverviewProjection>> listAssignments(#RequestParam(required = false) String search,
#RequestParam(required = false) String sort,
Pageable pageable) {
// Spring creates the Pageable object correctly for simple properties,
// but for nested properties we need to fix it manually
pageable = fixPageableSort(pageable, sort, Set.of("client.firstName", "client.age"));
Page<Assignment> assignments = assignmentRepository.filter(search, pageable);
Page<AssignmentsOverviewProjection> projectedAssignments = assignments.map(assignment -> projectionFactory.createProjection(
AssignmentsOverviewProjection.class,
assignment));
return resourceAssembler.toModel(projectedAssignments);
}
private Pageable fixPageableSort(Pageable pageable, String sortStr, Set<String> allowedProperties) {
if (!pageable.getSort().equals(Sort.unsorted())) {
return pageable;
}
Sort sort = parseSortString(sortStr, allowedProperties);
if (sort == null) {
return pageable;
}
return PageRequest.of(pageable.getPageNumber(), pageable.getPageSize(), sort);
}
private Sort parseSortString(String sortStr, Set<String> allowedProperties) {
if (StringUtils.isBlank(sortStr)) {
return null;
}
String[] split = sortStr.split(",");
if (split.length == 1) {
if (!allowedProperties.contains(split[0])) {
return null;
}
return Sort.by(split[0]);
} else if (split.length == 2) {
if (!allowedProperties.contains(split[0])) {
return null;
}
return Sort.by(Sort.Direction.fromString(split[1]), split[0]);
} else {
return null;
}
}
}
From Spring Data REST documentation:
Sorting by linkable associations (that is, links to top-level resources) is not supported.
https://docs.spring.io/spring-data/rest/docs/current/reference/html/#paging-and-sorting.sorting
An alternative that I found was use #ResResource(exported=false).
This is not valid (expecially for legacy Spring Data REST projects) because avoid that the resource/entity will be loaded HTTP links:
JacksonBinder
BeanDeserializerBuilder updateBuilder throws
com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot construct instance of ' com...' no String-argument constructor/factory method to deserialize from String value
I tried activate sort by linkable associations with help of annotations but without success because we need always need override the mappPropertyPath method of JacksonMappingAwareSortTranslator.SortTranslator detect the annotation:
if (associations.isLinkableAssociation(persistentProperty)) {
if(!persistentProperty.isAnnotationPresent(SortByLinkableAssociation.class)) {
return Collections.emptyList();
}
}
Annotation
#Retention(RetentionPolicy.RUNTIME)
#Target(ElementType.FIELD)
public #interface SortByLinkableAssociation {
}
At your project incluide #SortByLinkableAssociation at linkable associations that whats sort.
#ManyToOne(fetch = FetchType.EAGER)
#SortByLinkableAssociation
private Event event;
Really I didn't find a clear and success solution to this issue but decide to expose it to let think about it or even Spring team take in consideration to include at nexts releases.
I have a yaml file setup like this:
system:
locators:
- first.com
- 103
- 105
- second.com
- 105
I want to load this as #autowired configuration that looks something like this:
#Autowired
List<Locator> locators;
I'm thinking that the Locator class would look something like this:
class Locator {
String name;
List<String> ports;
}
But I'm not sure how to put this all together. Any help is appreciated!
Firstly, I believe your yaml file structure is invalid. In your Locator class you have given names to the fields - you should do the same in your yaml file. In the end it should look like this:
system:
locators:
- name: first.com
ports:
- 103
- 105
- name: second.com
ports:
- 105
Secondly, you can leverage Spring Boot's rather advanced properties-to-bean auto mapping. As in every Spring Boot app, you need to annotate your main class with #SpringBootApplication. Then, you can create a class representing your properties structure:
#Configuration
#ConfigurationProperties(prefix = "system")
public class SystemProperties {
private List<Locator> locators;
public List<Locator> getLocators() {
return locators;
}
public void setLocators(List<Locator> locators) {
this.locators = locators;
}
public static class Locator {
private String name;
private List<String> ports;
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public List<String> getPorts() {
return ports;
}
public void setPorts(List<String> ports) {
this.ports = ports;
}
}
}
Notice the #ConfigurationProperties annotation that defines the prefix which is the root of this classe's mapped configuration. It can of course be any node in a yaml tree of properties, not necessarily the main level as in your case.
For further reading I would suggest this blog post and the official documentation as there are more ppowerful abilities when it comes to property bean mapping.