I'm using jackson to retrive file .yml into pojo java.
It's working fine.
how could I handle environment variables in .yml when read to pojo? jackson has this implementation?
example:
attr1: ${ENV_ATTR} # read this value from environment when has ${}
Dependencie
implementation 'com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.1'
Implementation code
var fileYml = new File(getClass().getClassLoader().getResource("file.yml").toURI());
var mapper = new ObjectMapper(new YAMLFactory());
mapper.setPropertyNamingStrategy(PropertyNamingStrategies.KEBAB_CASE);
var entityFromYml = mapper.readValue(fileYmlContent, MyEntity.class);
note: I'm not using spring or spring boot.
There's Apache Commons' StringSubstitutor which does the job.
You can either pre-process the input string with it, or post-process each loaded string.
Post-processing obviously works only on values that load into Strings so you can't use it e.g. on a value that loads into an int.
Pre-processing, on the other hand, is dangerous because it doesn't protect you against YAML special characters in the env variable's value. In your example, set ENV_ATTR to
foo
bar: baz
and after substitution, you will have
attr1: foo
bar: baz
which might not be desired.
If you want to guard against that but also want to substitute in non-String values, you'll need to use SnakeYAML's API directly and specialize its Composer. Jackson is an abstraction API on top of SnakeYAML that restricts what you can do, so this is not possible with Jackson.
Related
I am using OpenAPI / Swagger to specify my API.
One thing that I could not find out is how to specify a Set.
I am using https://editor.swagger.io/ and I typed in the whole API. For a property that I want to specify as Set I wrote the following:
myProperty:
uniqueItems: true
type: array
description: some description
items:
type: string
I would have guessed that uniqueItems does the trick and a Set is generated, but this is not the case. Instead the following code is generated:
#JsonProperty("myProperty")
private List<String> myProperty = null;
Is there a way to generate something like
#JsonProperty("myProperty")
private Set<String> myProperty = null;
instead?
I found a possible solution here in SO, but this requires some configuration in a pom.xml. However, the online editor that I am using gives me only the option to generate code for different platforms but does not accept a pom file.
OpenAPI (version 3) supports the following data types:
string
number
integer
boolean
array
object
There is no support for set data type in OpenAPI v3. The closest data type is an array with property uniqueItems set to true (as you've suggested). But it's still an array with a constraint on the uniqueness of its items, not a set.
So, your request cannot be resolved on the OpenAPI level.
However, there might be an option on the code generator level, and you would need to address the issue to the code generator of your choice.
I have a property that looks like:
#Value(...)
Map<String, List<String>> classToMethods;
I want to populate this map via my application.yml in a Spring Boot application via the #Value("...") annotation. Some sample values are:
"java.lang.Double" => ["parseDouble", "toString"]
"java.lang.String" => ["toUpperCase"]
How do I represent this in my application.yml file? Also, I would like to put placeholder for environmentally injected values from our deployment tool. So for example, I currently have:
some:
property: ${ENV_VALUE:default_value_if_no_env}
Basically, this is the syntax I use for injecting values for the specific environment(value for ENV_VALUE varies for each environment). I am looking for a way to format the YML map such that this kind of environment replacements are also possible.
Alternative I am thinking of is just creating a String and manually parsing it during constructor injection but would like to avoid if possible.
I have a yaml file which consist of placeholders to be taken from environment variable. I want to parse it to a custom class. Currently I am using snakeyaml library to parse it and its populating the bean correctly, how can I resolve environment variables using snakeyaml or any other library in Java.
datasource:
url: {url_environment_variable}
foo: bar
username: {user_name_environment_variable}
password: {password_environment_variable}
#Getter
#Setter
public class DataSource {
private String url;
private String foo;
private String username;
private String password;
}
Parsing code below
Constructor c = new Constructor(MyDataSource.class);
Yaml yaml = new Yaml(c);
MyDataSource myData = yaml.loadAs(inputStream, MyDataSource.class);
The problem is I am yet to find a way to resolve placeholders. People were able to solve it using python and is available in question -
How to replace environment variable value in yaml file to be parsed using python script
How can I do the same in Java. I can add a new dependency if required.
PS - It's not a Spring Boot Project so standard Spring placeholder replacements can not be used.
The easiest way would be to do it in two passes. First deserialize into MyDataSource as you’re doing already. Then use reflection to iterate over all fields of the instance, and if the value starts with a curly brace and ends with one, extract the key, and get the value from System.getenv map. See this answer for code.
If you want to do it in one pass, then you need to use the snakeyaml event-driver parser. For every key, you resolve the value as described above, and then based on the key name, set the corresponding field in the MyDataSource class, either using reflection, or simple if-else. For an example of the event-driven parser, see this class; it’s not Java, it’s Kotlin, but it may be the only JVM language example you’ll find.
I use #JsonProperty to serialize data from a JSON through Spring RestTemplate's exchange.
#JsonProperty("ip_address")
public String ipAddress;
#JsonProperty("port")
public Integer port;
I need this property to recognize both upper and lowercase versions of the attribute names i.e. "ip_address" and "IP_ADDRESS" set in #JsonProperty should be recognized.
I have tried the following and none worked:
#JsonFormat(with=JsonFormat.Feature.ACCEPT_CASE_INSENSITIVE_PROPERTIES) applied on a class level does not work with existing issue reported in GitHub. Also does not work when applied to each attribute in the model class.
use MapperFeature.ACCEPT_CASE_INSENSITIVE_PROPERTIES on an ObjectMapper bean.
This example using ObjectMapper and RestTemplate also didn't work even with a combination code with the example on the item before.
All three just have null values for their respective attributes because I disabled error on unknown attributes (a.k.a. the different letter case) for the template.
You can tell jackson to convert all your property names into e.g. SNAKE_CASE variants and set your #JsonProperty accordingly:
Example:
in spring boot set the property in application.properties
spring.jackson.property-naming-strategy=SNAKE_CASE
or you can enable it for just your single class and annotate the class with:
#JsonNaming(PropertyNamingStrategy.SNAKE_CASE.class)
and then set #JsonProperty:
#JsonProperty(vale="ip_address")
I was able to get this working without changing my original pojo class #JsonProperty configs. Using the Object Mapper and Rest Template example you linked to but instead of a Property Naming strategy use the case insensitive mapper feature
ObjectMapper mapper = new ObjectMapper();
mapper.configure(MapperFeature.ACCEPT_CASE_INSENSITIVE_PROPERTIES, true);
For Spring Boot App to Ignore Case for JSON attributes names:
Step 1: Make sure your POJO/Domain/Model class has a constructor with respective parameters, and remove any zero arg constructor.
Step 2: Add Jackson lib as a dependency
Ex:
<dependency>
<groupId>com.fasterxml.jackson.module</groupId>
<artifactId>jackson-module-jaxb-annotations</artifactId>
<version>2.6.5</version>
</dependency>
Step 3: Enable in application.properties file as below
spring.jackson.mapper.accept_case_insensitive_properties=true
we have a table with 350+ columns. pojo class is generated and getters order get messed up. trying to use csvmapper from jackson, but it generates csv based on getter order. #JsonPropertyOrder is also not use feasible because of many columns.we maintain column ordering in xml and can generate field order array at runtime. can we override at runtime to provide array of fieldnames for property ordering? can we customize using annotation introspector?
What you are looking for is called a MappingFeature. You need to disable alphanumeric sorting of properties, which is enabled by default:
CsvMapper mapper = new CsvMapper();
mapper.disable(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY);
More on this you can find here: Add a feature in CsvSchema to allow definition of ordering #42
Just in case you get here in 2020, this comes straight from the documentation.
So how do you get a CSV Schema instance to use? There are 3 ways:
Create schema based on a Java class
Build schema manually.
Use the first line of CSV document to get the names (no types) for Schema Here
is code for above cases:
// Schema from POJO (usually has #JsonPropertyOrder annotation)
CsvSchema schema = mapper.schemaFor(Pojo.class);
// Manually-built schema: one with type, others default to "STRING"
CsvSchema schema = CsvSchema.builder()
.addColumn("firstName")
.addColumn("lastName")
.addColumn("age", CsvSchema.ColumnType.NUMBER)
.build();
// Read schema from the first line; start with bootstrap instance
// to enable reading of schema from the first line
// NOTE: reads schema and uses it for binding
CsvSchema bootstrapSchema = CsvSchema.emptySchema().withHeader();
ObjectMapper mapper = new CsvMapper();
mapper.readerFor(Pojo.class).with(bootstrapSchema).readValue(json);
Note that #JsonPropertyOrder does not necessarily have to include all properties, just ones you are to include for serialization. But to indicate what is to be serialized you may need to use combination of #JsonProperty (to indicate properties to serialize) and different visibility for inclusion (either via ObjectMapper.setVisibility() for defaults, or via #JsonAutoDetect for per-POJO).
But assuming you do not want to use #JsonPropertyOrder, you can:
Override method in JacksonAnnotationIntrospector that reads the annotation, provide your own implementation that uses other sources (does not need to come from annotations at all)
If using Jackson 2.8.0, there is new way to specify per-class defaults for some things (see ObjectMapper.configOverride() object), including property order
Similarly you could override method that looks for #JsonProperty (findNameForDeserialization() and/or findNameForSerialization()) if you want to use custom criteria for inclusion/exclusion.
There are other mechanisms for inclusion/exclusion as well, like JSON Views (#JsonView), JSON Filters.
I believe your only choice here is uniVocity-parsers, as it allows you to choose which columns to write and in what order:
CsvWriterSettings settings = new CsvWriterSettings();
// Sets the file headers (used for selection only, these values won't be written automatically)
settings.setHeaders("Year", "Make", "Model", "Description", "Price");
// Selects which fields from the input should be written. In this case, fields "make" and "model" will be empty
// The field selection is not case sensitive
settings.selectFields("description", "price", "year");
//configures the writer process java beans with annotations (assume TestBean has a few annotated fiedls)
settings.setRowWriterProcessor(new BeanWriterProcessor<TestBean>(TestBean.class));
// Creates a writer with the above settings;
CsvWriter writer = new CsvWriter(new File("/path/to/output.csv"), settings);
// Writes the headers specified in the settings
writer.writeHeaders();
//creates a bean instance for writing
TestBean bean = new TestBean();
bean.setPrice(new BigDecimal("500.33"));
bean.setDescription("Blah,blah");
bean.setYear(1997);
//writes it
writer.processRecord(bean);
writer.close();
Hope it helps.
Disclosure: I'm the author of this libary, it's open-source and free (Apache 2.0 License)