We are working on Spring based application which needs to access and read certain credentials stored in the yaml file which use the format below:
#yaml file content
...
secrets: username1:pw1, username2:pw2
...
we load the yaml file using PropertySource annotation like this
#PropertySources({
#PropertySource(value = "file:/someSecretPath/secrets.yaml", ignoreResourceNotFound = true) // secret yaml file
})
I have tried using the following code to parse it into array of map object since it is a list contains 2 object type,
#Repository
public class getSecretClass {
#Value("${secrets}")
private Map<String,String>[] credentials;
}
but it fails with this error:
... 'java.lang.string' cannot be converted into java.lang.map[] ....
On the other hand,
I tested reading from simple string array like this one:
#yaml file content
...
secrets: text1, text2
...
and the code below would work with no issue:
#Repository
public class getSecretClass {
#Value("${secrets}")
private String[] credentials;
...
private someReadSecretMethod(){
System.out.println( credentials[0] );
}
}
it will print "text1" as expected. But we do need to store the credentials as pairs in my use case, a plain text would not work for me.
Any suggestion or advice would be much appreciated !
Why not do something easier on the eyes, easier to program if not a little verbose:
secrets:
- username: username1
password: pwd1
- username: username2
password: pwd2
Have a class for your secret:
public class Secret {
private String username;
private String password;
public String toString() { return username + ":" password; }
}
And then injects as
#Value("${secrets}")
List<Secret> secrets;
If you must use map. I think array of map are probably not what you desired. Maybe you need one map which contains two keys, username1 and username2. The following might let you know how to obtain the map.
The yml file content is like that.
secrets: "{username1:pw1,username2:pw1}"
The Java code field is like that.
#Value("#{${secrets}}")
public Map<Integer,Integer> secrets;
Related
i am trying to read a Yaml template and replace certain fields in the template dynamically and create a new Yaml file. My resultant yaml file should reflect the template in all aspects including the double quotes. But I am missing double quotes for the required fields when I use snake yaml.
Can anyone please suggest to resolve this issue?
Example :
My yaml template is as shown below:
version: snapshot-01
kind: sample
metadata:
name: abc
groups:
id: "1000B"
category: category1
I am reading the above template and replacing the required fields dynamically as shown below.
Yaml yaml = new Yaml();
InputStream inputStream = this.getClass().getClassLoader().getResourceAsStream(yamlTemplateLocation);
Map<String, Object>yamlMap = yaml.load(inputStream);
Now I am replacing the required fields as shown below
yamlMap.put("version","v-1.0");
Map<String, Object> metadata = (Map<String, Object>) yamlMap.get("metadata");
metadata.put("name", "XYZ");
Map<String, Object> groups = (Map<String, Object>) yamlMap.get("groups");
groups.put("id","5000Z");
groups.put("category","newCategory");
DumperOptions options = new DumperOptions();
options.setDefaultFlowStyle(DumperOptions.FlowStyle.BLOCK);
options.setPrettyFlow(true);
Yaml yaml = new Yaml(options);
String output = yaml.dump(map);
System.out.println(output);
I am expecting output as shown below
Expected Output :
version: v-1.0
kind: sample
metadata:
name: XYZ
groups:
id: "5000Z"
category: newCategory
But I am actually getting output as below
version: v-1.0
kind: sample
metadata:
name: XYZ
groups:
id: 5000Z
category: newCategory
My problem here is, I am missing the double quotes for "id" node in the new yaml file.
When I use, options.setDefaultScalarStyle(ScalarStyle.DOUBLE_QUOTED), I am getting all fields double quoted which is not required. I need double quotes for id field only.
Can anyone please advice to resolve this issue.
Thanks
If your input is a template, it might be better to use a templating engine. As simple example, MessageFormat would allow you to write id: "{0}" and then interpolate the actual value into it, keeping the double quotes. You could use more sophisticated templating depending on your use-case.
That being said, let's look at how to do it with SnakeYAML:
If you want to control how a single item is rendered as scalar, you have to define a class like this:
class QuotedString {
public String value;
public QuotedString(String value) {
this.value = value;
}
}
And then create a custom representer for it:
class MyRepresenter extends Representer {
public MyRepresenter() {
this.representers.put(QuotedString.class, new RepresentQuotedString());
}
private class RepresentQuotedString implements Represent {
public Node representData(Object data) {
QuotedString str = (QuotedString) data;
return representScalar(
Tag.STR, str.value, DumperOptions.ScalarStyle.DOUBLE_QUOTED);
}
}
}
Modify your code to use the new class:
groups.put("id", new QuotedString("5000Z"));
And finally, instruct SnakeYAML to use your representer:
Yaml yaml = new Yaml(new MyRepresenter(), options);
This should do it.
I am using Jackson API in Vert.x core bundle to decode a JSON string to a java object. Normally this works in almost all cases but for one particular use case. I am constructing the JSON string from the user entered form data and using below line to map it to a java object.
MyClass myClass = io.vertx.core.json.Json.mapper.readValue(jsonString, MyClass.class)
MyClass.java
public class MyClass{
private String ID;
private String description;
//getter and setter methods
}
Input string
{
"description": "“success”,\n “data”: [\n {\n “severity”: “2\",\n “createdby”: “Online user\",\n “product”: “Google map”,\n “description”: “test”,",
"ID": "74085652"
}
When the value of the field description is another JSON string then the mapping fails with an exception.
com.fasterxml.jackson.databind.JsonMappingException: Unexpected character ('“' (code 8220 / 0x201c)): was expecting either valid name character (for unquoted name) or double-quote (for quoted) to start field name
I have tried adding these com.fasterxml.jackson.core.JsonParser.Feature configurations but it didn't work.
Json.mapper.configure(ALLOW_UNQUOTED_CONTROL_CHARS, true);
Json.mapper.configure(ALLOW_UNQUOTED_FIELD_NAMES, true);
Json.mapper.configure(ALLOW_BACKSLASH_ESCAPING_ANY_CHARACTER, true);
I am using Vert.x 3.4.1 and Java 8
Am I missing anything here? Any help is greatly appreciated!
i need get string from url where is "?" but controller does not accept "?"
I need send something like "Hello world?"
but I get only "Hello world"
I find solution for dot(.) -- value = "{textToTransform:.+}"
#RestController
#RequestMapping(textTransformCtrl.BASE_URI)
public class textTransformCtrl {
#Autowired
private TextTransformatorService textTransformatorService;
public static final String BASE_URI = "transform/text";
#RequestMapping(value = "{textToTransform:.+}")
public String getText(#PathVariable final String textToTransform) {
return textTransformatorService.transformText(textToTransform);
}
}
Question mark is a reserved character in URLs. It indicates where the query string starts.
If you want to send a ? as a parameter value and be able to read it on server side, you must URL encode it.
When URL encoded, Hello world? becomes Hello+world%3F.
You can use %3F to manually encode it or take a look at UriBuilder
The title talks by itself, I have a Config object (from https://github.com/typesafehub/config) and I want to pass it the a constructor which only supports java.util.Properties as argument.
Is there an easy way to convert a Config to a Properties object ?
Here is a way to convert a typesafe Config object into a Properties java object. I have only tested it in a simple case for creating Kafka properties.
Given this configuration in application.conf
kafka-topics {
my-topic {
zookeeper.connect = "localhost:2181",
group.id = "testgroup",
zookeeper.session.timeout.ms = "500",
zookeeper.sync.time.ms = "250",
auto.commit.interval.ms = "1000"
}
}
You can create the corresponding Properties object like that:
import com.typesafe.config.{Config, ConfigFactory}
import java.util.Properties
import kafka.consumer.ConsumerConfig
object Application extends App {
def propsFromConfig(config: Config): Properties = {
import scala.collection.JavaConversions._
val props = new Properties()
val map: Map[String, Object] = config.entrySet().map({ entry =>
entry.getKey -> entry.getValue.unwrapped()
})(collection.breakOut)
props.putAll(map)
props
}
val config = ConfigFactory.load()
val consumerConfig = {
val topicConfig = config.getConfig("kafka-topics.my-topic")
val props = propsFromConfig(topicConfig)
new ConsumerConfig(props)
}
// ...
}
The function propsFromConfig is what you are mainly interested in, and the key points are the use of entrySet to get a flatten list of properties, and the unwrapped of the entry value, that gives an Object which type depends on the configuration value.
You can try my scala wrapper https://github.com/andr83/scalaconfig. Using it convert config object to java Properties is simple:
val properties = config.as[Properties]
As typesafe config/hocon supports a much richer structure than java.util.propeties it will be hard to get a safe conversion.
Or spoken otherwise as properties can only express a subset of hocon the conversion is not clear, as it will have a possible information loss.
So if you configuration is rather flat and does not contain utf-8 then you could transform hocon to json and then extract the values.
A better solution would be to implement a ConfigClass and populate the values with values from hocon and passing this to the class you want to configure.
It is not possible directly through typesafe config. Even rending the entire hocon file into json does provide a true valid json:
ex:
"play" : {
"filters" : {
"disabled" : ${?play.filters.disabled}[
"play.filters.hosts.AllowedHostsFilter"
],
"disabled" : ${?play.filters.disabled}[
"play.filters.csrf.CSRFFilter"
]
}
}
That format is directly from Config.render
as you can see, disabled is represented twice with hocon style syntax.
I have also had problems with rendering hocon -> json -> hocon
Example hocon:
http {
port = "9000"
port = ${?HTTP_PORT}
}
typesafe config would parse this to
{
"http": {
"port": "9000,${?HTTP_PORT}"
}
}
However if you try to parse that in hocon - it throws a syntax error. the , cannot be there.
The hocon correct parsing would be 9000${?HTTP_PORT} - with no comma between the values. I believe this is true for all array concatenation and substitution
Need a quick help. I am a newbie in QuickFixJ. I have a FIX message in a txt file. I need to convert that into FIX50SP2 format. I am enclosing the code snippet.
String fixMsg = "1128=99=25535=X49=CME34=47134052=20100318-03:21:11.36475=20120904268=2279=122=848=336683=607400107=ESU2269=1270=140575271=152273=121014000336=2346=521023=1279=122=848=336683=607401107=ESU2269=1270=140600271=206273=121014000336=2346=681023=210=159";
System.out.println("FixMsg String:"+fixMsg);
Message FIXMessage = new Message();
DataDictionary dd = new DataDictionary("FIX50SP2.xml");
FIXMessage.fromString(fixMsg, dd, false);
System.out.println("FIXMessage Output:" + FIXMessage.toString()); // Print message after parsing
MsgType msgType = new MsgType();
System.out.println(FIXMessage.getField(msgType));
Here is the output:
FixMsg String:1128=99=15835=X49=CME34=47164052=2012090312102051175=20120904268=1279=122=848=336683=607745107=ESU2269=1270=140575271=123273=121020000336=2346=501023=110=205
FIXMessage Output:9=6135=X34=47164049=CME52=2012090312102051175=20120904268=110=117
quickfix.FieldNotFound: Field [35] was not found in message.
at quickfix.FieldMap.getField(FieldMap.java:216)
at quickfix.FieldMap.getFieldInternal(FieldMap.java:353)
at quickfix.FieldMap.getField(FieldMap.java:349)
at MainApp.main(MainApp.java:52)
I want to extract MsgType field (field 35). Could you please tell me where I am wrong? The thing I have observed is that after parsing to FIX50SP2 format, the convert FIX message is missing many data element (for details see the output)
Thanks
Like others mentioned the MsgType is an header field and you get it by using the following
String msgType = null;
if(FIXMessage.getHeader().isSetField(MsgType.FIELD)) {
msgType = FIXMessage.getHeader().getString(MsgType.FIELD);
}
System.out.println("MsgType is " + msgType);`
The reason you are missing many data element after parsing is, probably your message have some custom tags(like tag 2346), which is not defined in your data dictionary(FIXSP02.xml). hence the parsing of those tags failed and missing in the output.
To fix this, get the data dictionary from the party that is sending you the message and use it to parse the message
I'm not familiar with FIX messages and QuickFixJ, but glancing at the Javadoc, it seems like you should use the identifyType method :
String fixMsg = "1128=99=25535=X49=CME34=47134052=20100318-03:21:11.36475=20120904268=2279=122=848=336683=607400107=ESU2269=1270=140575271=152273=121014000336=2346=521023=1279=122=848=336683=607401107=ESU2269=1270=140600271=206273=121014000336=2346=681023=210=159";
MsgType msgType = Message.identifyType(fixMsg);
You may find FixB framework useful as it deals well with non-standard use cases of FIX.
As in your case, to extract only data you are interested in, you need to define a class that will represent this data and to bind it to FIX using annotations. E.g.:
#FixBlock
public class MDEntry {
#FixField(tag=269) public int entryType; // you could define an enum type for it as well
#FixField(tag=278) public String entryId;
#FixField(tag=55) public String symbol;
}
...
FixFieldExtractor fixExtractor = new NativeFixFieldExtractor();
List<MDEntry> mdEntries = fixExtractor.getGroups(fixMsg, List.class, 268, FixMetaScanner.scanClass(MDEntry.class))
In more common cases, FixSerializer interface should be used, but it requires a message with MsgType(35) tag and a class annotated with #FixMessage(type="...") accordingly. E.g.:
#FixMessage(type="X")
public class MarketData {
#FixGroup(tag=268) public List<MDEntry> entries;
}
...
FixMetaDictionary fixMetaDictionary = FixMetaScanner.scanClassesIn("my.fix.classes.package");
FixSerializer fixSerializer = new NativeFixSerializer("FIX.5.0.SP2", fixMetaDictionary);
MarketData marketData = fixSerializer.deserialize(fixMsg);
I hope you will find it useful.
If you need just a MsgTyp, you're sure the message is correct and you do not need any other field from the message, then I would recommend extracting MsgType from string using regexp.
e.g.: \u000135=(\w+)\u0001
It is MUCH FASTER than parsing (and validating) a string via QuickFix.