Develop Reference

How to best share and store enum-like data between microservices?

Situation:
A microservices architecture with:
Microservices are written in Java with Spring Boot and Hibernate
An options-service which provides information about Options via a REST interface
Option data which could be modelled as follows e.g. via an enum (representative)
Has some attributes like enabled status associated with it. It is unlikely that more attributes belonging to Option will be added in future which have to be directly tied to it.
New Options and OptionTypes have to be seldom added. They will never be deleted (just disabled if need be).
An Option should have a unique identifier which can be referenced
There should be no UNKOWN option if possible
enum OptionType {
TYPE_A,
TYPE_B,
TYPE_C;
}
enum Option {
TYPE_A_X1(TYPE_A),
TYPE_A_X2(TYPE_A),
TYPE_B_Z1(TYPE_B, false),
TYPE_B_Z2(TYPE_B);
TYPE_C_U1(TYPE_C);
TYPE_C_U2(TYPE_C);
TYPE_C_U3(TYPE_C);
private final OptionType type;
private final boolean enabled;
Option(OptionType type){
this.type = type;
this.enabled = true;
}
Option(OptionType type, boolean enabled){
this.type = type;
this.enabled = enabled;
}
}
Other microservices (currently 3) need to be able to access Option data. They need to know which Options exist and somehow reference an Option e.g. via its name or identifier
One of those services (example-service) needs to provide the Option data type as filter settings in its own REST interface to the outside world. The filter object in JSON would look something like this:
{
"typeA": "TYPE_A_X1",
"typeB": "TYPE_B_Z2",
"typeC": [ "TYPE_C_U1", "TYPE_C_U2"]
// more filter settings
}
Different approaches of storing and sharing this Option data between microservices as I see it:
options-service stores Option data in its own database. If I read my data from database into my Hibernate entities Option is only a String everywhere from there on.
Pro:
Easy to rename, add and remove Options
Con:
No type safety when working with Option in code e.g. when deserialising a response containing Option
example-service cannot easily offer Option data in its OpenAPI doc (just Strings)
Microservices need to query and cache Option data from options-service
Option data only lives in source code in an enum as e.g. modelled above and is shared between different services via a lib.
Pro:
Type safety everywhere where Options are needed. Really useful when deserializing reponses containing Options data but also for generating OpenAPI doc
Microservices can still reference an Option in its database via its name since it is unique
Con:
Editing the name of an Option is difficult
Removing an Option not possible
If a new Option/OptionType is added the order in which the services relying on that lib update their lib version matters. Since we cannot deserialize responses into an UNKNOWN Option type.
There is also the possibility of a mixed database and enum solution which comes with the big drawback that one has to maintain both sources of truth.
What is the best way to store and share the Option data between microservices? What is best practice?

While enums are very helpful in most programming languages, they can become a nightmare in inter-service communication.
If you use an enum in your microservice endpoints, it becomes part of the API specification. Changing them on one side will result in problems of serialization/deserialization on the other side.
So you should only use enums in your API if
you are definitely sure they will never change (e.g. weekdays) or
you are the owner of the code of all participating services and are able to orchestrate a big bang redeployment.
In these cases, you may create a shared lib between your services.
If this does not apply, make your life easier and consider most enum-like data (e.g. currency codes, country codes) as simple string values that are subject to change. If you want to have a central management for these values, why don't create a microservice for it that acts as a master? That's what microservice architectures are made for. Offer some endpoints that other services can query from time to time. You don't even need a persistence layer for that.

Another option : Use the first option, but with a specialised class.
This allow for type safety,
but does not require your enums to be known at compile time.
Still, if you need to check the validity of your Option instance, you need a service that has cached the list of allowed Options.
Example code:
import com.fasterxml.jackson.annotation.JsonValue;
public class Option {
#JsonValue
private String value;
Option(String value) {
this.value = value;
}
}

First Option :
Define or Create new Module / MicroServices / Project that only Contains your CommonMessage , String Message, Enum, Exception or everything that Common and reusable for your multiple microservices that using spring-boot.
But those Enum is not parameterizing so you can not Create Update Or Delete.
And you create your Option ENUM class for multiple purpose and you can inject those class from POM.xml as a dependency.
Second Option :
Store it in Database as an Entity Object , so you can manage those data.
if you want to use those data on other microservices, dont forget to use this artifact as dependency in POM.xml so other microservices can call your service or repository.

REST API Creation of Multiple Resources

I am currently working with an API where a POST request may create multiple resources, depending on the resource being passed. To give you an example, I have a Reservation resource, and two child resources, Ancillary and SpecialRequest. A reservation is uniquely identified by an alphanumeric string of 6 characters, and an Ancillary is identified by a unique ID within the reservation (i.e. ancillary IDs are only known to the parent reservation).
So, to create an Ancillary, my API endpoint looks like this:
POST /reservations/{reservationId}/ancillaries/
Usually REST states that the resource being created (Ancillary in this case) is the resource that should be returned. However, my use-case is somewhat more complicated than that, since the reservation system which my API is interfacing with is legacy, and is somewhat unpredictable.
There are certain ancillaries (bundles) which actually create multiple ancillaries. For example, an ancillary might be a package of two other ancillaries, which costs less than if you had to purchase the other two ancillaries. Moreover, an ancillary might also be linked automatically to a SpecialRequest.
I'm wondering what my options are ... so far I've come up with the following:
Return the entire Reservation, which is sure to include all sub resources which were created and/or modified as a result of the Ancillary creation. This is the "safest" option, but in doing so I wouldn't be able to tell the user which Ancillarys were created.
Return only the Ancillary which was created, however this approach is likely make my API less usable since the API user is extremely likely to perform a GET /reservations/{reservationId}
Return a Ancillary[] regardless of the ancillary type, although this still leaves out the link between the Ancillary and the SpecialRequest.
Thanks in advance for your thoughts and input

Know when value of any variable defined inside the class is changed

I have defined a class which acts like a model/pojo. The class has many keys/variable. I have implemented custom solution for storing the POJO on disk for future uses. Now what I want to do is that whenever any value in the class/POJO is changed, I should call a method which sync the fresh changes with file on disk.
I know I can define setter for each variable. But it's quite tedious to do for 100s of direct and sub fields, and even if I define setter for each field, I have to call sync function from all the setters.
What I need is single proxy setter or interceptor for all change pushes to variables in class.
I am using this in an android application, so whenever the user enters new details in his/her account I have to store those details at that specific instance of time for preventing the data loss. I am using GSON for serialising and de-serialising.
Sorry for using vague terminologies, never been to college :|.

The easiest solution is indeed to use a setter. You only have to create one for each field you want to monitor, and most IDEs generate them for you or you can use something like Koloboke, so it being tedious isn't really an argument.
A proxy class or reflection would also be possible, but that is pretty hacky. Another way would be an asynchronous watcher/worker that checks for changes in you POJO instances, but even that seems unnecessarily complicated.
Apart from that you might need to rethink your POJOs structure if it has that many fields.

The problem with persisting(in your case writting to a disk) entity on each property update is that most of the updates are modifying more then one property. So in case you have a code like this:
entity.setA(avalue);
entity.setb(bvalue);
entity.setc(cvalue);
You would write it to the disk 3 times, which is probably not a best way, as it takes more resources, and 2 out of 3 writes are unnecessary.
There are several ways to deal with it. Imagine you have some service for saving this data to a disk, lets name it entityRepository. So one option is manually call this entityRepository each time you want to save/update your entity. It seems to be very uncomfortable, comparing to calling this automatically on setter call, however, this approach clearly shows you when and why your entity is persisted/updated, in your approach it's unclear, and can lead to some problems future problems and mistakes, for example, in future you will decide that you now need to update one of the properties without immideately persisting, then it appears that you will need 2 setter, one with update, and one without...
Another way is to add version property, and when its setter is called inside this setter call entityRepository.save(this).
The other way is to look at AOP, however anyway I don't recommend persist entity on any change, without having control over it.

You are talking about data binding. There is no built-in way for that so you have indeed to sync it yourself. Look into How to Write a Property Change Listener. There are also lots of other approaches to this, but as said no built-in way.

Is using one class to store data safe?

So I was searching about storing data in one class, and found this. However, that's not what I'm looking for. What I wanted to know about this was whether it's bad practice, can cause performance issues in an application, or if there's another way to do it, etc... (and I'm not doing this on an Android).
Let's say I have a class that stores a HashMap<Enum, Object> and is initialized when I create the class in main.
public Main() {
// Creates the HashMap, initialized in the constructor 'MemoryContainer'
MemoryContainer m = new MemoryContainer();
m.getTestHash().put(SomeEnum.TEST, "Test"); // Using created HashMap
}
Other than casting the value every time I use it, would this cause major issues? If so, is there an alternative?

There's nothing wrong with storing static values in a class, however this is not a good practice.
To store the constants you should create an interface as every field in an interface is already a constant (public static final).
A better approach will be to store these values in properties files, and load them as needed.
A properties file can be stored externally and a person who isn't aware of your source code would be able to modify this properties file if needed. For example you can store database connection details in properties files and if server support admin determines that database instance is down, he/she can edit the properties file to point the application to a new one.
Finally for most flexibility you shouldn't store the configuration inside application at all. It can be stored in a database like MySql or in a fast data structure storage like Redis. This will allow multiple instances of your application to share the configuration data and it will also allow you to modify configuration on the fly by modifying them in the database.
Sometimes a Git repository is also used to store this kind of data (like in case of microservices). Git repository in addition to being shared among all the instances, also maintains the history of modifications.

I would not look too much at performance issues (of course, I do not know what else your application does or wants to do and how it achieves it).
What you should look at first is Mutability - in your example, nothing would stop me from changing the configuration at Runtime by calling
m.getTestHash().put(SomeEnum.TEST, "NotATestAnymore") - this would immediately change the behaviour for every other use of that specific setting.
I am also not sure why you would not just use a configuration class that would directly provide (typed) getters and, if you know all configuration settings at the launch of the app, one constructor containing all the settings.
Do you plan to read the configuration from an outside source (e.g. file)?

NO,
It won't cause major issues.
Also, it is a good practice to keep those variables (HashMap in your case) in a different class away from your main class (which contains your app logic).

Where to keep options values, paths to important files, etc [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
I'm creating a program that requires some options values to be set along with some paths to image files, path to a SQLite database, some information about the text on various buttons, information about which database to use (SQLite / MySQL for example), etc.
So far, I had a helper class in which I made everything static and I accessed it from everywhere in the program, but it was becoming a mess and I read that this is sort of bad practice since it doesn't follow object-oriented programming guidelines.
So what I did is make that class a singleton with its reference in my controller object. To access my options, I now call controller.getOptionName instead of calling Helper.getOptionName directly.
I have a feeling I'm approaching this the wrong way, especially because from what I've read, if many of my objects depend on a single class (the helper class), I'm not separating everything enough.
I don't know what I should be doing instead, is there a "standard" on where to keep all my options? I thought about using a XML file or something along those lines, but I'm going to end up having to access it from everywhere anyway, so it feels like this would create the same problem.

The Problem: Configuration Sprawl Over time, programs gain features and options. When they connect to external systems and services (e.g. databases, event brokers, cloud/web services), they must also keep a growing set of configs and credentials for those services handy.
The traditional places to store this information at runtime are global variables and OS environment variables. Both suck.
Config data logically is "global environment" or context in which the app is running, but you can't easily depend on either global or environment variables.
The other traditional mechanism, config files--whether XML, INI, .properties, or whatever--help store config data between runs, but do nothing to organize configs/options for the program code, or during its execution.
You can clean things up a bit by making options into properties of your application's classes. This is the traditional "next step." Unfortunately, it can take a lot of up-front "what goes where??" thinking. IMO, more than it's worth. Even if you get those choices right, it won't last. If you have a reasonably feature-ful app, the number of options and settings will become overwhelming over time. You'll spend a lot of time hand-coding defaults and options in object constructors and the arguments/code of other methods. Trying to perfectly partition configurations among classes not only expends effort, it can lead to highly interdependent classes. So much for clean encapsulation!
I've banged my head against this particular wall often, especially when aiming for code that has "reasonable" or "intelligent" defaults and behaviors for everything, that allows users to override defaults at any time, and that presents a simple interface that doesn't require understanding the complete interplay of app classes and components to use its services.
Solution: Delegate to a Config Object The best solution I've found is to encapsulate option/config data into its own designated object. A fancy way of describing this pattern: For configuration, settings, and option data, use delegation rather than inheritance or composition.
How you build a config mapping depends on the language you're working in. In many languages, constructing your own Config object gives you a nice "look":
if opts.verbose:
print "..."
which I find more readable than the more explicit "getter" opts.get("verbose") or "indexer" opts['verbose'] ways of accessing a property. But you usually don't have to make your own Config class, which basically is just a mapping.
● The Easy Way ● Use a generic mapping: e.g. in Python a dict, in Perl a %hash, in Java a Dictionary or HashMap. Even better, there are extensions of these designed for, or especially suited to, configuration data. In Python, e.g., I use stuf and TreeDict for their simple dot-access and other nice properties. In Java, Properties is a similar specific-for-configs extension. E.g.:
from stuf import stuf # stuf has attributes!
opts = stuf(
show_module=False, # comment explaining what show_module means
where=True, # ...
truncate=False, # ...
everything=False, # ...
allvars=False, # ...
allkeys=False, # ...
yaml=False, # ...
on=True, # ...
ret=None, # ...
)
if opts.truncate:
...
This way, all your config and option data is in one place, neatly accessible, and clearly delineated from all of the other program, class, instance, and function/method data it's used side-by-side with. This helps maintain clarity over time, as the program evolves. You can quickly determine "Is this part of the core data? Or is it related to the context in which the core data is being processed?"
To make things even better, if you pre-load config data from a config file, load or copy those values directly into your config object. And if you take arguments from the command line, load or copy those values directly into your config object. Now you have one unified source of all the "what does the user want me to do, with what options and settings?" information.
TL;DR - 90% of apps or services are just fine with a simple config/options mapping. Everything that follows is for advanced use cases. Because this was a design/patterns question, here's why this approach isn't a one-off, but extends to successively more sophisticated/intricate use cases.
● Per-Instance Config ● You can have multiple levels of config/option data. The most common use for this would be defaults set at a class or module level, then potentially different options for each instance. A server app might have an instance per user, with each user/instance needing its own customized settings. The config map is copied at instance creation/initialization, either automatically or explicitly.
●● Multiple Config Objects ●● You can partition config/option data into several config objects, if that makes sense. For example, you might partition options for data retrieval from those for data formatting. You can do this at the start of the design, but need not. You can start with one monolithic config object, then refactor over time (generally, as you start to refactor the underlying functions). Obviously you don't want to "go crazy" adding config objects, but you can have a few without adding much program complexity. If you partition config objects, you can proxy multiple config "domains" through a single API--giving you quality information decomposition internally, but a very simple outward appearance.
◆ Chain Gang ◆ More elegant than copying config data per instance: Use chainable or hierarchical mapping (e.g. in Python, ChainMap) that lets you "overlay" the values of one mapping with those of another (similar to "copy-on-write" schemes, or "union" and "translucent" file systems). Instance options then refer directly to class/default options--unless they are explicitly set, in which case they're instance-specific. Advantage: If class/default/global settings are changed during program execution, subsequent instance method invocations will "see" the changed defaults and use them (as long as they haven't been overridden at the instance level).
◆◆ Transient Config ◆◆ If you need configs/options changeable "on the fly"--say for the scope of a given method invocation--the method can extend the instance option chained mapping. In Python, that's what ChainMap.new_child() does. It sounds complicated, but as far as the method code is concerned, it's drop-dead-simple. There's still just a single config object to refer to, and whatever it says is the option, use it.
◆◆◆ Arbitrary Duration Overlay ◆◆◆ There's nothing magical about the temporal scope of method invocation. With the proper setup, any level of configuration can be transiently overlaid for as long as needed. For example, if there's some period during program run you'd like to turn on debugging, logging, or profiling, you can turn that on and off whenever you like--just for certain instances, or for all of them at once. This hors catégorie usage requires a Config object slightly beyond stock ChainMap--but not by much (basically just a handle to the chain mapping).
Happily, most code doesn't come close to needing these "black diamond" levels of config sophistication. But if you want to go three or four levels deep, delegating to separate config objects will take you there in a logical way that keeps code clean and orderly.

I would recommend placing your options into some sort of file (whether it be xml or a .properties) especially if the values can change (such as a database username, etc). I would also say that you can break up these configuration files by component. Just like you break up your code, the components that need database information likely do not need image paths. So you can have a file for database info, image path info, etc. Then have your components load the file that they need.
In the java-specific case, you can put these files on your classpath and have your components reference them. I like using .properties files because java has a nice class, Properties, for dealing with them.
So here's a tiny example for an image provider that gives you a BufferedImage, given the filename.
image.properties
icon.path=/path/to/my/icons
background.path=/path/to/my/backgrounds
Make sure this file is on your classpath. Then here's my provider class
public class ImageProvider {
private static final String PROP_FILE = "image.properties";
private static final String ICON_PATH = "icon.path";
private Properties properties;
public ImageProvider() {
properties = new Properties();
properties.load(getClass().getClassLoader().getResourceAsStream(PROP_FILE));
}
public BufferedImage getIcon(String icon) {
return ImageIO.read(properties.getProperty(ICON_PATH) + icon);
}
}

times when configuration was kept in static variables or when every component access them independently has ended long ago. since then mankind has invented IoC and DI. you take a DI library (e.g. spring), use it to load all the configuration (from files, jndi, environment, user, you name it) and inject it to every component that needs it. no manual resource loading, no static configuration - it's just evil