I have currently two queues and items traveling between them. Initially, an item gets put into firstQueue, then one of three dedicated thread moves it to secondQueue and finally another dedicated thread removes it. These moves obviously include some processing. I need to be able to get the status of any item (IN_FIRST, AFTER_FIRST, IN_SECOND, AFTER_SECOND, or ABSENT) and I implemented it manually by doing the update of the statusMap where the queue gets modified like
while (true) {
Item i = firstQueue.take();
statusMap.put(i, AFTER_FIRST);
process(i);
secondQueue.add(i);
statusMap.put(i, IN_SECOND);
}
This works, but it's ugly and leaves a time window where the status is inconsistent. The inconsistency is no big deal and it'd solvable by synchronization, but this could backfire as the queue is of limited capacity and may block. The ugliness bothers me more.
Efficiency hardly matters as the processing takes seconds. Dedicated threads are used in order to control concurrency. No item should ever be in multiple states (but this is not very important and not guaranteed by my current racy approach). There'll be more queues (and states) and they'll of different kinds (DelayQueue, ArrayBlockingQueue, and maybe PriorityQueue).
I wonder if there's a nice solution generalizable to multiple queues?
Does it make sense to wrap the queues with logic to manage the Item status?
public class QueueWrapper<E> implements BlockingQueue<E> {
private Queue<E> myQueue = new LinkedBlockingQueue<>();
private Map<E, Status> statusMap;
public QueueWrapper(Map<E, Status> statusMap) {
this.statusMap = statusMap;
}
[...]
#Override
public E take() throws InterruptedException {
E result = myQueue.take();
statusMap.put(result, Status.AFTER_FIRST);
return result;
}
That way status management is always related to (and contained in) queue operations...
Obviously statusMap needs to be synchronized, but that would be an issue anyway.
I see that your model might be improved in consistency, state control, and scaling.
A way of to implement this is accouple the item to your state, enqueue and dequeue this couple and create a mechanism to ensure state change.
My proposal can be see in figure below:
According with this model and your example, we can to do:
package stackoverflow;
import java.util.concurrent.LinkedBlockingQueue;
import stackoverflow.item.ItemState;
import stackoverflow.task.CreatingTask;
import stackoverflow.task.FirstMovingTask;
import stackoverflow.task.SecondMovingTask;
public class Main {
private static void startTask(String name, Runnable r){
Thread t = new Thread(r, name);
t.start();
}
public static void main(String[] args) {
//create queues
LinkedBlockingQueue<ItemState> firstQueue = new LinkedBlockingQueue<ItemState>();
LinkedBlockingQueue<ItemState> secondQueue = new LinkedBlockingQueue<ItemState>();
//start three threads
startTask("Thread#1", new CreatingTask(firstQueue));
startTask("Thread#2", new FirstMovingTask(firstQueue, secondQueue));
startTask("Thread#3", new SecondMovingTask(secondQueue));
}
}
Each task runs the operations op() of according with below affirmation on ItemState:
one of three dedicated thread moves it to secondQueue and finally
another dedicated thread removes it.
ItemState is a immutable object that contains Item and your State. This ensures consistency between Item and State values.
ItemState has acknowledgement about the next state creating a mechanism of self-controled state:
public class FirstMovingTask {
//others codes
protected void op() {
try {
//dequeue
ItemState is0 = new ItemState(firstQueue.take());
System.out.println("Item " + is0.getItem().getValue() + ": " + is0.getState().getValue());
//process here
//enqueue
ItemState is1 = new ItemState(is0);
secondQueue.add(is1);
System.out.println("Item " + is1.getItem().getValue() + ": " + is1.getState().getValue());
} catch (InterruptedException e) {
e.printStackTrace();
}
}
//others codes
}
With ItemState implemetation:
public class ItemStateImpl implements ItemState {
private final Item item;
private final State state;
public ItemStateImpl(Item i){
this.item = i;
this.state = new State();
}
public ItemStateImpl(ItemState is) {
this.item = is.getItem();
this.state = is.getState().next();
}
// gets attrs
}
So this way is possible build solutions more elegant, flexible and scalable.
Scalable because you can to control more states only changing next() and generalizing the moving task for increase the number of queue.
Results:
Item 0: AFTER_FIRST
Item 0: IN_FIRST
Item 0: IN_SECOND
Item 0: AFTER_SECOND
Item 1: IN_FIRST
Item 1: AFTER_FIRST
Item 1: IN_SECOND
Item 1: AFTER_SECOND
Item 2: IN_FIRST
Item 2: AFTER_FIRST
Item 2: IN_SECOND
... others
UPDATE(06/07/2018): analysing the use of map for search
Search in map using equals values like comparator might not work because usally the mapping between values and identity (key/hash) is not one-to-one(see figure bellow). In this way is need to create an sorted list for search values which results in O(n) (worst-case).
with Item.getValuesHashCode():
private int getValuesHashCode(){
return new HashCodeBuilder().append(value).hashCode();
}
In this case, you must keep Vector<ItemState> instead of Item and to use the key like the result of getValuesHashCode. Change the mechanism of state-control for keep first reference of the Item and the state current. See bellow:
//Main.class
public static void main(String[] args) {
... others code ...
//references repository
ConcurrentHashMap<Integer, Vector<ItemState>> statesMap = new ConcurrentHashMap<Integer, Vector<ItemState>>();
//start three threads
startTask("Thread#1", new CreatingTask(firstQueue, statesMap));
... others code ...
}
//CreateTask.class
protected void op() throws InterruptedException {
//create item
ItemState is = new ItemStateImpl(new Item(i++, NameGenerator.name()));
//put in monitor and enqueue
int key = is.getHashValue();
Vector<ItemState> items = map.get(key);
if (items == null){
items = new Vector<>();
map.put(key, items);
}
items.add(is);
//enqueue
queue.put(is);
}
//FirstMovingTask.class
protected void op() throws InterruptedException{
//dequeue
ItemState is0 = firstQueue.take();
//process
ItemState is1 = process(is0.next());
//enqueue
secondQueue.put(is1.next());
}
//ItemState.class
public ItemState next() {
//required for consistent change state
synchronized (state) {
state = state.next();
return this;
}
}
To search you must use concurrentMapRef.get(key). The result will the reference of updated ItemState.
Results in my tests for :
# key = hash("a")
# concurrentMapRef.get(key)
...
Item#7#0 : a - IN_FIRST
... many others lines
Item#7#0 : a - AFTER_FIRST
Item#12#1 : a - IN_FIRST
... many others lines
Item#7#0 : a - IN_SECOND
Item#12#1 : a - IN_FIRST
... many others lines
Item#7#0 : a - AFTER_SECOND
Item#12#1 : a - IN_FIRST
More details in code: https://github.com/ag-studies/stackoverflow-queue
UPDATED IN 06/09/2018: redesign
Generalizing this project, I can undestand that the state machine is something like:
In this way I decoupled the workers of the queues for improve concepts. I used an MemoryRep for keep the unique reference for item in overall processment.
Of course that you can use strategies event-based if you need keep ItemState in a physic repository.
This keep the previous idea and creates more legibility for the concepts. See this:
I understand that each job will have two queue (input/output) and relationship with a business model! The researcher will always find the most updated and consistent state of Item.
So, answering your ask:
I can find the consistent state of Item anywhere using MemoryRep (basically an Map), wrapping state and item in ItemState, and controlling the change state on job on enqueue or dequeue it.
The performace is keeped, except on running of next()
The state is allways consistent (for your problem)
In this model is possible use any queue type, any number of jobs/queues, and any number of state.
Additionaly this is beautiful!!
As previously answered, Wrap the queues or the item would be viable solutions or both.
public class ItemWrapper<E> {
E item;
Status status;
public ItemWrapper(Item i, Status s){ ... }
public setStatus(Status s){ ... }
// not necessary if you use a queue wrapper (see queue wrapper)
public boolean equals(Object obj) {
if ( obj instanceof ItemWrapper)
return item.equals(((ItemWrapper) obj).item)
return false;
}
public int hashCode(){
return item;
}
}
...
process(item) // process update status in the item
...
Probably a better way, already answered, is to have a QueueWrapper who update the queue status. For the fun I don't use a status map but I use the previously itemwrapper it seems cleaner (a status map works too).
public class QueueWrapper<E> implements Queue<E> {
private Queue<ItemWrapper<E>> myQueue;
static private Status inStatus; // FIRST
static private Status outStatus; // AFTER_FIRST
public QueueWrapper(Queue<E> myQueue, Status inStatus, Status outStatus) {...}
#Override
public boolean add(E e) {
return myQueue.add(new ItemWrapper(e, inStatus));
}
#Override
public E remove(){
ItemWrapper<E> result = myQueue.remove();
result.setStatus(outStatus)
return result.item;
}
...
}
You can also use AOP to inject status update in your queues without changing your queues (a status map should be more appropriate than itemwrapper).
Maybe I didn't answer well your question because an easy way to know where is your item could be to check in each queue with "contains" function.
Here's something different from what others have said. Taking from the world of queue services and systems we have the concept of message acknowledgement. This is nice, because it also gives you some built in retry logic.
I'll lay out how it would work from a high level, and if you need I can add code.
Essentially you'll have a Set to go with each of your queues. You'll wrap your queues in an object so that when you dequeue an item a few things happen
The item is removed from the queue
The item is added to the associated set
A task (lambda containing an atomic boolean (default false)) is scheduled. When run it will remove item from the set and if the boolean is false, put it back in the queue
The item and a wrapper around the boolean are returned to the caller
Once process(i); completes, your code will indicate receipt acknowledgement to the wrapper, and the wrapper will remove the item from the set and make the boolean false.
A method to return status would simply check which queue or set the item is in.
Note that this gives "at least once" delivery, meaning an item will be processed at least once, but potentially more than once if the processing time is too close to the timeout.
Related
I have a cache refresh logic and want to make sure that it's thread-safe and correct way to do it.
public class Test {
Set<Integer> cache = Sets.newConcurrentHashSet();
public boolean contain(int num) {
return cache.contains(num);
}
public void refresh() {
cache.clear();
cache.addAll(getNums());
}
}
So I have a background thread refreshing cache - periodically call refresh. And multiple threads are calling contain at the same time. I was trying to avoid having synchronized in the methods signature because refresh could take some time (imagine that getNum makes network calls and parsing huge data) then contain would be blocked.
I think this code is not good enough because if contain called in between clear and addAll then contain always returns false.
What is the best way to achieve cache refreshing without impacting significant latency to contain call?
Best way would be to use functional programming paradigm whereby you have immutable state (in this case a Set), instead of adding and removing elements to that set you create an entirely new Set every time you want to add or remove elements. This is in Java9.
It can be a bit awkward or infeasible however to achieve this method for legacy code. So instead what you could do is have 2 Sets 1 which has the get method on it which is volatile, and then this is assigned a new instance in the refresh method.
public class Test {
volatile Set<Integer> cache = new HashSet<>();
public boolean contain(int num) {
return cache.contains(num);
}
public void refresh() {
Set<Integer> privateCache = new HashSet<>();
privateCache.addAll(getNums());
cache = privateCache;
}
}
Edit We don't want or need a ConcurrentHashSet, that is if you want to add and remove elements to a collection at the same time, which in my opinion is a pretty useless thing to do. But you want to switch the old Set with a new one, which is why you just need a volatile variable to make sure you can't read and edit the cache at the same time.
But as I mentioned in my answer at the start is that if you never modify collections, but instead make new ones each time you want to update a collection (note that this is a very cheap operation as internally the old set is reused in the operation). This way you never need to worry about concurrency, as there is no shared state between threads.
How would you make sure your cache doesn't contain invalid entries when calling contains?? Furthermore, you'd need to call refresh every time getNums() changes, which is pretty inefficient. It would be best if you make sure you control your changes to getNums() and then update cache accordingly. The cache might look like:
public class MyCache {
final ConcurrentHashMap<Integer, Boolean> cache = new ConcurrentHashMap<>(); //it's a ConcurrentHashMap to be able to use putIfAbsent
public boolean contains(Integer num) {
return cache.contains(num);
}
public void add(Integer nums) {
cache.putIfAbsent(num, true);
}
public clear(){
cache.clear();
}
public remove(Integer num) {
cache.remove(num);
}
}
Update
As #schmosel made me realize, mine was a wasted effort: it is in fact enough to initialize a complete new HashSet<> with your values in the refresh method. Assuming of course that the cache is marked with volatile. In short #Snickers3192's answer, points out what you seek.
Old answer
You can also use a slightly different system.
Keep two Set<Integer>, one of which will always be empty. When you refresh the cache, you can asynchronously re-initialize the second one and then just switch the pointers. Other threads accessing the cache won't see any particular overhead in this.
From an external point of view, they will always be accessing the same cache.
private volatile int currentCache; // 0 or 1
private final Set<Integer> caches[] = new HashSet[2]; // use two caches; either one will always be empty, so not much memory consumed
private volatile Set<Integer> cachePointer = null; // just a pointer to the current cache, must be volatile
// initialize
{
this.caches[0] = new HashSet<>(0);
this.caches[1] = new HashSet<>(0);
this.currentCache = 0;
this.cachePointer = caches[this.currentCache]; // point to cache one from the beginning
}
Your refresh method may look like this:
public void refresh() {
// store current cache pointer
final int previousCache = this.currentCache;
final int nextCache = getNextPointer();
// you can easily compute it asynchronously
// in the meantime, external threads will still access the normal cache
CompletableFuture.runAsync( () -> {
// fill the unused cache
caches[nextCache].addAll(getNums());
// then switch the pointer to the just-filled cache
// from this point on, threads are accessing the new cache
switchCachePointer();
// empty the other cache still on the async thread
caches[previousCache].clear();
});
}
where the utility methods are:
public boolean contains(final int num) {
return this.cachePointer.contains(num);
}
private int getNextPointer() {
return ( this.currentCache + 1 ) % this.caches.length;
}
private void switchCachePointer() {
// make cachePointer point to a new cache
this.currentCache = this.getNextPointer();
this.cachePointer = caches[this.currentCache];
}
I've been reading up on State Machines since its likely I need to use for my next project. Most examples I find online show how to go from StateA to StateB. But what if your next desired state is not an adjacent state? Are there any common patterns/practices to achieve this? Ideally in Java, but I can read other programming languages just as well.
# Example States
WakeUp->Get Dressed->Get Car Keys->Get in Car->Drive to Work->Work
Current State: Get in Car
Problems to solve
# Scenario 1: Desired State == Work
Forgot car keys, so you have to return to previous state and then move forward in states again.
# Scenario 2: Desired State == Work
Have car keys, so move forward in states to get to Desired State.
It's very likely that State Machine may not solve this problem elegantly and I just need to hand-craft the logic, I don't mind, but thought I'd follow a common design pattern to help others understand it.
From the example above, I do not need to worry about 'internal' states, which is also true for the project I'm tackling; just in case that makes a difference in possible solutions.
Here is a simple way to define a state machine.
Define in an enum all the states that you want.
enum StateType {
WAKE_UP, GET_DRESSED, GET_CAR_KEYS, GET_IN_CAR, DRIVE_TO_WORK, WORK
}
Have a statemachine which controls states, and a state interface which performs an action on the statemachine. The state then returns the next state to go to.
interface State {
StateType next(StateMachine sm);
}
Implement this state for multiple types
class GetInCarState implements State {
#Override
public StateType next(StateMachine sm) {
if (sm.hasKeys()) {
return StateType.DRIVE_TO_WORK;
}
return StateType.GET_CAR_KEYS;
}
}
Now define the State Machine
class StateMachine {
private Map<StateType, State> states = new HashMap<StateType, State>() {{
put(StateType.WAKE_UP, new WakeUpState());
put(StateType.GET_DRESSED, new GetDressedState());
put(StateType.GET_CAR_KEYS, new GetCarKeysState());
put(StateType.GET_IN_CAR, new GetInCarState());
put(StateType.DRIVE_TO_WORK, new DriveToWorkState());
put(StateType.WORK, new WorkState());
}};
private StateType currentState = StateType.WAKE_UP;
private boolean hasCarKeys;
public boolean hasKeys() {
return hasCarKeys;
}
public void setHasKeys(boolean hasKeys) {
hasCarKeys = hasKeys;
}
public void update() {
currentState = states.get(currentState).next(this);
}
}
I have a requirement in Spring Batch where I have a file with thousands of records coming in a sorted order.The key field is product code.
The file may have multiple records of the same product code.The requirement is that I have to group the records that have the same
product Code in a collection (i.e List) and then send them over to a method i.e validateProductCodes(List prodCodeList).
I am looking for the best way to do this.The approach I thought of was to read every record in the Processor and then build a collection
of records for the same product code in the processor.If at any point in the processor,if the product code in the record is different than it would imply that
the productCode grouping is complete and the validateProductCodes() can be called for that group of records with the same product code.Also I am using a Step.So does
not that automatically mean that the process is multithreaded?Meaning Groups of records with same productCode will be processed in a multithreaded way.Please advise.
Thanks
There are two questions in your question: first, you want to know how to group the items together and second how they are processed.
In order to group them, you could create a group reader as Luca suggested or something like:
public class GroupReader<I> implements ItemReader<List<I>>{
private SingleItemPeekableItemReader<I> reader;
private ItemReader<I> peekReaderDelegate;
public void setReader(ItemReader<I> reader) {
peekReaderDelegate = reader;
}
#Override
public void afterPropertiesSet() throws Exception {
Assert.notNull(peekReaderDelegate, "The 'itemReader' may not be null");
this.reader= new SingleItemPeekableItemReader<I>();
this.reader.setDelegate(delegateReader);
}
#Override
public List<I> read() throws Exception {
State state = State.NEW;
List<I> group = null;
I item = null;
while (state != State.COMPLETE) {
item = reader.read();
switch (state) {
case NEW: {
if (item == null) {
// end reached
state = State.COMPLETE;
break;
}
group = new ArrayList<I>();
group.add(item);
state = State.READING;
I nextItem = reader.peek();
if (isItAKeyChange(item, nextItem)) {
state = State.COMPLETE;
}
break;
}
case READING: {
group.add(item);
// peek and check if there the peeked entry has a new date
I nextItem = peekEntry();
if (isItAKeyChange(item, nextItem)) {
state = State.COMPLETE;
}
break;
}
default: {
throw new org.springframework.expression.ParseException(groupCounter, "ParsingError: Reader is in an invalid state");
}
}
}
return group;
}
}
For every key, this reader will return a list with all elements matching this key. Therefore, the grouping ist done directly in the reader.
You cannot do that with a processor, as you described.
Your second question about multithreading.
Now, using a step does not necessarily mean, that the step is processed with several threads.
In order to do that, you need set an AsyncTaskExecutor and you have to set the throttle limit.
But if you do that, your reader must be threadsafe, or otherwise your grouping won't work. You could do that by simply defining the read method above as synchronized.
Another way could be to write a small SynchronizedWrapperReader, as suggested in this question: Parellel Processing Spring Batch StaxEventItemReader
Please note, depending on your target you are writing to, you probably also have to synchronize the writer, and if necessary to reorder the result.
I have something to do for work and I need your help.
We want to implement a FSM - Finite State Machine, to identify char sequence(like: A, B, C, A, C), and tell if it accepted.
We think to implement three classes: State, Event and Machine.
The state class presents a node in the FSM, we thought to implement it with State design pattern, every node will extend from the abstract class state and every class would handle different types of events and indicate transitions to a new state. Is it good idea in your opinion?
Second thing, we don't know how to save all the transitions. Again we thought to implement it with some kind of map, that hold the starting point and gets some kind of vector with the next states, but I'm not sure thats a good idea.
I would be happy to get some ideas of how to implement it or maybe you can give me some starting points.
How should I save the FSM, meaning how should I build the tree at the beginning of the program?
I googled it and found a lot of examples but nothing that helps me.
Thanks a lot.
The heart of a state machine is the transition table, which takes a state and a symbol (what you're calling an event) to a new state. That's just a two-index array of states. For sanity and type safety, declare the states and symbols as enumerations. I always add a "length" member in some way (language-specific) for checking array bounds. When I've hand-coded FSM's, I format the code in row and column format with whitespace fiddling. The other elements of a state machine are the initial state and the set of accepting states. The most direct implementation of the set of accepting states is an array of booleans indexed by the states. In Java, however, enumerations are classes, and you can specify an argument "accepting" in the declaration for each enumerated value and initialize it in the constructor for the enumeration.
For the machine type, you can write it as a generic class. It would take two type arguments, one for the states and one for the symbols, an array argument for the transition table, a single state for the initial. The only other detail (though it's critical) is that you have to call Enum.ordinal() to get an integer suitable for indexing the transition array, since you there's no syntax for directly declaring an array with a enumeration index (though there ought to be).
To preempt one issue, EnumMap won't work for the transition table, because the key required is a pair of enumeration values, not a single one.
enum State {
Initial( false ),
Final( true ),
Error( false );
static public final Integer length = 1 + Error.ordinal();
final boolean accepting;
State( boolean accepting ) {
this.accepting = accepting;
}
}
enum Symbol {
A, B, C;
static public final Integer length = 1 + C.ordinal();
}
State transition[][] = {
// A B C
{
State.Initial, State.Final, State.Error
}, {
State.Final, State.Initial, State.Error
}
};
EasyFSM is a dynamic Java Library which can be used to implement an FSM.
You can find documentation for the same at :
Finite State Machine in Java
Also, you can download the library at :
Java FSM Library : DynamicEasyFSM
You can implement Finite State Machine in two different ways.
Option 1:
Finite State machine with a pre-defined workflow : Recommended if you know all states in advance and state machine is almost fixed without any changes in future
Identify all possible states in your application
Identify all the events in your application
Identify all the conditions in your application, which may lead state transition
Occurrence of an event may cause transitions of state
Build a finite state machine by deciding a workflow of states & transitions.
e.g If an event 1 occurs at State 1, the state will be updated and machine state may still be in state 1.
If an event 2 occurs at State 1, on some condition evaluation, the system will move from State 1 to State 2
This design is based on State and Context patterns.
Have a look at Finite State Machine prototype classes.
Option 2:
Behavioural trees: Recommended if there are frequent changes to state machine workflow. You can dynamically add new behaviour without breaking the tree.
The base Task class provides a interface for all these tasks, the leaf tasks are the ones just mentioned, and the parent tasks are the interior nodes that decide which task to execute next.
The Tasks have only the logic they need to actually do what is required of them, all the decision logic of whether a task has started or not, if it needs to update, if it has finished with success, etc. is grouped in the TaskController class, and added by composition.
The decorators are tasks that “decorate” another class by wrapping over it and giving it additional logic.
Finally, the Blackboard class is a class owned by the parent AI that every task has a reference to. It works as a knowledge database for all the leaf tasks
Have a look at this article by Jaime Barrachina Verdia for more details
Hmm, I would suggest that you use Flyweight to implement the states. Purpose: Avoid the memory overhead of a large number of small objects. State machines can get very, very big.
http://en.wikipedia.org/wiki/Flyweight_pattern
I'm not sure that I see the need to use design pattern State to implement the nodes. The nodes in a state machine are stateless. They just match the current input symbol to the available transitions from the current state. That is, unless I have entirely forgotten how they work (which is a definite possiblilty).
If I were coding it, I would do something like this:
interface FsmNode {
public boolean canConsume(Symbol sym);
public FsmNode consume(Symbol sym);
// Other methods here to identify the state we are in
}
List<Symbol> input = getSymbols();
FsmNode current = getStartState();
for (final Symbol sym : input) {
if (!current.canConsume(sym)) {
throw new RuntimeException("FSM node " + current + " can't consume symbol " + sym);
}
current = current.consume(sym);
}
System.out.println("FSM consumed all input, end state is " + current);
What would Flyweight do in this case? Well, underneath the FsmNode there would probably be something like this:
Map<Integer, Map<Symbol, Integer>> fsm; // A state is an Integer, the transitions are from symbol to state number
FsmState makeState(int stateNum) {
return new FsmState() {
public FsmState consume(final Symbol sym) {
final Map<Symbol, Integer> transitions = fsm.get(stateNum);
if (transisions == null) {
throw new RuntimeException("Illegal state number " + stateNum);
}
final Integer nextState = transitions.get(sym); // May be null if no transition
return nextState;
}
public boolean canConsume(final Symbol sym) {
return consume(sym) != null;
}
}
}
This creates the State objects on a need-to-use basis, It allows you to use a much more efficient underlying mechanism to store the actual state machine. The one I use here (Map(Integer, Map(Symbol, Integer))) is not particulary efficient.
Note that the Wikipedia page focuses on the cases where many somewhat similar objects share the similar data, as is the case in the String implementation in Java. In my opinion, Flyweight is a tad more general, and covers any on-demand creation of objects with a short life span (use more CPU to save on a more efficient underlying data structure).
Consider the easy, lightweight Java library EasyFlow. From their docs:
With EasyFlow you can:
implement complex logic but keep your code simple and clean
handle asynchronous calls with ease and elegance
avoid concurrency by using event-driven programming approach
avoid StackOverflow error by avoiding recursion
simplify design, programming and testing of complex java applications
I design & implemented a simple finite state machine example with java.
IFiniteStateMachine: The public interface to manage the finite state machine
such as add new states to the finite state machine or transit to next states by specific actions.
interface IFiniteStateMachine {
void setStartState(IState startState);
void setEndState(IState endState);
void addState(IState startState, IState newState, Action action);
void removeState(String targetStateDesc);
IState getCurrentState();
IState getStartState();
IState getEndState();
void transit(Action action);
}
IState: The public interface to get state related info
such as state name and mappings to connected states.
interface IState {
// Returns the mapping for which one action will lead to another state
Map<String, IState> getAdjacentStates();
String getStateDesc();
void addTransit(Action action, IState nextState);
void removeTransit(String targetStateDesc);
}
Action: the class which will cause the transition of states.
public class Action {
private String mActionName;
public Action(String actionName) {
mActionName = actionName;
}
String getActionName() {
return mActionName;
}
#Override
public String toString() {
return mActionName;
}
}
StateImpl: the implementation of IState. I applied data structure such as HashMap to keep Action-State mappings.
public class StateImpl implements IState {
private HashMap<String, IState> mMapping = new HashMap<>();
private String mStateName;
public StateImpl(String stateName) {
mStateName = stateName;
}
#Override
public Map<String, IState> getAdjacentStates() {
return mMapping;
}
#Override
public String getStateDesc() {
return mStateName;
}
#Override
public void addTransit(Action action, IState state) {
mMapping.put(action.toString(), state);
}
#Override
public void removeTransit(String targetStateDesc) {
// get action which directs to target state
String targetAction = null;
for (Map.Entry<String, IState> entry : mMapping.entrySet()) {
IState state = entry.getValue();
if (state.getStateDesc().equals(targetStateDesc)) {
targetAction = entry.getKey();
}
}
mMapping.remove(targetAction);
}
}
FiniteStateMachineImpl: Implementation of IFiniteStateMachine. I use ArrayList to keep all the states.
public class FiniteStateMachineImpl implements IFiniteStateMachine {
private IState mStartState;
private IState mEndState;
private IState mCurrentState;
private ArrayList<IState> mAllStates = new ArrayList<>();
private HashMap<String, ArrayList<IState>> mMapForAllStates = new HashMap<>();
public FiniteStateMachineImpl(){}
#Override
public void setStartState(IState startState) {
mStartState = startState;
mCurrentState = startState;
mAllStates.add(startState);
// todo: might have some value
mMapForAllStates.put(startState.getStateDesc(), new ArrayList<IState>());
}
#Override
public void setEndState(IState endState) {
mEndState = endState;
mAllStates.add(endState);
mMapForAllStates.put(endState.getStateDesc(), new ArrayList<IState>());
}
#Override
public void addState(IState startState, IState newState, Action action) {
// validate startState, newState and action
// update mapping in finite state machine
mAllStates.add(newState);
final String startStateDesc = startState.getStateDesc();
final String newStateDesc = newState.getStateDesc();
mMapForAllStates.put(newStateDesc, new ArrayList<IState>());
ArrayList<IState> adjacentStateList = null;
if (mMapForAllStates.containsKey(startStateDesc)) {
adjacentStateList = mMapForAllStates.get(startStateDesc);
adjacentStateList.add(newState);
} else {
mAllStates.add(startState);
adjacentStateList = new ArrayList<>();
adjacentStateList.add(newState);
}
mMapForAllStates.put(startStateDesc, adjacentStateList);
// update mapping in startState
for (IState state : mAllStates) {
boolean isStartState = state.getStateDesc().equals(startState.getStateDesc());
if (isStartState) {
startState.addTransit(action, newState);
}
}
}
#Override
public void removeState(String targetStateDesc) {
// validate state
if (!mMapForAllStates.containsKey(targetStateDesc)) {
throw new RuntimeException("Don't have state: " + targetStateDesc);
} else {
// remove from mapping
mMapForAllStates.remove(targetStateDesc);
}
// update all state
IState targetState = null;
for (IState state : mAllStates) {
if (state.getStateDesc().equals(targetStateDesc)) {
targetState = state;
} else {
state.removeTransit(targetStateDesc);
}
}
mAllStates.remove(targetState);
}
#Override
public IState getCurrentState() {
return mCurrentState;
}
#Override
public void transit(Action action) {
if (mCurrentState == null) {
throw new RuntimeException("Please setup start state");
}
Map<String, IState> localMapping = mCurrentState.getAdjacentStates();
if (localMapping.containsKey(action.toString())) {
mCurrentState = localMapping.get(action.toString());
} else {
throw new RuntimeException("No action start from current state");
}
}
#Override
public IState getStartState() {
return mStartState;
}
#Override
public IState getEndState() {
return mEndState;
}
}
example:
public class example {
public static void main(String[] args) {
System.out.println("Finite state machine!!!");
IState startState = new StateImpl("start");
IState endState = new StateImpl("end");
IFiniteStateMachine fsm = new FiniteStateMachineImpl();
fsm.setStartState(startState);
fsm.setEndState(endState);
IState middle1 = new StateImpl("middle1");
middle1.addTransit(new Action("path1"), endState);
fsm.addState(startState, middle1, new Action("path1"));
System.out.println(fsm.getCurrentState().getStateDesc());
fsm.transit(new Action(("path1")));
System.out.println(fsm.getCurrentState().getStateDesc());
fsm.addState(middle1, endState, new Action("path1-end"));
fsm.transit(new Action(("path1-end")));
System.out.println(fsm.getCurrentState().getStateDesc());
fsm.addState(endState, middle1, new Action("path1-end"));
}
}
Full example on Github
Well this is an old question but while nobody mentioned here, I will advice to check two existing frameworks before you implement you own State Machines.
One is Spring State Machine most of you are familiar with Spring framework, which allow us to use several features of Spring like dependency injection and everything else that Spring can offer.
It is really great for modelling the lifecycle of an Apparat, with states like INITIALIZING, STARTED, ERROR, RECOVERING, SHUTTINGDOWN, etc.. but I see lots of people are trying to model a Shopping Chart, a Reservation System with it, the memory footprint a Spring State Machine is relatively big to model millions of Shopping Charts or Reservations.
One other disadvantage, Spring State Machine, while has a capability to persist itself for long running processes but it does not have any mechanism to adapt to changes in these processes, if you persist a process and you have to recover it lets say 10 days later with a change occurred in your business process because of new software release / requirement, you have no built in means to deal with it.
I have several blogs, blog1 blog2, demonstrating how you can program Spring State Machine, specially model driven way, if you want to check it.
Mainly because the disadvantages I mentioned, I advice you to look another framework first, Akka FSM (Finite State Machine) which is more fitting with its low memory footprint to have millions and millions of instances and has a capability to adapt changing long running processes.
Now you can develop with Akka framework with Java but believe me because of some missing language elements, you don't want to read the produced code, Scala is a much more fitting language to develop with Akka. Now I hear you saying Scala is too complex, I can't convince my project leads to develop with Scala, to convince you all this is an option, I developed a Proof of Concept application using a Java/Scala hybrid with all Scala Akka Finite State Machine code generated from an UML model, if you want to check it out here the links to the blogs, blog3 blog4.
I hope this information would help you.
Here is a SUPER SIMPLE implementation/example of a FSM using just "if-else"s which avoids all of the above subclassing answers (taken from Using Finite State Machines for Pattern Matching in Java, where he is looking for a string which ends with "#" followed by numbers followed by "#"--see state graph here):
public static void main(String[] args) {
String s = "A1#312#";
String digits = "0123456789";
int state = 0;
for (int ind = 0; ind < s.length(); ind++) {
if (state == 0) {
if (s.charAt(ind) == '#')
state = 1;
} else {
boolean isNumber = digits.indexOf(s.charAt(ind)) != -1;
if (state == 1) {
if (isNumber)
state = 2;
else if (s.charAt(ind) == '#')
state = 1;
else
state = 0;
} else if (state == 2) {
if (s.charAt(ind) == '#') {
state = 3;
} else if (isNumber) {
state = 2;
} else if (s.charAt(ind) == '#')
state = 1;
else
state = 0;
} else if (state == 3) {
if (s.charAt(ind) == '#')
state = 1;
else
state = 0;
}
}
} //end for loop
if (state == 3)
System.out.println("It matches");
else
System.out.println("It does not match");
}
P.S: Does not answer your question directly, but shows you how to implement a FSM very easily in Java.
I have something to do for work and I need your help.
We want to implement a FSM - Finite State Machine, to identify char sequence(like: A, B, C, A, C), and tell if it accepted.
We think to implement three classes: State, Event and Machine.
The state class presents a node in the FSM, we thought to implement it with State design pattern, every node will extend from the abstract class state and every class would handle different types of events and indicate transitions to a new state. Is it good idea in your opinion?
Second thing, we don't know how to save all the transitions. Again we thought to implement it with some kind of map, that hold the starting point and gets some kind of vector with the next states, but I'm not sure thats a good idea.
I would be happy to get some ideas of how to implement it or maybe you can give me some starting points.
How should I save the FSM, meaning how should I build the tree at the beginning of the program?
I googled it and found a lot of examples but nothing that helps me.
Thanks a lot.
The heart of a state machine is the transition table, which takes a state and a symbol (what you're calling an event) to a new state. That's just a two-index array of states. For sanity and type safety, declare the states and symbols as enumerations. I always add a "length" member in some way (language-specific) for checking array bounds. When I've hand-coded FSM's, I format the code in row and column format with whitespace fiddling. The other elements of a state machine are the initial state and the set of accepting states. The most direct implementation of the set of accepting states is an array of booleans indexed by the states. In Java, however, enumerations are classes, and you can specify an argument "accepting" in the declaration for each enumerated value and initialize it in the constructor for the enumeration.
For the machine type, you can write it as a generic class. It would take two type arguments, one for the states and one for the symbols, an array argument for the transition table, a single state for the initial. The only other detail (though it's critical) is that you have to call Enum.ordinal() to get an integer suitable for indexing the transition array, since you there's no syntax for directly declaring an array with a enumeration index (though there ought to be).
To preempt one issue, EnumMap won't work for the transition table, because the key required is a pair of enumeration values, not a single one.
enum State {
Initial( false ),
Final( true ),
Error( false );
static public final Integer length = 1 + Error.ordinal();
final boolean accepting;
State( boolean accepting ) {
this.accepting = accepting;
}
}
enum Symbol {
A, B, C;
static public final Integer length = 1 + C.ordinal();
}
State transition[][] = {
// A B C
{
State.Initial, State.Final, State.Error
}, {
State.Final, State.Initial, State.Error
}
};
EasyFSM is a dynamic Java Library which can be used to implement an FSM.
You can find documentation for the same at :
Finite State Machine in Java
Also, you can download the library at :
Java FSM Library : DynamicEasyFSM
You can implement Finite State Machine in two different ways.
Option 1:
Finite State machine with a pre-defined workflow : Recommended if you know all states in advance and state machine is almost fixed without any changes in future
Identify all possible states in your application
Identify all the events in your application
Identify all the conditions in your application, which may lead state transition
Occurrence of an event may cause transitions of state
Build a finite state machine by deciding a workflow of states & transitions.
e.g If an event 1 occurs at State 1, the state will be updated and machine state may still be in state 1.
If an event 2 occurs at State 1, on some condition evaluation, the system will move from State 1 to State 2
This design is based on State and Context patterns.
Have a look at Finite State Machine prototype classes.
Option 2:
Behavioural trees: Recommended if there are frequent changes to state machine workflow. You can dynamically add new behaviour without breaking the tree.
The base Task class provides a interface for all these tasks, the leaf tasks are the ones just mentioned, and the parent tasks are the interior nodes that decide which task to execute next.
The Tasks have only the logic they need to actually do what is required of them, all the decision logic of whether a task has started or not, if it needs to update, if it has finished with success, etc. is grouped in the TaskController class, and added by composition.
The decorators are tasks that “decorate” another class by wrapping over it and giving it additional logic.
Finally, the Blackboard class is a class owned by the parent AI that every task has a reference to. It works as a knowledge database for all the leaf tasks
Have a look at this article by Jaime Barrachina Verdia for more details
Hmm, I would suggest that you use Flyweight to implement the states. Purpose: Avoid the memory overhead of a large number of small objects. State machines can get very, very big.
http://en.wikipedia.org/wiki/Flyweight_pattern
I'm not sure that I see the need to use design pattern State to implement the nodes. The nodes in a state machine are stateless. They just match the current input symbol to the available transitions from the current state. That is, unless I have entirely forgotten how they work (which is a definite possiblilty).
If I were coding it, I would do something like this:
interface FsmNode {
public boolean canConsume(Symbol sym);
public FsmNode consume(Symbol sym);
// Other methods here to identify the state we are in
}
List<Symbol> input = getSymbols();
FsmNode current = getStartState();
for (final Symbol sym : input) {
if (!current.canConsume(sym)) {
throw new RuntimeException("FSM node " + current + " can't consume symbol " + sym);
}
current = current.consume(sym);
}
System.out.println("FSM consumed all input, end state is " + current);
What would Flyweight do in this case? Well, underneath the FsmNode there would probably be something like this:
Map<Integer, Map<Symbol, Integer>> fsm; // A state is an Integer, the transitions are from symbol to state number
FsmState makeState(int stateNum) {
return new FsmState() {
public FsmState consume(final Symbol sym) {
final Map<Symbol, Integer> transitions = fsm.get(stateNum);
if (transisions == null) {
throw new RuntimeException("Illegal state number " + stateNum);
}
final Integer nextState = transitions.get(sym); // May be null if no transition
return nextState;
}
public boolean canConsume(final Symbol sym) {
return consume(sym) != null;
}
}
}
This creates the State objects on a need-to-use basis, It allows you to use a much more efficient underlying mechanism to store the actual state machine. The one I use here (Map(Integer, Map(Symbol, Integer))) is not particulary efficient.
Note that the Wikipedia page focuses on the cases where many somewhat similar objects share the similar data, as is the case in the String implementation in Java. In my opinion, Flyweight is a tad more general, and covers any on-demand creation of objects with a short life span (use more CPU to save on a more efficient underlying data structure).
Consider the easy, lightweight Java library EasyFlow. From their docs:
With EasyFlow you can:
implement complex logic but keep your code simple and clean
handle asynchronous calls with ease and elegance
avoid concurrency by using event-driven programming approach
avoid StackOverflow error by avoiding recursion
simplify design, programming and testing of complex java applications
I design & implemented a simple finite state machine example with java.
IFiniteStateMachine: The public interface to manage the finite state machine
such as add new states to the finite state machine or transit to next states by specific actions.
interface IFiniteStateMachine {
void setStartState(IState startState);
void setEndState(IState endState);
void addState(IState startState, IState newState, Action action);
void removeState(String targetStateDesc);
IState getCurrentState();
IState getStartState();
IState getEndState();
void transit(Action action);
}
IState: The public interface to get state related info
such as state name and mappings to connected states.
interface IState {
// Returns the mapping for which one action will lead to another state
Map<String, IState> getAdjacentStates();
String getStateDesc();
void addTransit(Action action, IState nextState);
void removeTransit(String targetStateDesc);
}
Action: the class which will cause the transition of states.
public class Action {
private String mActionName;
public Action(String actionName) {
mActionName = actionName;
}
String getActionName() {
return mActionName;
}
#Override
public String toString() {
return mActionName;
}
}
StateImpl: the implementation of IState. I applied data structure such as HashMap to keep Action-State mappings.
public class StateImpl implements IState {
private HashMap<String, IState> mMapping = new HashMap<>();
private String mStateName;
public StateImpl(String stateName) {
mStateName = stateName;
}
#Override
public Map<String, IState> getAdjacentStates() {
return mMapping;
}
#Override
public String getStateDesc() {
return mStateName;
}
#Override
public void addTransit(Action action, IState state) {
mMapping.put(action.toString(), state);
}
#Override
public void removeTransit(String targetStateDesc) {
// get action which directs to target state
String targetAction = null;
for (Map.Entry<String, IState> entry : mMapping.entrySet()) {
IState state = entry.getValue();
if (state.getStateDesc().equals(targetStateDesc)) {
targetAction = entry.getKey();
}
}
mMapping.remove(targetAction);
}
}
FiniteStateMachineImpl: Implementation of IFiniteStateMachine. I use ArrayList to keep all the states.
public class FiniteStateMachineImpl implements IFiniteStateMachine {
private IState mStartState;
private IState mEndState;
private IState mCurrentState;
private ArrayList<IState> mAllStates = new ArrayList<>();
private HashMap<String, ArrayList<IState>> mMapForAllStates = new HashMap<>();
public FiniteStateMachineImpl(){}
#Override
public void setStartState(IState startState) {
mStartState = startState;
mCurrentState = startState;
mAllStates.add(startState);
// todo: might have some value
mMapForAllStates.put(startState.getStateDesc(), new ArrayList<IState>());
}
#Override
public void setEndState(IState endState) {
mEndState = endState;
mAllStates.add(endState);
mMapForAllStates.put(endState.getStateDesc(), new ArrayList<IState>());
}
#Override
public void addState(IState startState, IState newState, Action action) {
// validate startState, newState and action
// update mapping in finite state machine
mAllStates.add(newState);
final String startStateDesc = startState.getStateDesc();
final String newStateDesc = newState.getStateDesc();
mMapForAllStates.put(newStateDesc, new ArrayList<IState>());
ArrayList<IState> adjacentStateList = null;
if (mMapForAllStates.containsKey(startStateDesc)) {
adjacentStateList = mMapForAllStates.get(startStateDesc);
adjacentStateList.add(newState);
} else {
mAllStates.add(startState);
adjacentStateList = new ArrayList<>();
adjacentStateList.add(newState);
}
mMapForAllStates.put(startStateDesc, adjacentStateList);
// update mapping in startState
for (IState state : mAllStates) {
boolean isStartState = state.getStateDesc().equals(startState.getStateDesc());
if (isStartState) {
startState.addTransit(action, newState);
}
}
}
#Override
public void removeState(String targetStateDesc) {
// validate state
if (!mMapForAllStates.containsKey(targetStateDesc)) {
throw new RuntimeException("Don't have state: " + targetStateDesc);
} else {
// remove from mapping
mMapForAllStates.remove(targetStateDesc);
}
// update all state
IState targetState = null;
for (IState state : mAllStates) {
if (state.getStateDesc().equals(targetStateDesc)) {
targetState = state;
} else {
state.removeTransit(targetStateDesc);
}
}
mAllStates.remove(targetState);
}
#Override
public IState getCurrentState() {
return mCurrentState;
}
#Override
public void transit(Action action) {
if (mCurrentState == null) {
throw new RuntimeException("Please setup start state");
}
Map<String, IState> localMapping = mCurrentState.getAdjacentStates();
if (localMapping.containsKey(action.toString())) {
mCurrentState = localMapping.get(action.toString());
} else {
throw new RuntimeException("No action start from current state");
}
}
#Override
public IState getStartState() {
return mStartState;
}
#Override
public IState getEndState() {
return mEndState;
}
}
example:
public class example {
public static void main(String[] args) {
System.out.println("Finite state machine!!!");
IState startState = new StateImpl("start");
IState endState = new StateImpl("end");
IFiniteStateMachine fsm = new FiniteStateMachineImpl();
fsm.setStartState(startState);
fsm.setEndState(endState);
IState middle1 = new StateImpl("middle1");
middle1.addTransit(new Action("path1"), endState);
fsm.addState(startState, middle1, new Action("path1"));
System.out.println(fsm.getCurrentState().getStateDesc());
fsm.transit(new Action(("path1")));
System.out.println(fsm.getCurrentState().getStateDesc());
fsm.addState(middle1, endState, new Action("path1-end"));
fsm.transit(new Action(("path1-end")));
System.out.println(fsm.getCurrentState().getStateDesc());
fsm.addState(endState, middle1, new Action("path1-end"));
}
}
Full example on Github
Well this is an old question but while nobody mentioned here, I will advice to check two existing frameworks before you implement you own State Machines.
One is Spring State Machine most of you are familiar with Spring framework, which allow us to use several features of Spring like dependency injection and everything else that Spring can offer.
It is really great for modelling the lifecycle of an Apparat, with states like INITIALIZING, STARTED, ERROR, RECOVERING, SHUTTINGDOWN, etc.. but I see lots of people are trying to model a Shopping Chart, a Reservation System with it, the memory footprint a Spring State Machine is relatively big to model millions of Shopping Charts or Reservations.
One other disadvantage, Spring State Machine, while has a capability to persist itself for long running processes but it does not have any mechanism to adapt to changes in these processes, if you persist a process and you have to recover it lets say 10 days later with a change occurred in your business process because of new software release / requirement, you have no built in means to deal with it.
I have several blogs, blog1 blog2, demonstrating how you can program Spring State Machine, specially model driven way, if you want to check it.
Mainly because the disadvantages I mentioned, I advice you to look another framework first, Akka FSM (Finite State Machine) which is more fitting with its low memory footprint to have millions and millions of instances and has a capability to adapt changing long running processes.
Now you can develop with Akka framework with Java but believe me because of some missing language elements, you don't want to read the produced code, Scala is a much more fitting language to develop with Akka. Now I hear you saying Scala is too complex, I can't convince my project leads to develop with Scala, to convince you all this is an option, I developed a Proof of Concept application using a Java/Scala hybrid with all Scala Akka Finite State Machine code generated from an UML model, if you want to check it out here the links to the blogs, blog3 blog4.
I hope this information would help you.
Here is a SUPER SIMPLE implementation/example of a FSM using just "if-else"s which avoids all of the above subclassing answers (taken from Using Finite State Machines for Pattern Matching in Java, where he is looking for a string which ends with "#" followed by numbers followed by "#"--see state graph here):
public static void main(String[] args) {
String s = "A1#312#";
String digits = "0123456789";
int state = 0;
for (int ind = 0; ind < s.length(); ind++) {
if (state == 0) {
if (s.charAt(ind) == '#')
state = 1;
} else {
boolean isNumber = digits.indexOf(s.charAt(ind)) != -1;
if (state == 1) {
if (isNumber)
state = 2;
else if (s.charAt(ind) == '#')
state = 1;
else
state = 0;
} else if (state == 2) {
if (s.charAt(ind) == '#') {
state = 3;
} else if (isNumber) {
state = 2;
} else if (s.charAt(ind) == '#')
state = 1;
else
state = 0;
} else if (state == 3) {
if (s.charAt(ind) == '#')
state = 1;
else
state = 0;
}
}
} //end for loop
if (state == 3)
System.out.println("It matches");
else
System.out.println("It does not match");
}
P.S: Does not answer your question directly, but shows you how to implement a FSM very easily in Java.