Develop Reference

O(1) in a Java algorithm

I have two endpoints, one responsible for receive transactions and other responsible for generate stats based on transactions from the last minute only.
To store them, I'm using a ConcurrentNavigableMap:
#Component
#Log
public class DatastoreComponent {
private ConcurrentNavigableMap<Long, List<Transaction>> transactions;
public DatastoreComponent() {
this.transactions = new ConcurrentSkipListMap<>();
}
public synchronized List<Transaction> addTransaction(Transaction t){
log.info("Adding transaction: "+t);
List<Transaction> transactionAtGivenTime = transactions.get(t.getTimestamp());
if(transactionAtGivenTime == null) transactionAtGivenTime = new ArrayList<>();
transactionAtGivenTime.add(t);
return transactions.put(t.getTimestamp(), transactionAtGivenTime);
}
I use the timestamp as key, so that I can get all transactions from last minute just tailing the map, as follow:
public StatisticsFacade aggregate(){
List<Transaction> validTransactions = new ArrayList<>();
dataStore.getTransactions().tailMap(sixtySecondsAgo())
.values()
.parallelStream()
.forEach(list -> validTransactions.addAll(list));
statsAgg.aggreate(validTransactions);
return this;
}
so far, so good (I guess?). well anyway, the process happens in the statsAgg.aggreate() method, and this method should be O(1). My implementation is like that:
public synchronized void aggreate(List<Transaction> validTransactions) {
if(validTransactions == null || validTransactions.isEmpty())
return;
this.avg = validTransactions.parallelStream().mapToDouble(a -> a.getAmount()).average().getAsDouble();
this.sum = validTransactions.parallelStream().mapToDouble(a -> a.getAmount()).sum();
this.max = validTransactions.parallelStream().mapToDouble(a -> a.getAmount()).max().getAsDouble();
this.min = validTransactions.parallelStream().mapToDouble(a -> a.getAmount()).min().getAsDouble();
this.count = new Long(validTransactions.size());
}
I'm not really sure that this is O(1) since I'm running through the list 4 times...I tried extract validTransactions.parallelStream().mapToDouble(a -> a.getAmount()) to a variable and re-use it, but of course, once the stream is processed, it is closed and I can't do anything.
So the question is: is this O(1) and if not, is there a way to run through the stream and too all this calculations at once?

An algorithm that solves your problem has to be at least O(n) complexity, as you have to go through each element in validTransactions at least once.
And it wouldn't become O(1) even if you run
validTransactions.parallelStream().mapToDouble(a -> a.getAmount()) just once.

Iterate over two TreeMap at the same time in Java

I have two maps:
Map<Date, List<Journey>> journeyMap = new TreeMap<Date, List<Journey>>
Map<Date, List<Job>> jobMap = new TreeMap<Date, List<Job>>
I used TreeMap because that means they're sorted by date but I want to go through both maps at the same time, get the values of Journey/Job, then do some work.
I think i could use generics, storing the Job/Journey as an Object, then checking the instanceOf but I'm not sure if thats the solution?
Thanks.

Even though the others are right, that there are better, safer and more comfortable ways to achive whatever you want, it is possible to iterate over (the entries of) two Maps (aka Collections) at the same time.
//replace keySet() with your favorite method in for-each-loops
Iterator<Date> journeyIterator = journeyMap.keySet().iterator()
Iterator<Date> jobIterator = jobMap.keySet().iterator();
while(journeyIterator.hasNext() && jobIterator.hasNext()){
Date journeyDate = journeyIter.next()
Date jobDate = jobIterator.next();
//... do whatever you want with the data
}
This code does explicitly, what a for-each-loop can do implicitly for one Collection. It retrieves the Iterator and gets the element from the Collection from it, much like reading a file.

You're making an assumption that these maps are having values sorted in the very same way, but this is definitely not correct. At least if you want to write a logic like this you need to declare the same implementing class as a reference:
TreeMap<Date, List<Journey>> journeyMap = new TreeMap<Date, List<Journey>>
TreeMap<Date, List<Job>> jobMap = new TreeMap<Date, List<Job>>
but believe me you don't want to do it.
You're right! Instead doing 2 maps create 1, holding pair of Job/Journey objects - create a JobJourneyHolder class which holds both objects, this will be a good solution.

Yes, defining a new class for that is definitely the solution, because it composes related objects together, which is very welcomed in OOP. And you should not forget to implement hashCode() and equals() methods to make such classes work properly in Java collections:
public final class JourneyJob {
final Journey journey;
final Job job;
public JourneyJob(Journey journey, Job job) {
if (journey == null || job == null)
throw new NullPointerException();
this.journey = journey;
this.job = job;
}
public int hashCode() {
return Objects.hash(journey, job);
}
public boolean equals(JourneyJob other) {
return other.job.equals(job) && other.journey.equals(journey);
}
}
To add elements to common Map:
Map<Date, List<JourneyJob>> map = new TreeMap<>();
...
if (map.contains(date)) {
map.get(date).add(new JourneyJob(journey, job));
} else {
map.put(date, new ArrayList<>(Arrays.asList(new JourneyJob(journey, job)));
}
...
To retrieve JourneyJob objects:
for (List<JourneyJob> jjList : map.values()) {
for (JourneyJob jj : jjList) {
journey = jj.journey;
job = jj.job;
//... do your work here
}
}
Or, if you use Java 8, this can be done using nested forEach():
map.values().stream().forEach(list ->
list.stream().forEach(jj -> {
Journey journey = jj.journey;
Job job = jj.job;
//... do your work here
})
);

Remove Element From Map Using Filter

I have a java.util.Map inside an rx.Observable and I want to filter the map (remove an element based on a given key).
My current code is a mix of imperative and functional, I want to accomplish this goal without the call to isItemInDataThenRemove.
public static Observable<Map<String, Object>> filter(Map<String, Object> data, String removeKey) {
return Observable.from(data).filter((entry) -> isItemInDataThenRemove(entry,removeKey));
}
private static boolean isItemInDataThenRemove(Map<String, Object> data, String removeKey) {
for (Map.Entry<String,Object> entry : data.entrySet()) {
if(entry.getKey().equalsIgnoreCase(removeKey)) {
System.out.printf("Found element %s, removing.", removeKey);
data.remove(removeKey);
return true;
}
}
return false;
}

The code you have proposed has a general problem in that it modifies the underlying stream while operating on it. This conflicts with the general requirement for streams for non-interference, and often in practice means that you will get a ConcurrentModificationException when using streams pipelines with containers that remove objects in the underlying container.
In any case (as I learned yesterday) there is a new default method on the Collection class that does pretty much exactly what you want:
private static boolean isItemInDataThenRemove(Map<String, Object> data, String removeKey) {
return data.entrySet().removeIf(entry -> entry.getKey().equalsIgnoreCase(removeKey));
}

WORKING CODE:
private static boolean isItemInDataThenRemove(Map<String, Object> data, String removeKey) {
data.entrySet().stream().filter(entry ->
entry.getKey().equalsIgnoreCase(removeKey)).forEach(entry -> {
data.remove(entry.getKey());
});
return true;
}

Java HashMaps with only put and get - possibly concurrency issues?

I've worked with ConcurrentHashMaps, but I'm not quite sure if that will cover all of the bases here.
I have a Spring Component. This Component will contain a Map. This will simply be a quick reference for objects in an external service. If the map does not contain a matching String, it will make a call to the external service, retrieve the object, and store it in the mapping. Then other classes can use the mapping for quick retrieval and usage. As such, there are only put() and get() operations performed on the map. Entries are never removed.
That being said, I'm a little concerned that ConcurrentHashMap may not provide the atomic control I'd like. Fetching SomeObject from the external service is potentially expensive. I'd rather not have two separate threads calling in at nearly the same time, resulting in multiple calls for the same value to the external service.
The idea is this:
Map<String, SomeObject> map = Collections.concurrentHashMap(
new HashMap<String, SomeObject>());
public SomeObject getSomeObject(String key){
if (!map.containsKey(key)){
map.put(key, retrieveSomeObjectFromService(key));
}
return map.get(key);
Or this:
Map<String, SomeObject> map = new HashMap<String, SomeObject>();
public SomeObject getSomeObject(String key){
synchronized(map){
if (!map.containsKey(key)){
map.put(key, retrieveSomeObjectFromService(key));
}
}
return map.get(key);
}
The former is certainly simpler, but the latter would ensure that one two or more threads won't try to simultaneously trigger a fetching of the same SomeObject. Alternatively, I suppose I could try locking out only gets that attempt to retrieve a SomeObject that is already in the process of being fetched and does not block retrieving SomeObjects that already exist, but that would require a wait mechanism on the various string values and I'm not sure how to best implement that.

I would suggest you do a little bit of both!
Fast path, just 1 get out of the concurrent hashmap.
Slow path, full sync and lock
private final ConcurrentHashMap<String, Object> map = new ConcurrentHashMap<String, Object>();
private final ReentrantLock lock = new ReentrantLock();
public Object getSomeObject(String key) {
Object value = map.get(key);
if (value == null) {
try {
lock.lock();
value = map.get(key);
if (value == null) {
value = retrieveSomeObjectFromService(key);
map.put(key, value);
}
} finally {
lock.unlock();
}
}
return value;
}
Do you understand why we need the 2nd get inside of the lock? Leaving that out leaves a case where we end up making the inside object twice, and having different copies of it floating around.
Also doing the assign the result to value and nullcheck vs using the contains method - understand why that is better? If we do a .contains then a .get, we just did 2 hashmap lookup. If I just do a get, I can cut my hashmap lookup time in half.
Another version Peter suggested.. less lines of code, but not my personal preference:
private final ConcurrentHashMap<String, Object> map = new ConcurrentHashMap<String, Object>();
public Object getSomeObject(String key) {
Object value = map.get(key);
if (value == null) {
synchronized (map) {
value = map.get(key);
if (value == null) {
value = retrieveSomeObjectFromService(key);
map.put(key, value);
}
}
}
return value;
}

Is there an elegant way to remove nulls while transforming a Collection using Guava?

I have a question about simplifying some Collection handling code, when using Google Collections (update: Guava).
I've got a bunch of "Computer" objects, and I want to end up with a Collection of their "resource id"s. This is done like so:
Collection<Computer> matchingComputers = findComputers();
Collection<String> resourceIds =
Lists.newArrayList(Iterables.transform(matchingComputers, new Function<Computer, String>() {
public String apply(Computer from) {
return from.getResourceId();
}
}));
Now, getResourceId() may return null (and changing that is not an option right now), yet in this case I'd like to omit nulls from the resulting String collection.
Here's one way to filter nulls out:
Collections2.filter(resourceIds, new Predicate<String>() {
#Override
public boolean apply(String input) {
return input != null;
}
});
You could put all that together like this:
Collection<String> resourceIds = Collections2.filter(
Lists.newArrayList(Iterables.transform(matchingComputers, new Function<Computer, String>() {
public String apply(Computer from) {
return from.getResourceId();
}
})), new Predicate<String>() {
#Override
public boolean apply(String input) {
return input != null;
}
});
But this is hardly elegant, let alone readable, for such a simple task! In fact, plain old Java code (with no fancy Predicate or Function stuff at all) would arguably be much cleaner:
Collection<String> resourceIds = Lists.newArrayList();
for (Computer computer : matchingComputers) {
String resourceId = computer.getResourceId();
if (resourceId != null) {
resourceIds.add(resourceId);
}
}
Using the above is certainly also an option, but out of curiosity (and desire to learn more of Google Collections), can you do the exact same thing in some shorter or more elegant way using Google Collections?

There's already a predicate in Predicates that will help you here -- Predicates.notNull() -- and you can use Iterables.filter() and the fact that Lists.newArrayList() can take an Iterable to clean this up a little more.
Collection<String> resourceIds = Lists.newArrayList(
Iterables.filter(
Iterables.transform(matchingComputers, yourFunction),
Predicates.notNull()
)
);
If you don't actually need a Collection, just an Iterable, then the Lists.newArrayList() call can go away too and you're one step cleaner again!
I suspect you might find that the Function will come in handy again, and will be most useful declared as
public class Computer {
// ...
public static Function<Computer, String> TO_ID = ...;
}
which cleans this up even more (and will promote reuse).

A bit "prettier" syntax with FluentIterable (since Guava 12):
ImmutableList<String> resourceIds = FluentIterable.from(matchingComputers)
.transform(getResourceId)
.filter(Predicates.notNull())
.toList();
static final Function<Computer, String> getResourceId =
new Function<Computer, String>() {
#Override
public String apply(Computer computer) {
return computer.getResourceId();
}
};
Note that the returned list is an ImmutableList. However, you can use copyInto() method to pour the elements into an arbitrary collection.

It took longer than #Jon Skeet expected, but Java 8 streams do make this simple:
List<String> resourceIds = computers.stream()
.map(Computer::getResourceId)
.filter(Objects::nonNull)
.collect(Collectors.toList());
You can also use .filter(x -> x != null) if you like; the difference is very minor.

Firstly, I'd create a constant filter somewhere:
public static final Predicate<Object> NULL_FILTER = new Predicate<Object>() {
#Override
public boolean apply(Object input) {
return input != null;
}
}
Then you can use:
Iterable<String> ids = Iterables.transform(matchingComputers,
new Function<Computer, String>() {
public String apply(Computer from) {
return from.getResourceId();
}
}));
Collection<String> resourceIds = Lists.newArrayList(
Iterables.filter(ids, NULL_FILTER));
You can use the same null filter everywhere in your code.
If you use the same computing function elsewhere, you can make that a constant too, leaving just:
Collection<String> resourceIds = Lists.newArrayList(
Iterables.filter(
Iterables.transform(matchingComputers, RESOURCE_ID_PROJECTION),
NULL_FILTER));
It's certainly not as nice as the C# equivalent would be, but this is all going to get a lot nicer in Java 7 with closures and extension methods :)

You could write your own method like so. this will filter out nulls for any Function that returns null from the apply method.
public static <F, T> Collection<T> transformAndFilterNulls(List<F> fromList, Function<? super F, ? extends T> function) {
return Collections2.filter(Lists.transform(fromList, function), Predicates.<T>notNull());
}
The method can then be called with the following code.
Collection c = transformAndFilterNulls(Lists.newArrayList("", "SD", "DDF"), new Function<String, Long>() {
#Override
public Long apply(String s) {
return s.isEmpty() ? 20L : null;
}
});
System.err.println(c);