I got several collections of objects I'm receiving from external API. For this example, lets say they look like this. In real scenario I can't modify those classes.
#Data
public class ExternalResourceA {
private LocalDate date;
private String type;
}
#Data
public class ExternalResourceB {
private LocalDate date;
private String id;
}
And I'm having my own class, that combines those two based on few business rules that are not important here. Also, same as above, this is generated class, can't edit it. All I can do is write wrapper class and translate it later to original one.
#Data
public class MyResource {
private LocalDate date;
private String type;
private String id;
}
For this example, let's say this is the data I'm getting from API
private List<ExternalResourceA> externalCollectionA() {
final List<ExternalResourceA> collection = new ArrayList<>();
final var today = LocalDate.now();
for(int i = 0 ; i < 36; i++) {
collection.add(new ExternalResourceA(today.minusMonths(i), "type" + i));
}
Collections.shuffle(collection);
return collection;
}
private List<ExternalResourceB> externalCollectionB() {
final List<ExternalResourceB> collection = new ArrayList<>();
final var today = LocalDate.now();
for(int i = 0 ; i < 36; i++) {
collection.add(new ExternalResourceB(today.minusMonths(i), "id" + i));
}
Collections.shuffle(collection);
return collection;
}
Now, I need to combine data from ExternalResourceA and ExternalResourceB from year 2019 and save it into MyResource. Some of data might be missing, for example I got A for march 2019, but I dont have B for march 2019.
I managed to do that like this
void filterResources() {
final var resourceAin2019 = getAFrom2019(externalCollectionA());
final var resourceBin2019 = getBFrom2019(externalCollectionB());
final List<MyResource> myCollection = new ArrayList<>();
for(int i = 0; i <= 12; i++) {
final var resource = new MyResource();
findA(resourceAin2019, i).ifPresent(res -> {
resource.setDate(res.getDate());
resource.setType(res.getType());
});
findB(resourceBin2019, i).ifPresent(res -> {
resource.setId(res.getId());
});
myCollection.add(resource);
}
myCollection.forEach(System.out::println);
}
private Optional<ExternalResourceA> findA(List<ExternalResourceA> list, int index) {
return list.size() > index ? Optional.ofNullable(list.get(index)) : Optional.empty();
}
private Optional<ExternalResourceB> findB(List<ExternalResourceB> list, int index) {
return list.size() > index ? Optional.ofNullable(list.get(index)) : Optional.empty();
}
private List<ExternalResourceA> getAFrom2019(List<ExternalResourceA> resource) {
return resource.stream()
.filter(res -> res.getDate().isAfter(LocalDate.parse("2019-01-01") || res.getDate().isEqual(LocalDate.parse("2019-01-01")))
.sorted(Comparator.comparing(ExternalResourceA::getDate))
.collect(Collectors.toList());
}
private List<ExternalResourceB> getBFrom2019(List<ExternalResourceB> resource) {
return resource.stream()
.filter(res -> res.getDate().isAfter(LocalDate.parse("2019-01-01") || res.getDate().isEqual(LocalDate.parse("2019-01-01")))
.sorted(Comparator.comparing(ExternalResourceB::getDate))
.collect(Collectors.toList());
}
And it kinda works but even in this simple example, there is a lot of almost identical functions, that just operate on other classes. In real scenario, this will grow even more, as I got much more structures I need to deal with. I'm wondering if there isn't possibility to make this much cleanier and simplier?
Edit#
After further checking, my solution isn't working as I expected, its simplier to show than explain, here is result when I'm missing data from ExternalResourceB on March 2019, end empty value appears on last element instead.
Related
I have the following classes:
#Getter
public class SomeClass implements Serializable {
private LocalDate date;
private String smth;
List<PairKeyValue> quotaParams;
}
The class PairKeyValue is just:
#Getter
public class PairKeyValue implements Serializable {
private String key;
private String value;
}
I want to do the following:
1) Check if in SomeClass's date equals sysdate then check value under key="somekey" in list<PairKeyValue> is equals to 1 (somekey = 1) then left it in list.
2) Check if in SomeClass's date NOT equals sysdate then check value under key="somekey" in List<PairKeyValue> is equals to 0 (somekey = 0) then left it in list.
3) And ignore other values.
So in the end I need a filtered list of only current values within SomeClass.
I have my realization but I don't like it is not using only stream API:
availableQuotes = ArrayList();
if (CollectionUtils.isNotEmpty(availableQuotes)) {
availableQuotes = availableQuotes
.stream()
.filter(this::checkDate).collect(toList());
}
private boolean checkDate (SomeClass someClass){
if (someClass.getDate().equals(LocalDate.now())) {
return checkDuration(someClass, "0");
} else {
return checkDuration(someClass, "1");
}
}
private boolean checkDuration (SomeClass someClass, String param){
List<PairKeyValue> quotaParams = someClass.getPairKeyValues().stream()
.filter(spKeyValue -> spKeyValue.getKey().equals("duration"))
.filter(spKeyValue -> spKeyValue.getValue().equals(param))
.collect(toList());
return (CollectionUtils.isNotEmpty(quotaParams));
}
I know it looks awful and I know it can be more readable so please help.
If I understood your question right, the last 2 functions can be resumed to the following:
availableQuotes = availableQuotes.stream()
.filter(availableQuote -> availableQuote.getQuotaParams().stream()
.anyMatch(quotaParam -> quotaParam.getKey().equals("duration")
&& quotaParam.getValue().equals(availableQuote.getDate().equals(LocalDate.now()) ? "0" : "1")))
.collect(Collectors.toList());
I mostly took your code and re-arranged it into a single filter.
My Rest API works fine. However, I'm concerned about concurrency issues, though I've tested via scripts and have yet to see any. In my studies, I encountered some material with regards to utilizing Atomic Values with concurrentHasMap to avoid what amounts to dirty reads. My questions is twofold. First, should I be concerned, given my implementation? Second, if I should be, what would be the most prudent way to implement Atomic values, if indeed I should? I've contemplated dropping the wrapper class for the RestTemplate and simply passing a String back to the Angular 4 component as a catalyst for speed, but given I may use the value objects elsewhere, I'm hesitant. See, implementation below.
#Service
#EnableScheduling
public class TickerService implements IQuoteService {
#Autowired
private ApplicationConstants Constants;
private ConcurrentHashMap<String,Quote> quotes = new ConcurrentHashMap<String, Quote>();
private ConcurrentHashMap<String,LocalDateTime> quoteExpirationQueue = new ConcurrentHashMap<String, LocalDateTime>();
private final RestTemplate restTemplate;
public TickerService(RestTemplateBuilder restTemplateBuilder) {
this.restTemplate = restTemplateBuilder.build();
}
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
public Quote getQuote(String symbol) {
if (this.quotes.containsKey(symbol)){
Quote q = (Quote)this.quotes.get(symbol);
//Update Expiration
LocalDateTime ldt = LocalDateTime.now();
this.quoteExpirationQueue.put(symbol, ldt.plus(Constants.getQuoteExpirationMins(),ChronoUnit.MINUTES));
return q;
} else {
QuoteResponseWrapper qRes = this.restTemplate.getForObject( Constants.getRestURL(symbol), QuoteResponseWrapper.class, symbol);
ArrayList<Quote> res = new ArrayList<Quote>();
res = qRes.getQuoteResponse().getResult();
//Add to Cache
quotes.put(symbol, res.get(0));
//Set Expiration
LocalDateTime ldt = LocalDateTime.now();
this.quoteExpirationQueue.put(symbol, ldt.plus(Constants.getQuoteExpirationMins(),ChronoUnit.MINUTES));
return res.get(0);
}
}
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
public ConcurrentHashMap<String,Quote> getQuotes(){
return this.quotes;
}
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
#Scheduled(fixedDelayString = "${application.quoteRefreshFrequency}")
public void refreshQuotes(){
if (quoteExpirationQueue.isEmpty()) {
return;
}
LocalDateTime ldt = LocalDateTime.now();
//Purge Expired Quotes
String expiredQuotes = quoteExpirationQueue.entrySet().stream().filter(x -> x.getValue().isBefore(ldt)).map(p -> p.getKey()).collect(Collectors.joining(","));
if (!expiredQuotes.equals("")) {
this.purgeQuotes(expiredQuotes.split(","));
}
String allQuotes = quoteExpirationQueue.entrySet().stream().filter(x -> x.getValue().isAfter(ldt)).map(p -> p.getKey()).collect(Collectors.joining(","));
List<String> qList = Arrays.asList(allQuotes.split(","));
Stack<String> stack = new Stack<String>();
stack.addAll(qList);
// Break Requests Into Manageable Chunks using property file settings
while (stack.size() > Constants.getMaxQuoteRequest()) {
String qSegment = "";
int i = 0;
while (i < Constants.getMaxQuoteRequest() && !stack.isEmpty()) {
qSegment = qSegment.concat(stack.pop() + ",");
i++;
}
logger.debug(qSegment.substring(0, qSegment.lastIndexOf(",")));
this.updateQuotes(qSegment);
}
// Handle Remaining Request Delta
if (stack.size() < Constants.getMaxQuoteRequest() && !stack.isEmpty()) {
String rSegment = "";
while (!stack.isEmpty()){
rSegment = rSegment.concat(stack.pop() + ",");
}
logger.debug(rSegment);
this.updateQuotes(rSegment.substring(0, rSegment.lastIndexOf(",")));
}
}
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
private void updateQuotes(String symbols) {
if (symbols.equals("")) {
return;
}
System.out.println("refreshing -> " + symbols);
QuoteResponseWrapper qRes = this.restTemplate.getForObject( Constants.getRestURL(symbols), QuoteResponseWrapper.class, symbols);
for (Quote q : qRes.getQuoteResponse().getResult()) {
this.quotes.put(q.getSymbol(), q);
}
}
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
private void purgeQuotes(String[] symbols) {
for (String q : symbols) {
System.out.println("purging -> " + q);
this.quotes.remove(q);
this.quoteExpirationQueue.remove(q);
}
}
}
Changed implementation of IQuoteService and implementation TickerService to use concurrenHashMap with Atomic References:
#Autowired
private ApplicationConstants Constants;
private ConcurrentHashMap<AtomicReference<String>,AtomicReference<Quote>>
quotes = new ConcurrentHashMap<AtomicReference<String>,AtomicReference<Quote>> ();
private ConcurrentHashMap<AtomicReference<String>,AtomicReference<LocalDateTime>> quoteExpirationQueue = new ConcurrentHashMap<AtomicReference<String>,AtomicReference<LocalDateTime>>();
private final RestTemplate restTemplate;
The code works precisely as it did prior with the with the new implementation being that it "should" ensure that updates to values are not partially read prior to being completely written, and the values obtained should be consistent. Given, I could find no sound examples and acquire no answers on this topic, I will test this and post any issues I find.
The main concurrency risks with this code come about if refreshQuotes() was to be called concurrently. If this is a risk, then refreshQuotes just needs to be marked as synchronized.
Working on the premise that refreshQuotes() is only ever called once at a time, and that Quote/LocalDateTime are both immutable; then the question appears to be does updating immutable values within a ConcurrentHashMap risk dirty reads/writes. The answer is no, the values are immutable and ConcurrentHashMap handles the concurrency of updating the references.
For more information, I strongly recommend reading JSR133 (The Java Memory Model). It covers in some detail when data will and will not become visible between threads. Doug Lea's JSR133 Cookbook will almost certainly give you far more information than you ever wanted to know.
Let's say I have a class Item like this:
class Item {
private long id;
private BigDecimal value1;
private BigDecimal value2;
private BigDecimal value3;
}
Then I have a list with many itens, I want to know the sum of each of the values:
So, I know I could do something like
BigDecimal v1 = list.stream().map(Item::value1).reduce(BigDecimal.ZERO, BigDecimal::add);
However, this way I would need to do the same for each value, I'd like to know if there's some way of summing each attribute into only one Dto like:
class TotalItem {
private BigDecimal value1;
private BigDecimal value2;
private BigDecimal value3;
}
TotalItem t = list.stream().map(???).reduce(BigDecimal.ZERO, BigDecimal::add);
Is this possible?
thanks in advance.
I didn't test it but I think that you can implement add function on Item like:
public Item add(Item other) {
Item newItem = new Item(this.value1 + other.value1,
this.value2 + other.value2,
this.value3 + other.value3);
return newItem;
}
and then do:
Item t = list.stream().reduce(BigDecimal.ZERO, Item::add);
How about the following way?
TotalItem t = new TotalItem();
list.stream().forEach(item -> {
t.value1+ = item.value1;
t.value2+ = item.value2;
t.value3+ = item.value3;
});
I'm making the assumption that Item/TotalItem are very large objects, which would make writing a toTotalItem and a summarise(TotalItem,TotalItem) by hand a large and laborious job. One that is completely boilerplate and easy to get wrong.
Change the data structure to be a list or map - This makes summarisation simpler, at the cost of readability of the code and type safety.
Use reflection to iterate over the fields.
TotalItem toTotalItem(Item item) {
Field[] targetFields = TotalItem.class.getFields();
Collection<Field> sourceFields = Item.class.getFields().stream()
.filter(x=>fieldNameIsIn(x, targetFields)
.collect(Collectors.toList());
TotalItem returnItem = new TotalItem();
for(Field field : sourceFields) {
toTargetField(field, targetFields).set(returnItem, (BigDecimal) field.get(item));
}
return returnItem;
}
boolean fieldNameIsIn(Field sourceField, Field[] targetFields) // exercise for the reader
Field toTargetField(Field sourceField, Field[] targetFields) // exercise for the reader
This code above is not neat, but should show the concept. The same concept could be used to summarise.
This reduces the amount of code you need to write, but at the cost of runtime speed. Reflection is also hard to get right and magic (which some developers do not like).
The faster option would be a custom annotation which adds in summarisation. However this would be large chunk of work. If you have a large number of objects that need this, then it may make sense. Like reflection it is hard to get right and magic (which some developers do not like). Luckily you do not need a build step as Javac supports annotation processing natively.
This answer is inspired by JDK way of doing similar operations. Namely, I'll be referencing DoubleSummaryStatistics class.
First, we define a holder for information about BigDecimals we'll be collecting:
public class BigDecimalSummaryStats {
private long count;
private MathContext mc;
private BigDecimal sum = BigDecimal.ZERO;
private BigDecimal max;
private BigDecimal min;
public BigDecimalSummaryStats(MathContext mathCtx) {
mc = requireNonNull(mathCtx);
}
public Supplier<BigDecimalSummaryStats> supplier(MathContext ctx) {
return () -> new BigDecimalSummaryStats(ctx);
}
public void accept(BigDecimal value) {
requireNonNull(value);
count++;
sum = sum.add(value, mc);
min = min.min(value);
max = max.max(value);
}
public void combine(BigDecimalSummaryStats other) {
requireNonNull(other);
count += other.count;
sum = sum.add(other.sum, mc);
min = min.min(other.min);
max = max.max(other.max);
}
public long getCount() {
return count;
}
public BigDecimal getSum() {
return sum;
}
public BigDecimal getMax() {
return max;
}
public BigDecimal getMin() {
return min;
}
public BigDecimal getAverage() {
long c = getCount();
return c == 0 ? BigDecimal.ZERO : getSum().divide(BigDecimal.valueOf(c), mc);
}
}
This will provide with nice general utility suitable for collecting summary from arbitrary sequences of BigDecimal values.
Then, we can define a Summary class for our Item:
public class ItemSummaryStats {
private BigDecimalSummaryStats value1;
private BigDecimalSummaryStats value2;
private BigDecimalSummaryStats value3;
// ... other fields as needed
public ItemSummaryStats(MathContext math) {
value1 = new BigDecimalSummaryStats(math);
value2 = new BigDecimalSummaryStats(math);
value3 = new BigDecimalSummaryStats(math);
}
public void accept(Item item) {
value1.accept(item.value1);
value2.accept(item.value2);
value3.accept(item.value3);
// ... other fields as needed
}
public void combine(ItemSummaryStats other) {
value1.combine(other.value1);
value2.combine(other.value2);
value3.combine(other.value3);
}
public TotalItem get(
Function<BigDecimalSummaryStats, BigDecimal> v1Mapper,
Function<BigDecimalSummaryStats, BigDecimal> v2Mapper,
Function<BigDecimalSummaryStats, BigDecimal> v3Mapper) {
TotalItem t = new TotalItem();
t.value1 = v1Mapper.get(value1);
t.value2 = v2Mapper.get(value2);
t.value3 = v3Mapper.get(value3);
return t;
}
public TotalItem getSum() {
return get(BigDecimalSummaryStats::getSum,
BigDecimalSummaryStats::getSum,
BigDecimalSummaryStats::getSum);
}
public TotalItem getAverage() {
return get(BigDecimalSummaryStats::getAverage,
BigDecimalSummaryStats::getAverage,
BigDecimalSummaryStats::getAverage);
}
public TotalItem getMin() {
return get(BigDecimalSummaryStats::getMin,
BigDecimalSummaryStats::getMin,
BigDecimalSummaryStats::getMin);
}
//.... other methods basically all the same. You get the idea.
}
And finally we use this goodness like this:
TotalItem totals = list.stream().collect(
Collector.of(() -> new ItemStatsSummary(MathContext.DECIMAL64),
ItemStatsSummary::accept,
ItemStatsSummary::combine,
ItemStatsSummary::getSum)
)
Cons of this approach:
Slightly longer development time than adhoc solutions.
Are far outweighed by the pros, or at least I'm convinced of it:
Follows separation of concerns principle: Item Stats are not actually concerned how to collect summary of specific field: they can trust that BigDecimalSummary works
Testable: Each part can be tested in its own suite. You can trust that every field will work the same since they use the same API.
Flexible: get(Function...) method exposes a big list of possibilities: you can collect sum of first field, an averabe of second and min of third if needed.
I have this class that serves as a container which I will use the instance variable for processing later
class Data{
static int counter= 0;
boolean boolean1;
String string1;
public Data() {
counter++;
}
}
And I have this method that sets the values of Data
public Data setData()
{
Data data = null;
for (int i = 0; i < somecoutnerhere; i++) {
Data = new Data();
Data.boolean1 = some boolean put here;
Data.string1 = "some string to be put here";
}
return ProcessData(Data);
}
I also have this class ProcessData that will make use of Data and will construct the response
private class ProcessData
{
private final Map<String, List<?>> map = new HashMap<String, List<?>>();
int counter;
public ProcessData(Data data)
{
map.put("boolean1", data.boolean1);
map.put("String1", data.string1);
counter = data.counter;
}
public String someMethodToGenerateReturnData(){
// some code here to make use of the Data collected. Will basically use map to construct the return String
}
}
My problem is that I couldn't figure out how can I return all the instance variables created on the for-loop for Data on setData(). Any thoughts?
My problem is that I couldn't figure out how can I return all the instance variables created on the for-loop for Data on setData(). Any thoughts?
According to this your problem is not "returning all instance one variables in one call", as your title states, but rather a question about how returning all Data-Objects created in your for-loop, which is easier.
Your code is erronous though, so I went ahead & corrected it (I hope I didn't mess up). I also renamed a few things.
The changes I made are:
renamed "boolean1" and "string1" to "trueOrFalse" and "string"
added a public, fully parameterized constructor to the Data-class
added a ProcessData-list to the setData()-method, which is filled in the for-loop
(+ a comment)
However, I'd strongly recommend you to check your architecture, and also to learn a bit about naming conventions, or coding conventions in general. Names should point out the purpose or content of the method/variable/class, and "boolean1" isn't really doing that.
Regarding the architecture: The Data-class seems to exist solely for the counter, and you could easily change that, making the Data-class obsolete (unless it's used somewhere else).
Data class:
class Data {
static int counter = 0;
boolean trueOrFalse;
String string;
public Data() {
counter++;
}
public Data(boolean someBoolean, String someString) {
this.trueOrFalse= someBoolean;
this.string = someString;
counter++;
}
}
setData()-Method:
public List<ProcessData> setData() {
List<ProcessData> processedDataList = new ArrayList<ProcessData>();
for (int i = 0; i < someCounterHere; i++) {
processedDataList.add(new ProcessData(new Data(true, "testString"));
// a new Data-object is created (parameters true and "testString")
// a new ProcessData-object is created (parameter is the newly created Data-Object)
// the newly created ProcessData-object is added to the list
}
return processedDataList;
}
ProcessData-class:
private class ProcessData {
private final Map<String, List<?>> map = new HashMap<String, List<?>>();
int counter;
public ProcessData(Data data) {
map.put("trueOrFalse", data.trueOrFalse);
map.put("string", data.string);
counter = data.counter;
}
public String someMethodToGenerateReturnData() {
// some code here to make use of the Data collected. Will basically use map to construct the return String
}
}
Somehow my old question was closed, so I open a new one:
I am using Java Generics to implement a generic bidirectional Hash Map out of an SQL Query. It should be able to map any combination of String, Integer pairs back and forth. It should be used like this:
String sql = "SELECT string_val, int_val FROM map_table";
PickMap<String, Integer> pm1 = new PickMap<String, Integer>(sql);
String key1 = "seven";
Integer value1 = pm1.getLeft2Right(key1);
Integer key2 = 7;
String value2 = pm1.getRightToLeft(key2);
Of course it should be possible to create an pm (Integer, Integer) and so on...
My implementation of Pick Map looks like this (without the getter...):
public class PickMap<L, R> {
private final HashMap<L, R> left2Right = new HashMap<L, R>();
private final HashMap<R, L> right2Left = new HashMap<R, L>();
public PickMap(String sql) throws OException {
DTable d = new DTable(sql);
int colTypeL = d.t.getColType(1);
int colTypeR = d.t.getColType(2);
Extractor<L> extLeft = (Extractor<L>) getInstance(colTypeL);
Extractor<R> extRight = (Extractor<R>) getInstance(colTypeR);
int numRows = d.t.getNumRows();
for(int i=1;i<=numRows;i++) {
L leftVal = extLeft.extract(d, i);
R rightVal = extRight.extract(d, i);
this.left2Right.put(leftVal, rightVal);
this.right2Left.put(rightVal, leftVal);
}
}
private Extractor<?> getInstance(int type) {
if(type == 1)
return new IntExtractor();
else
return new StringExtractor();
}
}
interface Extractor<E> {
E extract(DTable source, int row);
}
class IntExtractor implements Extractor<Integer> {
#Override
public Integer extract(DTable source, int row) {
int value = 5;
return new Integer(value);
}
}
class StringExtractor implements Extractor<String> {
#Override
public String extract(DTable source, int row) {
String retVal = "hello";
return retVal;
}
}
I have no compiler errors and I'm pretty sure, that it will work this way. BUT I'm getting unchecked cast warnings on the "getInstance" methods Where I cast Extractor(E) to Extractor(L)...
How should I cast properly? Or what am I missing? Or should I just suppress those warnings?
You're getting warnings because what you're doing can't be proved to be safe. You're assuming that getInstance(colTypeL) will return an Extractor<L> - but that can't be verified at either compile-time or execution time.
You can use #SuppressWarnings("unchecked") as mentioned by others, but I would try to rethink the design somewhat.
You can use the following annotation to make the compiler not output those warnings:
#SuppressWarnings("unchecked")
See this related question which deals with the same issue. The answer there will explain everything you need to know.
If you are using the Spring Framework, you can use CastUtils:
import static org.springframework.data.util.CastUtils.cast;
obj.setString(cast(someObject));
String bob = cast(someObject);