Is it possible to query a collection that is dumped in a heap dump? I can obviously browse that collection using something like Eclipse MAT, but would really love to be able to actually call a getter on the collection object. This would obviously be a lot clearer than going through segments in a ConcurrentHashMap trying to find the mapping I need.
I suppose what I'm looking for is some way to 'rehydrate' the dumped state of a named collection so that it can then be manipulated using the standard APIs.
Does anyone know of any such utilities, or, if not, can someone provide some sample code or pointers as to how to achieve this?
You should be able to query all collections or single one with Object Query Language (OQL) in jhat.
You can't necessarily invoke arbitrary methods but you can write some complicated queries using the available functions.
A. Sundararajan has some interesting blog posts on the subject that showcase what you can do. See here and here.
For instance, you can find all instances of java.util.HashMap that have a key test with the following query:
select s from java.util.HashMap s where contains(s.table, function(it) {
if (it && it.key && it.key.value && it.key.value.toString() == 'test') {
return true;
}
return false;
})
This should find the same key in a java.util.concurrent.ConcurrentHashMap:
select s from java.util.concurrent.ConcurrentHashMap s where contains(s.segments, function(it) {
if (!it || !it.table) {
return false;
}
var i, e;
for (i=0; i < it.table.length; i = i + 1) {
e = it.table[i];
if (e) {
return e.key && e.key.value && e.key.value.toString() == 'test';
}
}
return false;
})
Related
I've got a query which returns data from several tables. In the application, these tables are classes one within another, for example a Client has several Orders, each Order has several OrderDetails, each OrderDetail has several Products, and so on... But I can't figure out a proper way to build the entire object in the app since the query returns one row for (let's just say) each product, so I have one client repeated over and over for every product it has bought.
So far I've tried this terribly inefficient code, and it works, problem is, it takes too much time for the app to process all of this information when it retrieves several clients.
boolean orderFound = false;
for (Order order1 : orders) {
if (order1 .getId() == order.getId()) {
orderFound = true;
if (od.getId() != 0) {
boolean odFound = false;
for (OrderDetail orderdetail : order1.getOrderDetail()) {
if (orderDetail.getId() == od.getId()) {
if (prod.getId() != 0) {
odFound = true;
boolean prodFound= false;
for (Product product: orderDetail.getProducts()) {
if (product.getId() == product.getId()) {
prodFound= true;
}
}
if (!prodFound) {
orderDetail.getProducts().add(dia);
}
}
if (!odFound) {
order1.getOrderDetail().add(od);
}
}
}
}
if (!orderFound) {
if (order.getId() != 0) {
orders.add(order);
This works, but there's gotta be a better way and I haven't found it. I've been told this can be solved using HashSets but I still don't know how to use them. Any help will be appreciated.
If you are open to using third party libraries, I think this is what you are looking for:
How to use hibernate to query for an object with a nested object that has a nested collection of objects
I'm trying to find an alternative to using java.utils.TreeMap in a threaded environment due to the memory TreeMap consumes and doesn't free, using Sun JDK 1.6. We have a constant resizing TreeMap, which needs to keep sorted by key:
public class WKey implements Comparable<Object> {
private Long ms = null;
private Long id = null;
public WKey(Long ms, Long id) {
this.ms = ms;
this.id = id;
}
#Override
public int hashCode() {
final int prime = 31;
int result = 1;
result = prime * result + ((id == null) ? 0 : id.hashCode());
result = prime * result + ((ms == null) ? 0 : ms.hashCode());
return result;
}
#Override
public boolean equals(Object obj) {
if (this == obj)
return true;
if (obj == null)
return false;
if (getClass() != obj.getClass())
return false;
WKey other = (WKey) obj;
if (id == null) {
if (other.id != null)
return false;
} else if (!id.equals(other.id))
return false;
if (ms == null) {
if (other.ms != null)
return false;
} else if (!ms.equals(other.ms))
return false;
return true;
}
#Override
public int compareTo(Object arg0) {
WKey k = (WKey) arg0;
if (this.ms < k.ms)
return -1;
else if (this.ms.equals(k.ms)) {
if (this.id < k.id)
return -1;
else if (this.id.equals(k.id)) {
return 0;
}
}
return 1;
}
}
Thread 1
-------------------------
Iterator<WKey> it = result.keySet().iterator();
if (it.hasNext()) {
WKey key = it.next();
/// Some processing here
result.remove(key);
}
Constantly retrieves the first element within the TreeMap and then
removes it.
Threads 2, 3, and 4
-------------------------
for (Object r : rs) {
Object[] row = (Object[]) r;
Long ms = ((Calendar) row[1]).getTimeInMillis();
Long id = (Long) row[0];
WKey key = new WKey(ms, id);
result.put(key, row);
}
Are bulk processing threads which process returned results from various
services, which are generally basic POJOs. POJOs are generated a key
based off their id and timestamp using the key above. I cannot
modify the POJO to implement a Comparator, so I must use this key.
After keys have been identified and process, they are inserted into a
shared tree map where they are getting pulled off in sorted order by
a processing thread.
We were using:
Map<WKey, Object[]> result =
Collections.synchronizedMap(new TreeMap<WKey, Object[]>());
We also tried using ConcurrentSkipListMap:
SortedMap<WKey, Object[]> result =
new ConcurrentSkipListMap<WKey, Object[]>();
We are experimenting with big data and need a collection which sufficiently utilizes memory any time remove or put is used in a threaded environment. We are inserting records by the hundred-thousands and removing elements from the top on a needed basis. We need a container which can scale. The problem with TreeMap is it never releases memory unless you recreate the container, new Collections.synchronizedMap(new TreeMap()) . This is an expensive operation to call in a threaded environment anytime a new entry is removed.
Alternatively, I've been experimenting with Javolution. It has a FastSortedMap, which seems to fit in nicely. However, I find their implementation and usage of the collection rather quirky and lacking sufficient documentation and examples.
They do have a few examples listed in the doc, which relate to the clases FastSortedMap is derived from, but nothing seems to work:
A high-performance hash map with real-time behavior. Related to FastCollection, fast map supports various views.
atomic() - Thread-safe view for which all reads are mutex-free and map updates (e.g. putAll) are atomic.
shared() - View allowing concurrent modifications.
parallel() - A view allowing parallel processing including updates.
sequential() - View disallowing parallel processing.
unmodifiable() - View which does not allow any modifications.
entrySet() - FastSet view over the map entries allowing entries to be added/removed.
keySet() - FastSet view over the map keys allowing keys to be added (map entry with null value).
values() - FastCollection view over the map values (add not supported).
I instantiated the following collection as a replacement to TreeMap:
private FastMap<WKey, Object[]> result =
new FastSortedMap<WKey, Object[]>().shared();
However, once another thread touches the container. All the member functions start to fail. I still encounter null values returned from result.iterator().next(), size() sometimes hangs, result.keySet().min() is very sluggish. result.get returns null. None of the examples in doc really show how the concurrent views are used, listed above. It's really frustrating.
I've looked a at Apache Collections, but I'm afraid I might experience the same issue as many of their sorting collections are derived from java.utils HashMaps and TreeMaps. I looked into Guava as well, but their sorted containers require you to implement comparable on both key and value. I was trying to avoid implementing comparable on the 'value'. I don't need to sort both objects. If I implemented comparable on the value, I would just use a sorted list, queue, or table. Highscale and Trove don't have ordered maps. Fastutils may be a candidate, but I'd have to synchronize everything manually, and I'm trying to save time.
I've reviewed others listed in the stackoverflow benchmark post, but the projects listed previously seem to be my best alternatives.
So far, I'm not convinced Javolution is everything they advertise on their site. My experience is that their implementation is very inconsistent, lacking documentation, and performs rather sluggish in threaded environments. TreeMap performs great; I just wish it wouldn't allocate in such large bursts and GC every now and then. However, I'm hoping there might be somebody out there to prove me wrong, may even demonstrate appropriate usage for Javolutions collections in a threaded environment.
Otherwise, if somebody knows a way around resizing Treemaps, without using 'new', or has solved similar/alternative instances working with threading and sorted maps, any info would be greatly appreciated!
I'm writing a function where I'm essentially doing the same thing over and over. I have the function listed below
public String buildGarmentsString(List<Garment> garments)
{
StringBuilder garmentString = new StringBuilder(10000);
for(int i=0;i<4;i++)
{
garmentString.append(this.garmentProductId(i,garments.get(i).getProductId()));
garmentString.append(this.garmentColor(i,garments.get(i).getColor()));
for(int j=0;j<garments.get(i).getSizes().size();j++)
{
//check xxsml
if(garments.get(i).getSizes().get(j).getXxsml() >0)
{
garmentString.append(this.garmentSizes(i, Size.xxsml(),garments.get(i).getSizes().get(j).getXxsml()));
}
//check xsml
if(garments.get(i).getSizes().get(j).getXsml() > 0)
{
garmentString.append(this.garmentSizes(i,Size.xsml(),garments.get(i).getSizes().get(j).getXsml()));
}
//check sml
if(garments.get(i).getSizes().get(j).getSml() > 0)
{
garmentString.append(this.garmentSizes(i,Size.sml(),garments.get(i).getSizes().get(j).getSml()));
}
//check med
if(garments.get(i).getSizes().get(j).getMed() > 0)
{
garmentString.append(this.garmentSizes(i,Size.med(),garments.get(i).getSizes().get(j).getMed()));
}
//check lrg
if(garments.get(i).getSizes().get(j).getLrg() > 0)
{
garmentString.append(this.garmentSizes(i,Size.lrg(),garments.get(i).getSizes().get(j).getLrg()));
}
//check xlrg
if(garments.get(i).getSizes().get(j).getXlg() > 0)
{
garmentString.append(this.garmentSizes(i,Size.xlg(),garments.get(i).getSizes().get(j).getXlg()));
}
//check xxlrg
if(garments.get(i).getSizes().get(j).getXxl() >0)
{
garmentString.append(this.garmentSizes(i,Size.xxlg(),garments.get(i).getSizes().get(j).getXxl()));
}
//check xxxlrg
if(garments.get(i).getSizes().get(j).getXxxl() >0)
{
garmentString.append(this.garmentSizes(i,Size.xxxlg(),garments.get(i).getSizes().get(j).getXxxl()));
}
}
}
}
This is my garmentSizes function:
public String garmentSizes(int garmentNumber, String size,int numberToSend)
{
String garmentSizes = "&garment["+garmentNumber+"][sizes]["+size+"]="+numberToSend;
return garmentSizes;
}
I'm trying to figure out how I can get this done with a lot less code. I've read that with functional programming you can do things like pass in functions to parameters to other functions. After doing some reading online, I think I want to do something like this but I'm not sure how or what the best approach would be.
I have done some reading here on stack overflow and I've seen people mention using either the Command pattern or FunctionalJava or LambdaJ for trying to approximate this feature in Java. I've read over the documentation for the two libraries and read the Wikipedia Article on the Command Pattern, but I'm still not sure how I would use any of those to solve my particular problem. Can somebody explain this to me? As somebody that has never done any functional programming this is a bit confusing.
You could use local variables to decrease the amount of repetition. Say bySize = garments.get(i).getSizes().get(j) for example.
instead of size.getXxsml(), size.getXsml() etc. you could use an enum for sizes and loop on sizes.
The whole thing would then look like:
for(int j=0;j<garments.get(i).getSizes().size();j++) {
bySize = garments.get(i).getSizes().get(j);
for (Size s : Size.values()) {
if (bySize.get(s) > 0) {
garmentString.append(garmentSizes(i, s, bySize.get(s)));
}
}
}
The bySize.get(s) method could be implemented either with a switch that directs to the right method or directly in the enum and you could get rid of the getXsml etc. methods.
The only thing which differs between all your checks is this:
getXxsml/xxsml, getXsml/xsml, getSml/sml, etc.
If you could pass these values (as strings) to some upper-level method, and if
that upper-level method could eval i.e. execute these strings, then you can just
have an array of these values and pass that array to that upper-level method.
In Java, you can do something similar with reflection.
All these checks could indeed be simplified to much less
code through the use of reflection.
Look at:
java.lang.Class
java.lang.reflect.Method
java.lang.reflect.Field
java.lang.reflect.Constructor
and you will see what I mean.
From your code it appears that some Class has the following methods:
xxsml(), xsml(), sml(), med(), ..., xxxlg()
to get the amounts (?) available for each size.
You can design your data better, like this:
Have a "Size" type, that enumerates all sizes (could be Enum or some class with attribute String key)
Have a method that returns a List of all known sizes.
replace the above methods with amountFor(Size) This could be backed by a Map<Size, Integer>
For backward compatibility, you could rewrite the old methods along the lines:
int xxsml() {
return amountFor(Size.XXSML); // assuming you have a singleton instance
// for each well known size
}
Of course, in getGarmentString, you would then loop through the List of all known sizes:
for (Size sz : Size.getAllKnownSizes()) {
if (garments.get(i).getSizes().get(j).amountFor(sz) > 0) {
... do whatever must be done here
}
}
In my java application I have a huge set of conditions which decides just one action. My question is how to make it look nice (I use NetBeans so I'd prefer solution that will not be broken by its code formatting function). I'd also like to have there as low amount of if/else statements as possible, because I think it will make it faster.
My original code was a mess, so I made an action diagram:. Take a copy if you want to play with it. Please keep in mind that the diagram is not perfect as to UML syntax, partly because I made it using google docs.
This is the code:
if (!config.get("checkForSpecials") || event.isNotSpecial()) {
if (config.get("filterMode").equals("blacklist")) {
if (!itemFilter.contains(event.getItem().getName())) {
item.process();
}
} else if (config.get("filterMode").equals("whitelist")) {
if (itemFilter.contains(event.getItem().getName())) {
item.process();
}
} else {
item.process();
}
}
There are two things I don't like about it - the conditions are not too clear (especially when I unfold full method names and config strings), and the fact that the process method call is there three times.
Factoring booleans out and caching return values from method calls can help clarify code.
In addition, plotting all the outcomes on a logic table can help. I use this tool to help.
With the linked tool:
A: config.get("filterMode").equals("blacklist")
B: config.get("filterMode").equals("whitelist")
C: filterContainsName (see below)
The tool churns out:
(!A && !B) || (!A && C) || (A && !C)
Which leads to the code below (with a small tweak that replaces (!A && C) with (B && C)):
boolean filterContainsName = itemFilter.contains(event.getItem().getName());
boolean useBlacklist = config.get("filterMode").equals("blacklist");
boolean useWhitelist = config.get("filterMode").equals("whitelist");
if (!config.get("safeMode") || event.isSafe()) {
if((!useBlackList && !useWhiteList) ||
( useWhiteList && filterContainsName) ||
( useBlackList && !filterContainsName)) {
item.process();
}
}
Use maps. The key of the map is the condition/case, the value is a single method class/anonymouse interface that contains the logic for that condition. Whenever you encounter a certain condition/case, you simply do a lookup in the map and execute the related function. This way you can even split up your logic-by-condition into seperate classes (if needed for sake of code beauty). As an added bonus you'll probably gain a performance bonus when the # of conditions > 10.
Looks good as it is to me. Perhaps you can isolate the valid conditions for calling item.process() to a method to make it more easier to understand.
if (!config.get("safeMode") || event.isSafe()) {
if (isItemValidForProcess(config, itemFilter, event)) {
item.process();
}
}
boolean isItemValidForProcess(config, itemFilter, event) {
String filterMode = config.get("filterMode");
if (filterMode.equals("whitelist")) {
return itemFilter.contains(event.getItem().getName());
}
if (filterMode.equals("blacklist")) {
return !itemFilter.contains(event.getItem().getName());
}
return true;
}
Believe it or not, the diagram is not that complex:)
There is no loop, and it is rather linear.
Here's a pseudo code that implements it
void action()
if <sort out specials>
if <is it special>
return;
if <check for unsafe items>
if not <safe items list contains item>
return;
if <filter status = on>
if < filter mode = whitelist>
if not <item is on filter>
return;
else // black list
if not <item is on filter>
return;
// finally!
[process item]
For really complex diagram, the answer is ... goto ...
I'm extending the LoadFunc. In the getNext function I'd like to skip returning a tuple under certain conditions - this way I could only load a sample of the data file. I tried returning null for the rows I don't want to return but the problem is that the method terminates after the first null Tuple is returned.
Does anyone know of a way to do this? Should I do it in a different method?
Thanks in advance.
(Assuming you mean LoadFunc in Pig ... )
I would suggest writing a new method that does what you want simply to not break the original documented use of the getNext() method.
You should look at the source for the Pig classes that extend LoadFunc and see how they implement getNext(). For example: TextLoader
From there it should be fairly trivial to do what you're trying to do.
Edit to try and offer a little more detailed help:
(This is using the TextReader as an example)
The getNext() method is reading from a RecordReader. It does this by calling RecordReader.nextKeyValue() to advance to the next record. You check to see if that's true (meaning it read a record) and if it is, you call RecordReader.getCurrentValue() to retrieve the value.
Lets say you only wanted every fifth one as a sample in getNext():
int count = 0;
Text myText = null;
whlie(myRecordReader.nextKeyValue() == true)
{
if (count == 4)
{
myText = (Text) myRecordReader.getCurrentValue();
break;
}
count++;
}
if (myText != null) // we didn't hit the end; we have a record
{
... // create the tuple
return myTuple;
}
else
return null;
(corrected my silly off-by-one mistake)