Develop Reference

How to get java object itself when its about to be collected

How can we execute a piece of code using the object (its state is needed) before it gets collected if we don't have control over its source (cant enforce implementing some interface or finally block)?
Java Reference types allow us to access an object if someone else makes it strongly reachable + if we use reference queues we can also be notified when the object is collected, unless my understanding is wrong that's all you can do with reference types, no matter what you use at any point the object is either strongly reachable or its gone and you have null.
All i really need is a way to get notified when specific object is about to be collected.

There is a reason why the Reference API doesn’t allow to retrieve the collected object: allowing to make a collected object reachable again, like happening with the finalize() method, is exactly what is not intended.
The standard approach is to create subclasses of the reference types to store the information associated with the referent, e.g. everything necessary to perform the cleanup action, within the specialized reference object. Of course, this information must not include strong references to the referent itself.
private static final ReferenceQueue<Integer> QUEUE = new ReferenceQueue<>();
static class IntegerPhantomReference extends PhantomReference<Integer> {
final int value;
public IntegerPhantomReference(Integer ref) {
super(ref, QUEUE);
value = ref.intValue();
}
public String toString() {
return "Integer[value="+value+"]";
}
}
private static final Set<IntegerPhantomReference> REGISTERED = new HashSet<>();
public static void main(String[] args) throws InterruptedException {
List<Integer> stronglyReferenced = new ArrayList<>();
for(int i = 0; i < 10; i++) {
Integer object = new Integer(i);
stronglyReferenced.add(object);
REGISTERED.add(new IntegerPhantomReference(object));
}
gcAndPoll("initial");
stronglyReferenced.removeIf(i -> i%2 == 0);
gcAndPoll("after removing even");
stronglyReferenced.clear();
gcAndPoll("after remove all");
if(REGISTERED.isEmpty()) System.out.println("all objects collected");
}
private static void gcAndPoll(String msg) throws InterruptedException {
System.out.println(msg);
System.gc(); Thread.sleep(100);
for(;;) {
Reference<?> r = QUEUE.poll();
if(r == null) break;
System.out.println("collected "+r);
REGISTERED.remove(r);
}
}
initial
after removing even
collected Integer[value=4]
collected Integer[value=8]
collected Integer[value=6]
collected Integer[value=2]
collected Integer[value=0]
after remove all
collected Integer[value=1]
collected Integer[value=5]
collected Integer[value=3]
collected Integer[value=7]
collected Integer[value=9]
all objects collected
For completeness, there is a hack that allows to resurrect a collected object, which will stop working in Java 9.
The documentation of PhantomReference says:
Unlike soft and weak references, phantom references are not automatically cleared by the garbage collector as they are enqueued.
It’s not clear why this has been specified and the get() method of PhantomReference has been overridden to always return null, exactly to disallow taking any benefit from the fact that this reference has not been cleared. Since the purpose of this special behavior is unclear, it has been removed from the specification in Java 9 and these references are automatically cleared like any other.
But for previous versions, it is possible to use Reflection with access override to access the referent, to do exactly what the API was not intended to allow. Needless to say, that’s just for informational purpose and is strongly discouraged (and as said, it stops working in Java 9).
private static final ReferenceQueue<Integer> QUEUE = new ReferenceQueue<>();
private static final Set<PhantomReference<Integer>> REGISTERED = new HashSet<>();
public static void main(String[] args)
throws InterruptedException, IllegalAccessException {
List<Integer> stronglyReferenced = new ArrayList<>();
for(int i = 0; i < 10; i++) {
Integer object = new Integer(i);
stronglyReferenced.add(object);
REGISTERED.add(new PhantomReference<>(object, QUEUE));
}
gcAndPoll("initial");
stronglyReferenced.removeIf(i -> i%2 == 0);
gcAndPoll("after removing even");
stronglyReferenced.clear();
gcAndPoll("after remove all");
if(REGISTERED.isEmpty()) System.out.println("all objects collected");
}
static final Field REFERENT;
static {
try {
REFERENT = Reference.class.getDeclaredField("referent");
REFERENT.setAccessible(true);
} catch (NoSuchFieldException ex) {
throw new ExceptionInInitializerError(ex);
}
}
private static void gcAndPoll(String msg)
throws InterruptedException, IllegalAccessException {
System.out.println(msg);
System.gc();
Thread.sleep(100);
for(;;) {
Reference<?> r = QUEUE.poll();
if(r == null) break;
Object o = REFERENT.get(r);
System.out.println("collected (and now resurrected)"+o);
REGISTERED.remove(r);
}
}

Why another branch is unreachable in my code?

Why the output of the following code is always suck. How to get happy as the output? Why the happy branch is unreachable?
public class HowToMakeStackoverflowBetter {
private static final int HUMAN_PATIENCE = 10;
private List<Member> members = new ArrayList<>();
private int atmosphere = -10;
private Random r = new Random();
public HowToMakeStackoverflowBetter(int size) {
for (int i = 0; i < size; i++) { members.add(new Member()); }
}
public Member pick() { return members.get(r.nextInt(members.size())); }
public class Member {
private int patience = HUMAN_PATIENCE;
private Question question = null;
public Member() { patience = r.nextInt(patience+1) + atmosphere; }
public void vote(Question q) {
if (patience >= 0) {
voteUp(q);
} else {
voteDown(q);
}
}
public void ask() {
question = new Question();
for (Member member : members) {
member.vote(question);
}
}
private void voteUp(Question q) { ++q.vote; }
private void voteDown(Question q) { --q.vote; }
public String toString() {
return (question.vote >= 0)? "Happy!" : "Suck!";
}
}
public class Question { private int vote; }
public static void main(String[] args) {
HowToMakeStackoverflowBetter stackoverflow = new HowToMakeStackoverflowBetter(100);
Member me = stackoverflow.pick();
me.ask();
System.out.println(me);
}
}
After a 1000 times loop, it gives us 1000 sucks. I remember 2 or 3 years ago, this was not the case. Something changed.

Two problems. First:
linkedList::linkedList(){
*sentinel.last=sentinel;
*sentinel.next=sentinel;
sentinel.str="I am sentinel!!";
};
sentinel is your member variable, and .last is its pointer to another node. This hasn't been initialised, so trying to use it is undefined behaviour. In practice, it's effectively pointing at a random address in (or out of) memory, and you attempt to dereference the pointer then copy the entire sentinel object over the node at the imagined pointed-to address: i.e. you try to copy the 3 pointers in the sentinel node member variable to a random address in memory.
You probably want to do this:
linkedList::linkedList()
{
sentinel.last = &sentinel;
sentinel.next = &sentinel;
sentinel.str = "I am sentinel!!";
}
Secondly, you explicitly call the destructor for linkedList, which results in undefined behaviour when the compiler-arranged destruction is performed as the object leaves the stack scope it's created in - i.e. at the end of main().
I suggest you change node.str to be a std::string, as in any realistic program you'll want to be able to handle variable text, and not just point to (constant) string literals. As is, if you mix string literals and free-store allocated character arrays, you'll have trouble knowing when to call delete[] to release the memory. You could resolve this by always making a new copy of the string data to be stored with new[], but it's safer and easier to use std::string.

Since you allocated it as a local variable, your mylist will be destroyed automatically upon exiting main. Since you've already explicitly invoked its destructor, that leads to undefined behavior (attempting to destroy the same object twice).
As a quick guideline, essentially the only time you explicitly invoke a destructor is in conjunction with placement new. If you don't know what that is (yet), that's fine; just take it as a sign that you shouldn't be invoking destructors.

You forgot to initialize sentinel
In code below you are trying to initialize sentinel (which is not yet constructed) with sentinel(same thing). So you have to pass something to constructor which can be used to initialize your member variable sentinel
*sentinel.last=sentinel;
Also no need to call destructor like this. Destructor will be called once your myList goes out of scope.
myList.~linkedList();

the program may crash, with this:
*sentinel.last=sentinel;
*sentinel.next=sentinel;
sentinel is not initialized sot i has random value on stack.

You're trying to de-reference the pointers last and next of member variable sentinel when they are not yet initialized.
And these de-references *sentinel.last=sentinel *sentinel.next=sentinel are causing the crash because without assigning the values to pointers you're changing the value pointed by the pointers.
You can do like this
sentinel.last=&sentinel;
sentinel.next=&sentinel;
And as pointed out by other explicit destructor calls aren't need here.

JIT Optimization and Weak References

I have the following piece of code:
private final List<WeakReference<T>> slaves;
public void updateOrdering() {
// removes void weak references
// and ensures that weak references are not voided
// during subsequent sort
List<T> unwrapped = unwrap();
assert unwrapped.size() == this.slaves.size();
// **** could be reimplemented without using unwrap() ****
Collections.sort(this.slaves, CMP_IDX_SLV);
unwrapped = null;// without this, ....
}
Method unwrap() just creates a list of T's referenced by the weak references in slaves
and as a side effect eliminates the weak references referencing null in slaves.
Then comes the sort which relies on that each member of slaves references some T;
otherwise the code yields a NullPointerException.
Since unwrapped holds a reference on each T in slaves, during sorting no GC eliminates a T. Finally, unwrapped = null eliminates the reference on unwrapped
and so releases GC again. Seems to work quite well.
Now my question:
If I remove unwrapped = null; this results in NullPointerExceptions when running many tests under some load. I suspect that the JIT eliminates List<T> unwrapped = unwrap();
and so GC applies to the T's in slaves during sorting.
Do you have another explanation? If you agree with me, is this a bug in the JIT?
I personally think that unwrapped = null should not be necessary, because unwrapped is removed from the frame as soon as updateOrdering() returns. Is there a specification what may be optimized and what is not?
Or did I do the thing in the wrong way? I have the idea to modify comparator that it allows weak references on null. What do you think about that?
Thanks for suggestions.
Add on (1)
Now I want to add some missing pieces of information:
First of all Java version:
java version "1.7.0_45"
OpenJDK Runtime Environment (IcedTea 2.4.3) (suse-8.28.3-x86_64)
OpenJDK 64-Bit Server VM (build 24.45-b08, mixed mode)
Then someone wanted to see method unwrap
private synchronized List<T> unwrap() {
List<T> res = new ArrayList<T>();
T cand;
WeakReference<T> slvRef;
Iterator<WeakReference<T>> iter = this.slaves.iterator();
while (iter.hasNext()) {
slvRef = iter.next();
cand = slvRef.get();
if (cand == null) {
iter.remove();
continue;
}
assert cand != null;
res.add(cand);
} // while (iter.hasNext())
return res;
}
Note that while iterating, void references are removed.
In fact i replaced this method by
private synchronized List<T> unwrap() {
List<T> res = new ArrayList<T>();
for (T cand : this) {
assert cand != null;
res.add(cand);
}
return res;
}
using my own iterator but functionally this should be the same.
Then someone wantet the stacktrace. Here is a piece of it.
java.lang.NullPointerException: null
at WeakSlaveCollection$IdxComparator.compare(WeakSlaveCollection.java:44)
at WeakSlaveCollection$IdxComparator.compare(WeakSlaveCollection.java:40)
at java.util.TimSort.countRunAndMakeAscending(TimSort.java:324)
at java.util.TimSort.sort(TimSort.java:189)
at java.util.TimSort.sort(TimSort.java:173)
at java.util.Arrays.sort(Arrays.java:659)
at java.util.Collections.sort(Collections.java:217)
at WeakSlaveCollection.updateOrdering(WeakSlaveCollection.java:183)
it points into the comparator, the line with the return.
static class IdxComparator
implements Comparator<WeakReference<? extends XSlaveNumber>> {
public int compare(WeakReference<? extends XSlaveNumber> slv1,
WeakReference<? extends XSlaveNumber> slv2) {
return slv2.get().index()-slv1.get().index();
}
} // class IdxComparator
and finally,
private final static IdxComparator CMP_IDX_SLV = new IdxComparator();
is an important constant.
Add on (2)
Observed now that indeed NPE occurs even if 'unwrapped = null' is present in updateOrdering().
Weak references may be removed by java runtime
if no strict reference holds after jit optimization.
The source code seems not important at all.
I solved the problem the following way:
public void updateOrdering() {
Collections.sort(this.slaves, CMP_IDX_SLV);
}
without any decoration inserted to prevent slaves to be garbage collected
and the comparator in CMP_IDX_SLV enabled to handle weak references to null:
public int compare(WeakReference<? extends XSlaveNumber> slv1,
WeakReference<? extends XSlaveNumber> slv2) {
XSlaveNumber sSlv1 = slv1.get();
XSlaveNumber sSlv2 = slv2.get();
if (sSlv1 == null) {
return sSlv2 == null ? 0 : -1;
}
if (sSlv2 == null) {
return +1;
}
assert sSlv1 != null && sSlv2 != null;
return sSlv2.index()-sSlv1.index();
}
As a side effect, ordering the underlying list List> slaves;
puts the void weak references at the end of the list, where it can be collected later.

I examine your source code, and I got NullPointerException when JIT compile my method corresponding to your method "updateOrdering" and GC occurs during sorting.
But I got NullPointerException when Collections.sort whether with or without unwrapped = null.
This maybe occurs difference between my sample source code and yours, or Java version difference. I will examine if you tell Java version.
I use java below version.
java version "1.7.0_40"
Java(TM) SE Runtime Environment (build 1.7.0_40-b43)
Java HotSpot(TM) 64-Bit Server VM (build 24.0-b56, mixed mode)
If you want to cheat on JIT compilation, the below code insert your source code instead unwrapped = null(e.g.). Then, JIT compilation doesn't eliminates unwrapped code.
long value = unwrapped.size() * unwrapped.size();
if(value * value % 3 == 1) {
//Because value * value % 3 always is 1 or 0, this code can't reach.
//Insert into this the source code that use unwrapped array, for example, show unwrapped array.
}
My examination result is below.
If JIT don't optimize my method corresponding to updateOrdering, no NullPointerException occurs.
If JIT optimize my method, then NullPointerException occurs at some point.
If JIT optimize my method inserting the above source code cheating JIT compiler, then no NullPointerException occurs.
So, I(and you) suggest JIT optimze eliminates unwrapped code, then NullPointerException occurs.
By the way, if you want to show JIT compiler optimization, you invoke java with -XX:+PrintCompilation.
If you want to show GC, with -verbose:gc.
Just for information, my sample source code is below.
public class WeakSampleMain {
private static List<WeakReference<Integer>> weakList = new LinkedList<>();
private static long sum = 0;
public static void main(String[] args) {
System.out.println("start");
int size = 1_000_000;
for(int i = 0; i < size; i++) {
Integer value = Integer.valueOf(i);
weakList.add(new WeakReference<Integer>(value));
}
for(int i = 0; i < 10; i++) {
jitSort();
}
GcTask gcTask = new GcTask();
Thread thread = new Thread(gcTask);
thread.start();
for(int i = 0; i < 100000; i++) {
jitSort();
}
thread.interrupt();
System.out.println(sum);
}
public static void jitSort() {
List<Integer> unwrappedList = unwrapped();
removeNull();
Collections.sort(weakList,
new Comparator<WeakReference<Integer>>() {
#Override
public int compare(WeakReference<Integer> o1,
WeakReference<Integer> o2) {
return Integer.compare(o1.get(), o2.get());
}
}
);
for(int i = 0; i < Math.min(weakList.size(), 1000); i++) {
sum += weakList.get(i).get();
}
unwrappedList = null;
// long value = (sum + unwrappedList.size());
// if((value * value) % 3 == 2) {
// for(int i = 0; i < unwrappedList.size(); i++) {
// System.out.println(unwrappedList.get(i));
// }
// }
}
public static List<Integer> unwrapped() {
ArrayList<Integer> list = new ArrayList<Integer>();
for(WeakReference<Integer> ref : weakList) {
Integer i = ref.get();
if(i != null) {
list.add(i);
}
}
return list;
}
public static void removeNull() {
Iterator<WeakReference<Integer>> itr = weakList.iterator();
while(itr.hasNext()) {
WeakReference<Integer> ref = itr.next();
if(ref.get() == null) {
itr.remove();
}
}
}
public static class GcTask implements Runnable {
private volatile int result = 0;
private List<Integer> stockList = new ArrayList<Integer>();
public void run() {
while(true) {
if(Thread.interrupted()) {
break;
}
int size = 1000000;
stockList = new ArrayList<Integer>(size);
for(int i = 0; i < size; i++) {
stockList.add(new Integer(i));
}
if(System.currentTimeMillis() % 1000 == 0) {
System.out.println("size : " + stockList.size());
}
}
}
public int getResult() {
return result;
}
}
}

As of Java 9, the correct way to prevent the JIT from discarding unwrapped is to use Reference.reachabilityFence:
public void updateOrdering() {
List<T> unwrapped = unwrap();
Collections.sort(this.slaves, CMP_IDX_SLV);
Reference.reachabilityFence(unwrapped);
}
The presence of the reachabilityFence call causes unwrapped to be considered strongly reachable before the call, preventing collection of unwrapped or its elements until the sort completes. (The strange way in which reachabilityFence's effects seem to propagate backward in time is because it behaves primarily as a JIT directive.) Without reachabilityFence, unwrapped can be collected once the JIT can prove it will never again be accessed, even though the variable is still in scope.

Your question
If I remove unwrapped = null; this results in NullPointerException when running many tests under some load.
According to my understanding I do not think so that unwrapped = null; makes any difference.
Yes, I have also read that making objects = null sometime increases the probability the object referenced will be GC'ed but I don't think it matters here because once the method ends, scope of unwrapped ends and is eligible for GC'ed and in your function sorting Collections.sort(this.slaves, CMP_IDX_SLV); is done prior to unwrapped = null; so it make no sense the you get NPE when adding or removing them.
I think it is just a coincidence that you get NPE, I believe if you run the test again you will get NPE with that statement also.
If you read Java Documentation
Weak reference objects, which do not prevent their referents from being made finalizable, finalized, and then reclaimed. Weak references are most often used to implement canonicalizing mappings.
Suppose that the garbage collector determines at a certain point in time that an object is weakly reachable. At that time it will atomically clear all weak references to that object and all weak references to any other weakly-reachable objects from which that object is reachable through a chain of strong and soft references. At the same time it will declare all of the formerly weakly-reachable objects to be finalizable. At the same time or at some later time it will enqueue those newly-cleared weak references that are registered with reference queues.
So it is really possible when you constructed the List from unwrap() some objects might have been marked finalized and while your Collection.sort is working some WeakRefrence are assigned null. And the point stated by Mattias Buelens is perfectly valid you'll always lose in a fight against the compiler.
If you agree with me, is this a bug in the JIT?
No surely not, I completely disagree with you.
I have the idea to modify comparator that it allows weak references on null. What do you think about that?
I think it will solve your one problem of NPE but your requirement removes void weak references and ensures that weak references are not voided during subsequent sort is not satisfied.
Rather try to call unwrap once again, this will reduce the window for NPE to almost zero,
List<T> unwrapped = unwrap();
unwrapped = unwrap(); //Again to eliminate the chances for NPE as now we would have
//already made strong refrences to all objects which have not been `null`

Object clone() in Java: why does this assertion fail?

From what I know, object clone() creates a new copy of the cloned object. In my case I'm trying to clone the matrix of Symbol (which is a simple enum). this.table is the original object, while t is the clone. When I write a new value into a cell of t I would expect that this.table remains unchanged. However this is not the case and the second assert fails. (I added the first assert only to ensure the correctness of the second one).
Here is the code:
#Override
public State applyAction(Action action) {
int x = ((TickAction)action).x;
int y = ((TickAction)action).y;
Symbol[][] t = this.table.clone();
assert this.table[x][y] != currentPlayer.getSymbol();
t[x][y] = currentPlayer.getSymbol();
assert t[x][y] != this.table[x][y] ;
TableState ts = new TableState(t,this.currentPlayer.getNextPlayer());
ts.setLastAction(action);
return ts;
}
Note: with debugger I checked that t and this.table actually have different id, however after a second check I noticed that, despite this, their single cells have the same id. Then I'm much confused about this. Could someone explain me what's happening?

You have an array of arrays of Symbol instances.
When you call clone() on this.table, you get a new array, t, but each of the arrays in t is the same as the array in this.table.
In order to check that, you can try assert t[0] == this.table[0];.
In order to get a deeper clone, you would have to create a new array and initialize it yourself:
Symbol[][] t = new Symbol[][this.table.length];
for (int i = 0; i < t.length; i++)
{
t[i] = new Symbol[this.table[i].length];
for (int j = 0; j < t[i].length; j++)
{
// Here I am sharing the Symbol objects between the two arrays.
// If you do not want that, define your own way to copy or clone the object.
t[i][j] = this.table[i][j];
}
}

I'm just guessing here, but Java makes a distinction between == and .equals() and everyone gets burned once or twice using == with some object reference that actually needs .equal. Give this a try...
assert this.table[((TickAction)action).x][((TickAction)action).y].equals( currentPlayer.getSymbol() );

you cant use clone as is, it wont help you if you did not implemented it yourself.
same for equals(except strings)

In Java, If I change an object reference during a recursive call, does it propogate up?

For example I make the recursive call defined below. The method finds the kth element from the last. If found, it assigns the current node to the object I pass to the recursive call. For some reason the node kth is null. Can you not do things this way? And why?
public void findKthFromLast(Node head, int k){
Node kth;
recrusiveHelper(head, k, kth);
System.out.println(kth.data); //this is null
}
public int recursiveHelper(Node n, int k, Node kthFromLast){
(if n == null){
return 0;
}
val = 1 + recursiveHelper(n.next, k, kthFromlast);
if(k == val){
kthFromLast = n;
}
return val;
}

If the object reference is local to the method, then changes to it wont be visible to caller or anyone else.
E.g:
void caller()
{
obj = new String("asdf");
doStuff(obj)
System.out.println(obj) // still prints "asdf"
}
void doStuff(String obj)
{
// obj is a local reference, changing it wont affect caller's ref
obj = new String("ghj");
}

Firstly, that code shouldn't compile because kth is not initialized and so cannot be used as argument for the recursiveHelper method call.
Secondly, any changes to the references in a called method are not propagated to the caller in Java, i.e.
private void caller()
{
StringBuilder s = new StringBuilder();
s.append("test");
calledMethod1(s);
System.out.println(s.toString());
calledMethod2(s);
System.out.println(s.toString());
}
private void calledMethod1(StringBuilder buffer)
{
buffer = new StringBuilder();
buffer.append("calledMethod1");
return;
}
private void calledMethod2(StringBuilder buffer)
{
buffer.append(", calledMethod2");
return;
}
Output:
test
test, calledMethod2
The reason is, in calledMethod1 you are merely changing what buffer reference points to but not making any changes to what buffer reference was pointing when the method was called. In calledMethod2, you are making changes to the object referred by buffer and hence the changes are visible in the caller.
If you are someone coming from C or C++ background, this is equivalent to assigning to a pointer argument in a called method which doesn't affect what was passed in the caller.

public void findKthFromLast(Node head, int k){
Node kth;
recrusiveHelper(head, k, kth);
System.out.println(kth.data); //this is null
}
You pass null to the method, whatever you do inside that method, outside this will remain being null. You cannot change the value of reference in a way that is visible outside of method doing so.