Develop Reference

How to block/bulk reset static variables using Java 7

I am parsing a text file looking for syntax matches. to complete this task, I created Variable class with static variables in it. Then I parse a file and I assign the parsed information to the variable class static elements.
Variable.name = "the parsed information";
Then I created a Baseline class to check and compare against the Variable elements to check if certain condition is met.
For example
if (BaseLine.x.equal(Variable.x) // do whatever.
Variable Class:
public class Variable {
static String name;
static String userID;
static Integer age
}
BaseLine Class:
public class BaseLine {
static String name;
static String userID;
static Integer age
}
Utilizing JavaFX I move between scenes to load a file, parse it, assign the parsed variable then compare it with my BaseLine class. Thus far, Everything works as expected until I hit the back button to go back to the original scene to load a new file. now, the issue that I am having how can I reset the variable inside my Variable class in bulk instead of doing it one by one inside my controller initialize statement? I want to do this to ensure that I don't capture any variable from the older file I just parsed before I hit the back button. what is the correct way of completing this task?
I was able to get what I am looking for when I reset the variables inside my initialize controller, but it seems to be a lengthy process to do for OOP I have over 100 variables (int/sets/strings...) to reset.
Here is what I did to reset the static Variables inside the controller initialize section.
#Override
public void initialize(URL url, ResourceBundle rb) {
Variable.name = null;
Variable.setName.clear();
Variable.age = null;

I was able to get what I am looking for when I reset the variables inside my initialize controller, but it seems to be a lengthy process to do for OOP I have over 100 variables (int/sets/strings...) to reset.
If you have hundreds of static variables, you are not doing OOP properly. In proper OO design, your application's state should be held in instance variables, and accessed via instance variables. Static variables should be kept to an absolute minimum. (You can eliminate them entirely, if you can use a dependency injection (DI) framework ...)
The bad news is that there is no >>good<< way to reset a large number of
static variables. There are a couple of >>bad<< ways; e.g. reflection, and messing around with classloaders ... but you just would be replacing one problem (clunky code) with a worse one (complex, fragile code). IMO.
But the good news is that you fix your design / implementation to be properly OO, then you won't have this nasty problem of resetting the variables. And a whole bunch of other things will be easier too ... like writing unit tests.

I am not aware of a way that would exists that would allow you to reset all static variables in one go. You would have to do one by one.
I think your problem lies somewhere else - and i believe you should refactor your code in that instance (if possible).
How about having all your variables as standard (non static) and set/get them like you normally would but when done just create new object and let the old one be collected by Garbage collector??
Edit:
Although you could perhaps use reflection ? Not entirely sure if that would work though.
Something like
Field[] fields = MyClass.class.getDeclaredFields();
for (Field field : fields) {
if (Modifier.isStatic(field.getModifiers() && isRightName(field.getName()) {
field = null;
}
}

How to set and get static variables from spark?

I have a class as this:
public class Test {
private static String name;
public static String getName() {
return name;
}
public static void setName(String name) {
Test.name = name;
}
public static void print() {
System.out.println(name);
}
}
Inside my Spark driver, I'm setting the name like this and calling the print() command:
public final class TestDriver{
public static void main(String[] args) throws Exception {
SparkConf sparkConf = new SparkConf().setAppName("TestApp");
// ...
// ...
Test.setName("TestName")
Test.print();
// ...
}
}
However, I'm getting a NullPointerException. How do I pass a value to the global variable and use it?

Ok, there is basically 2 ways to take a value known to the master to the executors:
Put the value inside a closure to be serialized to the executors to perform a task. This is the most common one and very simple/elegant. Sample and doc here.
Create a broadcast variable with the data. This is good for immutable data of a big size, so you want to guarantee it is send only once. Also good if the same data is used over and over. Sample and doc here.
No need to use static variables in either case. But, if you DO want to have static values available on your executor VMs, you need to do one of these:
If the values are fixed or the configuration is available on the executor nodes (lives inside the jar, etc), then you can have a lazy val, guaranteeing initialization only once.
You can call mapPartitions() with code that uses one of the 2 options above, then store the values on your static variable/object. mapPartitions is guaranteed to run only once for each partition (much better than once per line) and is good for this kind of thing (initializing DB connections, etc).
Hope this helps!
P.S: As for you exception: I just don't see it on that code sample, my bet is that it is occurring elsewhere.
Edit for extra clarification: The lazy val solution is simply Scala, no Spark involved...
object MyStaticObject
{
lazy val MyStaticValue = {
// Call a database, read a file included in the Jar, do expensive initialization computation, etc
4
}
}
Since each Executor corresponds to a JVM, once the classes are loaded MyStaticObject will be initialized. The lazy keyword guarantees that the MyStaticValue variable will only be initialized the first time it is actually requested, and hold its value ever since.

The copy of your class in your driver process isn't the copy in your executors. They aren't in the same ClassLoader, or even the same JVM, or even on the same machine. Setting a static variable on the driver does nothing to the other copies, hence you find it null remotely.

I would like to add one more approach this makes sense only when if you have a few variables which cab ne passed in runtime as arguments.
spark Configuration --> --conf "spark.executor.extraJavaOptions=-DcutomField=${value}"
and
when you need data in transformations you can call System.getProperty("cutomField");
you can find more details here
Note: above discussed does not make sense when we have a significant number of variables
. in those cases, I would prefer #Daniel Langdon approaches.

I would like to add one more point into DanielL's Answer
When declare a variable with static keyword the JVM loads it during the class loading so if you create a jar and set initial values of static fields in a Java /scala class are stored in the jar, workers can use it directly. However if you change the value of a static field in the driver program, workers can only see the initial value assigned into Jar and your changed value will not reflect , so you need to copy again new jar or need to copy class manually into all executors .

pattern for getting around final limitation of Java closure

I'm trying to write a very simple piece of code and can't figure out an elegant solution to do it:
int count = 0;
jdbcTemplate.query(readQuery, new RowCallbackHandler() {
#Override
public void processRow(ResultSet rs) throws SQLException {
realProcessRow(rs);
count++;
}
});
This obviously doesn't compile. The 2 solutions that I'm aware of both stink:
I don't want to make count a class field because it's really a local variable that I just need for logging purposes.
I don't want to make count an array because it is plain ugly.
This is just silly, there got to be a reasonable way to do it?

A third possibility is to use a final-mutable-int-object, for example:
final AtomicInteger count = new AtomicInteger(0);
....
count.incrementAndGet();
Apache Commons also have a MutableInteger I believe, but I have not used it.

You seem to already be aware of the solutions (they are different though); and you are probably aware of the reasons (it cannot capture local variables by reference because the variable might not exist by the time the closure is run, so it must capture by value (have multiple copies); it is bad to have the same variable refer to different copies in different scopes that each can be changed independently, so they cannot be changed).
If your closure does not need to share state back to the enclosing scope, then a field in the class is the right thing to do. I don't understand what your objection is. If the closure needs to be able to be called multiple times and it needs to increment each time, then it needs to maintain state in the object. A field (instance variable) properly expresses the storing of state in an object. The field can be initialized with the captured value from the outside scope.
If your closure needs to share state back to the enclosing scope (which is not a very common situation), then using a mutable structure (like an array) is the right thing to do, because it avoids the problem of the lifetime of the local variable.

I typically make count a class field but add a comment that it is only a field because it is used by an inner closure, Runnable etc...

`*this = rhs` in Java?

I'm coming from the C++ world and I can't find what is the Java alternative (if any) to the following:
struct SomeStruct
{
SomeStruct(){}
SomeStruct(const SomeStruct& rhs)
{
*this = rhs;
}
};
The reason why I need this is that I have a cache of existing objects, so I don't want to create another instance but just to 'clone' the existing one, something like this:
public class SomeObject
{
private static Hashtable _objects;
SomeObject()
{
SomeObject obj = _objects.get(some_key);
if (obj != null) {
// *this = obj;
// instead of:
// this.something = obj.something;
// this.something1 = obj.something1;
// this.something2 = obj.something2;
// a zillion fields....
}
}
};
EDIT:
Sorry, I confused some things (still need to learn both Java and C++).
Thank You

The closest is Object.clone() but read the relevant section from Effective Java first.
There are probably simpler ways to do what you want if you remove some of your requirements. For example, if you make your objects immutable then you don't need to copy the data into another object. Instead you can return a reference to the original object. This is much faster than a memberwise copy, and has lots of other benefits too, such as making it easier to write thread-safe code.

There's no direct equivalent. Object.clone() is the closest, but it creates a new instance and performs a shallow copy. You could write something generic using reflection though.

public class SomeObject
{
private static Hashtable _objects;
public static SomeObject getInstance(SomeKey some_key)
{
return _objects.get(some_key);
}
}

It is unclear from your question whether you are trying to achieve a pure cache of immutable objects, or keep a cache of 'template' objects, and return copies that can then be mutated by the client. I am going to assume the latter.
Assuming that you do want to return copies of originals. there is no really nice way to implement a copy constructor in Java. Clone is marginally nicer, so you should hide the constructor behind a static factory method:
public static SomeObject getInstance(...) {
SomeObject cached = ...;
if (cached != null) {
return cached.clone();
}
...
}
Perhaps in your particular case you can separate the immutable and stateful parts of the objects? If so, some changes to your object model can lead to cleaner (and more efficient) code?

In Java, if you want to have a constructor corresponding to a copy constructor, you have to implement it yourself. In some cases, this means you have to copy fields from one object instance into another, while in other cases it means you have to implement a full deep copy - recursively iterate through the reference fields of the parameter in the copy ctor.
This depends on what you want - should the objects hashtable be copied as well? Or should both objects share the reference to it? For more info, see this question.

I don't know of any generic way to copy all the contents of fields from one object to another existing without using;
reflections
generated code.
copy using the Unsafe class.
Reflections is easier to use at runtime, but not as efficient as generated code. (But pretty close) I generate my data model so I use this option.
The Unsafe class is faster than reflection, but not cross platform and unsafe. It is almost as fast as generated code. ;)
In terms of speed, if generated code is 1x, Unsafe is 1.5 - 2x and reflection is 2 - 4x slower for highly cached data. For poorly cached data you won't see a difference.

You should use the Singleton pattern. You create a static instance of SomeObject and work with that instance.
private static class SingletonHolder {
public static final SomeObject INSTANCE = new SomeObject();
}
public static SomeObject getInstance() {
return SingletonHolder.INSTANCE;
}
But don't fall in the Singleton trap, limit calls to getInstance()!

In Java, is new always new?

Is there any way in Java that a statement like
Thing thing = new Thing();
could not result in a new object being created (i.e. any way that thing could end up pointing to an already-existing object)?

The new operator allocates new heap space and calls the constructor. You will always get a new object that way (unless, as others pointed out, an Exception is thrown in the constructor).
The thing is a little different with static methods that return a reference, such as Integer.valueOf() which re-uses objects from an internal pool if possible.

If the constructor for Thing throws an exception, the object isn't created. However, thing will never point to another existing instance of Thing.

Several people are saying that an object won't be created if the constructor throws an exception. I would just like to point out that this is not true. As an example, take a look at this very bad code:
public class Test {
static int count;
static Set<Test> set = new HashSet<Test>();
int id = count++;
public Test() {
set.add(this);
throw new RuntimeException();
}
public static void main(String[] args) throws Exception {
for(int i = 0; i < 5; i++) {
try {
new Test();
} catch(Exception e) {
}
}
System.out.println(set);
}
public String toString() {
return "Test[" + id + "]";
}
}
The output is:
[Test[0], Test[1], Test[2], Test[4], Test[3]]
new creates a new object EVERY TIME.

Using new will always result in a new object being allocated and created. However, you might be referring to the concept of interning, where objects and stored in a pool and can be reused to save on space. For a good example, see this article on String interning in Java.

New is always new (maybe with some exceptions for primitive wrappers), but if reusing objects is a desired behaviour there are ways to do that through certain design patterns (singleton, factory, pool etc.).

I'm not sure if I understand your question correctly.
The definition of new is to allocate and initialize a new object.
I guess you might be able store a static reference to a object, and then clone it to make a new object, but that object would still be new.
Since you cannot modify the value of this (otherwise you could just say something like this = oldObject in your constructor), the only way you could do this is to throw an exception, like has been mentioned before.

as far as I know it's not like C++ where you can overload operator new or do placement new or other allocator things. (but I'm not familiar with the JLS)

new always creates new object. In java you can't reuse object using new. Really this leads to performance issue in certain java class. For e.g. Boolean class can practically stores only two values (true or false), still a user can create multiple objects with same value using new.

No, in the case there's no space in memory for it, you should get an OutOfMemoryError.
Of course, there could be other exceptions thrown by the Thing constructor.

No.
After executing that line of code thing will never reference an object that existed before that line.

The call to new can fail if this is the first time you are referring to the class and if the class has static blocks that fail to initialize. I have experienced behaviors wherein any runtime errors that occur in a static block or constructor will result in the new call throwing a ClassNotFound Exception in the context of usage within the websphere application server. The websphere class loader apparently rejects the class from loading because of the runtime exception.

In Java, the only thing I can think of is using the Singleton pattern. This would not create multiple new instances of Thing, but rather an instance of the same object every time.