Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 12 days ago.
Improve this question
public void bad() {
final ConcurrentMap<String, Integer> chm = new ConcurrentHashMap<>();
final String key = "1";
chm.computeIfAbsent(key, __ -> {
chm.remove(key);
return 1;
});
}
Of course i understand this looks very silly. It is just a super simplified version of some problematic code i was dealing with. I understand it makes no sense to do this but i am trying to understand the behaviour it causes.
When running this code you get stuck in an infinite loop on line 1107 after invoking http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/687fd7c7986d/src/share/classes/java/util/concurrent/ConcurrentHashMap.java#l1096
I am finding it very difficult to understand exactly what is happening which is causing this. Same behaviour when done on a seperate thread but waiting
public void bad2() {
final ConcurrentMap<String, Integer> chm = new ConcurrentHashMap<>();
final String key = "1";
Thread worker = new Thread(() -> chm.remove("1"));
chm.computeIfAbsent(key, __ -> {
worker.start();
try {
worker.join();
} catch (InterruptedException e) {
e.printStackTrace();
}
return 1;
});
}
Why in both cases is it not that chm.remove(key) completes normally returning null and then the value of 1 is set?
Interestingly this was addressed at some point and the first example throws java.lang.IllegalStateException: Recursive update when i ran with java 17.
This is called a 'contract error'. This happens when javadoc explicitly tells you NOT to do thing X, and leaves it at that; it does not specify what happens if you do X, just that you shouldn't do that. In this case, X is 'update the map in your passed-in function', and the javadoc explicitly spells out: DO NOT.
so the computation should be short and simple,
and must not attempt to update any other mappings of this map.
When you perform such a contract error, anything can happen. The spec is clear: Don't. So, if you do, the spec essentially claims the ability to do anything. Hard-crash, whistle a dixie tune from the speakers, you name it.
Hence why the behaviour changed (ordinarily, java does not change its behaviours without quite a big ordeal about breaking compatibility, but here, the spec behaviour has not changed, because the spec merely says 'do not do this, this thing does not perform according to spec if you fail to heed this warning' and 'loop endlessly' and 'throw an exception' are both just 'doing unspecified broken stuff' in this regard.
Okay, but why does it endlessly loop?
Because concurrent hashmap is 'smart' and uses a retry/CAS update model. Instead of acquiring a bunch of stuff, it just tries the operation without doing that, but will then check during/afterwards if it actually succeeded, or if, due to other threads modifying the same map at the same time in the same general area, its write got overwritten or otherwise didn't apply, in which case it'll do it again. In this case, removing the key is essentially 'eliminating a marker', which makes CHM think it updated a thing concurrently with another thing and therefore it should try again. Forever and ever.
That's what the 'cas' in line 1656 in your linked source file (casTabAt) stands for: Compare-And-Set. This is a concurrency primitive that can be a lot faster than locks: "If the current value is X, then set it to Y. Otherwise, do not set it at all. Tell me whether you set it or not" - all that, in one atomic operation, which is speedy because CPUs tend to support it as barebones machine code. No lock acquiry required. The general principle is to check what the current value is, do some bookkeeping, then set the new value, using CAS to ensure that the 'state' you checked is still the state we're in. If not, some other thread so happened to also be updating stuff, so, start over.
That's just one implementation. Tomorrow, it can change. You cannot rely on 'it will endlessly loop' because the spec do not guarantee it, and indeed, in JDK17, you get an exception instead.
Related
I have written the code below and want to test its concurrency.
Even though I have used ConcurrentHashMap, I am not confident that the code is threadsafe because get and update/replace are used on different lines.
Can anyone suggest how to test this?
ConcurrentHashMap<String, int> somemap = new ConcurrentHashMap<>();
boolean some_method() {
somemap.putifAbsent("somekey",value);
final int val = somemap.get("somekey");
if(val < total_count) {
val++;
}
somemap.replace("somekey", val);
}
Can anyone suggest how to test this?
Testing for thread safety is not reliable. The reason is, The specification for a programming language or library may promise that if your multi-threaded program obeys certain rules, then you can expect it to behave in a certain way, but you will practically never find a promise that if your program breaks the rules then it will fail to meet your expectations.
I personally have seen a software update go through weeks of unit testing and integration testing, and then be shipped out into the field, where it ran for half a year before a piece of "thread unsafe" code caused it to fail at a customer site.
The only way to be sure that your multi-threaded program will work, is to prove that (A) it says what you think it says, and (B) it obeys the rules.
Your example obeys the rules, but I don't think it does what you think it does. It has a race condition. Namely, two threads both could call get("somekey") at the same time, and both get the same value. Then they both could compute the same new value, and they both could put(...) it back. Two calls to some_method(), but the end result is that the value only goes up by one.
I saw many times that using the functional API in java is really verbose and error-prone when we have to deal with checked exceptions.
E.g: it's really convenient to write (and easier to read) code like
var obj = Objects.requireNonNullElseGet(something, Other::get);
Indeed, it also avoids to improper multiple invokation of getters, like when you do
var obj = something.get() != null ? something.get() : other.get();
// ^^^^ first ^^^^ ^^^^ second ^^^^
BUT everything becomes a jungle when you have to deal with checked exceptions, and I saw sometimes this really ugly code style:
try {
Objects.requireNonNullElseGet(obj, () -> {
try {
return invokeMethodWhichThrows();
} catch (Exception e) {
throw new RuntimeException(e);
}
});
} catch (RuntimeException r){
Throwable cause = r.getCause();
if(cause == null)
throw r;
else
throw cause;
}
which only intent is to handle checked exceptions like when you write code without lambdas. Now, I know that those cases can be better expressed with the ternary operator and a variable to hold the result of something.get(), but that's also the case for Objects.requireNonNullElse(a, b), which is there, in the java.util package of the JDK.
The same can be said for logging frameworks' methods which take Suppliers as parameters and evaluate them only if needed, BUT if you need to handle checked exceptions in those supplier you need to invoke them and explicitly check for the log level.
if(LOGGER.isDebugEnabled())
LOGGER.debug("request from " + resolveIPOrThrow());
Some similar reasonament can be maid also for Futures, but let me go ahead.
My question is: why is Functional API in java not handling checked exceptions?
For example having something like a ThrowingSupplier interface, like the one below, can potentially fit the need of dealing with checked exceptions, guarantee type consistency and better code readability.
interface ThrowingSupplier<O, T extends Exception> {
O get() throws T;
}
Then we need to duplicate methods that uses Suppliers to have an overload that uses ThrowingSuppliers and throws exceptions. But we as java developers have been used to this kind of duplication (like with Stream, IntStream, LongStream, or methods with overloads to handle int[], char[], long[], byte[], ...), so it's nothing too strange for us.
I would really appreciate if someone who has deep knowledge of the JDK argues about why checked exceptions have been excluded from the functional API, if there was a way to incorporate them.
This question can be interpreted as 'why did those who made this decision decide it this way', which is asking: "Please summarize 5 years of serious debate - specifically what Brian Goetz and co thought about it", which is impossible, unless your name is Brian Goetz. He does not answer questions on SO as far as I know. You can go spelunking in de archives of the lambda-dev mailing list if you want.
One could make an informed guess, though.
In-scope vs Beyond-scope
There are 3 transparancies that lambdas do not have.
Control flow.
Checked exceptions.
Mutable local variables.
Control flow transparency
Take this code, as an example:
private Map<String, PhoneNumber> phonebook = ...;
public PhoneNumber findPhoneNumberOf(String personName) {
phonebook.entrySet().stream().forEach(entry -> {
if (entry.getKey().equals(personName)) return entry.getValue();
});
return null;
}
This code is silly (why not just do a .get, or if we must stream through the thing, why not use .filter and .findFirst, but if you look past that, it doesn't even work: You cannot return the method from within that lambda. That return statement returns the lambda (and thus is a compiler error, the lambda you pass to forEach returns void). You can't continue or break a loop that is outside the lambda from inside it, either.
Contrast to a for loop that can do it just fine:
for (var entry : phonebook.entrySet()) {
if (entry.getKey().equals(personName)) return entry.getValue();
}
return null;
does exactly what you think, and works fine.
Checked exception transparency
This is the one you are complaining about. This doesn't compile:
public void printFiles(Path... files) throws IOException {
Arrays.stream(files).forEach(p -> System.out.println(Files.readString(p)));
}
The fact that the context allows you to throw IOExceptions doesn't help: The above does not compile, because 'can throw IOExceptions' as a status doesn't 'transfer' to the inside of the lambda.
There's a theme here: Rewrite it to a normal for loop and it compiles and works precisely the way you want to. So why, exactly, can't we make lambdas work the same way?
mutable local variables
This doesn't work:
int x = 0;
someList.stream().forEach(k -> x++);
System.out.println("Count: " + x);
You can neither modify local variables declared outside the lambda, nor even read them unless they are (effectively) final. Why not?
These are all GOOD things.. depending on scope layering
So far it seems really stupid that lambdas aren't transparent in these 3 regards. But it turns into a good thing in a slightly different context. Imagine instead of .stream().forEach something a little bit different:
class DoubleNullException extends Exception {} // checked!
public class Example {
private TreeSet<String> words;
public Example() throws DoubleNullException {
int comparisonCount = 0;
this.words = new TreeSet<String>((a, b) -> {
comparisonCount++;
if (a == null && b == null) throw new DoubleNullException();
});
System.out.println("Comparisons performed: " + comparisonCount);
}
}
Let's image the 3 transparencies did work. The above code makes use of two of them (tries to mutate comparisonCount, and tries to throw DoubleNullException from inside to outside).
The above code makes absolutely no sense. The compiler errors are very much desired. That comparator is not going to run until perhaps next week in a completely different thread. It runs whenever you add the second element to the set, which is a field, so who knows who is going to do that and which thread would do it. The constructor has long since ceased running - local vars are 'on the stack' and thus the local var has disappeared. Nevermind that the printing would always print 'comparisons made: 0' here, the statement 'comparisonCount++:' would be trying to increment a memory position that no longer holds that variable at all.
Even if we 'fix' this (the compiler realizes that a local is used in a lambda and hoists it onto heap, this is what most other languages do), the code still makes no sense as a concept: That print statement wouldn't print. Also, that comparator can be called from multiple threads so... do we now allow volatile on our local vars? Quite the can of worms! In current java, a local variable cannot possibly suffer from thread concurrency synchronization issues because it is not possible to share the variable (you can share the object the variable points at, not the variable itself) with another thread.
The reason you ARE allowed to mess with (effectively) final locals is because you can just make a copy, and that's what the compiler does for you. Copies are fine - if nobody changes anything.
The exception similarly doesn't work: It's the code that calls thatSet.add(someElement) that would get the DoubleNullException. The fact that somebody wrote:
Example ex;
try {
ex = new Example();
} catch (DoubleNullException e) {
throw new WrappedEx(e);
}
ex.add(null);
ex.add(null); // BOOM
The line with the remark (BOOM) would throw the DoubleNullEx. It 'breaks' the checked exception rules: That line would compile (set.add doesn't throw DNEx), but isn't in a context where throwing DNEx is allowed. The catch block that is in the above snippet cannot ever run.
See how it all falls apart, and nothing makes sense?
The key clue is: What happens to the lambda? Is it 'transported'?
For some situations, you hand a lambda straight to a method, and that method has a 'use it and lose it' mentality: That method you handed the lambda to will run it 0, 1, or many times, but the key is: It runs it right then and there and once the method you handed the lambda to returns, that lambda is gone. The thing you handed the lambda to did not store it in a field or hand it to other code that stores it in a field, nor did that method transport the lambda to another thread.
In such cases (the method is use-it-then-lose-it), the transparencies would certainly be handy and wouldn't "break" anything.
But when the method you hand the lambda to does transport it to a field (such as the constructor of TreeSet which stores the passed comparator in a field, so that future .add calls can call it), the transparencies break down and make no sense.
Lambdas in java are for both and therefore the lack of transparency (in all 3 regards) actually makes sense. It's just annoying when you have a use-it-then-lose-it situation.
POTENTIAL FUTURE JAVA FIX: I've championed it before but so far, it fell on mostly deaf ears. Next time I see Brian I might bring it up again. Imagine an annotation or other marker you can stick on the parameter of a method that says: "I shall use it or lose it". The compiler will then ensure you do not transport it (the only thing the compiler will let you do with that param is call .invoke() on it. You can't call anything else, nor can you assign it or hand it to anything else unless you hand it to a method that also marked that parameter as #UseItOrLoseIt. Then the compiler can make the transparency happen with some tactical wrapping for control flow, and for checked exception flow, just by not complaining (checked exceptions are a figment of javac's imagination. The runtime does not have checked exceptions. Which is why scala, kotlin, and other runs-on-the-JVM languages can do it).
Actually THEY CAN!
As your question ends with - you can actually write O get() throws T. So why do the various functional interfaces, such as Supplier, not do this?
Mostly because it's a pain. I'm honestly not sure why e.g. list's forEach is not defined as:
public <T extends Throwable> forEach(ThrowingConsumer<? super E, ? super T> consumer) throws T {
for (E elem : this) consumer.consume(elem);
}
Which would work fine and compile (with ThrowingConsumer having the obvious impl). Or even that Consumer as we have it is declared with the <O, T extends Exception> part.
It's a bit of a hassle. The way lambdas 'work' is that the compiler has to infer from context what functionalinterface you are implementing which notably includes having to bind all the generics out. Adding exception binding to this mix makes it even harder. IDEs tend to get a little confused if you're in the middle of writing code in a 'throwing lambda' and start red-underlining rather a lot, and auto-complete and the like is no help, because the IDE can't be useful in that context until it knows.
Lambdas as a system were also designed to backwards compatibly replace any existing usages of the concept, such as swing's ActionListener. Such listeners couldn't throw either, so having the interfaces in the java.util.function package be similar would be more familiar and slightly more java idiomatic, possibly.
The throws T solution would help but isn't a panacea. It solves, to an extent, the lack of checked exception transparency, but does nothing to solve either mutable local var transparency or control flow transparency. Perhaps the conclusion is simply: The benefits of doing it are more limited than you think, the costs are higher than you think. The cost/benefit analysis says: Bad idea, so it wasn't done.
I'm attempting to understand what's happening in this bit of Java code as its owner are no longer around and possibly fixing it or simplifying it. I'm guessing these blocks had a lot more in them at some point and what's left in place was not cleaned up properly.
It seems all occurrences of orElse(false) don't set anything to false and can be removed.
Then the second removeDiscontinued method is returning a boolean that I don't think is used anywhere. Is this just me or this is written in a way that makes it hard to read?
I'm hesitant removing anything from it since I haven't used much of the syntax like orElse, Lazy, Optional. Some help would be much appreciated.
private void removeDiscontinued(Optional<Map<String, JSONArrayCache>> dptCache, Lazy<Set<String>> availableTps) {
dptCache.map(pubDpt -> removeDiscontinued(pubDpt.keySet(), availableTps)).orElse(false);
}
private boolean removeDiscontinued(Set<String> idList, Lazy<Set<String>> availableTps) {
if (availableTps.get().size() > 0) {
Optional.ofNullable(idList).map(trIds -> trIds.removeIf(id -> !availableTps.get().contains(id)))
.orElse(false);
}
return true;
}
This code is indeed extremely silly. I know why - there's a somewhat common, extremely misguided movement around. This movement makes claims that are generally interpreted as 'write it 'functional' and then it is just better'.
That interpretation is obvious horse exhaust. It's just not true.
We can hold a debate on who is to blame for this - is it folks hearing the arguments / reading the blogposts and drawing the wrong conclusions, or is it the 'functional fanfolks' fanning the flames, so to speak, making ridiculous claims that do not hold up?
Point is: This code is using functional style when it is utterly inappropriate to do so and it has turned into a right mess as a result. The code is definitely bad; the author of this code is not a great programmer, but perhaps most of the blame goes to the functional evangelistst. At any rate, it's very difficult to read; no wonder you're having a hard time figuring out what this stuff does.
The fundamental issue
The fundamental issue is that this functional style strongly likes being a side-effect free process: You start with some data, then the functional pipeline (a chain of stream map, orElse, etc operations) produces some new result, and then you do something with that. Nothing within the pipeline should be changing anything, it's just all in service of calculating new things.
Both of your methods fail to do so properly - the return value of the 'pipeline' is ignored in both of them, it's all about the side effects.
You don't want this: The primary point of the pipelines is that they can skip steps, and will aggressively do so if they think they can, and the pipeline assumes no side-effects, so it makes wrong calls.
That orElse is not actually optional - it doesn't seem to do anything, except: It forces the pipeline to run, except the spec doesn't quite guarantee that it will, so this code is in that sense flat out broken, too.
These methods also take in Optional as an argument type which is completely wrong. Optional is okay as a return value for a functional pipeline (such as Stream's own max() etc methods). It's debatable as a return value anywhere else, and it's flat out silly and a style error so bad you should configure your linter to aggressively flag it as not suitable for production code if they show up in a field declaration or as a method argument.
So get rid of that too.
Let's break down what these methods do
Both of them will call map on an Optional. An optional is either 'NONE', which is like null (as in, there is no value), or it is a SOME, which means there is exactly one value.
Both of your methods invoke map on an optional. This operation more or less boils down, in these specific methods, as:
If the optional is NONE, do nothing, silently. Otherwise, perform the operation in the parens.
Thus, to get rid of the Optional in the argument of your first method, just remove that, and then update the calling code so that it decides what to do in case of no value, instead of this pair of methods (which decided: If passing in an optional.NONE, silently do nothing. "Silently do nothing" is an extremely stupid default behaviour mode, which is a large part of why Optional is not great). Clearly it has an Optional from somewhere - either it made it (with e.g. Optional.ofNullable in which case undo that too, or it got one from elsewhere, for example because it does a stream operation and that returned an optional, in which case, replace:
Optional<Map<String, JSONArrayCache>> optional = ...;
removeDiscontinued(thatOptionalThing, availableTps);
with:
optional.map(v -> removeDiscontinued(v, availableTps));
or perhaps simply:
if (optional.isPresent()) {
removeDiscontinued(optional.get(), availableTps);
} else {
code to run otherwise
}
If you don't see how it could be null, great! Optional is significantly worse than NullPointerException in many cases, and so it is here as well: You do NOT want your code to silently do nothing when some value is absent in a place where the programmer of said code wasn't aware of that possibility - an exception is vastly superior: You then know there is a problem, and the exception tells you where. In contrast to the 'silently do not do anything' approach, where it's much harder to tell something is off, and once you realize something is off, you have no idea where to look. Takes literally hundreds of times longer to find the problem.
Thus, then just go with:
removeDiscontinued(thatOptionalThing.get(), availableTps);
which will NPE if the unexpected happens, which is good.
The methods themselves
Get rid of those pipelines, functional is not the right approach here, as you're only interested in the side effects:
private void removeDiscontinued(Map<String, JSONArrayCache> dptCache, Lazy<Set<String>> availableTps) {
Set<String> keys = dptCache.keySet();
if (availableTps.get().size() > 0) {
keys.removeIf(id -> availableTps.get().contains(id));
}
}
That's it - that's all you need, that's what that code does in a very weird, sloppy, borderline broken way.
Specifically:
That boolean return value is just a red herring - the author needed that code to return something so that they could use it as argument in their map operation. The value is completely meaningless. If a styleguide that promises: "Your code will be better if you write it using this style" ends up with extremely confusing pointless variables whose values are irrelevant, get rid of the style guide, I think.
The ofNullable wrap is pointless: That method is private and its only caller cannot possibly pass null there, unless dptCache is an instance of some bizarro broken implementation of the Map interface that deigns to return null when its keySet() method is invoked: If that's happening, definitely fix the problem at the source, don't work around it in your codebase, no sane java reader would expect .keySet to return null there. That ofNullable is just making this stuff hard to read, it doesn't do anything here.
Note that the if (availableTps.get().size() > 0) check is just an optimization. You can leave it out if you want. That optimization isn't going to have any impact unless that dptCache object is a large map (thousands of keys at least).
Let say you have a method that checks if the argument (Answer) is correct and check if the question already have answers in the list that is also correct:
public void addAnswer(Answer answer) {
if (answer.isCorrect()) {
...
}
}
However, I only want one answer to be correct in the list. I have multiple options. I could throw an exception, I could ignore it, I could return some boolean value from the addAnswer that tells me if the operation was ok or not. How are you supposed to think in such scenarios?
The rule is pretty simple: Use exceptions on exceptional, erroneous, unpredicted failures. Don't use exceptions when you expect something to happen or when something happens really often.
In your case it's not an error or something truly rare that an answer is not correct. It's part of your business logic. You can throw an exception, but only as part of some validation (assertion) if you expect an answer at given point to always be correct and suddenly it's not (precondition failure).
And of course if some failure occurs while checking correctness (database connection lost, wrong array index) exception are desired.
This entirely depends on what you want to achieve. Should the caller of your method already have made sure that it doesn't add two correct answers? Is it a sign of a programming error if that happens? Then throw an exception, but definitely an unchecked exception.
If your method's purpose is to relieve the caller from enforcing the one-true-answer invariant (I doubt that, though), then you can just arrange to signal via a boolean return value, which makes it only an optional information channel for the caller.
If there is no way to know in advance whether there are other correct answers—for example, the answers are added concurrently from several threads or even processes (via a database)—then it would be meaningful to throw a checked exception.
Bottom line: there is no one-size-fits-all best practice, but there is a best practice for every scenario you want to accomplish.
The exception police will be down on you like a ton of bricks, and me for this answer, with statements like "don't use exceptions for flow control" and "don't use exceptions for normal conditions".
The trouble with the first statement is that exceptions are a form of flow control. This makes the argument self-contradictory, and therefore invalid.
The trouble with the second statement is that it seems to inevitably go along with endlessly redefining exceptional conditions as normal. You will find examples in this very site: for example, a lively discussion where the police insisted that EOF was 'normal' and therefore that EOFException shouldn't be caught, despite the existence of dozens of Java APIs that don't give you any choice in the matter. Travel far enough down this path and you can end up with nothing that is exceptional whatsoever, and therefore no occasion to use them at all.
These are not logical arguments. These are unexamined dogmas.
The original and real point, back in about 1989 when it was first formulated, was that you shouldn't throw exceptions to yourself, to be handled in the same method: in other words, don't treat it as a GOTO. This principle continues to have validity.
The point about checked exceptions is that you force the caller to do something about handling them. If you believe, on your own analysis, that this is what you want, use an exception. Or, if you are using an API that forces you to catch them, catch them, at the appropriate level (whatever that is: left as an exercise for the reader).
In other words, like most things in the real world, it is up to your discretion and judgment. The feature is there to be used, or abused, like anything else.
#Exception police: you will find me in the telephone book. But be prepared for an argument.
An exception thrown from a method enforces the callers to take some action in the anticipation of the exception occurring for some inputs. A return value doesn't enforce the same and so it is up to the caller to capture it and take some action.
If you want the callers to handle the scenario to take some corrective action, then you should throw a checked exception (sub class of java.lang.Exception).
The problem here is that your API is error prone. I'd use the following scheme instead:
public class Question {
private List<Answer> answers;
private int mCorrect;
// you may want a List implementation without duplicates
public void setAnswers(List<Answer> answers, int correct) {
this.answers = answers;
// check if int is between bounds
mCorrect = correct;
}
public boolean isCorrect(Answer answer) {
return answers.indexOf(answer) == mCorrect;
}
}
because an Answer by itself is simply a statement, and usually cannot be true of false without being associated to a Question. This API makes it impossible to have zero or more than one correct answers, and forces the user to supply the correct one when he adds answers, so your program is always in a consistent state and simply can't fail.
Before deciding how to signal errors, it's always better to design the API so that errors are less common as possible. With your current implementation, you have to make checks on your side, and the client programmer must check on his side as well. With the suggested design no check is needed, and you'll have correct, concise and fluent code on both sides.
Regarding when to use a boolean and when to use Exceptions, I often see boolean used to mirror the underlying API (mostly low level C-code).
I agree with Tomasz Nurkiewicz's response. I cant comment on it because I'm a new user. I would also recommend that if the addAnswer() method is not always going to add the answer (because they already exists a correct one), name it to suggest this behaviour. "add" is suggest normal collections behaviour.
public boolean submitAnswer(Answer answer); // returns true is answer accepted
Your exact solution may depend on the bigger picture about your application that we dont know about. Maybe you do want to throw an Exception but also make it the responsibility of the caller to check if adding the Answer is valid.
It's all a rich tapestry.
I would implement it in this way:
public class Question {
private int questionId;
private final Set<Answer> options = new HashSet<Answer>();
private final Set<Answer> correctAnswers = new HashSet<Answer>();
public boolean addAnswer(Answer answer) throws WrongAnswerForThisQuestionException {
if(!answer.isValid(questionId)) {
throw new WrongAnswerForThisQuestionException(answer, this);
}
if (answer.isCorrect(questionId)) {
correctAnswers.add(answer);
}
return options.add(answer);
}
}
I have a variable that very rarely gets an incorrect value. Since the system is quite complex I'm having trouble tracing all the code paths that value goes through - there are multiple threads involved, it can be saved and then loaded from a DB and so on. I'm going to try to use a code graph generator to see if I can spot the problem by looking at the ways the setter can be called, by may be there's some other technique. Perhaps wrapping the value with a class that traces the places and changes it goes through? I'm not sure the question is clear enough, but I'd appreciate input from somebody who encountered such a situation.
[Edit] The problem is not easily reproducible and I can't catch it in a debugger. I'm looking for a static analysis or logging technique to help track down the issue.
[Edit 2] Just to make things clearer, the value I'm talking about is a timestamp represented as the number of milliseconds from the Unix epoch (01/01/1970) in a 64-bit long variable. At some unknown point the top 32 bits of the value are truncated generating completely incorrect (and unrecoverable) timestamps.
[Edit 3] OK, thanks to some of your suggestions and to a couple of hours of pouring through the code, I found the culprit. The millisecond-based timestamp was converted into a second-based timestamp by dividing it by 1000 and stored in an int variable. At a later point in code, the second-based timestamp (an int) was multiplied by 1000 and stored into a new long variable. Since both 1000 and the second-based timestamps were int values, the result of the multiplication was truncated before being converted to long. This was a subtle one, thanks to everyone who helped.
If you are using a setter and only a setter to set your value you can add these lines in order to track the thread and stack trace:
public void setTimestamp(long value) {
if(log.idDebugEnabled) {
log.debug("Setting the value to " + value + ". Old value is " + this.timestamp);
log.debug("Thread is " + Thread.currentThread().getName());
log.debug("Stacktrace is", new Throwable()); // we could also iterate on Thread.currentThread().getStackTrace()
}
// check for bad value
if(value & 0xffffffff00000000L == 0L) {
log.warn("Danger Will Robinson", new IlegalValueException());
}
this.timestamp = value;
}
Also, go over the class that contains the field, and make sure that every reference to it is done via the setter (even in private/protected methods)
Edit
Perhaps FindBugs can help in terms of static analysis, I'll try to find the exact rule later.
The fact that 32 bits of the long get changed, rather than the whole value, suggests strongly that this is a threading problem (two threads update the variable at the same time). Since java does not guarantee atomic access to a long value, if two threads update it at the same time, it could end up with half the bits set one way and half the other. This means that the best way to approach the issue is from a threading point of view. Odds are that there is nothing setting the variable in a way that a static analysis tool will show you is an incorrect value, but rather the syncronization and locking strategy around this variable needs to be examined for potential holes.
As a quick fix, you could wrap that value in an AtomicLong.
I agree - if the value is only changed via a setter (no matter what the orgin) - and it better be - then the best way is to modify the setter to do the tracking for you (print stack trace at every setting, possibly only when the value set is a specific one if that cuts down on the clatter)
Multithreaded programming is jsut hard, but there are IDE tools to help. If you have intellij IDEA, you can use the analyze dataflow feature to work out where things gets changed. If won't show you a live flow (its a static analysis tool), but it can give you a great start.
Alternatively, you can use some Aspects and just print out the value of the variable everywhere, but the resulting debugging info will be too overwhelming to be that meaningful.
The solution is to avoid state shared between threads. Use immutable objects, and program functionally.
Two things:
First, to me, it smells as though some caller is treating their timestamp in an integer context, losing your high 32 bits. It may be, as Yishai surmised, threading-related, but I'd look first at the operations being performed. However, naturally, you need to assure that your value is being updated "atomically" - whether with an AtomicLong, as he suggested, or with some other mechanism.
That speculation aside, given that what you're losing is the high 32 bits, and you know it's milliseconds since the epoch, your setter can enforce validity: if the supplied value is less than the timestamp at program start, it's wrong, so reject it, and of course, print a stack trace.
1) Supposing that foo is the name of your variable, you could add something like this to the setter method:
try {
throw new Exception();
}
catch (Exception e) {
System.out.println("foo == " + foo.toString());
e.printStackTrace();
}
How well this will work depends on how frequently the setter is being called. If it's being called thousands of times over the run of your program, you might have trouble finding the bad value in all the stack traces. (I've used this before to troubleshoot a problem like yours. It worked for me.)
2) If you can run your app in a debugger and you can identify programatically bad values for your variable, then you could set a breakpoint in the setter conditional on whatever it is that makes the value bad. But this requires that you can write a test for badness, which maybe you can't do.
3) Since you said (in a subsequent edit) that the problem is the high 32 bits being zeroed, you can specifically test for that before printing your stack trace. That should cut down the amount of debugging output enough to be manageable.
In your question, you speak of a "variable" that has an incorrect value, and suggest that you could try "wrapping the value with a class". Perhaps I'm reading too much into your choice of words, but would like to see a bit more about the design.
Is the value in question a primitive? Is it a field of a large, complex object that is shared between threads? If it is a field of some object, is that object a DTO or does it implement domain-specific behavior?
In general, I'd agree with the previous comments re instrumenting the object of which the "variable" is a field, but more information about the nature and usage of this variable would help guide more precise suggestions.
Based on your description, I don't know if that means it's not feasible to actual debug the app in real time, but if it is, depending on your IDE there's a bunch of debugging options available.
I know that with Eclipse, you can set conditional breakpoints in the setter method for example. You can specify to suspend only when the value gets set to a specific value, and you can also filter by thread, in case you want to focus on a specific thread.
I will rather keep a breakpoint inside the setter. Eclipse allows you to do that.
There are some IDE which allows you to halt ( wait for execution of next instruction ) the program, if the value of variable is changed.
IMO the best way to debug this type of problem is using a field modification breakpoint. (Especially if you're using reflection extensively)
I'm not sure how to do this in eclipse, but in intellij you can just right click on the field and do an "add breakpoint".