Develop Reference

Why does ImmutableCollection.contains(null) fail?

Question ahead:
why does in Java the call coll.contains(null) fail for ImmutableCollections?
I know, that immutable collections cannot contain null-elements, and I do not want to discuss whether that's good or bad.
But when I write a Function, that takes a (general, not explicit immutable) Collection, it fails upon checking for nulls. Why does the implementation not return false (which is actually the 'correct' answer)?
And how can I properly check for nulls in a Collection in general?
Edit:
with some discussions (thanks to the commenters!) I realized, that I mixed up two things: ImmutableCollection from the guava library, and the List returned by java.util.List.of, being some class from ImmutableCollections. However, both classes throw an NPE on .contains(null).
My problem was with the List.of result, but technically the same would happen with guaves implementation. [edit: It does not]

I am distressed by this discussion!
Collections that do this have been a pet peeve of mine since before I wrote the first collections that eventually became Guava. If you find any Guava collection that throws NPE just because you asked it a perfectly innocent question like .contains(null), please file a bug! We hate that crap.
EDIT: I was so distressed that I had to go back to look at my 2007 changelist that first created ImmutableSet and saw literally this:
#Override public boolean contains(#Nullable Object target) {
if (target == null) {
return false;
}
ahhhhh.

why does in Java the call coll.contains(null) fail for ImmutableCollections?
Because the design team (the ones who have created guava) decided that, for their collections, null is unwanted, and therefore any interaction between their collections and a null check, even in this case, should just throw to highlight to the programmer, at the earliest possible opportunity, that there is a mismatch. Even where the established behaviour (as per the existing implementations in the core runtime itself, such as ArrayList and friends, as well as the javadoc), rather explicitly go the other way and say that a non-sequitur check (is this pear part of this list of apples?) strongly suggests that the right move is to just return false and not throw.
In other words, guava messed up. But now that they have done so, going back is potentially backwards compatibility breaking. It really isn't very - you are replacing an exception thrown with a false return value; presumably code could be out there that relies on the NPE (catching it and doing something different from what the code would do had contains(null) returned false instead of throwing) - but that's a rare case, and guava breaks backwards compatibility all the time.
And how can I properly check for nulls in a Collection in general?
By calling .contains(null), just as you are. The fact that guava doesn't do it right doesn't change the answer. You might as well ask 'how do I add elements to a list', and counter the answer of "well, you call list.add(item) to do that" with: Well, I have this implementation of the List interface that plays Rick Astley over the speaker instead of adding to the list, so, I reject your answer.
That's.. how java and interfaces work: You can have implementations of them, and the only guardianship that they do what the interface dictates they must, is that the author understands there is a contract that needs to be followed.
Now, normally a library so badly written they break contract for no good reason*, isn't popular. But guava IS popular. Very popular. That gets at a simple truth: No library is perfect. Guava's API design is generally quite good (in my opinion, vastly superior to e.g. Apache commons libraries), and the team actively spends a lot of time debating proper API design, in the sense that the code that one would write using guava is nice (as defined by: Easy to understand, has few surprises, easy to maintain, easy to test, and probably easy to mutate to deal with changing requirements - the only useful definition for nebulous terms like 'nice' or 'elegant' code - it's code that does those things, anything else is pointless aesthetic drivel). In other words, they are actively trying, and they usually get it right.
Just, not in this case. Work around it: return item != null && coll.contains(item); will get the job done.
There is one major argument in favour of guava's choice: They 'contract break' is an implicit break - one would expect that .contains(null) works, and always returns false, but it's not explicitly stated in the javadoc that one must do this. Contrast to e.g. IdentityHashMap, which uses identity equivalence (a==b) and not value equality (a.equals(b)) in its .containsKey etc implementations, which explicitly goes against the javadoc contract as stated in the j.u.Map interface. IHM has an excellent reason for it, and highlights the discrepancy, plus explains the reason, in the javadoc. Guava isn't nearly as clear about their bizarre null behaviour, but, here's a crucial thing about null in java:
Its meaning is nebulous. Sometimes it means 'empty', which is bad design: You should never write if (x == null || x.isEmpty()) - that implies some API is badly coded. If null is semantically equivalent to some value (such as "" or List.of()), then you should just return "" or List.of(), and not null. However, in such a design, list.contains(null) == false) would make sense.
But sometimes null means not found, irrelevant, not applicable, or unknown (for example, if map.get(k) returns null, that's what it means: Not found. Not 'I found an empty value for you'). This matches with what NULL means in e.g. SQL. In all those cases, .contains(null) should be returning neither true nor false. If I hand you a bag of marbles and ask you if there is a marble in there that is grue, and you have no idea what grue means, you shouldn't answer either yes or no to my query: Either answer is a meaningless guess. You should tell me that the question cannot be answered. Which is best represented in java by throwing, which is precisely what guava does. This also matches with what NULL does in SQL. In SQL, v IN (x) returns one of 3 values, not 2 values: It can resolve to true, false, or null. v IN (NULL) would resolve to NULL and not false. It is answering a question that can't be answered with the NULL value, which is to be read as: Don't know.
In other words, guava made a call on what null implies which evidently does not match with your definitions, as you expect .contains(null) to return false. I think your viewpoint is more idiomatic, but the point is, guava's viewpoint is different but also consistent, and the javadoc merely insinuates, but does not explicitly demand, that .contains(null) returns false.
That's not useful whatsoever in fixing your code, but hopefully it gives you a mental model, and answers your question of "why does it work like this?".

Swig behavior for methods that accept rvalue reference as a parameter

Swig documentation specifies that it intentionally ignores move constructors. However it does not specify what it does with methods that accept rvlaue reference, for example:
class AAA {
public:
AAA() {}; // generates java code
AAA(const AAA& a) {}; // generates java code
AAA(AAA&& a) {}; // ignored by swig, no java code generated
virtual void move(const std::string& str); // generates java code
virtual void move(std::string&& str); // also generates java code, but accepts SWIGTYPE_p_std__string as a parameter instead of String
};
Would these methods behave differently? Is there a go-to way of working with rvalue parameters in swig? Or should I just not use them?

Generally for well written C++ code you'd expect the same behavior of two overloaded functions or constructors to be equivalent, with the caveat that the rvalue reference variant will most likely be destructive to its input.
So for making an interface in another language, which probably doesn't really have any neat 1:1 mapping to the concept that rvalues are trying to express then the vast majority of the time the right answer is to simply ignore these overloads. Internally in some cases the SWIG generated wrappers might end up using these overloads, or could be written to do so with some effort on your part, but it's just an optimization. And if you're jumping across a JNI boundary then that's probably unlikely to be your biggest performance bottleneck. (Measure it to be sure before doing anything).
In some language (e.g. Python, Lua?) you might be tempted to use the fact that your objects are reference counted to your advantage and (transparently to these languages) write a typemap that picks which overload to use based on if we're the only ones holding a reference to an object still. I'd argue that even this case is wrong (and also premature optimization) though because:
There's a race condition around weak references if the language supports such a construct. It's hard to spot and avoid this.
Even though you have only one reference to your (e.g.) Python object there could still be multiple references to the same C++ object, for instance:
struct a {};
a& blah() {
static a a;
return a;
}
With
import test
a=test.blah()
b=test.blah()
c=test.blah()
Has both a,b, and c only having 1 reference, yet moving those would clearly be wrong. And it's almost impossible to to prove when it would be safe.
So what I'm saying is: ignore them unless you have no choice. Sometimes you might come across cases where the only option is to call a function that takes an rvalue reference. You can wrap those functions, but it needs to be done on a case by case basis as you need to understand the semantics of the function/constructor being wrapped.

Valid usage of Optional type in Java 8

Is this a valid (intended) usage of Optional type in Java 8?
String process(String s) {
return Optional.ofNullable(s).orElseGet(this::getDefault);
}

I'll take another swing at this.
Is this a valid usage? Yes, in the narrow sense that it compiles and produces the results that you're expecting.
Is this intended usage? No. Now, sometimes things find usefulness beyond what they were originally for, and if this works out, great. But for Optional, we have found that usually things don't work out very well.
Brian Goetz and I discussed some of the issues with Optional in our JavaOne 2015 talk, API Design With Java 8 Lambdas and Streams:
link to video
link to slides
The primary use of Optional is as follows: (slide 36)
Optional is intended to provide a limited mechanism for library method return types where there is a clear need to represent "no result," and where using null for that is overwhelmingly likely to cause errors.
The ability to chain methods from an Optional is undoubtedly very cool, and in some cases it reduces the clutter from conditional logic. But quite often this doesn't work out. A typical code smell is, instead of the code using method chaining to handle an Optional returned from some method, it creates an Optional from something that's nullable, in order to chain methods and avoid conditionals. Here's an example of that in action (also from our presentation, slide 42):
// BAD
String process(String s) {
return Optional.ofNullable(s).orElseGet(this::getDefault);
}
// GOOD
String process(String s) {
return (s != null) ? s : getDefault();
}
The method that uses Optional is longer, and most people find it more obscure than the conventional code. Not only that, it creates extra garbage for no good reason.
Bottom line: just because you can do something doesn't mean that you should do it.

Since this is more or less an opinion-based question, I'll throw mine in. If you're trying to say
if (id == 1) {
Foo f = new Foo(id, "Bar", "US");
return "Bar".equals(f.getName()) && "US".equals(f.getCountryCode());
} else {
return false;
}
then just say that. Making things "functional" doesn't automatically make things clearer or better. By introducing a needless Optional, a couple lambdas, and some Optional methods that I had to look up, you've made the code more convoluted and difficult to understand. I don't think the designers of Java "intended" for people to use Optional to help make code more obscure.
EDIT: After reading some responses, I think it's worth adding some comments. This is not a functional programming idiom I'm familiar with, which would make it harder to understand. The idioms I am familiar with mostly involve Java streams, or (in other languages) functional idioms applied to multiple values in arrays or lists or other collections of multiple values. In those cases, once you get past the unfamiliarity, the functional syntax can be seen as an improvement because it allows some details to be hidden (loop indexes, iterators, running pointers, accumulator variables). So overall, it can simplify things. This example, by itself, doesn't do any such simplification.
However, some of the Optional features are useful in stream contexts. Suppose we had a parseInt() method that returns an Optional<Integer>, which is empty if the input string is invalid. (Java 8 really should have provided this.) This would make it easy to take an array of strings and produce an array of integers in which the strings that don't parse are simply eliminated from the result--use parseInt in a stream map(), and use a stream filter to filter out the empty Optionals. (I've seen multiple StackOverflow questions asking how to do this.) If you want to keep only the positive values, you could use an Optional.filter() to change the nonpositives to Optional.empty() before using the stream filter (although in this case, you could add another stream filter afterwards, but in a more complex case the Optional filter could be more useful). That's what I see as the main benefit of Optional from a functional standpoint. It allows you to work with a collection of values all at once, by giving you a way to represent "non-values" and write a function that will still work with them. So I guess the main use of Optional, besides a replacement for null, would be to represent empty spaces in a sequence of values while you're applying functions to the entire sequence as a whole.

Asking whether it's "valid" is rather opinion-based, but as to whether it's the intended use case: no, it's not.
Brian Goetz, Oracle's language architect for Java, has stated that the use case for Optional is for when you need a "no value" marker, and when using null for this is likely to cause errors. Specifically, if a reasonable user of your method is not likely to consider the possibility that its result is null, then you should use Optional. It was explicitly not intended to be a general "Maybe"-type object, as you're using it here.
In your case, the method that returns the Optional is private. That means it can only be used by the implementers of the class, and you can assume that they have good knowledge of the class' methods — including which of them may return null. Since there's no reasonable risk of confusion, Brian Goetz would (probably) say that he would not consider this a valid use case.

Its a little contrived, but 'valid' (as in 'syntactically') , but as #yshavit pointed to, it was intended for use in library development.
Previous answer was due to FP code being difficult to read. Below is commented(a little verbose, b/c that is the javadoc comments) but still. Much easier to read IMHO. (2nd is no-comments, and at least alignment to help readability)
private boolean isFooValid(final Integer id) {
return getFoo(id)
// filter if 'f' matches the predicate, return Optional w/f if true, empty Optional if false
.filter(f -> "Bar".equals(f.getName()) && "US".equals(f.getCountryCode()))
// If a value is present, apply the provided mapping function to it,
// If non-null, return an Optional describing the result.
.map(f -> true)
// Return the value if present, otherwise return other.
.orElse(false);
}
Or at least line it up so its more apparent what is going on and easier to read.
private boolean isFooValid(final Integer id) {
return getFoo(id)
.filter(f -> "Bar".equals(f.getName()) && "US".equals(f.getCountryCode()))
.map(f -> true)
.orElse(false);
}

Would it be a good idea if compiler resolved nulls when Optional<Object> is expected as argument?

That would be so obviously useful that I am starting to think I am missing a rationale to avoid it, since I am sure Oracle would have made it that way. It would be the most valuable feature on Optional for me.
public class TestOptionals{
public static void main(String[] args) {
test(null);
}
public static void test(Optional<Object> optional){
System.out.println(optional.orElse(new DefaultObject()));
}
}
(This throws a NullPointerException)
Without that feature I see too verbose using Optional for the argument.
I prefer a simple Object optional signature and
checking it by if (null = optional) that creating the object Optional for comparing later. It is not valuable if that doesn't help you checking the null

There was a HUGE discussion of Optional on all the various Java mailing lists, comprising hundreds of messages. Do a web search for
site:mail.openjdk.java.net optional
and you'll get links to lots of them. Of course, I can't even hope to summarize all the issues that were raised. There was a lot of controversy, and there was quite a breadth of opinion about how much "optionality" should be added to the platform. Some people thought that a library solution shouldn't be added at all; some people thought that a library solution was useless without language support; some people thought that a library solution was OK, but there was an enormous amount of quibbling about what should be in it; and so forth. See this message from Brian Goetz on the lambda-dev mailing list for a bit of perspective.
One pragmatic decision made by the lambda team was that any optional-like feature couldn't involve any language changes. The language and compiler team already had its hands full with lambda and default methods. These of course were the main priorities. Practically speaking, the choices were either to add Optional as a library class or not at all.
Certainly people were aware of other languages' type systems that support option types. This would be a big change to Java's type system. The fact is that for most of the past 20 years, reference types have been nullable, and there's been a single, untyped null value. Changing this is a massive undertaking. It might not even be possible to do this in a compatible way. I'm not an expert in this area, but most such discussions have tended to go off into the weeds pretty quickly.
A smaller change that might be more tractable (also mentioned by Marko Topolnik) is to consider the relationship between reference types and Optional as one of boxing, and then bring in the support for autoboxing/autounboxing that's already in the language.
Already this is somewhat problematic. When auto(un)boxing was added in Java 5, it made a large number of cases much nicer, but it added a lot of rough edges to the language. For example, with auto-unboxing, one can now use < and > to compare the values of boxed Integer objects. Unfortunately, using == still compares references instead of values! Boxing also made overload resolution more complicated; it's one of the most complicated areas of the language today.
Now let's consider auto(un)boxing between reference types and Optional types. This would let you do:
Optional<String> os1 = "foo";
Optional<String> os2 = null;
In this code, os1 would end up as a boxed string value, and os2 would end up as an empty Optional. So far, so good. Now the reverse:
String s1 = os1;
String s2 = os2;
Now s1 would get the unboxed string "foo", and s2 would be unboxed to null, I guess. But the point of Optional was to make such unboxing explicit, so that programmers would be confronted with a decision about what to do with an empty Optional instead of having it just turn into null.
Hmmm, so maybe let's just do autoboxing of Optional but not autounboxing. Let's return to the OP's use case:
public static void main(String[] args) {
test(null);
}
public static void test(Optional<Object> optional) {
System.out.println(optional.orElse(new DefaultObject()));
}
If you really want to use Optional, you can manually box it one line:
public static void test(Object arg) {
Optional<Object> optional = Optional.ofNullable(arg);
System.out.println(optional.orElse(new DefaultObject()));
}
Obviously it might be nicer if you didn't have to write this, but it would take an enormous amount of language/compiler work, and compatibility risk, to save this line of code. Is it really worth it?
What seems to be going on is that this would allow the caller to pass null in order to have some specific meaning to the callee, such as "use the default object" instead. In small examples this seems fine, but in general, loading semantics onto null increasingly seems like a bad idea. So this is an additional reason not to add specific language support for boxing of null. The Optional.ofNullable() method mainly is there to bridge the gap between code that uses null and code that uses Optional.

If you are committed to using the Optional class, then see the other answers.
On the other hand, I interpreted your question as, "Would it be a good idea to avoid the syntactic overhead of using Optional while still obtaining a guarantee of no null pointer exceptions in your code?" The answer to this question is a resounding yes. Luckily, Java has a feature, type annotations, that enables this. It does not require use of the the Optional class.
You can obtain the same compile-time guarantees, without adding Optional to your code and while retaining backward compatibility with existing code.
Annotate references that might be null with the #Nullable type
annotation.
Run a compiler plugin such as the Checker Framework's Nullness Checker.
If the plugin issues no errors, then you know that your code always checks for null where it needs to, and therefore your code never issues a null pointer exception exception at run time.
The plugin handles the special cases mentioned by #immibis and more, so your code is much less verbose than code using Optional. The plugin is compatible with normal Java code and does not require use of Java 8 as Optional does. It is in daily use at companies such as Google.
Note that this approach requires you to supply a command-line argument to the compiler to tell it to run the plugin. Also note that this approach does not integrate with Optional; it is an alternate approach.

Why does java/javascript/python force the use of () after a method name, even if it takes no arguments?

One of my most common bugs is that I can never remember whether something is a method or a property, so I'm constantly adding or removing parentheses.
So I was wondering if there was good logic behind making the difference between calling on an object's properties and methods explicit.
Obviously, it allows you to have properties and methods that share the same name, but I don't think that comes up much.
The only big benefit I can come up with is readability. Sometimes you might want to know whether something is a method or a property while you're looking at code, but I'm having trouble coming up with specific examples when that would be really helpful. But I am a n00b, so I probably just haven't encountered such a situation yet. I'd appreciate examples of such a situation.
Also, are there other languages where the difference isn't explicit?
Anyways, if you could answer, it will help me be less annoyed every time I make this mistake ^-^.
UPDATE:
Thanks everyone for the awesome answers so far! I only have about a week's worth of js, and 1 day of python, so I had no idea you could reference functions without calling them. That's awesome. I have a little more experience with java, so that's where I was mostly coming from... can anyone come up with an equally compelling argument for that to be the case in java, where you can't reference functions? Aside from it being a very explicit language, with all the benefits that entails :).

All modern languages require this because referencing a function and calling a function are separate actions.
For example,
def func():
print "hello"
return 10
a = func
a()
Clearly, a = func and a = func() have very different meanings.
Ruby--the most likely language you're thinking of in contrast--doesn't require the parentheses; it can do this because it doesn't support taking references to functions.

In languages like Python and JavaScript, functions are first–class objects. This means that you can pass functions around, just like you can pass around any other value. The parentheses after the function name (the () in myfunc()) actually constitute an operator, just like + or *. Instead of meaning "add this number to another number" (in the case of +), () means "execute the preceding function". This is necessary because it is possible to use a function without executing it. For example, you may wish to compare it to another function using ==, or you may wish to pass it into another function, such as in this JavaScript example:
function alertSomething(message) {
alert(message);
}
function myOtherFunction(someFunction, someArg) {
someFunction(someArg);
}
// here we are using the alertSomething function without calling it directly
myOtherFunction(alertSomething, "Hello, araneae!");
In short: it is important to be able to refer to a function without calling it — this is why the distinction is necessary.

At least in JS, its because you can pass functions around.
var func = new Function();
you can then so something like
var f = func
f()
so 'f' and 'func' are references to the function, and f() or func() is the invocation of the function.
which is not the same as
var val = f();
which assigns the result of the invocation to a var.
For Java, you cannot pass functions around, at least like you can in JS, so there is no reason the language needs to require a () to invoke a method. But it is what it is.
I can't speak at all for python.
But the main point is different languages might have reasons why syntax may be necessary, and sometimes syntax is just syntax.

I think you answered it yourself:
One of my most common bugs is that I can never remember whether something is a method or a property, so I'm constantly adding or removing parentheses.
Consider the following:
if (colorOfTheSky == 'blue')
vs:
if (colorOfTheSky() == 'blue')
We can tell just by looking that the first checks for a variable called colorOfTheSky, and we want to know if its value is blue. In the second, we know that colorOfTheSky() calls a function (method) and we want to know if its return value is blue.
If we didn't have this distinction it would be extremely ambiguous in situations like this.
To answer your last question, I don't know of any languages that don't have this distinction.
Also, you probably have a design problem if you can't tell the difference between your methods and your properties; as another answer points out, methods and properties have different roles to play. Furthermore it is good practice for your method names to be actions, e.g. getPageTitle, getUserId, etc., and for your properties to be nouns, e.g., pageTitle, userId. These should be easily decipherable in your code for both you and anyone who comes along later and reads your code.

If you're having troubles, distinguishing between your properties and methods, you're probably not naming them very well.
In general, your methods should have a verb in them: i.e. write, print, echo, open, close, get, set, and property names should be nouns or adjectives: name, color, filled, loaded.
It's very important to use meaningful method and property names, without it, you'll find that you'll have difficulty reading your own code.

In Java, I can think of two reasons why the () is required:
1) Java had a specific design goal to have a "C/C++ like" syntax, to make it easy for C and C++ programmers to learn the language. Both C and C++ require the parentheses.
2) The Java syntax specifically requires the parentheses to disambiguate a reference to an attribute or local from a call to a method. This is because method names and attribute / local names are declared in different namespaces. So the following is legal Java:
public class SomeClass {
private int name;
private int name() { ... }
...
int norm = name; // this one
}
If the () was not required for a method call, the compiler would not be able to tell if the labeled statement ("this one") was assigning the value of the name attribute or the result of calling the name() method.

The difference isn't always explicit in VBA. This is a call to a Sub (i.e. a method with no return value) which takes no parameters (all examples are from Excel):
Worksheets("Sheet1").UsedRange.Columns.AutoFit
whereas this is accessing an attribute then passing it as a parameter:
MsgBox Application.Creator
As in the previous example, parentheses are also optional around parameters if there is no need to deal with the return value:
Application.Goto Worksheets("Sheet2").Range("A1")
but are needed if the return value is used:
iRows = Len("hello world")

Because referencing and calling a method are two different things. Consider X.method being the method of class X and x being an instance of X, so x.method == 'blue' would'nt ever be able to be true because methods are not strings.
You can try this: print a method of an object:
>>> class X(object):
... def a(self):
... print 'a'
...
>>> x=X()
>>> print x.a
<bound method X.a of <__main__.X object at 0x0235A910>>

Typically properties are accessors, and methods perform some sort of action. Going on this assumption, it's cheap to use a property, expensive to use a method.
Foo.Bar, for example, would indicate to me that it would return a value, like a string, without lots of overhead.
Foo.Bar() (or more likely, Foo.GetBar()), on the other hand, implies needing to retrieve the value for "Bar", perhaps from a database.
Properties and methods have different purposes and different implications, so they should be differentiated in code as well.
By the way, in all languages I know of the difference in syntax is explicit, but behind the scenes properties are often treated as simply special method calls.