I'd like to use Kotlin & Scala together in projects, and maybe some other languages, but I've seen no good way of doing it. The only way I thought of was compiling one language and decompiling it into Java to work with the other. Are there any alternatives?
For the sake of completeness and not putting words into someone else's mouth, I wanted to weigh in.
I agree with the last sentence of ziggystar's answer. The right thing to do is to take a component-based approach and not try to combine multiple languages in one component or project.
From a technical perspective, each of the JVM languages has their own compiler. Some, such as Scala's, can compile both Scala and Java files. However, this may or may not be true for other compilers. In order to avoid strange build processes, a good approach would be to use a single language for every built module.
Since you're sticking to JVM languages, every languages can be compiled into a JAR, so you can easily distribute your executable binary as a single JAR file, with all of the components wrapped up inside it. This is the Fat JAR approach (see this question on Stack Overflow, this post on Java Code Geeks).
From a human readability perspective, this should also make your software more easily understood. Not only have you decomposed it into logical building blocks (each component), but someone making modifications only needs to understand the language that the component they are working on is written in and the public interface of the components they need to interact with. There's no mental context switching between languages.
You can use Scala and Java simultaneously, since scalac understands and compiles Java files. The same probably holds for other languages. Problems might arise when using multiple alternative JVM languages, since, e.g., the Kotlin compiler probably can't understand the Scala files and vice versa.
I think the best way would be to split the project into different modules, and use at most one alternative language per module.
What do I mean with module?
With module, I mean a set of source files that gets translated into one (binary) artifact, i.e. a jar file. Under different circumstances I would simply call a "module" a project. Note that a module may depend on other modules on the binary level (e.g. has some jar files as dependencies).
Multi module support in IDEs
I think it should be possible with most major IDEs to work on different modules simultaneously, even if each module uses a different language. Terminology varies across IDEs.
Terminology
For Intellij IDEA, one of my modules is called "module". For Eclipse it would be called "project".
Related
I have zero experience with Java, but when trying to understand a certain "apocalyptic" vulnerability, I ended up with a fundamental question about imports in Java, so please bear with me.
My question is, as given in the title, why a Java package can not be updated with a single central patch.
For comparison, two hypothetical diametric cases that I think I understand reasonably well:
If, say, a python library had some vulnerability, then it should suffice (on well-maintained systems that use centralized libraries located on PYTHONPATH) to update that single library and any code that imports it should, in general, be fixed.
On the other hand, if a C library had a vulnerability, then it would be necessary to replace every single binary whose source includes the vulnerable library with a patched binary.
Now, as far as I could tell, Java is actually closer to the former category of languages, where external imports are not included in compiled sources.
If this is the case, then why can't a single patch be applied to fix an entire system (au contraire, our IT department forwarded a gigantic list of software for us to check individually)? Is it because of multiple decentralized copies of identical libraries being installed, or is there some other reason? Or am I misunderstanding the issue?
Java applications themselves are separate processes. In principle, all these processes can use different VM's. This is often the case for larger applications, which are tested against a specific VM. In principle, Java runtimes (J2SE implementations) should remain as compatible as possible with each other, but it is certainly possible for developers or libraries to muck this up, e.g. by using "Sun" inner classes or by assuming things not specified for the API calls. Personally hate these kind of J2SE inclusions; I'd rather have applications that are created to remain compatible.
Smaller applications usually just run on one of the installed JRE's. However, they usually still need additional libraries or components - say, for instance, Log4J from Apache. These are often offered as separate .jar files (or "artifacts" in Maven speak). These libraries may also get updates; there is however not a common way of updating these on most systems; there is no single "application" set of shared libraries although it is certainly possible to create one. On Linux for instance there may be a set of libraries in /usr/share/java (by version, with generic names pointing to the latest one).
Many web applications - I those running on a specific application server such as Tomcat, Glassfish etc. do share a common "classpath", where application specific .jar files are put in specific folder. In that case an update of a library in the shared folder will affect all applications.
Java has had a framework for specific class-loaders, and in principle any framework can define their own set, so where the libraries are stored can depend on the framework. Java is very flexible and doesn't really have one single way of handling applications.
All this has previous little to do with import statements. These are just use as a shorthand notation, basically. You might as well use java.util.List as import java.util.List followed by List further in the code. Class files contain references to other classes (etc.), and those are resolved (found and loaded) at runtime; see the description from Oracle here.
We're currently migrating from Java 8 to Java 11. However, upgrading our services was less painful, than we anticipated. We basically only had to change the version number in our build.gradle file and the services were happily up and running. We upgraded libraries as well as (micro) services that use those libs. No problems until now.
Is there any need to actually switch to modules? This would generate needless costs IMHO. Any suggestion or further reading material is appreciated.
To clarify, are there any consequences if Java 9+ code is used without introducing modules? E.g. can it become incompatible with other code?
No.
There is no need to switch to modules.
There has never been a need to switch to modules.
Java 9 and later releases support traditional JAR files on the
traditional class path, via the concept of the unnamed module, and will
likely do so until the heat death of the universe.
Whether to start using modules is entirely up to you.
If you maintain a large legacy project that isn’t changing very much,
then it’s probably not worth the effort.
If you work on a large project that’s grown difficult to maintain over
the years then the clarity and discipline that modularization brings
could be beneficial, but it could also be a lot of work, so think
carefully before you begin.
If you’re starting a new project then I highly recommend starting with
modules if you can. Many popular libraries have, by now, been upgraded
to be modules, so there’s a good
chance that all of the dependencies that you need are already available
in modular form.
If you maintain a library then I strongly recommend that you
upgrade it to be a module if you haven’t done so already, and if all of
your library’s dependencies have been converted.
All this isn’t to say that you won’t encounter a few stumbling blocks
when moving past Java 8. Those that you do encounter will, however,
likely have nothing to do with modules per se. The most common
migration problems that we’ve heard about since we released Java 9 in
2017 have to do with changes to the syntax of the version
string and to the removal or
encapsulation of internal APIs
(e.g., sun.misc.Base64Decoder) for which public, supported
replacements have been available for years.
I can only tell you my organization opinion on the matter. We are in the process of moving to modules, for every single project that we are working on. What we are building is basically micro-services + some client libraries. For micro-services the transition to modules is somehow a lower priority: the code there is already somehow isolated in the docker container, so "adding" modules in there does not seem (to us) very important. This work is being picked up slowly, but it's low priority.
On the other hand, client libraries is an entirely different story. I can not tell you the mess we have sometimes. I'll explain one point that I hated before jigsaw. You expose an interface to clients, for everyone to use. Automatically that interface is public - exposed to the world. Usually, what I do, is have then some package-private classes, that are not exposed to the clients, that use that interface. I don't want clients to use that, it is internal. Sounds good? Wrong.
The first problem is that when those package-private classes grow, and you want more classes, the only way to keep everything hidden is to create classes in the same package:
package abc:
-- /* non-public */ Usage.java
-- /* non-public */ HelperUsage.java
-- /* non-public */ FactoryUsage.java
....
When it grows (in our cases it does), those packages are way too big. Moving to a separate package you say? Sure, but then that HelperUsage and FactoryUsage will be public and we tried to avoid that from the beginning.
Problem number two: any user/caller of our clients can create the same package name and extend those hidden classes. It happened a few times to us already, fun times.
modules solves this problem in a beautiful way : public is not really public anymore; I can have friend access via exports to directive. This makes our code lifecycle and management much easier. And we get away from classpath hell. Of course maven/gradle handle that for us, mainly, but when there is a problem, the pain will be very real. There could be many other examples, too.
That said, transition is (still) not easy. First of all, everyone on the team needs to be aligned; second there are hurdles. The biggest two I still see is: how do you separate each module, based on what, specifically? I don't have a definite answer, yet. The second is split-packages, oh the beautiful "same class is exported by different modules". If this happens with your libraries, there are ways to mitigate; but if these are external libraries... not that easy.
If you depend on jarA and jarB (separate modules), but they both export abc.def.Util, you are in for a surprise. There are ways to solve this, though. Somehow painful, but solvable.
Overall, since we migrated to modules (and still do), our code has become much cleaner. And if your company is "code-first" company, this matters. On the other hand, I have been involved in companies were this was seen as "too expensive", "no real benefit" by senior architects.
I am a student and I hate not knowing how things are organized. I'd like to be able to create a full on java project from scratch on the command line. I'd like to be able to import jars and set the classpath, make packages and import them. ALso learn about environment variables. I currently do not know much about organizing code. I just know how to code in Java.
Is there a textbook, online article or the like that allows one to learn how to organize a java project?
I do not want any involvement with eclipse or any IDE. I am willing to learn Maven, XML, or the likes to accomplish my goal.
If you are a student willing to have a Java programming career, it might help to learn how to do things from command line, e.g. edit the files, compiling the classes, testing and building the project. Oracle tutorials provide example on this matter: https://docs.oracle.com/javase/tutorial/getStarted/cupojava/win32.html#win32-2
However, I strongly advise you to embrace an IDE as your Java career will mostly reside in an IDE as real life projects are BIG! There are tons of helpful things the IDE does to you out of the box or to simplify things. Since you are a student, I will give you one basic example besides compiling: a class with 10 fields requires you some typing for getters, setters, hashCode, equals. Alternative? Few keystrokes to instruct the IDE to generate them for you. And that's one basic example.
Regarding the project structure, embracing the (since you mentioned it) Maven project structure of src/main/{java,resources}, src/test/{java,resources} even if you do NOT use Maven. This will let you forget about organizing the files around.
If you were asking about structuring the classes in the right packages, you will figure out yourself as you gain experience. Rule of thumb is to group classes together by functionality they provide. Additionally, if the packages are organized right, if you change something and touching a few classes, ideally you'd want the changed classes to be located in a single package if possible.
Learning Maven is a good choice as it is a powerful tool for building a project and keeping things organized (project structure, project dependencies, etc.).
A simple Java program can be compiled trivially by javac MyMainClass.java, provided that your CLASSPATH list directories and jars with its dependencies.
Compiling a large Java project is not trivial. There are several tools intended to make it simpler.
Gradle: very widely used, uses its own language, very powerful and complex.
Maven: Still widely used. Uses XML to describe everything.
Apache Ant is lower level and lower abstraction power.
The power of these tools lies exactly in hiding the boilerplate of the Java project building process. They generate a skeleton of a build for you, and provide higher-level operations.
Of course you can start with simplest and watch the steps these tools make.
Reading the docs for javac and jar thoroughly does help, too.
I'm new to modern Java compilers and Virtual Machines, so I'm curious, what technical issues do large Java projects (5000+ sizable classes) encounter, during compilation and at runtime, as the gordian knot of package dependencies grows?
In large C++ projects, you can get yourself into technical trouble (all maintainability concerns aside) if you stray far from an acyclic library (or package) dependency graph in large projects.
Some examples
compilation can run out of memory if most of a source tree is included
linking can too if too many object archives are included (object archives generally correlate with packages in C++ projects)
The problem is considerably exacerbated with inline template instantiation. Modern workstations aren't equipped to compile and link a project that pulls most of 5000 sizable classes together in either phase of the build.
The Java developers I've asked do not believe technical limitations are a reason to avoid circular package dependencies (other motivations apply). Are there any?
The Java compiler (javac) does not compile all the classes at the same time, but rather one by one, dynamically discovering uncompiled or stale .class files.
There is no linking. Instead all the .class files are packaged together in a jar file once compiled. This is basically a ZIP compression and this step isn't even required.
The Java compiler is moderately simple due to simple language syntax and semantics. There isn't much metaprogramming, type inference, etc. Scala compiler, for example, is much slower because the language itself is much more complicated.
That being said I can't find any technical limitations of compiling large, tangled projects. Obviously the build time grows and once it exceeds 10 minutes it becomes a pain, but that isn't really an issue.
The real problem with tangled, circular, cross-references is source code maintainability. Mainly it is much harder to refactor code. Once the project reaches certain size (5000+ classes is probably around half million LOC) developers will try to split it into pieces. Extract libraries, modules and layers. If the dependencies are so strong, this process is close to impossible.
There is really no such thing as package dependencies in Java. There are only class (and interface) dependencies. When you import a package in Java, you are only telling the compiler how to resolve names (so you don't need to fully qualify every class or static import name you use).
Circular dependencies between thousands of classes would probably bring a compiler to its knees.
I am working on a legacy framework and apparently there are two libraries, which are inter-dependent. By that I mean libA import from libB, and libB import from libA. First i think it is a terrible design, but why would somebody do something like this? Rather which conditions can lead somebody to write this ?
edit:
Each library depends on classes in the other, so they do import packages and have the other library jar in their build path.
It's easier to do in this case, because the two parties are independent. If they don't talk to each other, it's not hard to create cycles. You have to be mindful to avoid them.
Cyclic dependencies aren't hard to create. Look at Java itself: java.lang, java.util, and java.io have cycles. Will you stop writing Java, since it's so "terrible"?
It means that you can never use libA without libB and vice versa. They've become one big library. Same with packages in Java and other systems: once you have a cycle, you have to use all those packages together as if they were one.
The guys who write Spring pay a lot of attention to cycles. They design and refactor their framework to eliminate them.
So - what's the harm? Juergen Heller says they're bad, and he's right. But from your point of view, what evil is visited upon you? It means you have to use both when you run and test. You can't test class A without class B and vice versa when there's a cycle between them. It makes testing and running harder.
You can choose an alternative that doesn't have the cycle. If you can change the source, you can refactor and maintain it. But that's it.
You should check your own code to see if you've done it to yourself. IntelliJ has nice analysis tools which can be applied to a code base. Check it out.
While developing lib A, the developer found that the class Foo from lib B was useful. And while developing lib B, the developer found that the class Bar from lib A was useful.
I'm not saying it's a wise thing to do, but your question asks why anybody would do that. This is probably the answer.
Both the libraries were written at a the same time, possibly by different developers. Or the same developer at different times, or by a developer who treated both libraries as one big code base, and wasn't concerned about avoiding circular dependencies. e.g. They had a hard enough time writing something which worked without worrying about niceties.
Most likely it will be inexperience.
Modern build tools like Maven preclude circular dependencies between artifacts.