I'm wondering if it would make sense to do some low level or OS stuff (a project) using Java. Reason why I ask is because I would like to expand my knowledge in Java and I'm into doing stuff like file compressor, bulk file renamer, etc. Are there any examples out there that I can look at or play with? Or should I just be using C or C++ instead?
stuff like file compressor, bulk file
renamer, etc.
I wouldn't consider that "low level or OS stuff".
In my book, "low level or OS stuff" means things like device drivers. For that kind of thing, Java is very badly suited because it runs in a VM and simply does not have access to the OS API and the hardware (well, unless you run a Java-based OS).
For the two examples you name, Java could work quite well, but you could also easily run into limitations that are hard or impossible to overcome: Java's filesystem API dictates what you can do, and if that's not enough, the only thing left is to call native (i.e. C/C++) code via JNI or Runtime.exec().
I'm wondering if it would make sense to do some low level or OS stuff(a project) using Java.
Generally speaking, no. The low-level stuff is either taken care of by the JVM (or the operating system) or is impossible to do in pure Java.
However, if you are really interested in this kind of stuff, wander across to the JNode project and take a look at the various student projects on offer. JNode is a complete operating system that boots and runs on a "bare-metal" PC and is implemented (almost) entirely in Java.
Low level OS stuff is not very related to Java; Java uses a lot of abstractions (and hence making it higher level). You can, however, use the Java language and VM to interact with lower level API's using Java Native Access.
The problem with "low level" is: what is low level? Do you want to execute assembly instructions? And then there is Java: completely platform independent. Using more lower level API's in Java means that you lose the independency from your platform (think of: OS or hardware).
You can, however, also learn more about Java bytecodes: this is also quite low level.
If you provide more information on your project, I can give you a more specific answer.
Related
I am an embedded programmer and working with an embedded JVM.
This enables running Java files on constrained devices.
These Java files are first compiled to bytecode into .class files which are then further optimized and uploaded to the device which has a micro JVM to run the optimized bytecode.
The micro JVM does not support all features, e.g., no reflection.
The main benefit is obvious: this allows programming in Java for constrained devices.
However, I was thinking that plenty of languages compile to bytecode, some are listed https://en.wikipedia.org/wiki/Java_bytecode.
So in theory these languages could also be used to program.
I'd like to obtain a list of common languages that compile down to bytecode and was wondering if you could help.
For example, Python has special implementations that reduce to Java Bytecode, if I'm not mistaken, and stuff like C to Java virtual machine compilers also exist.
So what languages would you think are logical to try and run on the devices? Any pointers on how to or similar experiences?
Also, I'm not clear what the difference is from reading Wikipedia between (Python) bytecode and Java bytecode, could anybody help explain that?
I'm agree with you about the overall idea and it would be nice to develop an embedded application using any language that can run on a JVM. But there are some practical issues that you should consider and I think that's why none of major vendors or open source initiatives have any active/serious project on this (as far as I know).
As you mentioned, a JVM implementations that can run on embedded devices, each have their own constraints and limitations. The most obvious one is that some packages may not be available at runtime. In order to apply such a constraint, you should either control it in the compile process or have a toolchain (sort of an SDK) which accepts the bytecode and checks such constraints.
This situation would be worth when a developer tries to use a third party library that is available for that specific language. It's not easy to guess if a library is safe for use against such a framework or not.
One great facility for developers would be to have their IDE check such issues on the fly (something like inspection in IntelliJ Idea). This makes it much more smoother to move toward using such a solution. But again the problem is that for each such languages there need to be a specific plugin compatible with their own syntax.
Also some of JVM languages that are actually implementation of an existing language (e.g. Jython or JRuby) are most of the time out of sync with the original language in case of supporting libraries/syntax changes of that language.
Anyway, I think in order to have a list of JVM languages you could easily find them on Wikipedia. Maybe you mean those who may worth considering in this regard by having a large community and tools support. In my opinion, you should focus on the following JVM languages as those who may worth to include in your final list:
Groovy
Kotlin
Scala
These are all pure JVM languages and are only using different syntax than Java.
Regarding the topic in general, I should say that when you search for embedded JVM implementations, you'll notice that it's also a fairly academic concepts and they're so many publications in this topics regarding the overall architecture, threading support, toolchain, error handling, memory management, etc. This means that you should have a very great experiences/background on both embedded systems and programming language concepts and implementation to be able to devise a proper architecture for such a platform.
About your last question regarding the difference between Python bytecode and Java bytecode (if I understand your question correctly), these are both conceptually the same but each has its own syntax and constraints. The bytecode concept refers to the piece of software that is the output of the compiler and is the platform independent representation of the original code and can be run/interpreted at runtime by another software component which is the virtual machine. In Java world, this software is called the Java Virtual Machine (JVM). I'm from the Java world so I don't know what it's called in Python vocabulary but it should be something similar (e.g. Python virtual machine).
I think due to the complexity of developing such a toolchain and also considering the unprecedented development of new IoT and SoC devices, many of them capable of running a more higher level operating systems, maybe in a long run most developers prefer to develop for a more high end devices using more high level APIs and SDKs. Who knows! In that case, we would have a same situation that we're in today for PCs. Languages like C and Assembly are still in use, but they have their own domain of applications. I mean throughout the time, layers of abstraction are being added on top of the previous one. The same thing can happen for embedded devices.
Is it possible to write an antivirus program in Java such as that it can intercept a program from being executed? Can I have such a deep control of the OS in Java?
update:
what about c#? same restrictions apply or that is a better way?
Having such influence on the OS is possible. There is only the problem, that you will lose the platform independency or at least have to write the code for every given platform due to the reason that such actions require quite deep access of the system which could be achived with JNI, which would tie the method you use it in to the OS.
I don't think that sort of control is possible with Java, primarily because it uses a VM and is shielded from the OS. Or rather the OS is shielded from the Java VM. This is by design.
Edited to add for clarity: I am assuming that you want to write the entire solution in Java, and not mix languages.
I am not convinced that it would work even with JNI.
In the case of "intercepting" when the OS starts a new process (or writes to a file or whatever), you need to write some kind of driver or kernel module which hooks into the OS. That driver/module is most certainly written in native compiled code. So the OS is the one in charge here, and will eventually call your native module.
So, as I see it, Java is not even involved here.
Thats the basic approach anyway. It may be possible using something like pam in Linux which is configurable to do almost anything related to security and file/process permissions and can call other processes to do its bidding. Seems far fetched though to run a JVM instance for each new process the OS tries to start.
As HalloDu said, this is technically possible with the use of JNI. However, IIRC, most antivirus programs use some sort of driver to intercept opened files and scan them before allowing the OS to continue using the file. This being the case, the amount of native code you would have to write (in C or possibly C++) would be substantial and is likely to outstrip your Java code in size.
When writing low-ish level apps, I'd stick to C. However, it might make sense to code things like the GUI in a higher level language, though Java wouldn't be my choice there either, because it's kind of a pain to interface with C. Personally, I'd do the whole damn thing in C just because mixing languages tends to be a pain. If I had to mix languages, my choices would by C and python, simply because ctypes makes interfacing with C really easy.
It is possible with the JNI. You would mostly be using Java for a GUI and C/C++ for any other sort of antivirus work though.
What is the point in making your own Antivirus? It is a lot of work, but I guess it would be cool if you made it a portable one that block and removes all the more nasty ones. If you must persist, ClamAV, it is an open source and pretty good AV (no realtime protection) but programmed in C++.
Your best bet might be to write the GUI and much of the logic in Java, then have a C or C++ back-end that does the scans.
You can then re-use the front-end across platforms and keep the platform specific stuff in the lower levels.
This way you can use the strengths of both languages--Java's platform independence and ease of use and C/C++'s ability to directly access the underlying platform.
Alright everyone,
Let's say i wanted to make a system that used only assembly and java, my question would be, as long as i included all of the jvm folders, classes, jars, etc.. java should still function effectively?
i understand there are those things that are compiled platform specifically but this is why i am asking, is it possible, using assembly to replicate all of the .exe, or other executable files that java has included into a pure assembly/java system?
If you are asking whether it is possible to build a system in Java and assembly language that runs on "bare metal", the answer is yes. There are a couple of current examples:
JavaOS is targeted primarily at the embedded systems domain. (Sun consider SunOS to be a "legacy" product line these days.)
JNode aims are broader, and encompass embedded systems, desktop systems, servers and cloud computing.
Be aware that building a system of this kind is a multi-year, multi-person project requiring deep understanding of virtual machine internals, compilers, garbage collectors, hardware architectures, device driver writing and so on.
If you are asking about something else, please be more explicit.
EDIT: responding to the OP's followup question:
It is not practical to use the Java and other "exe" files per se. They require a fully fledged operating system underneath them; e.g. Windows, Linux, whatever. If you had access to the source code, you could conceivably rewrite as required to make them run on "bare metal", but that would entail significant architectural changes, especially if you want to write device drivers, etc in Java. (Besides, the core of Sun's JRE is implemented in C++ ... ).
You cannot directly use the existing Java class library JAR files, because they include a significant amount of platform specific code. However, you can build your own Java class library JARs from an existing open-source version of the Java class libraries (e.g. the OpenJDK 6.0 J2SE libraries). You deal with the platform specific code by providing your own versions as native libraries or (as JNode does) as Java classes.
If I understand your question correctly, you mean something like JavaOS. Sure, its possible to implement the JVM raw on the hardware, not sure why you would, though. And if you did, why you wouldn't use C instead of Assembly for most of the work.
Its theoretically possible to implement the jvm in a whole other language. The best example I can think of is Python/Jpython where there is the original C implementation and a pure Java implementation of the language.
The main argumant against this is -- its a ton of work for not much benefit.
The official Sun jvm and supporting jni libraries are written mostly in C, you would need to provide native assembler implementations for most of the C POSIX APIs at the very least.
Also the original design goal of C was 'a portable assembly language' and to a large extent it still meets these goals. C produces efficient machine code and most C compilers will let code machine instructions inline with the C code.
Another benefit of C is the number of cross compilers available, you dont need to run the development environment on tHe target architecture, you can deveop and unit test on your favourite paltform/IDE, when you are ready you can then export your executables to the target platform.
Jikes RVM and Sun's Maxine provide a JVM implementation with little (of the order of 1 kloc) native code. However, both VMs require an OS and are only research implementations. The process of creating a stream of octets that form machine code, is obviously achievable in Java.
Have a look at JNode. They have been working on this for years.
http://www.jnode.org/
The Java runtime provides a set of standard system libraries for use by programs. To what extent are these libraries similar to the system calls of an operating system, and to what extent are they different???
Half the point of java was to make it platform independent, so what it tries to do is provide an api that remains the same regardless of the OS underneath it.
If the OS is underpowered, Java will add library code to compensate for it.
If the OS has an implementation that doesn't map, Java will do it's best to map it.
If a new function becomes popular and Java users need to provide access to it, a new library can be created through which you can access the new functionality. If this library is popular, it will be restructured and added into the Java SDK at some point
For instance, an implementation of some concurrency libraries became popular, and soon they were voted upon and added to the standard libraries. This happens all the time.
That obviously depends on the OS you're running on, since the system calls are generally different for every OS :-).
That said, I believe Java was mostly inspired by Unix conventions (not surprinsingly, as Sun is a Unix vendor), so some Java system libraries are similar to Unix sytem calls.
E.g. java.nio.MappedByteBuffer was probably inspired by Unix's mmap() call. But ultimately most concepts are present on most OSes, so you cannot really say what inspired what.
Some of Java's "low-level" functions are basically "wrappers" around some OS
system calls.
I don't see an objective way (and reason) to "compare" both.
If you are interested in this topic, you can search the Java source code
for the native keyword, which indicates some "hidden"
(mostly OS-dependend) functionality.
Java's standard library often has a similar feature set compared to the native library but there are several important differences.
Java is Object Oriented, whether you like it or not. The advantage of this is that certain concepts are easier to manage. For example, most file related operations are found directly in the File object. Compare this to Posix, where a FILE is a handle which is really just a number; an index into your process's open file list. The Posix approach is very close to how the OS actually implements stuff. But in Java you don't see that or know it or care.
Java has certain lowest-common-denominator behaviours in certain cases. There are many AWT APIs that are the way they are because AWT needed to be identical on a number of separate platforms. That turned out to be madness, and Sun quasi-deprecated most of AWT, because supporting platform equally meant supporting every platform crappily. The newer library, Swing, implements almost everything in pure Java, and thus is far better at cross-platform stuff, and thus has a richer API. And that API is very different from the native windowing library. Also, Swing doesn't integrate too well because it uses so little of the native OS.
Java has certain limitations that the native libraries don't have. For example, you don't have function pointers. Thus you have Listeners and Runnable and other Java patterns for doing things that in C++ would involve function pointers. So any API that needs one of these features will be significantly different in Java than in the native OS.
So in conclusion, Java often has libraries that offer similar behaviour to the native OS, and sometimes offer completely different behaviour, but it's best to think of Java as a platform in its own right. Sometimes you need advanced performance, such as OpenGL or super-fast data transfer, in which case you'll want a specific Java API (Jogl, nio), but most of the time you should evaluate Java as its own thing.
Like the subject of this post suggests, I am looking at developing a suite like nero which helps burn bluray discs. I am kind of clueless as to where to start. Is there anything in Java API that lets you do this? If I were to start from scratch, would I need to start with the bluray disc spec? Are there any open source tools which are already doing this? I tried searching at sourceforge.net and found nothing useful. Any help is much appreciated.
To start with the obvious: Know your requirements and tools. I try guessing here, maybe.
Requirements:
Should burn BluRay discs
Graphical user interface
Preferred tool:
Java
Now, Java, being perhaps the prime example of a VM language from the 90es, achieves its relatively good platform-agnosticism by virtue of its VM. It's a language designed to run on a virtual hardware to ease portability to real hardware.
Now, what comes with this fact is that you abstract away many things you would have to care about, like memory-management details and architecture or platform-specifics. Among those things you can't reliably get access to is hardware. After all, you abstracted most of that away.
Now, to burn a BluRay disc you have to access hardware, in particular the BluRay writer. Not that it's impossible but Java is, in my humble opinion, not the right tool for this. You can go out of your way by implementing a library in C or C++ and using JNI/JNA to access that but looking at that, what do you really gain?
Java is usually a choice when you need a fairly modern high-level language with a large standard library and you also need your programs to run on more than one platform. Those are the primary use cases. It's not impossible with other technologies, but perhaps harder to achieve, depending on what exactly you need.
If you implement a native library to talk to the BluRay writer and talk to that from Java, then you necessarily need to re-implement it for other platforms as well (assuming that's what you want—if not, then again: Why Java?).
TL/DR version: My point is that it's not too surprising that you can't find much on exactly that topic. For one, Java wasn't really designed to do that sort of things. Most of the Java/native interop lies in the JVM and that's already an awful lot of code. Don't expect Java to natively support very rare usage scenarios such as CD/DVD/BluRay burning. Secondly, BluRay is a relatively new technology with writers not yet common hardware in computers such as CD/DVD writers, so the lack of libraries and tools may also be a mirror of the current demands of the market.
Low-level hardware access is simply not possible in pure Java unless it's in the standard API, which Bluray isn't.
Therefore, you will have to use non-Java code to access the hardware; at that point you lose the platform-independance of Java, and necessarily have a multi-language system, which is always more painful to program than using just a single language.
However, if you can find (or, I guess, develop) a multi-platform Bluray writing API or command line tool in (most likely) C, then it might still make sense to write the rest of the app in Java as a GUI wrapper with added functionality.