Introduction
I heard something about writing device drivers in Java (heard as in "with my ears", not from the internet) and was wondering... I always thought device drivers operated on an operating system level and thus must be written in the same language as the OS (thus mostly C I suppose)
Questions
Am I generally wrong with this
assumption? (it seems so)
How can a driver in an "alien"
language be used in the OS?
What are the requirements (from a
programming language point of view)
for a device driver anyway?
Thanks for reading
There are a couple of ways this can be done.
First, code running at "OS level" does not need to be written in the same language as the OS. It merely has to be able to be linked together with OS code. Virtually all languages can interoperate with C, which is really all that's needed.
So language-wise, there is technically no problem. Java functions can call C functions, and C functions can call Java functions. And if the OS isn't written in C (let's say, for the sake of argument that it's written in C++), then the OS C++ code can call into some intermediate C code, which forwards to your Java, and vice versa. C is pretty much a lingua franca of programming.
Once a program has been compiled (to native code), its source language is no longer relevant. Assembler looks much the same regardless of which language the source code was written in before compilation. As long as you use the same calling convention as the OS, it's no problem.
A bigger problem is runtime support. Not a lot of software services are available in the OS. There usually is no Java virtual machine, for example. (There is no reason why there technically couldn't be, but usually, but usually, it's safe to assume that it's not present).
Unfortunately, in its "default" representation, as Java bytecode, a Java program requires a lot of infrastructure. It needs the Java VM to interpret and JIT the bytecode, and it needs the class library and so on.
But there are two ways around this:
Support Java in the kernel. This would be an unusual step, but it could be done.
Or compile your Java source code to a native format. A Java program doesn't have to be compiled to Java bytecode. You could compile it to x86 assembler. The same goes for whatever class libraries you use. Those too could be compiled all the way to assembler. Of course, parts of the Java class library requires certain OS features that won't be available, but then use of those classes could be avoided.
So yes, it can be done. But it's not straightforward, and it's unclear what you'd gain.
Of course another problem may be that Java won't let you access arbitrary memory locations, which would make a lot of hardware communication pretty tricky. But that could be worked around too, perhaps by calling into very simple C functions which simply return the relevant memory areas as arrays for Java to work on.
Writing Solaris Device Drivers in Java covers a A RAM disk device written in Java.
Another one for Linux. Goes more in depth on why you might want a DD in Java as well (since some people were wondering by the looks of the other posts and comments)
A device driver could be a lot of things
I actually write device drivers in java for a living: drivers for industrial devices, such as scales or weighing devices, packaging machines, barcode scanners, weighing bridges, bag and box printers, ... Java is a really good choice here.
Industrial devices are very different from your home/office devices (e.g. scanners, printers). Especially in manufacturing (e.g. food), companies opt more and more for a centralized server which runs an MES application (e.g. developed in Java) The MES server needs to interface with the devices of the production line, but also contains business logic. Java is a language that can do both.
Where your home/office devices are often built-in to your computer or connected with an USB cable, these industrial devices usually use Ethernet or RS232 connectors. So, in essence, pretty much every language could do the job.
There is not much standardisation in this area yet. Most vendors prefer to create their own protocol for their devices. After all they are hardware builders, not software geniuses. The result is that there is a high diversity of protocols. Some vendors prefer simple plain-text protocols, but others prefer complex binary protocols with CRC codes, framing, ... Sometimes they like to stack multiple protocols (e.g. a vendor specific handshaking algorithm on top of an OPC layer). A strong OOP language has a lot of advantages here.
E.g. I've seen java print at a continuous speed of 100ms/cycle. This includes generating a unique label, sending it to the printer, receiving a confirmation, printing it on paper and applying it to the product using air pressure.
In summary, the power of java:
It is useful for both business logic as complex interfacing.
It is just as reliable in communication with sockets as C.
Some drivers can benifit from Java's OOP power.
Java is fast enough.
It's not impossible, but possibly hard and possibly makes not much sense.
Possible is it, because Java is a normal programming language, as long as you have some way to access the data, it's no problem. Normally in a modern OS the kernel has a layer to allow raw access to hardware in some way. Also already exist drivers in userspace, at least the userspace-part should be no problem to implement in Java.
It makes possibly not too much sense, because the kernel has to start a JVM to execute the driver. Also JVM-implementations normally eat up much memory.
You could also use Java-code compiled to be executed natively on the platform (not with the help of a JVM). This is usually not that efficient, but it could be suitable for a device-driver.
The question is, does it make sense to implement the driver in Java? Or stated in another way: What is the benefit you hope for, if you use Java for implementing the driver instead of another alternative? If you can answer this question, you should find a way to make it possible.
At the end the hint to JNode, a project that tries to implement a complete OS purely based on Java.
You have a too narrow view of device drivers.
I have written such device drivers on top of MOST in an automotive application. A more widespread use might be drivers for USB devices if Java ever gets a decent USB library.
In these cases there is a generic low-level protocol which is handled in native code, and the Java driver handles the device specifics (data formats, state machines, ...).
For the motivation, please remember that there is plenty of fast languages which are better than C for programming; they might not be as fast as C, but they are safe languages: if you make a mistake you don't get undefined behavior. And "undefined behavior" includes executing arbitrary code supplied by some attacker which formats your HD.
Many functional languages are usually compiled to native code.
Device drivers contain the most bugs in an OS kernel - I know that for Linux (Linus Torvalds and others keep saying so) and I heard that for Windows. While for a disk or Ethernet driver you need top-notch performance, and while in Linux drivers today are the bottleneck for 10G Ethernet or SSD disks, most drivers don't need that much speed - all computers wait at the same speed.
That's why there are various projects to allow writing drivers which run outside of the kernel, even if that causes a slowdown; when you can do that, you can use whatever language you want; you will just then need Java bindings for the hardware control library you use - if you were writing the driver in C, you would still have a library with C bindings.
For drivers in kernel mode proper, there are two problems that I've not yet seen mentioned:
Garbage Collection, and that's a tough requirement. You need to write an in-kernel Garbage Collector; some GC algorithms rely on Virtual Memory, and you cannot use them. Moreover, you probably need to scan the whole OS memory to find roots for the GC. Finally, I would only trust an algorithm guaranteeing (soft) real-time GC, which would make the overhead even bigger.
Reading the paper which was mentioned about Java Device Drivers on top of Linux, they just give up, and require programmers to manually free memory. They try to argue that this will not compromise safety, but I don't think their argument is convincing - it's not even clear whether they understand that Garbage Collection is needed for a safe language.
Reflection and class loading. A full Java implementation, even when running native code, needs to be able to load new code. This is a library you can avoid, but if you have an interpreter or JIT compiler in kernel (and there's no real reason that makes it technically impossible).
Performance. The paper about a JVM on Linux is very bad, and their performance numbers are not convincing - indeed, they test a USB 1.1 network driver, and then show that performance is not so bad! However, given enough effort something better can surely be done.
Two last things:
I'd like to mention Singularity, which is a complete OS written in a C# variant, with just a Hardware Abstraction Layer in a native language.
About picoJava, it's a bad idea to use it unless your system is a really memory constrained one, like a smart card. Cliff Click already explained why: it gives better performance to write a good JIT, and nowadays even smartphones can support that.
Have you perhaps heard a reference to the JDDK?
Writing a device driver 100% in Java is not possible without native code to provide the interaction between (1) the OS-specific driver entry points and conventions, and (2) the JVM instance. The JVM instance could be started "in-process" (and "in-process" may have different meanings depending on the OS and on whether the driver is a kernel-mode or user-mode driver), or as a separate user-land process with which a thin, native driver adaptation layer can communicate and onto which the said driver adaptation layer can offload actual user-land work.
It is possible to compile java code to hardware native (i.e. not JVM bytecode) instructions. See for instance GCJ. With this in hand, you're a lot closer to being able to compile device drivers than you were before.
I don't know how practical it is, though.
Possible?
Yes but only in special circumstances. Because you can write an operating system in Java and C#, and then, should be able to write device drivers for it. The memory hit to these drivers and operating systems would be substantial.
Probable?
Not likely. Atleast not in the world of Windows or MacOS or even Linux... At least not anytime soon. Because languages like C# and Java depend on the CLR and JVM. The way these languages work means that they cannot effectively be loaded into ring0.
Also, the performance hit would be rather large if managed languages were employed in device drivers.
Device drivers have to be written in a language which can execute in the kernel, either compiled into it, or loaded as a module at runtime. This usually precludes writing device drivers in Java, but I suppose you theoretically could implement a JVM inside a device driver and let it execute Java code. Not that any sane person would want to do that.
On Linux there are several user-land (i.e. non-kernel) implementations of filesystems which uses a common abstraction layer called (fuse) which allows user-land programs to implement things which are typically done in the kernel.
The Windows Driver Foundation (WDF) is a Microsoft API that does allow both User and Kernel mode device drivers to be written. This is being done today, and it is now compatible with w2k and later (used to not have w2k as a supported target). There is no reason that JNI calls can't be made to do some work in the JRE . . . ( assuming that JNI is still the way to call Java from C/C++ . . . my knowledge is dated in that arena). This could be an interesting way to have high level algorithms directly munch on data from a USB pipe for something to that effect . . . cool stuff!
PCIe user space device drivers can be written in Pure Java. See JVerbs for details about memory-based direct hardware access, in the context of OFED. This is a technique that can be used to create very high performance systems.
You can examine the PCI bus to determine the memory regions for a given device, what ports it has, etc. The memory regions can be mapped into the JVM's process.
Of course, you're responsible for implementing everything yourself.
I didn't say easy. I said possible. ;)
See also Device Drivers in User Space, which discusses using the UIO framework to build a user space driver.
First of all, note that I'm not an expert on device drivers (though I wrote a few myself back in the day), much less an expert on Java.
Let's leave the fact that writing device drivers in a high-level language is not a good idea (for performance and possibly many other reasons) aside for a moment, and answer your question.
You can write device drivers in almost any language, at least in theory.
However, most device drivers need to do plenty of low-level stuff like handling interrupts and communicating with the OS using the OS APIs and system calls, which I believe you can't do in Java.
But, if your device communicates using, say, a serial port or USB, and if the OS doesn't necessarily need to be aware of the device (only your application will access the device*), then you can write the driver in any language that provides the necessary means to access the device.
So for example you probably can't write a SCSI card driver in Java, but you can write a driver for a proprietary control device, USB lava lamp, license dongle, etc.
* The obvious question here is, of course, does that count as a driver?
Related
All the operating systems till date have been written in C/C++ while there is none in Java. There are tonnes of Java applications but not an OS. Why?
Because we have operating systems already, mainly. Java isn't designed to run on bare metal, but that's not as big of a hurdle as it might seem at first. As C compilers provide intrinsic functions that compile to specific instructions, a Java compiler (or JIT, the distinction isn't meaningful in this context) could do the same thing. Handling the interaction of GC and the memory manager would be somewhat tricky also. But it could be done. The result is a kernel that's 95% Java and ready to run jars. What's next?
Now it's time to write an operating system. Device drivers, a filesystem, a network stack, all the other components that make it possible to do things with a computer. The Java standard library normally leans heavily on system calls to do the heavy lifting, both because it has to and because running a computer is a pain in the ass. Writing a file, for example, involves the following layers (at least, I'm not an OS guy so I've surely missed stuff):
The filesystem, which has to find space for the file, update its directory structure, handle journaling, and finally decide what disk blocks need to be written and in what order.
The block layer, which has to schedule concurrent writes and reads to maximize throughput while maximizing fairness.
The device driver, which has to keep the device happy and poke it in the right places to make things happen. And of course every device is broken in its own special way, requiring its own driver.
And all this has to work fine and remain performant with a dozen threads accessing the disk, because a disk is essentially an enormous pile of shared mutable state.
At the end, you've got Linux, except it doesn't work as well because it doesn't have near as much effort invested into functionality and performance, and it only runs Java. Possibly you gain performance from having a single address space and no kernel/userspace distinction, but the gain isn't worth the effort involved.
There is one place where a language-specific OS makes sense: VMs. Let the underlying OS handle the hard parts of running a computer, and the tenant OS handles turning a VM into an execution environment. BareMetal and MirageOS follow this model. Why would you bother doing this instead of using Docker? That's a good question.
Indeed there is a JavaOS http://en.wikipedia.org/wiki/JavaOS
And here is discuss about why there is not many OS written in java Is it possible to make an operating system using java?
In short, Java need to run on JVM. JVM need to run on an OS. writing an OS using Java is not a good choice.
OS needs to deal with hardware which is not doable using java (except using JNI). And that is because JVM only provided limited commands which can be used in Java. These command including add, call a method and so on. But deal with hardware need command to operate reg, memory, CPU, hardware drivers directly. These are not supported directly in JVM so JNI is needed. That is back to the start - it is still needed to write an OS using C/assembly.
Hope this helps.
One of the main benefits of using Java is that abstracts away a lot of low level details that you usually don't really need to care about. It's those details which are required when you build an OS. So while you could work around this to write an OS in Java, it would have a lot of limitations, and you'd spend a lot of time fighting with the language and its initial design principles.
For operating systems you need to work really low-level. And that is a pain in Java. You do need e.g. unsigned data types, and Java only has signed data types. You need struct objects that have exactly the memory alignment the driver expects (and no object header like Java adds to every object).
Even key components of Java itself are no longer written in Java.
And this is -by no means- a temporary thing. More and more does get rewritten in native code to get better performance. The HotSpot VM adds "intrinsics" for performance critical native code, and there is work underway to reduce the overall cost of native calls.
For example JavaFX: The reason why it is much faster than AWT/Swing ever were is because it contains/uses a huge amount of native code. It relies on native code for rendering, and e.g. if you add the "webview" browser component it is actually using the webkit C library to provide the browser.
There is a number of things Java does really well. It is a nicely structured language with a fantastic toolchain. Python is much more compact to write, but its toolchain is a mess, e.g. refactoring tools are disappointing. And where Java shines is at optimizing polymorphism at run-time. Where C++ compilers would need to do expensive virtual calls - because at compile time it is not known which implementation will be used - there Hotspot can aggressively inline code to get better performance. But for operating systems, you do not need this much. You can afford to manually optimize call sites and inlining.
This answer does not mean to be exhaustive in any way, but I'd like to share my thoughts on the (very vast) topic.
Although it is theoretically possible to write some OS in pure java, there are practical matters that make this task really difficult. The main problem is that there is no (currently up to date and reliable) java compiler able to compile java to byte code. So there is no existing tool to make writing a whole OS from the ground up feasible in java, at least as far as my knowledge goes.
Java was designed to run in some implementation of the java virtual machine. There exist implementations for Windows, Mac, Linux, Android, etc. The design of the language is strongly based on the assumption that the JVM exists and will do some magic for you at runtime (think garbage collection, JIT compiler, reflection, etc.). This is most likely part of the reason why such a compiler does not exist: where would all these functionality go? Compiled down to byte code? It's possible but at this point I believe it would be difficult to do. Even Android, whose SDK is purely java based, runs Dalvik (a version of the JVM that supports a subset of the language) on a Linux Kernel.
I'm a front end developer that's looking to get into some other languages such as Java or C++. I have an idea for a program and was just looking for an answer to something. What I would like to do is build a program and boot directly to that program. For example I have an old computer and I wipe the hard drive clean. So they is nothing currently on it. Not even an OS. I want to build a program that I can install to the hard drive that will boot straight into the program once started. Would this be considered an OS?
No you don't. Unless you want to spend many years, writing drivers for your graphics card, harddisk controller, usb controller, dma controller and all the other hardware your computer have.
What you want is a minimal operation system, which include just the kernel, and a runtime library and which start your program and nothing else on startup. A minimal Linux such as linux from scratch or bsd would be a good starting point.
First of all you need to decide your your program needs what. I mean should operate in Protected mode or the routine you have is tiny, so it is enough to run before entering protected mode (i.e. in real mode).
Here you can do three things
Modify bootloader to jump the execution to your code . Then Your code can resume normal os initialization.
Modify your os kernel early initialization code So that it executes your code before entering protected mode
I think your code will not be harmed if a bit of os portion is running. So you can write your routine before full kernel initialization.
Now note that for the later two point you need to modify your kernel, which is not easy (not even always possible)
Now the problem in first approach: Nothing will be ready for you, not even a regular c library or divice drivers , so you have to write every raw bit of code by hand which is crude.
This is off course not possible in java. Because the jvm will not be ready for you.
Now practically: there are lot of tiny os available, use one of them and modify as per your need. use this link to get a complete list of what is available for you.
First, Java is right out. You cannot possibly do this in Java without enormous amounts of tool-building. Java is not suited for this task at all.
You can do it in C++ or C. The search terms you are looking for is operating system development. This would probably not technically be considered developing an Operating System since it wouldn't run other programs, but the information about how to get through the boot-up procedure and establish a minimal environment are going to be most easily found in the category of operating system development. Some reasonable starting resources for that can be found at the OS Dev Wiki.
Alternately, you could take an existing small open-source OS and modify what it does after the boot-up sequence completes. If your program is intending to do anything more than just use the keyboard and the screen in text mode, there need to be device drivers. Thus, depending on the project, changing an existing OS may be the easiest route because you won't need to write your own device drivers for any devices you want to use.
Java can't run without Environment. If you want to run you program on you machine without OS, Java is a wrong choice.
C++ program can run without OS, but it's difficult to write a bootable program in C++.
If you want to write your own bootable program, you should use assembly for boot and load function, with some knowledge to use hardware in low level.
You have to have an operating system, so your program would be the operating system (or you would have to use another one and write it for that). It's certainly possible in C++, but it's not really possible to write an operating system in java.
Unless you want write something in (for example) Open Firmware and Forth or say a ROM BASIC. You'll probably qualify as a boot loader. Your application may qualify as an operating system. In my opinion, and a modern context, it entirely depends on how much functionality it provides to hosted applications. I'm not sure that something like FreeDOS would be considered an operating system (no pre-emptive task scheduling or GUI for example) given modern computers (I don't care to argue the point either way).
I never did kernel programming. I am a good programmer in the Java language and frequently use it. Now i feel to do something interesting with kernels. A kernel resides between hardware and OS. It communicates with hardware using system calls. Every programming language require a compiler to compile the code written in high level language and then it generate low level code, which is generally assembly language code. Here comes my doubt, if we have kernel written in C, then should we have a C compiler installed on the machine? At the end, when kernel interacts with hardware it uses assembly language, can i create kernel in Java language? If yes, then what are the requirements for the same? Thank you.
A kernel resides between hardware and OS
Usually, the kernel is considered to be part of the operating system.
It communicates with hardware using system calls
System calls are the interface that is provided by the OS to user applications. The operating system communicates with the hardware through other mechanisms (for example interrupts or memory-mapped registers).
Every programming language require a compiler to compile the code written in high level language and then it generate low level code, which is generally assembly language code.
The compiler output is typically either native machine code or a language-specific bytecode (like in the case of Java). Sometimes, compilers also target another programming language such as C or Javascript (transpilation).
Here comes my doubt, if we have kernel written in C, then should we have a C compiler installed on the machine?
That's not necessary. The C compiler produces output that can execute directly on the hardware without interpretation.
At the end, when kernel interacts with hardware it uses assembly language
The CPU doesn't understand assembly. It understands machine code.
can i create kernel in Java language?
It has been done.
If yes, then what are the requirements for the same?
If you want to write a kernel in Java, then you either have to
compile your entire Java codebase to machine code
get yourself a CPU that can execute Java bytecode
find or build a Java VM and runtime that can run on bare metal and run your Java code in it (if you do it cleverly, you can write much of the runtime and maybe also parts of the VM in Java itself).
Now to the unspoken, almost rhethorical question:
Is this a good idea?
Probably not. Why? First of all, because it would take ages to set up. Second, because you couldn't just code the way you develop an average business application. You'd have to think about performance of very time-critical code (e.g. context switching, which often requires hand-tuned assembly to be fast enough), manual memory management (as in: your MRU might expect you to give it the physical address where the page table lies), system-/hardware-specific mechanisms (how to access a XYZ controller on this particular architecture?), ...
So you would lose many of the advantages that Java has over a low-level language like C in the first place.
Yes a kernel can be written in Java, see the JNode. It would have the advantage of having no problems with: dangling pointers, mix up of pointers and array addresses, unitialised data, and many more features of C.
This question already has answers here:
Why are drivers and firmwares almost always written in C or ASM and not C++?
(15 answers)
Closed 6 years ago.
As I know in order to write device drivers people usually use c++ or assembly? The choice of assembly is clear for me. But why c++? I guess it is possible to do in java (for example), or in other high level language as well. So why c++ is so common? Is there a specific feature of C++ programming language that is necessary for writing drivers?
I would say that C is much more commonly used as a language for device drivers (over C++).
There are a couple of reasons for this:
Most of the major operating systems are written in a combination of C and assembler and provide C language interfaces to the operating system.
Writing a device driver usually involves communicating with a device which involves direct memory manipulation of registers and the like. C and C++ allow you to do this natively.
Edit - Example of Direct Memory Manipulation:
Let me make up some fictitious device. Say it has a register, that when you write certain values onto it, makes a light on the device turn on. This register is 32 bits wide. The top 31 bits say how bright to make the light, the lowest bit turns it on and off.
When you plug the device into the computer, the operating system assigns that register a particular memory location, (let's say 0x00FF00FF00 on a 32 bit OS). In order to turn the light on, the device driver would do something like this:
int* lightRegister = 0x00FF00FF00; // You would normally ask the OS for this address, not hardcode it.
int brightnessValue = 256; // Only Even numbers, the low bit must be zero.
*lightRegister = brightnessValue | 1; //Turn it on.
*lightRegister = 0; // Turn it off.
Higher level languages like Java, don't generally let you write into some random memory location like this.
There is a good discussion on the pros and cons of using C++ for device drivers in Windows here:
C++ for Kernel Mode Drivers: Pros and Cons
A lot of the C++ vs. C discussion is "religious" and a question of personal preference. In general working at a lower level gives you better control over things like memory and performance but may also require more time duplicating facilities that may be available in higher level languages. Making those trade-offs and choosing the right tool for the right problem is part of what software development is about.
There is no fundamental reason why you couldn't use Java for writing a device driver. It's more of a question of run-time support for the language, the libraries and the kind of work you need to get done inside the driver (interrupt handlers, I/O etc.). Performance may also be an issue. You would need to develop a very large amount of infrastructure to do that. While Java doesn't offer low level facilities (such as pointers) these could be replaced with calls to a native function.
C++ can produce very efficient assembly code AND provide higher-level constructs such as templates, RAII and OOP. Assembly is much too slow to write by hand for modern programs and does't provide many things that modern programmers expect, like OOP, whereas Java would be MUCH too slow to write a device driver in.
I don't know why? and whether it's actually true, but I would guess, C++ combination of efficiency and high level of abstraction makes it a very good candidate for tasks that require performance and can benefit from high level abstractions.
Because C++ is the last well spread language (except C) which translates directly to machine code without too many complications.
If so, then how? I'm doing a team project for school. I thought that Java couldn't actually access hardware directly, since that would make it hard to be cross-platform. I need to know this, because after some quick Googling, I haven't found anything, and my team members(who want to do this and want to use Java) seem unsure of how to proceed- after apparently much more searching than I've done.
Your right in that you can't access hardware directly from Java (unless your calling on native code, but that's not what your after) since it runs in a sandboxed environment, namely the Java Virtual Machine (JVM).
However you can get some basic info from the JVM that it gathers from the underlying OS.
Take a look at using Java to get OS-level system information
What you are looking for is SIGAR API
Overview
The Sigar API provides a portable interface for gathering system information such as:
System memory, swap, cpu, load
average, uptime, logins
Per-process memory, cpu, credential
info, state, arguments, environment,
open files
File system detection and metrics
Network interface detection,
configuration info and metrics
TCP and UDP connection tables
Network route table
This information is available in most operating systems, but each OS has their own way(s) providing it.
SIGAR provides developers with one API to access this information regardless of the underlying platform.
The core API is implemented in pure C with bindings currently implemented for Java, Perl, Ruby, Python, Erlang, PHP and C#.
The amount of info you'll be able to get using native Java API's is pretty small, as your program generally only knows about the VM it's sitting on.
You can, however, call out to the command line, run native apps, and parse the results. It's not particularly good form in a Java app, however, as you lose the cross-platform benefits usually associated with the language.
You can get some basic information regarding the Processor/s using System.getEnv().
You can also use Runtime.getRuntime() - See the response of R. Kettelerij for details.
Another option is to use JMX. The MemoryMXBean for example provides some information regarding the RAM usage (heap and non-heap).
To anyone looking for a library that is still being maintained as of 2023 check out OSHI