Related
For reasons that are beside the point, a company has bought an Exadata Eighth Rack. Some of the managers thought that this would improve performance of current applications. The problem is that hardly any application makes intensive database work (yes, this is a good moment for looking at facepalm animated gifs). So, at the moment, migrations have proven just little benefit.
The question is obvious. Most of the applications are written in Java, and some of them make intensive use of Solr and Cassandra. For what I know, Exadata is intended for storing data, while Exalogic can hold applications too. Anyway, I'm wondering if there is some way of taking advantage of mentioned infrastructure.
Replace Solr with Oracle Text.
Before I get down-voted: normally I would not recommend replacing existing code built with a popular, open-source program with a seldom-used, proprietary product. But if you want to use a lot of space and CPU on your database servers then Oracle Text can definitely help.
As more generic advice, the primary role of a database is not to store data. A file system can do that. Databases are built to join data. If an application is reading a large amount of data and doing ad hoc joins, those are the jobs you want to move to the database.
Exadata -> Oracle Database extreme performance.
Exalogic -> Fusion Middleware extreme performance. (Java goes here)
Your best move will be refactoring the application to put as much workload as possible on the DB (PL/SQL).
Another thing I could think of, but this would be a radical approach I have never really tried it myself (Yes I work with Exadatas too) maybe you can give it a shot and let us know here...
What about using all those GBs on the Exadata's RAM and start tuning your Java application's latency? I mean with that gruesome amount of Memory you can try and set a real nice amount of heap and avoid Garbage Collection induced latency. Please do let me know here what comes out if you actually try this.
Which protocol do the Java applications use to connect to Oracle?
If it's not IPC (inter process communication, aka BEQUEATH, aka shared memory), but maybe TCP and you have many fast & tiny roundtrips, than this would be your low-hanging fruit - eliminate the network stack.
edit: just realized that exadata cannot run java applications by default (only ODA does) - so it wouldn't be possible to make use of IPC. However, perhaps you're able to test the impact of IPC in one of your applications using the former infrastructure?
Exadata cannot host any customer application. You cannot install anything there. You only can host Oracle database on Exadata.
It means you can use database features like DBFS (file system over Oracle database), Java option (storing and executing java code in database). But you need to check what options you have license for. And internal JVM is used, which cannot be customized or upgraded.
Exadata is database appliance designed to work with large amount of differently accessed data in very effective and manageable way.
Will google Go use less resources than Python and Java on Appengine? Are the instance startup times for go faster than Java's and Python's startup times?
Is the go program uploaded as binaries or source code and if it is uploaded as source code is it then compiled once or at each instance startup?
In other words: Will I benefit from using Go in app engine from a cost perspective? (only taking to account the cost of the appengine resources not development time)
Will google Go use less resources than Python and Java on Appengine?
Are the instance startup times for go faster than Java's and Python's
startup times?
Yes, Go instances have a lower memory than Python and Java (< 10 MB).
Yes, Go instances start faster than Java and Python equivalent because the runtime only needs to read a single executable file for starting an application.
Also even if being atm single threaded, Go instances handle incoming request concurrently using goroutines, meaning that if 1 goroutine is waiting for I/O another one can process an incoming request.
Is the go program uploaded as binaries or source code and if it is
uploaded as source code is it then compiled once or at each instance
startup?
Go program is uploaded as source code and compiled (once) to a binary when deploying a new version of your application using the SDK.
In other words: Will I benefit from using Go in app engine from a cost
perspective?
The Go runtime has definitely an edge when it comes to performance / price ratio, however it doesn't affect the pricing of other API quotas as described by Peter answer.
The cost of instances is only part of the cost of your app. I only use the Java runtime right now, so I don't know how much more or less efficient things would be with Python or Go, but I don't imagine it will be orders of magnitude different. I do know that instances are not the only cost you need to consider. Depending on what your app does, you may find API or storage costs are more significant than any minor differences between runtimes. All of the API costs will be the same with whatever runtime you use.
Language "might" affect these costs:
On-demand Frontend Instances
Reserved Frontend Instances
Backed Instances
Language Independent Costs:
High Replication Datastore (per gig stored)
Outgoing Bandwidth (per gig)
Datastore API (per ops)
Blobstore API storge (per gig)
Email API (per email)
XMPP API (per stanza)
Channel API (per channel)
The question is mostly irrelevant.
The minimum memory footprint for a Go app is less than a Python app which is less than a Java app. They all cost the same per-instance, so unless your application performs better with extra heap space, this issue is irrelevant.
Go startup time is less than Python startup time which is less than Java startup time. Unless your application has a particular reason to churn through lots of instance startup/shutdown cycles, this is irrelevant from a cost perspective. On the other hand, if you have an app that is exceptionally bursty in very short time periods, the startup time may be an advantage.
As mentioned by other answers, many costs are identical among all platforms - in particular, datastore operations. To the extent that Go vs Python vs Java will have an effect on the instance-hours bill, it is related to:
Does your app generate a lot of garbage? For many applications, the biggest computational cost is the garbage collector. Java has by far the most mature GC and basic operations like serialization are dramatically faster than with Python. Go's garbage collector seems to be an ongoing subject of development, but from cursory web searches, doesn't seem to be a matter of pride (yet).
Is your app computationally intensive? Java (JIT-compiled) and Go are probably better than Python for mathematical operations.
All three languages have their virtues and curses. For the most part, you're better off letting other issues dominate - which language do you enjoy working with most?
It's probably more about how you allocate the resources than your language choice. I read that GAE was built the be language-agnostic so there is probably no builtin advantage for any language, but you can get an advantage from choosing the language you are comfortable and motivated with. I use python and what made my deployment much more cost-effective was the upgrade to python 2.7 and you can only make that upgrade if you use the correct subset of 2.6, which is good. So if you choose a language you're comfortable with, it's likely that you will gain an advantage from your ability using the language rather than the combo language + environment itself.
In short, I'd recommend python but that's the only app engine language I tried and that's my choice even though I know Java rather well the code for a project will be much more compact using my favorite language python.
My apps are small to medium sized and they cost like nothing:
I haven't used Go, but I would strongly suspect it would load and execute instances much faster, and use less memory purely because it is compiled. Anecdotally from the group, I believe that Python is more responsive than Java, at least in instance startup time.
Instance load/startup times are important because when your instance is hit by more requests than it can handle, it spins up another instance. This makes that request take much longer, possibly giving the impression that the site is generally slow. Both Java and Python have to startup their virtual machine/interpreter, so I would expect Go to be an order of magnitude faster here.
There is one other issue - now Python2.7 is available, Go is the only option that is single-threaded (ironically, given that Go is designed as a modern multi-process language). So although Go requests should be handled faster, an instance can only handle requests serially. I'd be very surprised if this limitation last long, though.
If so, then how? I'm doing a team project for school. I thought that Java couldn't actually access hardware directly, since that would make it hard to be cross-platform. I need to know this, because after some quick Googling, I haven't found anything, and my team members(who want to do this and want to use Java) seem unsure of how to proceed- after apparently much more searching than I've done.
Your right in that you can't access hardware directly from Java (unless your calling on native code, but that's not what your after) since it runs in a sandboxed environment, namely the Java Virtual Machine (JVM).
However you can get some basic info from the JVM that it gathers from the underlying OS.
Take a look at using Java to get OS-level system information
What you are looking for is SIGAR API
Overview
The Sigar API provides a portable interface for gathering system information such as:
System memory, swap, cpu, load
average, uptime, logins
Per-process memory, cpu, credential
info, state, arguments, environment,
open files
File system detection and metrics
Network interface detection,
configuration info and metrics
TCP and UDP connection tables
Network route table
This information is available in most operating systems, but each OS has their own way(s) providing it.
SIGAR provides developers with one API to access this information regardless of the underlying platform.
The core API is implemented in pure C with bindings currently implemented for Java, Perl, Ruby, Python, Erlang, PHP and C#.
The amount of info you'll be able to get using native Java API's is pretty small, as your program generally only knows about the VM it's sitting on.
You can, however, call out to the command line, run native apps, and parse the results. It's not particularly good form in a Java app, however, as you lose the cross-platform benefits usually associated with the language.
You can get some basic information regarding the Processor/s using System.getEnv().
You can also use Runtime.getRuntime() - See the response of R. Kettelerij for details.
Another option is to use JMX. The MemoryMXBean for example provides some information regarding the RAM usage (heap and non-heap).
To anyone looking for a library that is still being maintained as of 2023 check out OSHI
How can I get the CPU utilization of client's machine.
I have created a Java web application and where I need to get the CPU utilization of client's machine.
Can it be done using either Javascript or Java?
Whatever you use, Java or Javascript, you can't get CPU utilization of the client's machine. That will be a huge security risk. The only way to do it will be to install a plug-in. This wouldn't help too much however, as reading the CPU utilization is platform specific.
#Daniel provides a link to a method that (on a good day) would give an approximate measure of CPU usage.
#kgiannakakis says that this would be a serious security risk.
But I wonder. How could an attacker exploit CPU usage levels to compromise the user machine's security, or steal sensitive information about the user? Off the top of my head, I cannot think of anything realistic.
The only possible risk I can think of would be if the attacker had already installed/launched some spy software somewhere else on the user's machine. The ability to measure CPU usage could be used to implement a "covert channel" to get information from the spy s/w to the outside world. But this is only of theoretical interest (unless you are trying to implement A1 / A2 level security).
It may look like a security risk, but there is a technique to get an estimate of the CPU usage with JavaScript inside the browser. You may want to check this article:
Ajaxian - JPU: JavaScript CPU Monitor (Demo here)
As kgiannakakis noted in a comment below, this calculation is based on the delay of setInterval() calls.
I remember seeing a similar implementation on Mindmeister.com (showing the CPU usage while editing a mind map.)
There is simple example of a CPU usage monitor demonstrates how to use XML DOM capabilities of client JavaScript in a BSP application to dynamically request data from server and adjust corresponding elements of the screen without reloading the whole page.
This article might be helpful.
jni concept for cpu utilization
http://www.javaworld.com/javaworld/javaqa/2002-11/01-qa-1108-cpu.html?page=1
try this link
If its java look at OperatingSystemMXBean using the please be aware this is Java 1.6 only.
OperatingSystemMXBean operatingSystemInfo =
(com.sun.management.OperatingSystemMXBean)ManagementFactory.getOperatingSystemMXBean();
I am not sure how well this works in an applet.
I once looked into creating a Web Start application that could be launched by a user to "scan" their system hardware. Luckily, the targeted systems were Windows boxes, so I used the Jacob package (Google or search this site for info) to do a WMI query for things like serial numbers, etc.
I created a proof of concept Swing client app, but never got around to the Web Start portion.
Introduction
I heard something about writing device drivers in Java (heard as in "with my ears", not from the internet) and was wondering... I always thought device drivers operated on an operating system level and thus must be written in the same language as the OS (thus mostly C I suppose)
Questions
Am I generally wrong with this
assumption? (it seems so)
How can a driver in an "alien"
language be used in the OS?
What are the requirements (from a
programming language point of view)
for a device driver anyway?
Thanks for reading
There are a couple of ways this can be done.
First, code running at "OS level" does not need to be written in the same language as the OS. It merely has to be able to be linked together with OS code. Virtually all languages can interoperate with C, which is really all that's needed.
So language-wise, there is technically no problem. Java functions can call C functions, and C functions can call Java functions. And if the OS isn't written in C (let's say, for the sake of argument that it's written in C++), then the OS C++ code can call into some intermediate C code, which forwards to your Java, and vice versa. C is pretty much a lingua franca of programming.
Once a program has been compiled (to native code), its source language is no longer relevant. Assembler looks much the same regardless of which language the source code was written in before compilation. As long as you use the same calling convention as the OS, it's no problem.
A bigger problem is runtime support. Not a lot of software services are available in the OS. There usually is no Java virtual machine, for example. (There is no reason why there technically couldn't be, but usually, but usually, it's safe to assume that it's not present).
Unfortunately, in its "default" representation, as Java bytecode, a Java program requires a lot of infrastructure. It needs the Java VM to interpret and JIT the bytecode, and it needs the class library and so on.
But there are two ways around this:
Support Java in the kernel. This would be an unusual step, but it could be done.
Or compile your Java source code to a native format. A Java program doesn't have to be compiled to Java bytecode. You could compile it to x86 assembler. The same goes for whatever class libraries you use. Those too could be compiled all the way to assembler. Of course, parts of the Java class library requires certain OS features that won't be available, but then use of those classes could be avoided.
So yes, it can be done. But it's not straightforward, and it's unclear what you'd gain.
Of course another problem may be that Java won't let you access arbitrary memory locations, which would make a lot of hardware communication pretty tricky. But that could be worked around too, perhaps by calling into very simple C functions which simply return the relevant memory areas as arrays for Java to work on.
Writing Solaris Device Drivers in Java covers a A RAM disk device written in Java.
Another one for Linux. Goes more in depth on why you might want a DD in Java as well (since some people were wondering by the looks of the other posts and comments)
A device driver could be a lot of things
I actually write device drivers in java for a living: drivers for industrial devices, such as scales or weighing devices, packaging machines, barcode scanners, weighing bridges, bag and box printers, ... Java is a really good choice here.
Industrial devices are very different from your home/office devices (e.g. scanners, printers). Especially in manufacturing (e.g. food), companies opt more and more for a centralized server which runs an MES application (e.g. developed in Java) The MES server needs to interface with the devices of the production line, but also contains business logic. Java is a language that can do both.
Where your home/office devices are often built-in to your computer or connected with an USB cable, these industrial devices usually use Ethernet or RS232 connectors. So, in essence, pretty much every language could do the job.
There is not much standardisation in this area yet. Most vendors prefer to create their own protocol for their devices. After all they are hardware builders, not software geniuses. The result is that there is a high diversity of protocols. Some vendors prefer simple plain-text protocols, but others prefer complex binary protocols with CRC codes, framing, ... Sometimes they like to stack multiple protocols (e.g. a vendor specific handshaking algorithm on top of an OPC layer). A strong OOP language has a lot of advantages here.
E.g. I've seen java print at a continuous speed of 100ms/cycle. This includes generating a unique label, sending it to the printer, receiving a confirmation, printing it on paper and applying it to the product using air pressure.
In summary, the power of java:
It is useful for both business logic as complex interfacing.
It is just as reliable in communication with sockets as C.
Some drivers can benifit from Java's OOP power.
Java is fast enough.
It's not impossible, but possibly hard and possibly makes not much sense.
Possible is it, because Java is a normal programming language, as long as you have some way to access the data, it's no problem. Normally in a modern OS the kernel has a layer to allow raw access to hardware in some way. Also already exist drivers in userspace, at least the userspace-part should be no problem to implement in Java.
It makes possibly not too much sense, because the kernel has to start a JVM to execute the driver. Also JVM-implementations normally eat up much memory.
You could also use Java-code compiled to be executed natively on the platform (not with the help of a JVM). This is usually not that efficient, but it could be suitable for a device-driver.
The question is, does it make sense to implement the driver in Java? Or stated in another way: What is the benefit you hope for, if you use Java for implementing the driver instead of another alternative? If you can answer this question, you should find a way to make it possible.
At the end the hint to JNode, a project that tries to implement a complete OS purely based on Java.
You have a too narrow view of device drivers.
I have written such device drivers on top of MOST in an automotive application. A more widespread use might be drivers for USB devices if Java ever gets a decent USB library.
In these cases there is a generic low-level protocol which is handled in native code, and the Java driver handles the device specifics (data formats, state machines, ...).
For the motivation, please remember that there is plenty of fast languages which are better than C for programming; they might not be as fast as C, but they are safe languages: if you make a mistake you don't get undefined behavior. And "undefined behavior" includes executing arbitrary code supplied by some attacker which formats your HD.
Many functional languages are usually compiled to native code.
Device drivers contain the most bugs in an OS kernel - I know that for Linux (Linus Torvalds and others keep saying so) and I heard that for Windows. While for a disk or Ethernet driver you need top-notch performance, and while in Linux drivers today are the bottleneck for 10G Ethernet or SSD disks, most drivers don't need that much speed - all computers wait at the same speed.
That's why there are various projects to allow writing drivers which run outside of the kernel, even if that causes a slowdown; when you can do that, you can use whatever language you want; you will just then need Java bindings for the hardware control library you use - if you were writing the driver in C, you would still have a library with C bindings.
For drivers in kernel mode proper, there are two problems that I've not yet seen mentioned:
Garbage Collection, and that's a tough requirement. You need to write an in-kernel Garbage Collector; some GC algorithms rely on Virtual Memory, and you cannot use them. Moreover, you probably need to scan the whole OS memory to find roots for the GC. Finally, I would only trust an algorithm guaranteeing (soft) real-time GC, which would make the overhead even bigger.
Reading the paper which was mentioned about Java Device Drivers on top of Linux, they just give up, and require programmers to manually free memory. They try to argue that this will not compromise safety, but I don't think their argument is convincing - it's not even clear whether they understand that Garbage Collection is needed for a safe language.
Reflection and class loading. A full Java implementation, even when running native code, needs to be able to load new code. This is a library you can avoid, but if you have an interpreter or JIT compiler in kernel (and there's no real reason that makes it technically impossible).
Performance. The paper about a JVM on Linux is very bad, and their performance numbers are not convincing - indeed, they test a USB 1.1 network driver, and then show that performance is not so bad! However, given enough effort something better can surely be done.
Two last things:
I'd like to mention Singularity, which is a complete OS written in a C# variant, with just a Hardware Abstraction Layer in a native language.
About picoJava, it's a bad idea to use it unless your system is a really memory constrained one, like a smart card. Cliff Click already explained why: it gives better performance to write a good JIT, and nowadays even smartphones can support that.
Have you perhaps heard a reference to the JDDK?
Writing a device driver 100% in Java is not possible without native code to provide the interaction between (1) the OS-specific driver entry points and conventions, and (2) the JVM instance. The JVM instance could be started "in-process" (and "in-process" may have different meanings depending on the OS and on whether the driver is a kernel-mode or user-mode driver), or as a separate user-land process with which a thin, native driver adaptation layer can communicate and onto which the said driver adaptation layer can offload actual user-land work.
It is possible to compile java code to hardware native (i.e. not JVM bytecode) instructions. See for instance GCJ. With this in hand, you're a lot closer to being able to compile device drivers than you were before.
I don't know how practical it is, though.
Possible?
Yes but only in special circumstances. Because you can write an operating system in Java and C#, and then, should be able to write device drivers for it. The memory hit to these drivers and operating systems would be substantial.
Probable?
Not likely. Atleast not in the world of Windows or MacOS or even Linux... At least not anytime soon. Because languages like C# and Java depend on the CLR and JVM. The way these languages work means that they cannot effectively be loaded into ring0.
Also, the performance hit would be rather large if managed languages were employed in device drivers.
Device drivers have to be written in a language which can execute in the kernel, either compiled into it, or loaded as a module at runtime. This usually precludes writing device drivers in Java, but I suppose you theoretically could implement a JVM inside a device driver and let it execute Java code. Not that any sane person would want to do that.
On Linux there are several user-land (i.e. non-kernel) implementations of filesystems which uses a common abstraction layer called (fuse) which allows user-land programs to implement things which are typically done in the kernel.
The Windows Driver Foundation (WDF) is a Microsoft API that does allow both User and Kernel mode device drivers to be written. This is being done today, and it is now compatible with w2k and later (used to not have w2k as a supported target). There is no reason that JNI calls can't be made to do some work in the JRE . . . ( assuming that JNI is still the way to call Java from C/C++ . . . my knowledge is dated in that arena). This could be an interesting way to have high level algorithms directly munch on data from a USB pipe for something to that effect . . . cool stuff!
PCIe user space device drivers can be written in Pure Java. See JVerbs for details about memory-based direct hardware access, in the context of OFED. This is a technique that can be used to create very high performance systems.
You can examine the PCI bus to determine the memory regions for a given device, what ports it has, etc. The memory regions can be mapped into the JVM's process.
Of course, you're responsible for implementing everything yourself.
I didn't say easy. I said possible. ;)
See also Device Drivers in User Space, which discusses using the UIO framework to build a user space driver.
First of all, note that I'm not an expert on device drivers (though I wrote a few myself back in the day), much less an expert on Java.
Let's leave the fact that writing device drivers in a high-level language is not a good idea (for performance and possibly many other reasons) aside for a moment, and answer your question.
You can write device drivers in almost any language, at least in theory.
However, most device drivers need to do plenty of low-level stuff like handling interrupts and communicating with the OS using the OS APIs and system calls, which I believe you can't do in Java.
But, if your device communicates using, say, a serial port or USB, and if the OS doesn't necessarily need to be aware of the device (only your application will access the device*), then you can write the driver in any language that provides the necessary means to access the device.
So for example you probably can't write a SCSI card driver in Java, but you can write a driver for a proprietary control device, USB lava lamp, license dongle, etc.
* The obvious question here is, of course, does that count as a driver?