Develop Reference

Languages that compile to Java Bytecode and can run on the JVM

I am an embedded programmer and working with an embedded JVM.
This enables running Java files on constrained devices.
These Java files are first compiled to bytecode into .class files which are then further optimized and uploaded to the device which has a micro JVM to run the optimized bytecode.
The micro JVM does not support all features, e.g., no reflection.
The main benefit is obvious: this allows programming in Java for constrained devices.
However, I was thinking that plenty of languages compile to bytecode, some are listed https://en.wikipedia.org/wiki/Java_bytecode.
So in theory these languages could also be used to program.
I'd like to obtain a list of common languages that compile down to bytecode and was wondering if you could help.
For example, Python has special implementations that reduce to Java Bytecode, if I'm not mistaken, and stuff like C to Java virtual machine compilers also exist.
So what languages would you think are logical to try and run on the devices? Any pointers on how to or similar experiences?
Also, I'm not clear what the difference is from reading Wikipedia between (Python) bytecode and Java bytecode, could anybody help explain that?

I'm agree with you about the overall idea and it would be nice to develop an embedded application using any language that can run on a JVM. But there are some practical issues that you should consider and I think that's why none of major vendors or open source initiatives have any active/serious project on this (as far as I know).
As you mentioned, a JVM implementations that can run on embedded devices, each have their own constraints and limitations. The most obvious one is that some packages may not be available at runtime. In order to apply such a constraint, you should either control it in the compile process or have a toolchain (sort of an SDK) which accepts the bytecode and checks such constraints.
This situation would be worth when a developer tries to use a third party library that is available for that specific language. It's not easy to guess if a library is safe for use against such a framework or not.
One great facility for developers would be to have their IDE check such issues on the fly (something like inspection in IntelliJ Idea). This makes it much more smoother to move toward using such a solution. But again the problem is that for each such languages there need to be a specific plugin compatible with their own syntax.
Also some of JVM languages that are actually implementation of an existing language (e.g. Jython or JRuby) are most of the time out of sync with the original language in case of supporting libraries/syntax changes of that language.
Anyway, I think in order to have a list of JVM languages you could easily find them on Wikipedia. Maybe you mean those who may worth considering in this regard by having a large community and tools support. In my opinion, you should focus on the following JVM languages as those who may worth to include in your final list:
Groovy
Kotlin
Scala
These are all pure JVM languages and are only using different syntax than Java.
Regarding the topic in general, I should say that when you search for embedded JVM implementations, you'll notice that it's also a fairly academic concepts and they're so many publications in this topics regarding the overall architecture, threading support, toolchain, error handling, memory management, etc. This means that you should have a very great experiences/background on both embedded systems and programming language concepts and implementation to be able to devise a proper architecture for such a platform.
About your last question regarding the difference between Python bytecode and Java bytecode (if I understand your question correctly), these are both conceptually the same but each has its own syntax and constraints. The bytecode concept refers to the piece of software that is the output of the compiler and is the platform independent representation of the original code and can be run/interpreted at runtime by another software component which is the virtual machine. In Java world, this software is called the Java Virtual Machine (JVM). I'm from the Java world so I don't know what it's called in Python vocabulary but it should be something similar (e.g. Python virtual machine).
I think due to the complexity of developing such a toolchain and also considering the unprecedented development of new IoT and SoC devices, many of them capable of running a more higher level operating systems, maybe in a long run most developers prefer to develop for a more high end devices using more high level APIs and SDKs. Who knows! In that case, we would have a same situation that we're in today for PCs. Languages like C and Assembly are still in use, but they have their own domain of applications. I mean throughout the time, layers of abstraction are being added on top of the previous one. The same thing can happen for embedded devices.

Using IPC to combine multiple languages

This is a general "noob" question about software design, so I apologise if it seems vague,
but I would really appreciate the advice. Note the system described below is purely an example, not a specific product I have in mind.
I often have a need to combine the functionality of several libraries or utilities, written in different languages. For example, if I want to code a high-performance audio processing application for the desktop, I will write it in C / C++. Then, I want to add a nice GUI. But I don't want to learn Qt. I like the look and feel of Adobe Air, and would like to use that. Later, I have a need to access a USB device. But the USB library I have only has an API in Java. How can I combine all these elements together, to take advantage of their relative strengths?
Clearly, I cannot compile these various elements into one single executable. So I need to create and run them seperately, and give them a means to communicate. The most common way to do this seems to be using IPC (Inter Process Communication), eg shared memory or sockets. I prefer the idea of sockets, as the programs could potentially run on seperate machines on a network.
So I decide to create a local client / server system, with a custom API, to allow these elements to communicate. For example, the Air application will receive a message from the C application, telling it to update it's UI. The USB application running in Java will use the sockets to stream audio from the USB hardware, into the C application.
My question : is using local sockets in this way a typical way to design such a system?
Will the performance be much worse than a truly native application (e.g. everything in Java or C, in a single executable) ? It also seems likely that such an approach would be prone to bugs, and difficult to maintain?
I frequently find myself coming up against the limits of existing software libraries (e.g. a graphics library with a pretty, flexible UI but no way to access low-level hardware, or a media library that can mix many audio streams, but has no support for video playback), and find it very frustrating. If anyone could advise the best way to combine arbitrary software libraries like this, I would really appreciate it.
Thanks in advance!

As you have correctly identified, combining libraries from different language or platforms is hard. There are several ways to do it, but none are ideal. Examples:
Native call interfaces (e.g. JNI / JNA) - very fast but tricky to make work correctly, and you have the problem that the data types used typically don't map cleanly across different platforms. Adds native dependencies.
Socket based IPC with text protocol (XML, JSON, etc) - works OK and common formats are likely to be supported at both ends, but adds a lot of overhead. Can be a pain to maintain custom schema mappings etc.
Socket based IPC with binary protocol (e.g. Google protocol buffers) - quite efficient, needs a lot of work to get a custom protocol working correctly on both ends
Communication via a 3rd system (e.g. database, message queue, filesystem) - lots of overhead, can get fragile, introduces a major dependency on a 3rd system.
In my experience, it usually isn't worth integrating a new language / platform just to get one specific library or feature. Take your user interface example - no matter how nice Adobe Air looks, I doubt it is worth trying to integrate it with an existing C/C++ application.
Even if you get it to work, it will significantly complicate the future maintenance and devlopment of your application. Builds become more complex. You need to maintain additional communication / "glue" code. You need to manage more dependencies. Your users will get hit by many more configuration issues. Testing becomes more difficult. It becomes harder to teach someone new about how the whole system works. You need to maintain your skills in more languages / frameworks etc.
I'd recommend the following strategy:
Pick a primary platform
Whenever you need a new library or feature, look for something on your primary platform first. Hopefully (usually?) there is something good available - but even if not then it might be worth coding something yourself if the requirement is quite small.
Only if there is no reasonable option on the primary platform, then you can start to think about integrating a new language/platform
In terms of primary platform, I'd normally suggest a JVM language like Java, Scala or Clojure since the JVM is very well engineered, offers great performance, is highly portable and has the largest / most cohesive library ecosystem (most of which is open source). The JVM is therefore probably the best "general purpose" choice unless you have some very specific requirement which is unlikely to be possible on the JVM, e.g.:
If you are doing lots of embedded / realtime / systems programming wthat requires hardware access you probably need to go for C/C++
If you are coding purely for web-based clients, you probably want to use JavaScript (if you are also writing code on the server side you can consider JavaScript code generation frameworks/libraries that can work on the JVM, e.g. Vaadin or ClojureScript)

the answer is pretty much depends on the technologies you're using and there is no silver-bullet solution for this.
In general, this solutions will fall into one of the following categories:
Some interprocess communication techniques
Integrations provided by the language/platform itself
Database/some common storage (even files :) )
Example of the first:
Sockets/pipes/whatever you operating system allows.
CORBA - allows to write distributed code in different languages.
Google protobuf - allows serialization/deserialization of data-objects and its language agnostic
For the second it really depends on language/ecosystem you're using.
Examples for java:
JNI - Java Native interface - allow to execute code (dlls/so) outside the JVM.
JCA - if you're in the enterprise environment - you can write the integration with the legacy systems in this.
For languages that are compiled into the native code its less tricky - you can write and compile some code, say in Pascal, and then use the DLL in C.
Sometimes when we're talking about Java there is a plethora of languages that have their own syntax and compiler, but their compiler compiles into java binary code that can be run inside the jvm. So if your solution is based on these languages the integration will be easier. Languages like Scala, Groovy, Closure, Jython and so on are falling into this category.
The last but not the least technology to be mentioned is Web Services. This is a very popular tool for integration of different system, although its more used in enterprise environment.
Basically its an abstraction over the sockets layer that allows to send data objects in XML/JSON format between the processes/servers. Both of XML and JSON are language agnostic, so its not an issue to create an XML in a program written in C++ and then consume it in JAVA.
Hope this helps

Java to EXE good or bad idea?

I have been wondering for a long time about converting Java projects to EXE.
The advantages relies in the faster deployment on Windows where the user simply double clicks the EXE and the application is launched where is with Java, he has to run certain commands.
But EXE is really not what the Java was intended for which is portability.
So what do you think, Java to EXE good or bad idea?
I found some interesting article here.
Update
Wow, so may contradicting views so far. I would like you guys to add the pros and cons of the JAVA to EXE.

Since my expertise is with Java Web Start, which is for launching desktop apps. with a GUI, please consider my advice to be targeted mostly at those types of apps.
Other people have commented on the OS specific nature of an EXE. I always have to wonder why people choose Java to develop Windows specific desktop apps., since the Visual Studio software for Windows would probably make both GUI development (no x-plat Java layouts to bend your head around) and deployment (just guessing it can produce an EXE) easier.
OTOH only you can say what is the best development tool/language for this use-case.
As to the potential disadvantages of creating an EXE, I note at the JavaFAQ on EXEs.
There are a number of good reasons not to package your application in an executable. Daniel Sjöblom notes:
It will probably not be any faster. Modern virtual machines don't interpret bytecodes, they actually employ a JIT compiler to produce native, compiled code. Check Sun's site for further information on JIT compilers.
Static compilation increases the size of your application multifold, since all of the libraries you are using need to be linked into the application.
You lose 'free' upgrades to your program. Anytime your user downloads a new faster virtual machine, your app gets a speed boost. If you are using an exe, you will not get this benefit.
Jon A. Cruz details some of the extra steps in the development process required to create an exe. He points out that developers making native exe's need to:
Validate the latest versions of the compilation product from the vendor. If critical bugs are found, it can't be used to build a shipping product until those are addressed. Work that needs to be done each time a revision comes out from the vendor.
Submit the software through a QA cycle. Once engineering thinks things are done, they need to be verified. So every shipping version and update of a product needs to go through complete testing cycles.
Further, since native compilation is per target platform, the QA cycle needs to be done completely for each target platform, which multiplies effort required.
Shelf space. Maybe not a big deal nowadays, but could be.
Then one needs to get all customers to upgrade to the proper version. Either have free updates (in which case the business needs to absorb the cost of producing updates) or alternatively needs to handle clients not all updating.
Jon notes futher: When you ship standard Java bytecodes, VM problems are the responsibility of the platform or VM vendor. However, when you ship compiled binaries, they become your responsibility (even if they're actually bugs in the vendor's compilation product).
...
Of course, my first choice for deploying Java rich client apps. is using Java Web Start. Putting some of the benefits/features of web-start in point form:
JWS provides many appealing features including, but not limited to:
splash screens
desktop integration
file associations
automatic update (including lazy downloads and programmatic control of updates)
partitioning of natives & other resource downloads by platform, architecture or Java version,
configuration of run-time environment (minimum J2SE version, run-time options, RAM etc.)
easy management of common resources using extensions
..
I decided to highlight auto-update since with the gradual shift from apps. delivered on disk to apps. delivered over a network, auto-update is becoming more common. JWS still provides the best update experience (very configurable, mostly transparent to the user) I've seen.
And of course, JWS works on OS' for desktop PCs for which Java is available.
Update
..does Java web apps require internet connection?
(Note that name is 'Java Web Start'.)
Sure it does. At least for the initial installation. Update checks can be specified to continue to launch the previously installed version of the app. if the user is not currently connected.
But then, (in my estimation) there are more machines (such as Netbooks) with no CD/DVD drive, than there are without internet connections. If you want to sell to the larger market, look to the network to deliver the app.

It depends on your needs. We had written a little barcode client scanner app here for our customer. They run it on two Windows-PCs. They are happy having their well-known exe-files. We coded it in Java and created an EXE-file for them.
Both parties are happy with it - so why not doing it?
When there are good reasons for it and nothing against it except dogmatism then it is ok in my opinion.

I am the author of the article you linked to - glad you've found it interesting!
As my article states, and as others have already pointed out in their answers, there are multiple ways to simplify deployment of Java apps - JNLP, EXE wrappers, installers bundling a private JRE, and so on. But true native compilation is the only option that also provides protection against Java decompilers - you simply do not ship the bytecodes.
Of course, that does not make reverse engineering of and tampering with your code impossible, just much more costly in terms of required skillset and time.
As far as application performance is concerned, native compilation can make a big difference if you target embedded systems. This also applies to memory and disk footprint, albeit to a smaller extent. On the desktop you would typically get better startup, but in most other scenarios and aspects the results would depend on your app.

If it has a good reason why not? Even Eclipse has an EXE on windows and (and platform dependent binaries for linux, mac, etc) Of course you loose portability but if that is not important then go ahead.
UPDATE
The question is what do you want to achieve by creating an exe :
Convenience : users on windows prefer to click on icons, this is especially true for the non geeks. On the other hand non geeks don't care what the link does internally if it starts up an exe or something else. You can have an application icon for non native Java applications too. The alternatives would be
Web start
Creating an installer package,e.g.: http://www.advancedinstaller.com/java.html This can also solve the problem of installing the JRE
Performance : If you compile your Java application into a native solution you may gain a bit on performance but it depends on what technology you use. For example Swing tends to be slow but compiling that to native is rather tricky. If you use SWT instead of Swing that is already using native components therefore no need for further native compilation. On the other hand recent JVMs perform very well and can compile java to native to further improve the performance bottlenecks. This is done silently on the background you dont need to worry about that.
Sum : in some cases it might be the only solution, but if you choose the right technologies there will be many Java based alternative solutions to reach the same goal.

The page behind the link in the question is written by a company that sells products that compile java to native code. I would not base a decision on that alone.
The question also says that the advantage of the exe a better user experience, because the user can just double click to launch the application.
That is possible with executable jar file. In fact, its actually quite easy with standard tools in the java runtime. You just have to add a manifest to a jar file, and specify the class with the main in it. You can also specify other jar files in the classpath relative to the location of the main jar file. You can also specify an image to use as a splash screen as a resource.
e.g.
Class-Path: lib/derby.jar lib/derbytools.jar lib/jcalendar-1.3.2.jar l
ib/joda-time-1.4.jar lib/log4j-1.2.14.jar lib/looks-2.2.1.jar lib/swi
ng-layout-1.0.jar
SplashScreen-Image: resources/splash.png
Main-Class: com.you.pkg.app.Main
The basic ant project in Netbeans will do all but the spash-screen for you if you use it. If your some reason you want to do all of that by hand, make sure you understand the format of the manifest file, its a bit finicky.

As Linux, mac., Solaris user I think this is bad idea.
If you want faster deploy on windows, just create installer.

Jar files provide many benefits including:
Compact: The whole application (i.e. all the class files) is stored in one archive file (which can incorporate image and sound files if required).
Ease of use: The application can be run by double-clicking.
Compression: The jar format allows you to compress your files for efficient storage.
Security: You can digitally sign the contents of a jar file. Users who recognise your signature can then optionally grant your software security privileges it wouldn't otherwise have.
I would not convert to exe.
Most Windows applications run from a .exe file (Word, Internet Explorer, FireFox, NetBeans, ...)
Java itself has no support for doing this as the executable file will then be platform dependent (i.e. it won’t run on Macs)
However, there are (free) applications that can do this for you.

Minecraft does it, so it must be a good idea!
All jokes aside, understand that it's not 'conversion' that you are looking for, but using a custom launcher. The article you linked does a nice job of explaining the different approaches and pros/cons of each. As a general idea, it requires the extra work of creating the launcher (and a different version for each different OS architecture), but it gives you a little more control (version checking is a nice feature, also you may update the application jar rather easily, like Minecraft does). Overall it's a good idea if you think it's worth the effort, and the (little) loss in portability.
Edit: the 'Custom Java Launchers And Wrappers' approach is the one that you should use if you don't need the really nifty extra features offered by those below it.

Depends on the user base. If they are tech-related in anyway then giving them a .jar file (which could be run by double click) is a good idea for mobility.
If your users are less techy but you still need it to run on multiple platforms then wrap it as exe for Windows and as .app for Mac.
Important: I would suggest making a script to wrap it into exe, so you run it each time you have a new version.

Android Source Not Compilable?

So I've been looking at code from various parts of the Android git, but when I try to load and compile many of them (camera, media player, etc) they reference classes and parts of the SDK that just aren't available even with my SDK version updated to 2.2. (for example the bitmap config option inNativeAlloc)
Is there a reason that these are not in the SDK given out to developers?

If you want to compile applications from the android open source project (ASOP), like the browser, camera, media player etc. You have to build on a Linux or Mac. You cannot compile the source on Windows.
See Get Android Source Code on info for building the source and the OS requirements requirements

Is there a reason that these are not
in the SDK given out to developers?
There are several possibilities, depending on what "these" are.
I am under the impression that most of the cases are that they simply have not gotten around to it yet. By "gotten around to it yet", I mean they haven't finalized the classes and method signatures they want to stick with for all time. The core Android team tries to minimize the amount of API breakage between releases, and generally succeeds (breakage tends to be more changed behaviors than changes in method signatures). Getting buy-in from all stakeholders for a given hunk of code takes a fair bit of effort. There is a ton of code in Android, some of which probably could go in the SDK, but only so much time gets allocated to that chore, compared to advancing Android capabilities.
In some cases, the answer is security -- that beyond not having a stable API, they do not know how they want to secure it yet, so they make it be available solely to firmware developers.
In a few cases, the answer may be performance -- they have their required use cases tuned appropriately, but there are others that outside developers might try that have not been optimized.
I am sure there are other possible reasons as well.

Emulator Framework

Are there any good open source frameworks for developing computer system emulators? I am particularly interested in something written in Python or Java that can reduce the effort involved in developing emulators for 8-bit processors (e.g. 6502, 6510, etc.).

Isn't the 6510 in the C64?
You might be able to make use of the java libraries that emulate c64 code
http://www.dreamfabric.com/c64/
http://www.jac64.com/jac64-java-based-c64-emulator.html
If you aren't afraid of C++ try this general purpose one:
http://cef.sourceforge.net/index.php

You may want to check out VICE, which can emulates a variety of Commodore 8-bit computers: "the C64, the C64DTV, the C128, the VIC20, almost all PET models, the PLUS4 and the CBM-II (aka C610)". That includes 6502, 6510 and 8502 processors. VICE is released under GPL and is written in C.

I've developed a complete emulator for the MIX machine (Knuth's imaginary computer from TAOCP) in Perl a few years ago. The source code is well documented and the simulator is runnable, so one can practice with examples. It wasn't too difficult and I don't recall needing any special framework. The machine's registers are just state variables in the simulator, and the rest is interpreting instructions and changing this internal state.
Do you have more specific questions? Perhaps it will then be easier to point you in the right direction.