matlab and enterprise applications - java

I have long background in enterprise engineering, abut as circumstance has it have have found my role changing. I have been tasked to lead a quantitative finance group, performing time series evaluation of proprietary data.
Our application stack (on the engineering side, which I have no influence on, but yet need to interface with) is JAVA (or SCALA) to Hibernate 3.x (annotations and xml) running on tomcat. Tons of experienced software guys...
I need data from them for two functions
research (i imagine pulling straight from the DB)
as parameters to any algorithms developed (described below)
My team is mostly folks with math and computational finance degrees, a couple w/ limited java experience (I have considerable .NET experience as well).
We are tasked to:
developed (multiple) algorithms that generate discreet trading signals (events) out of our underlying data
apply those algorithms to events coming from our web applications in real time
raise any trading signals (events) back to the application stack as they occur
a. display events visually in the application
b. send events to clients over the internet (somehow)
The best case is that any tool (MATLAB) used for the purpose of algorithm research and development, will also be used in the production environment - and be completely integrated to our production systems (as a listener to events, and then again as a source of events feeding back in).
The worst case is that any algorithm we develop needs to be reimplemented in the JAVA/SCALA space for integration.
My questions are
is matlab integration with java sufficient for this? They are not using an application server (like JBOSS), so i guess each tomcat machine is logically and physically its own JVM instance. So I don't see any JVM constraints (as in MATLAB owning its own instance) as a major obstacle
has anybody interfaced matlab to a database over Hibernate?
does .NET a better choice for interfacing with matlab? If so which features does it offer that java integration does not?
what capabilities are there in Matlab to "compile" your work into modules, and add to standard unit testing and automated build processes (ie HUDSON)?
Thanks

MATLAB's Java integration is sufficient for your aims. There is no issue in using
Java classes from the MATLAB JVM interacting with JBOSS as well.
Yes through JBOSS.
Never touched .NET, but you won't get the seamless support as seen for Java. Using Java you may use MATLAB as Java scripting engine, similar to projects like Groovy, or use instances of proxy classes using api calls.
Use MATLAB Builder JA in order to generate Java classes from your MATLAB code. The compiled code may be tested with any black box testing tool.

Regarding #4: For testing inside the MATLAB environment, I recommend Steve Eddins' test framework: http://www.mathworks.com/matlabcentral/fileexchange/22846

Related

Websockets Server in Java (or PHP)

I could need your experience according server implementations for the currently evolving Websockets in HTML5. We want to use this technology for a project using common webbrowsers on the client site. What we now need is a webserver that feeds our requirements. These are:
written in Java or PHP since these are the languages which currently are used and mastered by all the programmers in the project. Since it seems that there are no promising PHP implementations yet, I tend to concentrate on the Java site.
as fast as possible. We don't expect too heavy usage in production but we need to be able to synch two browsers on independent host in nearly real-time.
easy expandable. We will need to implement our own protocol on top of the websockets so we would like the most possible flexibility.
easy to use, well documented and actively developed. Since we have to come to a first running implementation rather quickly and since I expect some changes in the developing people during the time, it would be great if it's not to hard to get into the server programming.
I already had a look on some servers/frameworks which look promising, e.g. Jetty, Netty and jWebSocket. I also found some older discussions of this topic. However, I would be glad about some current experiences or recommendations of those of you who already have used Java (or PHP) to work with Websockets. Is there some implementation which is especially well-fitting or one that we absolutely shouldn't use?
Edit:
For clarity: When I speak of implementing our own protocol on top of Websockets I mean having some kind of plugin mechanism so that we can use a Websocket to send XML-messages to the server and then use a plugin to interpret them into some server actions as we like. I guess every server implementation also supports this but it would be good if it was not too difficult to write such plugins.
I don't have any experience with PHP so I'll give you my opinion for the Java (and python side).
For Java:
You have your very common and safe choice of Tomcat. It's one of the most actively developed servers, its open source, lots of support/help from its very large community, from what I've read (but not tested) it can be fairly scalable and it also has a very mature console to monitor different metrics for your website.
You also have other choices such as Caucho's Resin, who have a very small yet smart team who actively develop their webserver. Their new heartbeat service allows a good degree of scalability (again from what I've read).
And then you also have Google's appengine which is all about scalability. Their API (for both, the server and client side) has started to focus on realtime web applications so it's a bit easier to for creating high performance web applications without having to worry about the server related optimization that you often have to make with other servers. They have APIs for creating "channels" between the server and client side for very realtime applications. The AppEngine team also has a new MapReduce API for analyzing data quickly/efficiently. For these reasons and the personal opinion that they are pioneering new web technologies/standards, I'm personally been leaning towards AppEngine. It's free if you're keeping cpu/data usuage low but if you need full scale deployment it does start to get pricey so do research. If your project is more for fun/school, its free quotas are enough and all its development tools/APIs are free so it's perfect.
For Python:
I'm more Java than Python, but Python is definitely a great language and very very very easy to learn. And the reason I'm recommending Python for consideration here is because AppEngine also allows developers the choice of Python. The author/creater of the Python language is now on the AppEngine team so, even though the Python and Java APIs are pretty much exactly the same, they sometimes have new/better APIs for Python. For example, I think the datastore API offered with Python is a bit easier and optimized for the Google datastore. Also, since you'd be using python, you can use Django with AppEngine.
The second choice that I've recently become aware of is the tornado web server which was developed by FriendFeed and then bought and used by Facebook. It's also actively developed. But the reason I bring this up is that Tornado's focus, much like AppEngine's, is high web application performance and scalability. I have no experience with it but I've become interested in its technology simply because no one can contest that Facebook requires a very high performance/scalable web server.
Some last thoughts:
In general you'll find that it isn't very difficult to build your own protocol on top of the APIs provided from most web servers (even Asp.NET MVC). But from what I've been able to test around with AppEngine, it really focuses on being flexible on what kind of protocol is build on top of it. To me, its pretty much just feels like a RPC server which which works on top of HTTP. And that becomes even more apparent if you use Python just because of some of the language constructs.

advantages of play framework for people coming from php / ruby / python

One of the strongest selling points of play framework is it's development cycle, just fix the code, go back to your browse, hit reload, and go one, all without the write, compile, deploy, hassle typical of j2ee web development.
Nevertheless, for web developers coming from php, ruby or python this is hardly surprissing, moreover it's considered the natural way to develop web apps.
Moreover, many of the advantages of play, (efficient template system, resolve errors quickly, stateless, fun & productive) are taken for granted in those frameworks...
So I'd like to know what do you think are the main advatages that play framework brings to web developer coming from dynamic languages.
From my point of view these are the main advantages
it's java, so if you don't feel confortable with java, and you are not willing to learn it, that might not be an advantage. On the other hand, if you HAVE to use java, coming from php or rails you will feel right at home with play
it's static (except for templates and routes, that's about to change on the next version), so you have compiler checks and safe refactoring.
tool support: IDE support (eclipse, intellij, netbeans)
endless collection of libraries to do everything you want, well, it's just java
runs on the jvm, which has years of experience and optimizations
it's compiled, so you may expect better performance
it's really fullstack: it comes with everything you need to develop a web app, compiler, web server, in memory db server, offline documentation, integrated database manager, local and distributed cache management, fixtures, evolutions, dependency management, lots of utility libraries, unit tests, selenium tests, ide support. You need no apache or mysql setup to get up and running
it has advanced features, like asynchronous requests, web sockets, asynchronous jobs
zero development environment setup: just unzip play.zip, play new my app, play run myapp
excellent documentation
great and fast growing community
scala is firmly in it's horizon, if you come from a dynamic language you might feel more comfortable with a language like scala, even though it's static
can be deployed to any servlet container, like tomcat, but also comes with it's own lightweight server
it's got an ever growing collection of community contributed modules
it's cloud friendly, works on gae, heroku, playapps, cloudbee, dotcloud, etc... (see this question)
source code: this is one of it's main advantages, even though it's often neglected. The source code is pretty readable, understandable and not over engineered. Even without much java knowledge, it's possible to have a look at the code, understand it, and begin contributing to the project
In my particular case, coming from a php & classic asp background, play allowed me get a web application up and running while learning java at the same time. it really let me enjoy my transition from dynamic languages to java...
Play has some advantages compared to other dynamic web frameworks, but I think that if you are proficient in codeignter, rails, django, etc, and if you are not willing to benefit from java's (or scala) ecosystem, you should stick to what you know...
A similar debate has been running on Google Groups recently, but more specifically comparing Play and Ruby (on Rails). I have tried to collate some of the feedback the community has provided on this topic, rather than just give me personal view. Hopefully this will make for a more rounded answer!
statically types business logic layer (and using the new templating engine, even your views will be statically typed). This has therefore the advantage of better IDE support as well.
Can use Java as well as Scala, so you are not restricted to a single technology stack.
Play just works out of the box, but is also highly configurable, so does not force you to work in a particular method.
In memory DB. Great for new developers, who just want to get started. No need to go through setting up WAMP/LAMP server etc, just to get started. Just download play, and you are ready to go!
Its Java (or Scala)! I got hooked on Play because I used PHP begrudgingly because it was the easiest way to host a personal website, but hated that I was building software in a way that I did not feel was well structured. Play allowed me to use Java without the bulk of a J2EE application.
Play framework. Play is heavily inspired by Ruby on Rails and Django. A developer familiar with any of these frameworks will feel at home.
1) Play leverages the power of Java to build web applications,By lifting away the Java EE constraints, Play provides developers with an easy-to-develop and elegant stack aimed at productivity.
2) It focuses on developer productivity and targets RESTful architectures. Play is a perfect companion to agile software development.
3) Folder structure of the play mostly match with the Rails.
4) Play loads the code dynamically. Just fix the bug in Java file and hit reload, the change will get reflected instantly on webpage. No need to go through those sluggish rebuild / deployment / server restart.
5) No configuration: No configuration required just download, unpack and start development.
6) Error discovery is very easy, When an error occurs, play shows the error message, the file path, line number, and relevant code snippet right in the browser.

Server-side Java dev on Windows

I'm carrying out a feasibility study on writing, let's say it's a lightweight run-in-browser MMORPG. (It's not exactly an MMORPG, but would take longer to explain, and the requirements are similar.) I'm trying to figure out the required technology stack.
Client side, it runs in the browser, so the client is Javascript. That was nice and easy :-)
Server side, I'm looking at Java. The common Lamp stack was designed for RESTful applications, as I gather were typical Java web frameworks, and this application is different - it needs a continual stream of data going back and forth between the server and all the clients. I think what I need in this case is Java of the non-framework, full no-holds-barred Java EE variety, someone please correct me if my understanding is incorrect or incomplete? I would need something that is commonly available on reasonably cheap hosting, Java EE fits this description, right?
(Figuring on using MySQL for the database, as this is what's most commonly available. Also I might actually write the code in Scala, being a Java-compatible but supposedly slightly nicer language. I assume neither of these makes any difference?)
Supposing I were writing a website in Lamp, doing at least the initial development on Windows, then I'd install Xamp, which gives you a running copy of the entire server stack right there on your desktop, so you can just alt-tab back and forth between your editor/IDE and browser for testing.
Now my question is: What's the best equivalent setup for Java, for developing something like this on Windows?
Right, in a sense it is. What I think I'm really saying is that almost all discussion of server-side Java seems to talk about JSP, EJB, Glassfish, Google app server etc which are frameworks in the sense that they put restrictions on what your code can do, whereas Java EE puts no such restrictions, you can use as much or as little of the standard library as you want, but it doesn't stop you running arbitrary persistent Java code. Is that correct?
You've thrown out a bunch of terms there:
Glassfish is an application server that implements (all of) Java EE.
JSP is a specification that is part of Java EE, and implemented by application servers such as Glassfish as well as web servers such as Tomcat and (I think) Jetty.
EJB is another specifications that are part of Java EE. It is typically implemented by application servers.
"Google app server" is really "Google Application Engine" (GAE), and is really a platform for implementing web servers in a cloud computing environment. If that's not what you want / need to do, GAE is probably a blind alley for you.
Glassfish, JBoss, Tomcat, Jetty and so on are all platforms that implement some or all of Java EE. None of them stop you implementing arbitrary persistent Java code.
GAE on the other hand does restrict what you can run, because the platform only allows you to run standard Java classes in a whitelist. If your "arbitrary" code depends on other standard Java classes, you are out of luck. (And hence my warning about blind alleys.)
And of course, there are various other Java-based frameworks that are targeted at web development in one form or another. Some are compatible with Java EE servlets and other EE technology, and some have gone off in a different direction.
My recommendation would be to start with something straight forward using plain servlets on a stock platform. Only look at the high performance stuff if and when performance looks like it will be a real concern (not just a "nice to have"). It is better to have a simple and non-scalable prototype, than a high performance solution that you don't have the time and skills to get working. And you can treat the prototype as a learning exercise.
A little hard to tell from the requirements given, but I would look at the following based on your description:
http://www.playframework.org/
http://www.zkoss.org/
http://code.google.com/webtoolkit/
Play Framework is a nice web-centric framework that provides a complete stack (you can code in Java or Scala). ZKoss (ZK) and GWT both provide user interface frameworks. ZK, GWT, and Play all run nicely in Tomcat/MySQL and should work just fine in typical Java/MySQL hosting environment.
... not sure what your definition of 'cheap' is but, that and 'no holds barred Java EE' don't exactly go together. Also, 'frameworks' by themselves don't put restrictions on running any arbitrary java on the server side. When I say 'frameworks' I mean Spring, Struts, etc. Servers will be a little different story. If you need EJBs you'll need Glasfish or JBoss or another EJB compliant container. Hosting with these will be more expensive compared to getting by with Tomcat. IMHO easiest and quickest way to get started is with NetBeans. Comes with Tomcat and / or Glassfish out of the box, all you need is a db.

What may be the difficulties, if any, with the web applications developed in Java EE?

I have a few questions to understand better Java's usage in context of web applications:-
Is Java EE web development suitable for small start-up (with less human resource) looking to develop an web application ?
What kind of difficulties may arise in Java EE web development, deployment & maintenance ?
What kind of things should be kept in mind/ considerations to be made when moving from PHP background to Java ?
Why Java web applications are not so popular today? ( or in case I perceived it wrongly, please list any major deployments beside linkedIn and ebay)
and Finally, What are some of the most important things to learn before starting web development in Java EE ?
Thank you
Generally the answer to the question of "what technology to use" is "the one which you have most experience with". However, Java EE is huge and clunky, and definitely not good for rapid prototyping, which you will be doing if you're doing a startup.
Personally I would recommend a more modern and dynamic environment. If you're coming from PHP, you should be able to pick up Ruby on Rails or Django (Python) easily. These two choices are in my opinion orders of magnitude better than Java EE. If you want to stick with Java, at least go with the Play framework then.
Is Java EE web development suitable for small start-up (with less human resource) looking to develop an web application ?
Yes, I worked in a startup where I was the only full time programmer.
What kind of difficulties may arise in Java EE web development, deployment & maintenance ?
The same as in any other web development shop. Of course, the problems have their Java flavor. For instance, one bug we discovered was caused by different minor version of JDK used on the live system than on our test system.
What kind of things should be kept in mind/ considerations to be made when moving from PHP background to Java ? Do not code the PHP way. Java's strength is OOP and its many libraries/ open source frameworks. Use that.
Why Java web applications are not so popular today? ( or in case I perceived it wrongly, please list any major deployments beside linkedIn and ebay)
I don't know why you think that, but Java is used everywhere. It is one of the few languages that Google officially uses. They use PHP as well, but it has a "lower" status.
and Finally, What are some of the most important things to learn before starting web development in Java EE ? Use Java's strong points which I mentioned above.
Updated after comment
I cannot make the choice for you. If you are in doubt and in a big hurry you should not go with Java. This is common sense. However, it is an opportunity for your team to learn and grow. Maybe there is a PHP/other client for Cassandra. I knew a former PHP programmer in a startup, who switched to Java. Not saying anything bad about PHP programmers in general, but he did all kinds of strange things, such as not leveraging the power of Java web frameworks and writing lots of procedural code mixed with HTML and SQL. Obviously there are lots of Java programmers who would do the same thing. The point is that your team will probably learn new ways to do things and benefit from it in the future.
Allow me to answer these from the perspectives of a developer/architect in a small start-up, experiencing a bunch of these issues.
What kind of difficulties may arise in Java EE web development, deployment & maintenance ?
How do you decide on which toolset/framework to use? Do you need an IDE? Which version control system and why? Do you want to develop at some place and deploy somewhere else, or develop directly on the server? Do you buy a linux box for this, or rent some cloud? How much do they cost, in terms of licenses and training?
What kind of things should be kept in mind/ considerations to be made when moving from PHP background to Java ?
How would your servlet send out an e-mail? It's much simpler in PHP. Need secure transfer of encrypted objects? Java is your friend. What about session tracking? Use cookies, or have a dedicated class do it? How do you access the database? Want to use hibernate? What other tools is hibernate dependent on? What are their costs (license+learning)? Can you use JDBC directly? What are the pros and cons? Which db to use to why.
Why Java web applications are not so popular today? ( or in case I perceived it wrongly, please list any major deployments beside linkedIn and ebay)
I am not sure if this is the case, but possible reasons could be the availability of .net and integration with C# based systems and Apple ditching Java from its SDK. But that is my speculation, don't quote me on it. I am developing a large scale system myself with Java 6.
and Finally, What are some of the most important things to learn before starting web development in Java EE ?
(This is my opinion) have a test or trial set up of the entire architecture. Is the GUI web-brower based? Is it an applet? Standalone application talking to a server? JNLP system downloading archives and JRE off the net? You will find some stuff do not work on Windows 7, some do not on Vista, W3C have deprecated the applet tag from HTML but Sun/Oracle asks you to use it, different browsers do not support contents of your style sheet, etc.
Firewall set up is another major challenge - you start using thread pooling using Spring libraries and your capabilities to use DBvisualizer to check on DB tables are gone! Now you need a DBA and a sys-admin to fix these who you do not have!
Personally I found the LAMP architecture (Linux-Apache-MySQL-PHP) the fastest way to go for smaller applications, but if you need heavier guns for your app (security, GUI with swing, multithreading, etc), replace the P in LAMP with a tomcat container. The hardest thing I find is to judge the value of a tool in the context of my application - I do not need a tool that generates Java files with getter/setter methods given a list of variables - to me that is yet another level of indirection, but then JUnit in eclipse is helpful for debugging.
Just shared some of my thoughts - hope this helps, - M.S.
For a startup an interesting choice could be the Lift web framework, which is used for developing "Java Web Applications" (although in Scala).

PHP to Java migration

We are occasionally in a position to take over PHP based projects, but as we are a Java-house we are searching for ways to turn a PHP-project (or codebase) into a Java-project.
The approaches we came up to work in a mixed Java/PHP context are :
PHP in frontend and Java in backend with separate front (PHP+JavaScript) and back (Java+SQL) teams
using both technologies in a Java webapp, for example via Quercus with one or multiple teams
migrating everything to Java
We haven't tried any of these approaches as we've been lucky enough to have enough Java-only projects to work on.
Do you see other approaches, or have you tried any of the described approaches?
I've used Quercus. I think that if it supports something like Drupal then it's probably fairly mature. If your PHP app works out of the box, then it offers you a very gradual upgrade path to Java since you can write your own plugins in Java and expose them to the PHP layer, such as using a JDBC back-end.
How easy it is all depends on how well separated the layers are in the PHP application. For example, if the view layer is well separated, you might be able to replace both controller and model logic with a Spring MVC application that uses a 'QuercusView' for the view - you may be able to re-use a lot of the PHP view logic.
You should also consider how you can make a survey of the PHP code - maybe a script that extracts all the function calls, imports etc. so you can quickly test the Quercus support for them.
Sorry, I haven't used the PHP-Java bridge, but I think the Quercus library is pretty mature these days, so that would be my first choice for a staged migration.
My experiences with Querces are not so good. Maybe it has improved, but when I tried it something like two years ago (a long time, I know) it was far from complete and did not support all functions yet. Also, if your application is using some extensions (e.g. from PECL) you will experience difficulties getting this up and running under Querces.
We're currently in the process of migrating a web application from PHP to Java. We're designing a SOA and we'll probably replace some data access objects in the PHP application with a version that talks to internal webservices. Currently we're using Thrift as protocol for our internal webservices, a framework that also has a client available for PHP. We expect this to increase flexibility during migration (allowing us to do step-by-step migration, rather than all-at-once)
PHP/Java Bridge may be of your interest:
The PHP/Java Bridge is an
implementation of a streaming,
XML-based network protocol, which can
be used to connect a native script
engine, for example PHP, Scheme or
Python, with a Java virtual machine.
It is up to 50 times faster than local
RPC via SOAP, requires less resources
on the web-server side. It is faster
and more reliable than direct
communication via the Java Native
Interface, and it requires no
additional components to invoke Java
procedures from PHP or PHP procedures
from Java.
NUMITON may be useful for you:
http://java.dzone.com/announcements/automated-translations-php-jav
The shortest route from PHP to Java
Numiton offers an automated alternative to migrating PHP codebase. This way,
established applications can benefit from the advantages offered by Java in its
enterprise-level capacity.
Some of the risks inherent to any software migration are avoided by using an automated
translation tool. Our PHPtoJava product performs variable type inference,
objectualization and other operations in a uniform manner, the resulting appearance and
behavior being identical to what the users already know.
Of course, the human factor still plays an important role in the post-translation
phases: application fine-tuning and functional testing. The speed and accuracy of the
entire process surpass however those of a manual translation.
One of the applications we have migrated this way is the well-known forum engine
phpBB. The translation result, nBB2, powers our own forum and was recently donated to the
open-source community as a SourceForge project.
PHP in frontend and Java in backend
with separate front (PHP+JavaScript)
and back (Java+SQL) teams
This is technically feasible using SOAP.
using both technologies in a Java
webapp, for example via Quercus with
one or multiple teams
I have no idea about this
migrating everything to Java
This is better option, but it will take time depending on the size & complexity of your project.
I do not have any experience running PHP inside a JVM, but I am betting that IBM does. :)
You might consider Project Zero.
http://www.projectzero.org/php/
Seems similar to Querces or that other thing someone linked. The only other thing that you didn't mention that I can think of is to refactor the PHP code because I'm assuming that its an older codebase written in a PHP 4 manner.
I know this is 2 years old question but i still want to contribute. We are migrating from full java portal to php+java. We have 14 million users. This design doesn't need web services because we use java as json provider for php and js. We will see the result soon...
PHP in frontend and Java in backend with separate front
(PHP+JavaScript) and back (Java+SQL) teams

Categories

Resources