I am attempting to create a program that can manage EC2 instances (create, stop, terminate). I am unfamiliar with the service, and after looking through documentation and searching the web have not found any general advice on creating a management service. My questions are:
What programming environment/language would best be suited to creating a management program?
How can this program interact with EC2 instances? (I've looked into AWS EC2 command line tools. Can these be used from a program to create/terminate instances?)
Any general advice in accomplishing this is appreciated (links to examples especially).
What programming environment/language would best be suited to creating a management program?
This question cannot be answered in a definite way, rather you should either choose the language you are most comfortable with or that's best suited to your environment/team instead - there are plenty to choose from, most popular major languages are covered by a dedicated SDK (currently Java, .NET, Node.js, Python, PHP, Ruby), see Tools for Amazon Web Services for the detailed listing and links to further information about each.
Please note that there are also two IDE Toolkits listed that might make it easier to get started if you happen to develop in Java or C#, specifically the AWS Toolkit for Eclipse and the AWS Toolkit for Microsoft Visual Studio.
How can this program interact with EC2 instances?
All the SDKs (and also the command line tools build on top of these) do use the AWS APIs to interact with the respective services (each one has a separate API, but most of them are structured very similar), see Documentation for a listing of all currently available services and links to their documentation:
Welcome to the Amazon Web Services (AWS) Documentation. Whether you are new to AWS or an advanced user, you can find useful information about the services ranging from introductions to advanced features.
To learn how you can get started with AWS, see our Getting Started with AWS guide. If you are interested in learning more about our AWS Free Usage Tier, see our AWS free usage tier article.
I guess you are new to the cloud and aws world, you can use AWS Command line tools to management the services. Also to make things pretty simpler you can make use of the readily available aws management console if you are only worry is to start, stop or terminate the instance. Also people have build some thing called Config management system like opscode chef which is built of ruby or you can use puppet built of puppet lab's custom DSL.
I am not sure if you really need a program to do that. The Amazon Management Console ( https://console.aws.amazon.com ) is pretty straight forward, simple to use, you can spin thousands of instances in a very short time. I created 20 instances in about 3 minutes and deleted(terminated) 9 during my first interaction with the technology. Would you provide a use case where this is necessary? Its like you want to create a missile to kill a chicken(rather than buy a knife)
HTH, Thanks,
Bles
I used to work on a project that deals a lot with EC2 and one of my responsibility was to manage instances remotly( deploy/start/stor/reboot). I used Java and the Amazon API to write a communication module for the application.
In addition I made a few general ant scripts to deal with code that has to be deploed into an instance. Kind a old fashion way, but works perfectly :)
Related
I'm on a student research team currently working on a Java project in which we want to be able to gather some info about specific external programs in CentOS 7. Some potential examples of data we might want to gather would be the URL from firefox, or the coordinates from Google Earth. Thus far, looking into how to implement this has mostly led me to using JNA and ptrace.
Is this a good route to take to tackle this problem? And if so, what can I use to find the location of specific information about the programs in memory?
JNA and ptrace are quite low level things. To run system applications you should use:
exec method of Runtime class
Apache Common Exec
Last one is more useful.
But it seems you don't need to run any application at all. To get information from Google Earth you can just use special web API - Google Maps APIs Web Services. If you want to send HTTP request you should try Apache Http Client. It's very powerful library and i'm pretty sure it's all for you need.
PS. More convenient client Java Client for Google Maps Services
PS1. I think java is the worst choice to work with OS and native applications. I see 2 ways to use it:
write agent on C/C++ which will be gather this information for java manager. It's a hard way
use hacks and tricks like here is there a way to get the url from current tab in google-chrome or here Possible to catch URLs in linux?. It's more easy but you will have restricted amount these hacks. And it will work not for all applications and cases.
I'm looking for something that can monitor the traffic of a java web application in order to estimate cloud computing prices.
It would be great if it can categorize the traffic in different categories, e.g. database, static resources, pages, etc.
Ideally, it should be something working in the same way New Relic does. Unfortunately, New Relic only monitors times and not traffic...
Does something like this exist?
Thanks.
You don't need any java-specific software for doing that. You can use any network monitor tool and just run your application locally! For monitoring different kind of traffic, I would use different tools too. There are a lot of DB-monitor tools out there... Sorry for not being more specific.
There are quite a lot solutions for monitoring Java web applications. I have tried a few of them but finally stopped at two: Zabbix and JavaMelody. Both are appropriate to monitor and categorize a traffic of apps, although they are completely different in how they work. Zabbix allows watching an app over the long term via JMX. JavaMelody can be built into an app and provides complete insight into business app processes.
Your final decision about Java app monitoring platform depends on the prioritized app features that you’d like to monitor. I recommend you to read the review that helps you look at both solutions in details: http://cases.azoft.com/enterprise-system-monitoring-solutions-business-apps/
I have a few questions to understand better Java's usage in context of web applications:-
Is Java EE web development suitable for small start-up (with less human resource) looking to develop an web application ?
What kind of difficulties may arise in Java EE web development, deployment & maintenance ?
What kind of things should be kept in mind/ considerations to be made when moving from PHP background to Java ?
Why Java web applications are not so popular today? ( or in case I perceived it wrongly, please list any major deployments beside linkedIn and ebay)
and Finally, What are some of the most important things to learn before starting web development in Java EE ?
Thank you
Generally the answer to the question of "what technology to use" is "the one which you have most experience with". However, Java EE is huge and clunky, and definitely not good for rapid prototyping, which you will be doing if you're doing a startup.
Personally I would recommend a more modern and dynamic environment. If you're coming from PHP, you should be able to pick up Ruby on Rails or Django (Python) easily. These two choices are in my opinion orders of magnitude better than Java EE. If you want to stick with Java, at least go with the Play framework then.
Is Java EE web development suitable for small start-up (with less human resource) looking to develop an web application ?
Yes, I worked in a startup where I was the only full time programmer.
What kind of difficulties may arise in Java EE web development, deployment & maintenance ?
The same as in any other web development shop. Of course, the problems have their Java flavor. For instance, one bug we discovered was caused by different minor version of JDK used on the live system than on our test system.
What kind of things should be kept in mind/ considerations to be made when moving from PHP background to Java ? Do not code the PHP way. Java's strength is OOP and its many libraries/ open source frameworks. Use that.
Why Java web applications are not so popular today? ( or in case I perceived it wrongly, please list any major deployments beside linkedIn and ebay)
I don't know why you think that, but Java is used everywhere. It is one of the few languages that Google officially uses. They use PHP as well, but it has a "lower" status.
and Finally, What are some of the most important things to learn before starting web development in Java EE ? Use Java's strong points which I mentioned above.
Updated after comment
I cannot make the choice for you. If you are in doubt and in a big hurry you should not go with Java. This is common sense. However, it is an opportunity for your team to learn and grow. Maybe there is a PHP/other client for Cassandra. I knew a former PHP programmer in a startup, who switched to Java. Not saying anything bad about PHP programmers in general, but he did all kinds of strange things, such as not leveraging the power of Java web frameworks and writing lots of procedural code mixed with HTML and SQL. Obviously there are lots of Java programmers who would do the same thing. The point is that your team will probably learn new ways to do things and benefit from it in the future.
Allow me to answer these from the perspectives of a developer/architect in a small start-up, experiencing a bunch of these issues.
What kind of difficulties may arise in Java EE web development, deployment & maintenance ?
How do you decide on which toolset/framework to use? Do you need an IDE? Which version control system and why? Do you want to develop at some place and deploy somewhere else, or develop directly on the server? Do you buy a linux box for this, or rent some cloud? How much do they cost, in terms of licenses and training?
What kind of things should be kept in mind/ considerations to be made when moving from PHP background to Java ?
How would your servlet send out an e-mail? It's much simpler in PHP. Need secure transfer of encrypted objects? Java is your friend. What about session tracking? Use cookies, or have a dedicated class do it? How do you access the database? Want to use hibernate? What other tools is hibernate dependent on? What are their costs (license+learning)? Can you use JDBC directly? What are the pros and cons? Which db to use to why.
Why Java web applications are not so popular today? ( or in case I perceived it wrongly, please list any major deployments beside linkedIn and ebay)
I am not sure if this is the case, but possible reasons could be the availability of .net and integration with C# based systems and Apple ditching Java from its SDK. But that is my speculation, don't quote me on it. I am developing a large scale system myself with Java 6.
and Finally, What are some of the most important things to learn before starting web development in Java EE ?
(This is my opinion) have a test or trial set up of the entire architecture. Is the GUI web-brower based? Is it an applet? Standalone application talking to a server? JNLP system downloading archives and JRE off the net? You will find some stuff do not work on Windows 7, some do not on Vista, W3C have deprecated the applet tag from HTML but Sun/Oracle asks you to use it, different browsers do not support contents of your style sheet, etc.
Firewall set up is another major challenge - you start using thread pooling using Spring libraries and your capabilities to use DBvisualizer to check on DB tables are gone! Now you need a DBA and a sys-admin to fix these who you do not have!
Personally I found the LAMP architecture (Linux-Apache-MySQL-PHP) the fastest way to go for smaller applications, but if you need heavier guns for your app (security, GUI with swing, multithreading, etc), replace the P in LAMP with a tomcat container. The hardest thing I find is to judge the value of a tool in the context of my application - I do not need a tool that generates Java files with getter/setter methods given a list of variables - to me that is yet another level of indirection, but then JUnit in eclipse is helpful for debugging.
Just shared some of my thoughts - hope this helps, - M.S.
For a startup an interesting choice could be the Lift web framework, which is used for developing "Java Web Applications" (although in Scala).
We are occasionally in a position to take over PHP based projects, but as we are a Java-house we are searching for ways to turn a PHP-project (or codebase) into a Java-project.
The approaches we came up to work in a mixed Java/PHP context are :
PHP in frontend and Java in backend with separate front (PHP+JavaScript) and back (Java+SQL) teams
using both technologies in a Java webapp, for example via Quercus with one or multiple teams
migrating everything to Java
We haven't tried any of these approaches as we've been lucky enough to have enough Java-only projects to work on.
Do you see other approaches, or have you tried any of the described approaches?
I've used Quercus. I think that if it supports something like Drupal then it's probably fairly mature. If your PHP app works out of the box, then it offers you a very gradual upgrade path to Java since you can write your own plugins in Java and expose them to the PHP layer, such as using a JDBC back-end.
How easy it is all depends on how well separated the layers are in the PHP application. For example, if the view layer is well separated, you might be able to replace both controller and model logic with a Spring MVC application that uses a 'QuercusView' for the view - you may be able to re-use a lot of the PHP view logic.
You should also consider how you can make a survey of the PHP code - maybe a script that extracts all the function calls, imports etc. so you can quickly test the Quercus support for them.
Sorry, I haven't used the PHP-Java bridge, but I think the Quercus library is pretty mature these days, so that would be my first choice for a staged migration.
My experiences with Querces are not so good. Maybe it has improved, but when I tried it something like two years ago (a long time, I know) it was far from complete and did not support all functions yet. Also, if your application is using some extensions (e.g. from PECL) you will experience difficulties getting this up and running under Querces.
We're currently in the process of migrating a web application from PHP to Java. We're designing a SOA and we'll probably replace some data access objects in the PHP application with a version that talks to internal webservices. Currently we're using Thrift as protocol for our internal webservices, a framework that also has a client available for PHP. We expect this to increase flexibility during migration (allowing us to do step-by-step migration, rather than all-at-once)
PHP/Java Bridge may be of your interest:
The PHP/Java Bridge is an
implementation of a streaming,
XML-based network protocol, which can
be used to connect a native script
engine, for example PHP, Scheme or
Python, with a Java virtual machine.
It is up to 50 times faster than local
RPC via SOAP, requires less resources
on the web-server side. It is faster
and more reliable than direct
communication via the Java Native
Interface, and it requires no
additional components to invoke Java
procedures from PHP or PHP procedures
from Java.
NUMITON may be useful for you:
http://java.dzone.com/announcements/automated-translations-php-jav
The shortest route from PHP to Java
Numiton offers an automated alternative to migrating PHP codebase. This way,
established applications can benefit from the advantages offered by Java in its
enterprise-level capacity.
Some of the risks inherent to any software migration are avoided by using an automated
translation tool. Our PHPtoJava product performs variable type inference,
objectualization and other operations in a uniform manner, the resulting appearance and
behavior being identical to what the users already know.
Of course, the human factor still plays an important role in the post-translation
phases: application fine-tuning and functional testing. The speed and accuracy of the
entire process surpass however those of a manual translation.
One of the applications we have migrated this way is the well-known forum engine
phpBB. The translation result, nBB2, powers our own forum and was recently donated to the
open-source community as a SourceForge project.
PHP in frontend and Java in backend
with separate front (PHP+JavaScript)
and back (Java+SQL) teams
This is technically feasible using SOAP.
using both technologies in a Java
webapp, for example via Quercus with
one or multiple teams
I have no idea about this
migrating everything to Java
This is better option, but it will take time depending on the size & complexity of your project.
I do not have any experience running PHP inside a JVM, but I am betting that IBM does. :)
You might consider Project Zero.
http://www.projectzero.org/php/
Seems similar to Querces or that other thing someone linked. The only other thing that you didn't mention that I can think of is to refactor the PHP code because I'm assuming that its an older codebase written in a PHP 4 manner.
I know this is 2 years old question but i still want to contribute. We are migrating from full java portal to php+java. We have 14 million users. This design doesn't need web services because we use java as json provider for php and js. We will see the result soon...
PHP in frontend and Java in backend with separate front
(PHP+JavaScript) and back (Java+SQL) teams
I want to take the next step in java web development, I am hoping to get insight & feedback on: what my next steps should be and how best to take them.
While learning the basics of java web development, I put together a simple web app that performs simple accounting and financial calculations. The web app is on a single jvm, uses Tomcat, and has standard web functionality - i.e. login/logout, basic security, etc.
How can I make this web app more "enterprise ready" - distribute functionality of tiers over different servers/jvms, HA, balance-able, etc.
What do I need to know/learn? - i.e. EJB3 or Spring Framework (seems spring is better option), REST and/or SOAP, etc.
How would one recommend (books, websites, etc.) I learn the "requirements" (see preceding line)?
Thanks!
In my opinion, you should try different approaches for a same problem, so you could compare the pros and cons of different tools and frameworks.
For instance, try to build an application using EJB, and then the same application using Spring. Take the presentation layer of your code written with JSF and then rewrite it using Tapestry.
I think this will be very helpful to you, as you'll be able to make best decisions when choosing tools for your future developments.
A few things to consider, as food for thought:
How good is the error handling/logging of the application? For example, if the user tries to put in X in for a currency value, what does the application do?
What is configurable within the application from the user and what is in configuration files and what is in a database with regards to configuration? Do you have passwords encrypted within the application?
What patterns would used in building this application? Are there patterns you could see using now that you have a prototype?
Is this application ready to handle different currencies and languages?
What happens if someone leaves the screen for a few hours and tries to use a form?
What administrative functionality does the application have?
Does it handle the case where the user has JavaScript disabled?
What are the limitations of your application, IOW what can't it handle the way it is?
Have you considered trying to write a manual for the application?
EJB or Spring? gets asked quite a lot nowadays, here's a decent related question about them.
Have some real users using your application. You'll be amazed on how many "new" features/improvements can be performed in your app ( and the technologies you'll learn to satisfy those requirements ) by having real users using it.
I would suggest the following books/tutorials are a must for every Java developer:
Manning: Spring in Action - 2nd edition
Manning: Java Persistence with Hibernate
Core JavaServer Faces
Adobe Flex ( Adobe website video tutorials )
Effective Java
Apart from the standard technologies above you must be familiar with
Different testing frameworks , JUnit is a must
Build tools like ANT and Maven
Also you can build small projects by downloading trial versions of MyEclipse or Flex Builder.
I suggest that you create small java experiment projects for each new framework/library that you want to learn.
I've had good success using maven to help me quickly and consistently create java projects that I use to experiment with one technology at a time, such as Spring, Hibernate, etc. I use maven's site life cycle to record notes about what I learned and to document how to build and run each project. So, now I have 20 or so projects that I can use as baseline projects, one for each framework, to build upon.
Also, I prefer buying and reading books rather than relying on google and websites to learn new frameworks. Seems that I'm able to learn a lot faster this way.
I also suggest that you write web apps that you, yourself, would want to use. Or write a web app that solves a problem you've been having. I've found that I learn a lot more this way rather than simply copying and pasting from examples in a text book.
Hope that helps,
- Dave
Spring or Tapestry would be good options for new learnings. Does your app use any web services? If not work those in. Work with other application servers like JBoss and Weblogic and note their nuances with java. I'd also recommend learning Maven and work that into your build/deploy process.
Have fun,
Mike
You could vastly reduce the time taken to build your apps by learning some Test-Driven Development.
Try learning JUnit - it's becoming a core skill now, even in unagile shops.
If you're focussing on the web, try out Selenium - which has a Java controller to drive your tests from Java test cases.
After investing a bit of time in TDD will pay off no matter which frameworks or apps you work on. If you learn to test drive your code, you'll end up with smaller, cleaner code and less debugging.
You might look at AppFuse, which is a bundle of Java things together.
Or, you might take a look at a few more technologies to play with and add in:
Version Control - SVN
Tools - Ant or Maven
Framework - Spring, Seam, Struts
ORM - Hibernate or iBatis
Test Driven Development - JUnit, Emma
Continuous Integration - Hudson
I'd also read the Pragmatic Programmer and/or Code Complete.
Allow me to state that "enterprise"-ready does not necessarily imply scale-out solutions, many, many enterprise Java applications are running on larger systems requiring long-running-systems skill of their own.
I recommend mastering the Java language and runtime, understanding how bytecodes and loading traverse the JVM vs. focus on any given framework.
Speaking of frameworks and if you really have the time, try recreating an application framework yourself. Try and re-invent the wheel. IMO it is an excellent lesson in why frameworks themselves exist and teaches one to employ the features instead of always trying to work around them.
One more thing, never forget the database. I don't care what that looks like Oracle, MySQL or NoSQL, but become also an equal master at the data store.