I have to run a lot of relational-database-based reports for my job. In order to make my life easier, I have written a handful of scripts and programs to run a report (simple SQL query, complicated super-specific cross-cutting query, and full JasperReports anything-goes-style reports), compress and/or encrypt it, and deliver it via SMTP to one or more recipients.
Some of this has been written in Java, some in PHP, and some as bash shell scripts. I'd like to unify a lot of this, plus add a single report-description format that can include configuration information like which report (or reports: sometimes I have to run several reports at once and combine them into a single "package" for the client) to run, what any parameters for the report should be set to (customer id, for instance), who should receive the report via email/SFTP/etc.
Note that tools like JasperReports are only part of the solution: the actual report-generation part. I'm more interested in something that can be used to script the whole process and be run from a cron job.
Do any products (OSS would be great) exist to help me with this kind of thing? Or, is this kind of thing so unique to a particular environment as to require the kind of customized tools that I have already built?
Use BIRT that integrates with your Java/J2EE application to produce compelling reports. And the second option is DataVision that is similar to popular Crystal Reports.
It might be a little off what you were thinking about, but have you considered (or heard of) R ? R is an open source statistical programming language that has many features, but specific to your problems is;
Is open source
Has packages that enable interacting with SQL databases (generic ODBC, or specific Oracle, MySQL, Postgres etc)
Can do many kinds of data processing steps with the data
Can produce
tables and graphs
Can produce documents in several ways (LaTex,
Markdown, ODF)
Can be used from the command line, (for instance I
produce documents with make)
Can be integrated into a web server
(RApache)
Can run Java code (RJava)
the only problem is this would be roll your own, there's no specific implementation that offers the features you want.
some frameworks that might help you with configuring report delivery workflow are.
Apache Service Mix http://servicemix.apache.org/ see achitecture diagram froma a previous version http://servicemix.apache.org/home.data/ServiceMix3.png
Spring integration http://static.springsource.org/spring-integration/reference/htmlsingle/
Both should give you a framework for building a decoupled architecture so that the part does report compression and the part that does report encryption and report generation don't know about each other but can be configured via the framework to work in a particular workflow.
Maybe you can check Gradle http://www.gradle.org/. Since it is based on Groovy you can send Mails, package archives and call JasperReports easily.
From Gradle web site:
Gradle is build automation evolved. Gradle can automate the building,
testing, publishing, deployment and more of software packages or other
types of projects such as generated static websites, generated
documentation or indeed anything else.
Gradle combines the power and flexibility of Ant with the dependency
management and conventions of Maven into a more effective way to
build. Powered by a Groovy DSL and packed with innovation, Gradle
provides a declarative way to describe all kinds of builds through
sensible defaults. Gradle is quickly becoming the build system of
choice for many open source projects, leading edge enterprises and
legacy automation challenges.
You might want to take a look at Pentaho: http://www.pentaho.com/explore/pentaho-business-analytics/
Related
I have a Java application (a quite large one with many external .jar dependencies as well as dependencies on images) and I need to package it up so that someone can double click to run, for example. Or something easy like that.
It uses Java Persistence, so it requires a sql connection which is specified in the Persistence.xml file in the Java Project.
How can I package this up? I was thinking:
the installation process should validate that the user has MySQL installed and if not, direct them to install it
the installation process could ask the user to enter credentials for any database and then I could update the Persistence.xml at run time
These were two ideas I had...but I wasn't sure if there was a known solution to this problem. Any help would be much appreciated!
I think you should take a look at embedded database solutions, like H2. Also, you can package your application using maven's shadowing or jar plugin, having the jar-with-dependencies profile activated.
This will nicely rid you of checking for database servers running on the client machine, and also will give you the proper means of bundling the application in one nice JAR, albeit a little large.
Maven is a build ecosystem and toolset especially designed for building Java applications and executing the code -- and generally doing whatever else you can imagine that's possible to do with and to your code.
It has a rich API for developing plugins and many developers have exploited this feature. There are numerous plugins for building -- and launching -- and packaging your application as well as helping you manage your applications dependencies.
Maven's shadowing comes in the form of maven-shade-plugin, available here. What it does is that it helps you create a single JAR file from all your dependencies. Also, there is the maven-jar-plugin which offers a profile jar-with-dependencies. It is also accessible from here.
H2, on the other hand is a full-fledged RDBMS. This is the website: http://www.h2database.com/html/main.html, and here is a tutorial.
You can find information on embedding the database here:
How to embed H2 database into jar file delivered to the client?
Embedding the Java h2 database programmatically
h2 (embedded mode ) database files problem
I would also suggest you use a combination of H2/Hibernate/Spring which is a very easy setup and provides you with really rich features and an easy-to-use API.
I hope this helps you :)
Building a sophisticated installer that checks lots of dependencies, and runs on lots of different platforms (which I assume you want) is complicated.
I suggest that you look at an installer generator; see What is the best installation tool for java?
Another alternative that I've seen in a few products is to write a (non-GUI) installer or configurer in a scripting language like Perl.
I wrote an installer using ANT, but has no GUI. Also, I used Iz Pack (good option), so I think that depends on how smart do you want it to be, if you are supposed to use it, or a non-technical person, etc.
I'm in charge of a portion of a large software development project. We have multiple environments (dev,test,pre-prod,prod, etc.) and many different software components. Most of them are based on Java/Weblogic (with a smattering of other application servers thrown in).
We currently have no good way to migrate configuration & application code from environment to environment. Code is checked into an SCM. To perform a migration someone checks it out and manually migrates it. This is obviously lengthy and very error prone - mistakes happen all the time.
Does anyone have experience with migration tools that they might be able to share? We don't mind doing scripting (obviously) but some sort of GUI to make this easy, along with a head start for situations like environment-specific config, would really help.
You can have all settings in property-files (one for dev, one for test etc). After it prepare deployment scripts (sh, ant, etc) that will take environment as a parameter and build/deploy the whole distributive with correct property file. Now when you have an automated way to do it you can setup jobs in any CI system (like Hudson) to run them using single click.
My company is trying to determine the best strategy for implementing batch Java programs. We have a few hundred (and growing) separate Java programs. Most of them are individual Jasper Reports but some are bigger batch Java jobs. Currently, each Java Project is packaged an independent JAR file using Eclipse's export option. Those JARs are then deployed to our Linux server manually where they are tested. If they pass testing, they are then migrated up through QA and onto Production through a home grown source code control system.
Is this the best strategy for doing batch Java? Ongoing maintenance can be a hassle since searching Jar files is not easy and different developers are creating new Java Projects (new reports) every week.
Importing existing projects from the Jar files into Eclipse is a tricky process as well. We would like these things to be easier. We have thought about packaging all the code into 1 big project and writing an interface to be able to execute the desired "package" (aka program) maybe using a Web Server.
What are other people/companies doing out there with their batch Java programs? Are there any best practices out there on this stuff? Any help/ideas/working models would be appreciated.
I would say that you should be able to create one web based app for access Jasper reports, rather than a bunch of batch processes. Then, when you need to deploy a new report, just deploy a minor update that accesses a new compiled Jasper report file.
That said, you should be checking your code, not your binaries, into a Subversion or Git repository. Dump the "home grown" source control repository. Life is too short to try to home grow stuff like that. Just use Git or Subversion, they're proven, simple, and functional. When you import a new project, just pull it down from Subversion, don't try to import the JAR file from your Eclipse IDE.
Put your JAR files into a Maven repository such as Nexus, and deploy to QA and Production from there. Create automated builds for every project (be that with Maven or something else). Don't depend upon an IDE to export your JAR files. IDE's change and exporting from an IDE introduces more opportunity for human error. Also, different developers will prefer different IDE's. By standardizing on something like Maven, you're a bit more IDE agnostic.
Mhy company has standardized Java Batch execution using IBM Websphere Extended Deployment.
Here http://www.ibm.com/developerworks/websphere/techjournal/0801_vignola/0801_vignola.html is an article introducing techniques for programming and deploying java batch.
Introduction to batch programming using WebSphere Extended Deployment Compute Grid
Christopher Vignola, WebSphere
Architect, IBM
Commonly thought of as a
legacy "mainframe" technology, batch
processing is showing itself to be a
venerable workload style with growing
demand in Java™ and distributed
environments. This article introduces
an exciting new capability for Java
batch processing from IBM®, the leader
in batch processing systems for the
last 40 years. This content is part of
the IBM WebSphere Developer Technical
Journal.
WebSphere Extended Deployment Compute
rid provides a simple abstraction of
a batch job step and its inputs and
outputs. The programming model is
concise and straightforward to use.
The built-in checkpoint/rollback
mechanism makes it easy to build
robust, restartable Java batch
applications.
The Batch Simulator utility provided
with this article offers an
alternative test environment that runs
inside your Eclipse (or Rational
Application Developer) development
environment. Its xJCL generator can
help jump start you to the next phase
of testing in the Compute Grid unit
test server.
But even if you are not interested in the product, the article is a must read anyway.
Here we are talking about three main products:
JasperReports: which is just a library to generate reports and fill it
iReport: which is GUI tool to develop templates used by the above library
JasperServer: which is full application with web interface for managing reports with users permissions and other features,
documentation is really not that good, i want to use the best of the above products to integrate JasperReports with my application (Spring/Hibernate), so please advice what is the best approach to do this,
1- design reports using iReport using embedded sql generation, and make it as a separate project from mine, I tried to use Hibernate or Spring within iReport but also I faced a lot of problems and no clear documentation for this part
2- attach it with my project, to use same Hibernate configuration, already created entities
3- try to use JasperServer which seems to be complicated, and then expose reports as webservices
or any other better, guaranteed approach, i know it's dependant on the way my project is dealing, but i've shortage in time, so it depends on the approach having better documentations or tutorials.
You can write your own Jasper framework in a module or subproject. You can maintain report definitions separately from queries and feed them with data produced by your code.
This way all data and handling stays in your project where it's easier to track or edit (e.g. when you only need to change an aspect of the query, add a filter or rewrite it for better performance). Also, this way you can use report data for something else (e.g. add CSV or XML output).
See http://en.wikipedia.org/wiki/JasperReports for the "What is" part of your question.
I faced the same problems when considering to use Jasper-reports in my application, I ended up given it up completely.
Anyway, take a look at this post for more information about different report frameworks for Java. Generating Reports - What works for you?
I wholeheartedly recommend that you take a look at BIRT http://www.eclipse.org/birt/phoenix/
I've used it with both Spring and Hibernate, and have no complain so far.
I can't comment on the other two approaches, but we are using Jasperserver and that seems to be more appropriate if you want to actually use a portal with which people will access their reports.
With this, you create reports in ireport and publish them to the portal. People log in to the portal to access their reports. I don't think you'd use Jasperserver if you were simply embedding it in your application.
I've started using JasperServer-pro recently and tbh I quite like it. As far as I know there seem to be plenty of options in the way of integrating it with existing apps :-
1 -Deploy as standalone server, and
use the WebService API to call into
to generate and return your reports
(these have to be predefined, either
through iReport or JasperServer's
own AdHoc Report Editor.
2- Deploy as standalone server, and do URL
calls through a kicked-off browser
to pull up reports (again
predefined)
3 - (Pro-only) -
Download the WAR BIN installer and
rebrand it.
4 - Download the source
code for JasperServer and build it
from ground up
5 - Download the
source code / JasperReports jars and
borrow the parts you want to use in
your source code.
For JasperReports can you buy documentation. You'll get a book and sample code that can help a lot. I just looked at the website (it's been some time ago I worked with JasperRepors, currently no reporting to do), they have a professional design application as well now: http://www.jaspersoft.com/jasperreports-professional.
Is it possible to set up continuous build of projects written in .NET and Java on single build server with single set of applications?
I've seen CruiseControl has support for both world but as far as I know these are two separate applications. Should I go with a separate machine to build Java projects (a machine for .NET projects is already set up)?
Hudson has support for both Ant (out of the box IIRC) and NAnt (through plugin) so that's an option for you.
CruiseControl supports several different build options include Ant, Maven, NAnt, Phing (php), Rake, XCode, and then the generic "exec" which you can use to invoke any command-line script/tool you want. Not a problem mixing Java and .NET on the same server.
Of course this is not unique to CruiseControl. There are lots of CI tools that support multiple build technologies as you can see on this matrix of features.
Disclaimer: I work on CruiseControl. OTOH since I don't make money on it I don't much care which tool people choose. I care more about advancing the state of CI practices which is why I organize the Continuous Integration and Testing Conference (CITCON).
Ant and NAnt can reasonably easily execute arbitrary processes, including each other, so the actual build part shouldn't be too hard.
I'd expect the tricky bit to be getting the reports (including unit test results) into an appropriate format. How's your XSLT? ;)
EDIT: Now that I think about it, my first agile project had a continuous build server (just CruiseControl, I believe) which must have been doing some of this... I suspect at the time we directly invoked Visual Studio to build the code and NUnit to test it. If I were at the same company I'd check, but that was two jobs ago :(
You could checkout Atlassian Bamboo. Unfortunately, its not free, unless you are applying for an opensource/community license for use with opensource software.
You can use two different products on the same machine. Or you can run a single builds system across multiple machines. It is really up to the load you place on your CI.