My current task is to produce a report on a collection of fairly large linear data set (ie where value 10 depends upon some average factors from 9 8 7 6 5 4 3 etc
The problem also is that this logic (lets call it moving average now) is not fixed and will most probably change. I need to not only generate a report in HTML but in Excel and perhaps email it to use automatically (so some of the variables will be relative, ie generate a report of last month for example)
So my question is should I use a reporting engine or not? (Jasper, Pentaho, Birt)
My drivers here are
-The reporting logic will change, but it will be changed by the developers. It will be complex logic so it will be easiest done in code in Java.
-The report has to be saved and emailed in excel format.
-The reporting task will be huge (some thing where ETL might be helpful)
-Future reporting requirements are not a consideration here, the system will go a rewrite if it needs to be generic and extend-able.
I could just create an entity called report, store its type (an enum) and the parameters list (from the restful URL) and other metardata (who created, what date etc, where to send, and relative parameters are stored) And then when some processing has been done, store it in DB, just to retrieve the processed data for controller to consume. It greatly simplifies the technologies involved (I use spring mvc and it should be enough with a little sprinkle of POI). But I wonder if using reporting engine would be simpler.
My experience with Jasper and Birt (and the commercial Windward Reports) is that they will not help you much with the number crunching part of the report; they are best suited to presenting data that you have already transformed into a suitable structure. Jasper, Birt and Windward Reports can easily handle running totals and averages, but any calculations more complex than that are a real pain (and may be impossible depending on your calculations). I am not familiar with Pentaho.
In your situation I would either implement the calcuations in custom Java code, or in an ETL process.
You could then use a reporting engine to present the results of the calculations in HTML and Excel formats. But if you don't have Jasper or Birt, etc, already integrated into your application then I wouldn't recommend adding one just for this. It would be quicker to manually write out the report data using JSP and POI respectively. Jasper and Birt reports are fairly easy to create, but only once you've got the reporting engine integrated properly!
I've never used Birt, but I have used Jasper, and my one comment would be: using a reporting engine has a learning curve, and adds overhead of it's own. So if you're doing some one-off report, and if the actual report layout is fairly simple, you may not gain a lot from a reporting engine. But, on the flipside, Jasper IS really good at things like letting you write the logic once, then easily generate your report in multiple formats (including Excel).
If there's a strong requirement that the generated report "look pretty" and if it has a really complex layout, and if you need to generate lots of formats, then you may find the reporting engine valuable.
If you can justify spending a couple of hours on it, download Jasper (or Birt or whatever) and run through some of the tutorials / demos and get a feel for it. That'll probably be more valuable to you than all the comments you get here.
Related
What i'm trying to do is to integrate anomaly detection module in the existing java application to allow user to chose from different algorithms and forecasting models
Egads library looks quite optimistic, but I'm not sure whether it fits my purposes, in case new data coming in should I store and update the existing model or to pass the whole data once again. Also what if I would like to forecast only 15 min time window, by passing only 15 min data in the results won't be precise for sure.
Might be there're any other useful techniques and someone could share his experience of similar tasks. Unfortunately can't find any other java libs for that purposes.
What I found out is that we can't store the model trained initially and apply it against any incoming data, as soon as the initial time series is changed the exception is thrown. That's why the only possible option here to to train the model every time new data comes in, fortunately it doesn't have great performance impact on our system yet.
Library itself looks fine and could be used as a base in building anomaly detection systems, but its still not that flexible as it Python competitors, however it's open sourced and can be modified anytime depending on your needs.
I am working on salesforce UI’s.
How can I ensure that whatever UI’s created are able to handle a load.
Do I need to do performance testing?
Please suggest tools , process so that 500 user load / performance testing if any?
Tool choice depends on many factors, i.e.
ability to integrate flawlessly into your existing SDLC (i.e. if you have Java stack a Python-based tool would be an "alien)
support of all required use cases (nearly all load testing tools are suitable for web applications testing, however when it comes to more "exotic" tasks like communicating with database, WebSockets, handle AJAX requests, etc.) the choice might be less wide
bugdet (there are some proprietary solutions which cost millions of coins and free/open source options)
skills (the tool might require coding skills which you and your team may or may not have)
You can find quite a list for instance here: http://www.opensourcetesting.org/category/testing-tools-overview/performance/?menu-page=overview
Given you have java tag I would recommend to stick to one of the following:
The Grinder
Gatling
Apache JMeter
Check out Open Source Load Testing Tools: Which One Should You Use? article which highlights main features of the above tools followed by comparison matrix, sample scripts, reports, etc., hopefully it will help you to choose the right option.
Are there any libraries (built-in or third party) for writing .html reports with JBoss apps quickly and easily?
I have a list of entities in a database that I want to display in different tables. There will need to be things like links within the report to go from some parts of the report to other parts, sorting/paging, and some nice CSS to make the report pretty would be nice.
I have an existing application that produces the .html and .css programmatically, but due to the huge size of the data (124Mb in some cases), the writing of the .html takes way too long (40+ minutes) and often freezes up the browser. So I am looking for an alternative.
I guess one solution is to write JSF forms that fill rich tables with bean data, but I am not sure if this is the best approach.
Replies highly valued! Thanks
Have you looked into JQuery? It offers large number of plug-ins to achive pagination, tables, sorting etc. and it sound like it might be a good fit for your case.
JasperReports is open source, flexible, and perfect for my needs (I am reporting using xml data, and a simple transformation will be perfect to JSXML format to produce reports in this way).
I'm trying to implement part of the facebook ads api, the auto complete function ads.getAutoCompleteData
Basically, Facebook supplies this 39MB file which updated weekly, and which contains targeting ads data including colleges, college majors, workplaces, locales, countries, regions and cities.
Our application needs to access all of those objects and supply auto completion using this file's data.
I'm thinking of preferred ways to solved this. I was thinking about one of the following options:
Loading it to memory using Trie (Patricia-trie), the disadvantage of course that it will take too much memory on the server.
Using a dedicated search platform such as Solr on a different machine, the disadvantage is perhaps over-engineering (Though the file size will probably increase largely in the future).
(Fill here cool, easy and speed of light option) ?
Well, what do you think?
I would stick with a service oriented architecture (especially if the product is supposed to handle high volumes) and go with Solr. That being said, 39 MB is not a lot of hold in memory if it's going to be a singleton. With indexes and all this will get up to what? 400MB? This of course depends on what your product does and what kind of hardware you wish to run it on.
I would go with Solr or write your own service that reads the file into a fast DB like MySQL's MyISAM table (or even in-memory table) and use mysql's text search feature to serve up results. Barring that I would try to use Solr as a service.
The benefit of writing my own service is that I know what is going on, the down side is that it'll be no where as powerful as Solr. However I suspect writing my own service will take less time to implement.
Consider writing your own service that serves up request in a async manner (if your product is a website then using ajax). The trouble with Solr or Lucene is that if you get stuck, there is not a lot of help out there.
Just my 2 cents.
I'm looking for a simple (free) way to convert an arbitrary document to a PDF from within a program. There are any number of free PDF printers, but I need to be able to call the conversion within a program without human intervention. The program is being developed in Java, but will run exclusively in a Windows environment so calling an exe seems like a good solution if such a conversion program exists.
I have had some success with JodConverter, which is a Java-based wrapper around the OpenOffice.org API. Basically, you can run OpenOffice as a server and automate the action of opening a document in OpenOffice (which supports many many types) and saving it as PDF. JodConverter makes that a lot easier and has built-in support for running as a web service if you're interested in that.
Downsides: 1) Like OpenOffice itself, the conversion for certain complicated proprietary documents is not perfect; some of your Word documents may not look exactly identical as PDFs. 2) OpenOffice as a server is not entirely stable; if you hit it with a bunch of requests it will crash. One (somewhat expensive -- I think a few thousand dollars US) alternative is Sun's StarOffice Server, which does exactly the same thing as JodConverter (wrap OpenOffice) but adds pooling of OpenOffice instances and other stability support.
The most accurate PDF conversion tools are made by Adobe (and they do have server-based converters with API support), but they are very expensive - tens of thousands of dollars US.
simple... free... pdf... arbitrary input... At least the requirements are easy and reasonable.
Seriously, those requirements just aren't going to be met. If you are willing to pay money for a library that does some of this, you can check out Amyuni - It's a great library, but the type of stuff you are asking for is squarely in native win32 land - not something that's going to happen in Java. And even with that in place, it's not going to be simple.
I suppose you could do something with Ghostscript as well (many of the free PDF converters use it). But even then, you still have to deal with the conversion from arbitrary input issue.
There are other libraries available that can display lots of different file formats (even without the native application available) - perhaps something like that would work. Here's one (owned by Oracle now, so you know it's gotta be good ;-) : Outside In.
(BTW - iText is most definitely not going to do what you are asking about. I love iText, I use iText - heck, I'm a developer for part of iText - but it's most definitely not a PDF print driver, which is more in line with what you are going for).
for Java, the most recommended is iText