I would like to populate the data store. Yet all the examples and instructions for populating the data store are concerned with Python projects. Is there a way to upload bulk data using the AppEngine Java tools? (At the moment the data is in CSV format, but I can easily reformat the data as needed.)
It would be especially useful if it could be done within the Eclipse IDE.
Thanks.
I'm having the same problem as you with this one. According to the discussion at http://groups.google.com/group/google-appengine-java/browse_thread/thread/72f58c28433cac26
there's no equivalent tool available for Java yet. However it looks like there's nothing stopping you from using the Python tool to just populate the datastore and then accessing that data as normal through your Java code, although this assumes you're comfortable with Python, which could be the problem.
Related
My task is to write a Java based web application which will produce various charts like WaferMap, Histogram, Overlay Chart etc.
The front end is ExtJS and the chart generation part is taken care by JFreeChart.
The data for charts will be in multiple .CSV files which are stored in the file system.
My questions are:
The .CSV files size will be in GB's. Can I store these files in HDFS and query them during run-time and display data in frontend?
Is using Hadoop ecosystem is a feasible solution for my above requirement? Should I also consider Apache pig or hive for querying the CSV file?
Yes you can (Apache Hive)
It all depends but Hive seems like what you're looking for. It was designed with a SQL like feel and can include SQL clauses. It is widely used with major corps like Facebook, Netflix, FINRA, etc. In your case, supporting SQL syntax also means that you can integrate with Java's JDBC driver real easy and query data from your CSV files.
http://www.tutorialspoint.com/hive/
Setting up Hive can be a bit difficult at first if you're not too familiar with the Hadoop environment. The above link is a great reference link to understand Hive better and get you in the right direction.
Hope this was helpful!
I had an idea and wondered if anyone could point me at a good java lib to achieve this?
I've written a package that allows multiple sections of dynamic content to be added into emails and distributed to it's audiences as required. One requirement that has been mentioned in the past was to produce graphs in these emails. (We don't have Oracle BI, or the budget to buy it). I would like to consider adding graphs based on an object or data (Maybe a view object) that could be passed into a Java operation and produce a BLOB of data i could pass into the email and distribute? Running on Oracle 10g R2. Any ideas would be greatly appreciated.
Cheers!
You could use any Java library that can be loaded in the Database JVM.
I've used http://www.jfree.org/jfreechart a while ago and it worked quite well.
If you wanna go PL/SQL only this is also possible and this http://www.adp-gmbh.ch/blog/2005/november/22.html article might be a good start.
I have a problem where I am supposed to create an application (Preferably Java) which extracts data (Excel sheet) from onedrive account of different users and store that in a database.
I went through different resources on internet to look for the same. But i couldn't find any REST API for JAVA Applications (Although there is for Android).
Any pointers in solving above problem would be really helpful.
Thanks
Excel Datasource
As much of a fan as I am of OneDrive, it is really about storing file information not relational data. It sounds like your problem would be best solved with a connected Excel data source as seen below in this example. This will require that you setup an accessible datasource (Setting up a DB, API, or file backed source). Using the following menu should allow you to creation the relationship with the excel presentation.
Straight-forwards APIs
Assuming that you don't have the time/control needed to architect a completely new way of using this content I can give you some basic pointers around the following scenarios, 1 getting an up-to-date file from OneDrive, 2 interacting with the excel data and 3 then naturally doing what you need to do with that information.
1) Working with OneDrive
If you can interact with the file inside of your OneDrive using your machine, install the sync client for OneDrive. This will work on a Mac, Windows 7, 8, and 8.1 machine and might just get you up and syncing with ease.
If you can't use your personal account/deploy to a controlled machine you will want to use and SDK like the LiveSDK. However, I would actually recommend the Live-SDK-for-Windows, while it is not java, C# is pretty close in a lot of ways as long as you can depend on running on a windows machine.
Lastly, if you need to be able to run in a non-windows environment or have to run using the Java stack then I'd say that you should look at porting the LiveSDK-for-Android into a java only version, this is a large amount of work as you'll need to remove all the android dependencies, but you should be able to target your specific scenario around getting/updating files. You could technically look at reverse engineering how the service interactions work with a program like Fiddler.
2) Interacting with Excel files
Look around the Excel SDKs these are your best functionality bets to dig into your spreadsheets and getting the data that you'll need. There is a high likelihood that you will need to use C# or a .Net language to use them however.
Find a java compatible library to use, a quick Bing search turned up J-Integra, note that this still requires a Windows environment for execution and will have their own restrictions and usages.
with the platform j2ee,data of datagrid is from java dao(from the database),so which way is better,to export the data of datagrid to Excel with java dao or flex?thx, and forget my poor English...
I would suggest using the server-side implementation approach. In the company I'm working we tried both (but okay it was .NET). By now we had so many problems with the export of documents (also PDF [isn't it ironic??]) in combination with Flex. I remember following problems:
you might need a more current version of your Flash plugin (>= 10.0) (could cause legacy problems)
there were problems specifying different data types like dates
in the end it took more man-days than intended
A server side implementation might be more effort (in the first step), but
Java is more powerful
it's faster
more configuration is possible
(garbage collection works a lot better)
your real qestion is where it have more support.
in flex the for me i found 2 solutions for this problem:
is using the action script library as3xls
http://code.google.com/p/as3xls/ - it export a real excel document but it have a cons
it from my expirent with him it support only english chars it's not export unicoded chars
is saving a feak xls file that contain an html table format yes it works!
I've also dealt with this issue with my flex app on whether to do the datagrid export on the client side or server side.
I had originally exported the datagrid on the client side using as3xls (as mentioned above). It worked well at first, but then started giving me real headaches once it's limitations were reached (aka not being able to export non ASCII characters). This is when I stopped using as3xls.
If you do not require the datagrid export have any specific formatting, another option would be to export the datagrid contents to a comma delimited string and save the string to a CSV file, all from within the client. This way you avoid issues with specific data types (dates), have complete control over the text exported, and write out to a really loose and flexible file format. And you save yourself having to code the infrastructure to round-trip the datagrid contents to your server and back in cases where the size and formatting of exported data may not really justify it.
If you do require formatting or the vastly superior power of java to handle your export, I would suggest using an MS Office API like Apache POI to give you the ability to format your data much better into standard XLS or XLSX documents. This ended up being the solution we went with (except we are using SmartXLS as our Excel API), for the greater ability to control exactly how the data exported was to be laid out and formatted, plus delivering XLS/XLSX files to clients is more professional, and it's easier to provide to those less computer savvy clients than is a CSV.
I am familier with java programming language I like to extract the data from a website and store it to my database running on my machine.Is that possible in java.If so which API I should use. For example the are number of schools listed on a website How can I extract that data and store it to my database using java.
What you're referring to is commonly called 'screenscraping'. There are a variety of ways to do this in Java, however, I prefer HtmlUnit. While it was designed as a way to test web functionality, you can use it to hit a remote webpage, and parse it out.
I would recommend using a good error handling html parser like Tagsoup to extract from the HTML exactly what you're looking for.
You definitely need a good parser like NekoHTML.
Here's an example of using NekoHTML, albeit using Groovy (a Java-based scripting language) rather than Java itself:
http://www.keplarllp.com/blog/2010/01/better-competitive-intelligence-through-scraping-with-groovy
You can use VietSpider XML from
http://sourceforge.net/projects/binhgiang/files/
Download VietSpider3_16_XML_Windows.zip or VietSpider3_16_XML_Linux.zip
VietSpider Web Data Extractor: Software crawls the data from the websites ((Data Scraper)), format to XML standard (Text, CDATA) then store in the relational database. Product supports the various of RDBMs such as Oracle, MySQL, SQL Server, H2, HSQL, Apache Derby, Postgres …VietSpider Crawler supports session (login, query by form input), multi-downloading, JavaScript handling, proxy (and multi-proxy by auto scan the proxies from website)…
Depending on what you are really trying to do, you can use many different solutions.
If you juste wanna fetch the HTML code of a web page, then URL.getContent() may be your solution. Here is a little tutorial :
http://www.javacoffeebreak.com/books/extracts/javanotesv3/c10/s4.html
EDIT : didn't understand he was searching for a way to parse the HTML code. Some tools have been suggested above. Sorry for that.