I have a problem where I am supposed to create an application (Preferably Java) which extracts data (Excel sheet) from onedrive account of different users and store that in a database.
I went through different resources on internet to look for the same. But i couldn't find any REST API for JAVA Applications (Although there is for Android).
Any pointers in solving above problem would be really helpful.
Thanks
Excel Datasource
As much of a fan as I am of OneDrive, it is really about storing file information not relational data. It sounds like your problem would be best solved with a connected Excel data source as seen below in this example. This will require that you setup an accessible datasource (Setting up a DB, API, or file backed source). Using the following menu should allow you to creation the relationship with the excel presentation.
Straight-forwards APIs
Assuming that you don't have the time/control needed to architect a completely new way of using this content I can give you some basic pointers around the following scenarios, 1 getting an up-to-date file from OneDrive, 2 interacting with the excel data and 3 then naturally doing what you need to do with that information.
1) Working with OneDrive
If you can interact with the file inside of your OneDrive using your machine, install the sync client for OneDrive. This will work on a Mac, Windows 7, 8, and 8.1 machine and might just get you up and syncing with ease.
If you can't use your personal account/deploy to a controlled machine you will want to use and SDK like the LiveSDK. However, I would actually recommend the Live-SDK-for-Windows, while it is not java, C# is pretty close in a lot of ways as long as you can depend on running on a windows machine.
Lastly, if you need to be able to run in a non-windows environment or have to run using the Java stack then I'd say that you should look at porting the LiveSDK-for-Android into a java only version, this is a large amount of work as you'll need to remove all the android dependencies, but you should be able to target your specific scenario around getting/updating files. You could technically look at reverse engineering how the service interactions work with a program like Fiddler.
2) Interacting with Excel files
Look around the Excel SDKs these are your best functionality bets to dig into your spreadsheets and getting the data that you'll need. There is a high likelihood that you will need to use C# or a .Net language to use them however.
Find a java compatible library to use, a quick Bing search turned up J-Integra, note that this still requires a Windows environment for execution and will have their own restrictions and usages.
Related
Before I started learning and using Java I used VB. In VB, a program can open a Microsoft Access database file and view/edit the records. VB has a component that handles the file and parsing. What I am looking for is a Java program that can open a database file directly. What I am not looking for is Java communicating with a SQL server a database server or any other kind of server. I have tried to search the net for Java and databases but all I am able to find is Java protocols that communicate with a server or a Java server service. Those items fill the search records on all of the search engines I have tried. I have found that DB's can be XML, and that Java can open them but, it seems that it no longer preserves the relational aspects. I may be wrong on that fact, I am uncertain.
A. Can Java open a local DB file?
B. Can that preserve the relational aspects?C. Can multiple Java programs open the same DB file?
D. If a Java program on computer A modifies the DB file will the Java program on computer B know about that change and update itself?
Assuming they are not modifying the same record. The type of DB file is not that important to but it would be nice if it could be opened by MS Access. It would also be nice if it could be open by JAVADB. Neither of those two are required.
E. what kind of components or projects could meet this need?
Pure Java hasn't graphic design. Some degree of design of databases have IDE, for example Eclipse in perspectives: Database Debug, Database development. Can view tables, design columns etc.
It isn't full desktop tool for (semi) amateur computer worker.
In Java can write full comfortable user database program (in example Open Office "Base" has some dependencies to Java), with filling similar to entry level access.
For simpler database like sqlite there are several java projects that you can use, for example sqlite 4 java.
Sqlite is pretty good these days and for small databases a good choice.
I'm curious to know if it's possible to fetch data from other applications to my Java program. I know java is running in a virtual environment, hence JVM and therefore have problems communicating with other applications unless you were to use the Robot class or so forth.
What I would like to do, for starters as educational purposes is to take let's say a music application like Spotify/iTunes, fetch the playlist (text data) and send/display it in a text file. I've tried a few things so far, and the only thing I've come close to is by using the Robot class, opening the application, doing Ctrl+A, pasting it in to a text document and so forth but that's more like a macro. I would like to make a java application that would do this automatically. Is that possible in any sense with Java or are you just better off changing languages? I wish to do it with Java though because it's the language I've studied for the past year and what I'm trying to master. (Sorry for the long explanation.)
It's nothing to do with Java and the JVM. Any language has the same problems and solutions for this sort of situation.
The thing you need to talk to has to provide you a way to talk to it. You need to talk to it using that way.
Methods include pipes, custom network protocols, SOAP and Restful web services, etc.
Just because an application runs in a virtual machine doesn't mean it cannot access external data through an API provided by an external program. For example, iTunes has a COM-based API for accessing playlists, and here is an example of using it from C#. You'll need something which allows accessing COM objects from Java.
(Please note I know nothing about this topic, this is just what I found with a little searching...)
You need to know which data interfaces provides the application you want to connect to.
As an example, maybe the application writes files to disk or stores some info in a database.
Then with Java you can read files, query the database, use an API (web services, REST, etc..) and so on...
I have a situation where I need to execute some code that for a number of reasons (that I won't get into here) is better done in Java than in PL/SQL. As I see it there are two options:
Create a jar with all my compiled code and other supporting files/other jars, load the jar into Oracle (we're running 10g), and execute the Java from a stored procedure.
Pros: The Java code integrates very nicely with the rest of the system, can be called from existing PL/SQL.
Cons: I have very little experience with running Java in Oracle.
Leave the Java in a separate jar and execute it through shell scripts.
Pros: I've written Java like this before so I'm familiar with it.
Cons: Poor integration with everything else, will probably require extra manual steps to run and manage.
The Java code will have to read XML data from Oracle tables, and write data (non-XML) to other tables, so the amount of database integration made me think loading the Java code into the database might be a good idea but I'm just not sure...
What experiences do people have loading and running Java code from within Oracle? How easy is it to test and debug? Are there are any special tools required? Any "gotchas" I should be aware of?
I would go with option number 1: loading your java code in your database. I have had good experiences with this approach and you don't need that much experience to get a good solution working with this method.
The only part where embedding Java in your database gets complicated (you'll have to set special permissions for your Java code) is when you need access to external resources (network, file i/o to name a few) and it is clearly not your situation in this problem.
For your scenario, doing it within the database seems the right approach. Reasons to do it outside could be:
dependence on Java libs that are not compatible with Oracle's built-in JVM
need to run it under a different linux user account than Oracle's
but I don't see either in your scenario
I'm looking for a document repository that supports physical data separation. I was looking into Open Text's LiveLink, but that price range is out of my league. I just started looking into DSpace! Its free and open source and works with PostgreSQL.
I'm really trying to figure out if DSpace supports physically separating the data. I guess I would create multiple DSpace Communities, and I want the documents for each Community store on different mounts. Is this possible? Or is there a better way?
Also, I wont be using DSpace's front end. The User will not even know DSpace is storing the docs. I will build my own front-end and use Java to talk to DSpace. Does anyone know a good Java API Wrappers for DSpace.
This answer is a pretty long time after you asked the question, so not sure if it's of any use to you.
DSpace currently does not support storing different parts of the data on different partitions, although your asset store can be located on its own partition separate from the database and the application. You may get want you need from the new DuraSpace application which deals with synchronising your data to the cloud.
In terms of Java APIs, DSpace supports SWORD 1.3 which is a deposit-only protocol, and a lightweight implementation of WebDAV with SOAP. Other than that, it's somewhat lacking. If you are looking for a real back-end repository with good web services you might look at Fedora which is a pure back end, with no native UI, and probably more suited to your needs. DSpace and Fedora are both part of the DuraSpace organisation so you can probably benefit from that also. I'm not knowledgeable enough about the way that Fedora stores data to say whether you can physically separate the storage, though.
Hope that helps.
Looking for any recommendations for an ETL system for 200+ distributed systems (Windows, AS400, Linux etc).
We collect data each month from all of our customers (regardless of system type), bring it back, process it all together and send the aggregate solutions back to them. I'm tasked with automating this system - any suggestions on how to do this robustly, I really don't want to re-invent the wheel. I don't own any of the systems I'm pulling data from, which has made this task more difficult but can install a client.
I've created a prototype client/server architecture in Java with FTP for transport but it feels brittle to me. I should note that all of the extract/transformation code for the different systems already exists in Java (albeit legacy).
I should mention we pull data once per month currently, but working towards weekly.
Any insight is appreciated.
I think it would depend on how the project will become. If this porject will be adding more requirement and there is some money involved, the ETL tool might be good idea.
However, if you have fixed output(the report) now and it is not intended to go anywhere, the custom ETL might be worth it. The reason is the most ETL tools have various output format(Chart, text file etc) and convinience to use the tool but the bottom line is Data moving part is almost universal for all the tools. Even with any other ETL tool, you need to implement same query you are doing now, plus you need to learn the tool. Who knows? Some tool might involved in 200+ site installation.
Recently, our company spent a lot of money to buy report tools & servers & human resource to build good ETL since our in-house ETL has been critisized for the slowness and not professional looking(You know it is not using popular ETL tools. It is bunch of script command). With all the money spending, the project faced on almost dead end.
One more thing. I don't understand how Java & FTP is involved in this process.
Can you directly connect the DB in your customer system using SQL?
If you could, using SQL & stored procedure is always better idea than using JAVA & FTP.
Hope it would help.