Develop Reference

Creating a new local database with Java

My aim is to create a local database that can be read and written to with Java. I have some experience with manipulating a local sqlite database with Python, and with interacting with existing networked databases on Microsoft Azure via VB.Net, but the Java formulation for creating a database is escaping me.
Most sources (like the JDBC Docs) seem to assume that you are accessing a database through a network protocol, or a database hosted on localhost. My desired implementation is to create and store the database in a file (or collection of files), so that it can be stored and accessed locally, without network connectivity (presumable through the "file:" protocol).
The JDBC Tutorial looks like it will be very useful once I am up and running, but is currently beyond my scope, since I don't even have an existing database yet.
Many sources have suggested solutions like H2, MySQL, Derby, or Hypersonic DB. However, I'm loath to install extensions (if that's the right term) for a number of reasons:
This project is initially intended to help me learn my way around Java - widening the scope of the project will dilute my experience with the "base" language and, probably, increase the temptation to engage in "cargo cult programming"
If this project does ever get distributed to other users (admittedly unlikely, but still!), I don't want to force them into installing more than the core of Java.
I simply don't know how to install extensions (add-ons? modules?) in Java - one baby-step at a time!
For similar reasons, installing Microsoft SQL Server would not be productive.
This answer looks close to what I'm aiming for; however, it gives the error:
java.sql.SQLException: No suitable driver found for jdbc:mysql://localhost/?user=root&password=rootpassword
and trying "jdbc:file://targetFile.sql" gives a similar error.
I've seen the term "embedded" database, which I think is a subset of "local database" (i.e. a local database is stored on the same system - an embedded database is a local database that is only used by a single application) - if I've got those definitions wrong, please feel free to correct me!

Most likely, the reason for which you are getting the error, is due to the fact that you are not registering the driver (using reflection...) before actually using it for establishing a connection and so on.
Presumably you will want to do something along the lines of Class.forName("driver")
and then cast that if necessary and then registering it in the DriverManager before calling the getConnection() method.
Here is a very useful link that might help you out in solving the issue:
http://www.kfu.com/~nsayer/Java/dyn-jdbc.html
However, if you really want to use a local database/file you might want to have a look at SQLite, that might be one way to go about it, although I recommend going for the MySQL approach, as it is a lot easier to configure and learn how stuff works with JDBC.
If you are still considering SQLite check this out:
Java and SQLite
I see you need some guidance in importing external .jar files into your code (i.e. 3rd party libraries like the ones you will be using for a JDBC driver). Are you using an IDE (e.g. Eclipse, Netbeans, etc.) or are you writing in a text editor and compiling manually?

A number of embedded pure Java databases appeared recently, which have a really simple interface, usually just java.util.Map, don't involve using JDBC or other SQL artifacts, and store their data in a single file or directory:
Chronicle Map
JetBrains Xodus
MapDB
The main downside is that most of such databases provide only the simples key-value model.

DBC can be used with any database that has a JDBC driver, which isn't necessarily a database in "network mode", it can be used with embedded databases as well.
Here are some Java and embeddable databases:
http://www.h2database.com/html/main.html
http://db.apache.org/derby/
http://hsqldb.org/

Java's JDK does not include any implementation of a database nor drivers to access it. It only provides JDBC as an abstraction to connect to a "database". Is up to you to include all the needed libraries in your code.
If you want to have a self contained code you can simply include the .jar file of a embeddable database in you classpath. That way you can create the instance of the database in your code and minimize the external dependencies.
You can find here a list of java embeddable databases
You can find here an example of how to embed HSQLDB in your code.

Switching between embedded Databases in Java with JPA

Im currently working my way towards JPA 2.0 and I start of liking how easy it is to maintain persistent data.
What I'm currently trying to accomplish is using JPA in a basic desktop application. The application should allow me to open embedded databases which are on my file system. I chose H2 databases for now, but I can really live switching to JavaDB or anything else.
What Im trying to accomplish is, that one can open the database file without previously define a persistence-unit in the persistence.xml file.
I can easily define a unit and persist objects, but it needs to be configured first.
I want to write some sort of database browser which allows opening without preconfiguration and recompiling.
http://www.objectdb.com/java/jpa/start/connection
I saw that ObjectDB allows access for this type of PersistenceFactory creation, but I was not able to transfer this example to other databases.
Am I totally wrong with the way I approach this probblem? Is JPA not designed with on-the-fly database access?
Thank you for your help,
Johannes

Not part of the JPA standard. Some implementations may offer their own API to do it. For example with DataNucleus if you go to this page http://www.datanucleus.org/products/accessplatform_3_0/jpa/persistence_unit.html at the end you can create dynamic persistence-units (and hence EMFs), and that implementation obviously allows persistence to the widest range of datastores you'll get anywhere

You can pass a Map of properties to createEntityManagerFactory() call that defines the database connection info, etc. The property names are the same as in the persistence.xml. I assume most JPA providers support this, EclipseLink does.
You will still need to define the set of classes for the database and map them.
If you do not have any classes either, than you could look into EclipseLink's dynamic support,
http://wiki.eclipse.org/EclipseLink/Examples/JPA/Dynamic

If you want to make a database browser accessing different databases, you can't use a PU/Entity Manager (imo).
You'll need a dialogue asking a user for the IP/Port of the database, the username/password, the database name to access, and the type of database.
Then all you need to do is create a socket, send requests over the socket, and parse the response into a view.
Since both the request and the response are database specific, the user has to select the proper database driver.

Properties file for database connection GWT

Hi is it possible to use a properties file in GWT that contains database details and then just change it anytime if for example, your database details change? This file would be read by java.sql.Connection so that the system could connect to the database. Thanks

It is possible to use files to configure GWT. Every GWT "module" has a definition xml file, and you can always define constants in your Java code too.
On the server side, you can use all of the Java tools you're used to, as long as your server supports them! GWT does not place any additional constraints on your server code.

I suggest you to try with an ORM (object-relational mapping) in order to handle your database communication. In this case you will naturally use a configuration-property file.
There is a good article that provides some info about using GWT with Hibernate.

Alternative of Storing data except databases like mysql,sql etc

I had completed my project Address Book in Java core, in which my data is stored in database (MySql).
I am facing a problem that when i run my program on other computer than tere is the requirement of creating the hole data base again.
So please tell me any alternative for storing my data without using any database software like mysql, sql etc.

You can use an in-memory database such as HSQLDB, Derby (a.k.a JavaDB), H2, ..
All of those can run without any additional software installation and can be made to act like just another library.

I would suggest using an embeddable, lightweight database such as SQLite. Check it out.
From the features page (under the section Suggested Uses For SQLite):
Application File Format. Rather than
using fopen() to write XML or some
proprietary format into disk files
used by your application, use an
SQLite database instead. You'll avoid
having to write and troubleshoot a
parser, your data will be more easily
accessible and cross-platform, and
your updates will be transactional.

The whole point of StackOverflow was so that you would not have to email around questions/answers :)
You could store data in a filesystem, memory (use serialisation etc) which are simple alternatives to DB. You can even use HSQLDB which can be run completely in memory

If you data is not so big, you may use simple txt file and store everything in it. Then load it in memory. But this will lead to changing the way you modify/query data.

Database software like mysql, sql etc provides an abstraction in terms of implementation effort. If you wish to avoid using the same, you can think of having your own database like XML or flat files. XML is still a better choice as XML parsers or handlers are available. Putting your data in your customised database/flat files will not be manageable in the long run.
Why don't you explore sqlite? It is file based, means you don't need to install it separately and still you have the standard SQL to retrieve or interact with the data? I think, sqlite will be a better choice.

Just use a prevayler (.org). Faster and simpler than using a database.

I assume from your question that you want some form of persistent storage to the local file system of the machine your application runs on. In addition to that, you need to decide on how the data in your application is to be used, and the volume of it. Do you need a database? Are you going to be searching the data different fields? Do you need a query language? Is the data small enough to fit in to a simple data structure in memory? How resilient does it need to be? The answers to these types of questions will help lead to the correct choice of storage. It could be that all you need is a simple CSV file, XML or similar. There are a host of lightweight databases such as SQLite, Berkelely DB, JavaDB etc - but whether or not you need the power of a database is up to your requirements.

A store that I'm using a lot these days is Neo4j. It's a graph database and is not only easy to use but also is completely in Java and is embedded. I much prefer it to a SQL alternative.

In addition of the others answers about embedded databases I was working on a objects database that directly serialize java objects without the need for ORM. Its name is Sofof and I use it in my projects. It has many features which are described in its website page.

How do I put data into the datastore of Google's app engine?

I have a little application written in php+mysql I want to port to AppEngine, but I just can't find the way to port my mysql data to the datastore.
How am I supposed to save the data into my datastore? Is that even possible? I can only see documentation for persistence of Java objects, does that mean I have to port my database to a bunch of fake objects, one per line?
Edit: I say fake objects because I don't want to use them, they're just a way to get over a shortcoming of the GAE design.
I have a 30 megs table I need to check on every GET, by using objects I would need to create an object for every row, so I'd have a java class of maybe 45 megs with thousands upon thousands of lines like:
Row Row23423 = new Row (123,346,75,34,"a cow");
I just can't believe this is the only way.
Here's an idea, what about populating the data store by POST-ing the objects one by one? I mean, like the posts in a blog. You write a class that generates and persists the data, and then you Curl the url with the data, one by one. Slow, but it may work?

How to upload data with the bulk loader is described here. It's not supported directly in Java yet, but that doesn't have to stop you - just do the following:
Create an app.yaml that looks something like this:
application: myapp
version: upload
runtime: python
api_version: 1
handlers:
- url: /remote_api
script: $PYTHON_LIB/google/appengine/ext/remote_api/handler.py
login: admin
Make sure the application name is the same as your Java app's, and the version is not the same as the version you're using for Java. Upload this 'empty' app using appcfg.py.
Now, follow the directions for bulk loading in the page linked to above. When it comes time to run the tool, specify the server address with --server=upload.latest.myapp.appspot.com .
Since multiple versions of the same app share the same datastore - even across runtimes - the data uploaded with the Python version will be accessible to the Java one.

There is documentation on the datastore here.
I can't see anything about a raw data-porting service but if you can extract the data from your MySQL database into text files, then it should be relatively easy to write a script to import it into the app engine's data store using the persistence frameworks provided by it.
Your script would take your raw data, convert into a (Java) object model and imprt those Java objects into the store.

Migrating an application to Googles App Engine I think would be quite some task. As you have seen the App Engine does not have a relational database instead it uses BigTable. This will likely involve exporting it to Java objects (serialized in some way) and the inserting them.
You say "fake" objects in your post but I as you will have to use Java objects anyway I don't think they would be fake unless you plan on using one set of objects for the migration and a new set for the application.

There is no (good) general answer to the question of how to port a relational application to the GAE datastore, because the notion of "data" is incompatible between the two. Relational databases are all about the schema. GAE doesn't even have one. It's a schemaless persistent object datastore with very specific APIs. The environment is great for certain types of apps if you're developing from scratch, but it's pretty tricky to port to.
That said, you can import CSV files, as Nick explains, which you should be able to export from MySQL fairly easily. GAE supports Java and Python "at the same time" using the versions mechanism. So you can set up your data store in Python, and then run against it for your application in Java. (A Java version of the bulk loader is under development.)