Official documentation leaves lots of questions unanswered.
I need a in-memory Ignite storage where I can keep some data, loaded from 3rd party database. I understand two things:
I know how to connect to Ignite DB via JDBC driver, how to write and execute DDL statements, how to insert and query data with H2-compatible SQL statements.
I know how to initialize Ignite cache using DataStreamers and how to query data using SQLFieldsQuery
But I have no idea how to combine these two features to make them work together. I don't even know if it is possible. If it's impossible, how should I initialize database for future access via JDBC from external app?
Yes, it's possible. If you're able to query cache using SQLFieldsQuery, then you definitely can use the same SQL query to access it using JDBC Driver.
Here is example, that shows how to access data that was inserted from key-value api with SQL: https://github.com/dmagda/ignite_world_demo, just replace accessing from SQLFieldsQuery with accessing cache from JDBC Driver.
Related
I'm trying copy table from one database to another(On different machines), and using JDBC Template to execute query, but this request is specific to Oracle:
COPY FROM username1/passwd1#//192.168.3.17:1521/PROD_SERVICE to username2/passwd2#//192.168.4.17:1521/SANDBOX_SERVICE INSERT TABLE_C (*) USING (SELECT * FROM TABLE_C);
And I get error:
Caused by: java.sql.SQLSyntaxErrorException: ORA-00900: invalid SQL statement
How can I use specific to Oracle syntax in JDBC?
Like some of the comments have already clarified, COPY is a sqlplus command, and it has been deprecated for a while. You cannot use it inside JAVA, because this command is not part of the SQL engine, it's just a kind of additional feature available only in sqlplus. It is still available, but only for backwards compatibility.
If you want to copy a table using Java, you need to understand first some things:
Java, or any external engine for that matter, can't connect at the same time to both databases. Either it connects to one or to the other.
You need to have a kind of bridge between both databases, so that your Java program is only acting as trigger.
Copying tables between databases is something related to the database, so you should think in using tools provided by your database engine. You have some options, like Datapump or RMAN, although I consider Datapump the best suitable for your scenario.
However, if you insist in using Java, first you need to have a database link between both databases. Then you can use Java to invoke an insert from one database to another.
https://docs.oracle.com/database/121/SQLRF/statements_5006.htm#SQLRF01205
If you don't want to depend on thsnames entries in the server, here an example of database links:
CREATE DATABASE LINK to_my_remote_user
CONNECT TO remote_user IDENTIFIED BY password
USING '(DESCRIPTION=
(ADDRESS=(PROTOCOL=TCP)(HOST=remote_server)(PORT=remote_port))
(CONNECT_DATA=(SERVICE_NAME=remote_service_name))
)';
Once you have the dblink created, then you can connect from java to the database where the link is available and copy the data to the remote database
INSERT INTO remote_user.remote_table#to_my_remote_user
select * from local_user.local_table ;
Important: Normally dblinks are not allowed on Production systems, because they increase security risks. Also remember that DDL operations over a database link require an extra step, such as using the procedure DBMS_UTILITY.EXEC_DDL_STATEMENT#dblink('create table ...);
Another option outside of Java is using SQL Developer copy feature. Although I only recommend it for small tables. If you want to use it with big tables, it will probably hang. You can read here an good example :
copy from one database to another using oracle sql developer - connection failed
Working on a JAX-RS application which uses Microsoft-SQL-Server as a Database.
It does not use any ORM frameworks, just plain old JDBC.
Most of the application's operations involve store and retrieve data as XML into DB tables.
I have a use case where I have to run this application offline. So there wont be any connection available to DB SQL-Server.
Whilst looking into my options thought I would embed the DB and ship with the application EAR.
Looking into options I learned that, SQL-Server-Compact does not have a proper JDBC driver.
Is there any other In-Memory DB that could serve my purpose?
I want avoid any code changes like: changing the SQL queries (written specifically for Microsoft-SQL-Server).
Is there any solution which I can use and ship my application just by changing the DataSource to embedded DB?
note: I could not find any useful post on stackoverflow for this query, If it's already been discussed. Please point me to the post and I will delete this duplicate question.
I just recently switch from MySQL to MongoDB, I'm wondering with MySQL I stored the player data inside a hashmap and retrieved name, coins etc; like that so I don't have to constantly query the database to retrieve the data.
Now with MongoDB would I need to do the same thing store the values inside a hashmap and retrieve it the same way I did with MySQL?
It depends on your requirement. You have migrated to mongodb from mysql, this doesnt means that your reads would be superfast. If there would have been any significant I/O improvement in mongodb, mysql developers would have adopted it as well. MongoDB provide flexibility over mysql and there are some more advantages there. So If your load remains the same, you should have a caching layer before mongodb layer. Both Mysql and mongodb come with in-built caching which caches results on the basis of query just like a hashmap, but rest data is on disk and as mentioned mongodb doesnt have any technical advantage over mysql in terms of I/O. So have a caching layer to avoid excessive querying to db.
i have a project am working on, its all about querying a data from multiple databases from different vendors (i mean querying databases like mysql, hsqldb, microsoft sql, oracle, etc at the same time using one query statement).
Though i have achieved this by loading each driver of the database connector sequentially and execute the query sequentially across the databases. But the project architecture is such that when i sent a query statement, it shouldgo simultaneously to each database and retrieve the item ifavailable in all databases involved.
I came across this unityjdbc software, a mediation software but dont know how to implement it in my java source file so that to achieve my aim. I have read the unityjdbc user manual but is not clear and straight-forward.
Please can anyone advise how toimplement this unityjdbc driver in my java application and use it to successful query multiple databases.
Suggestions for any other way to simultaneously query their multiple databases with a single statement would also be welcome.
UnityJDBC allows you to query multiple databases in one SQL query. You cannot do this using separate threads as you would then be responsible for merging the data from the multiple databases yourself in your Java program.
The setup steps are easy:
Use the SourceBuilder application to specify the JDBC connection information to your databases.
Test a sample query that accesses multiple databases. Standard SQL is supported. To reference tables in different databases use databaseName.tableName in your FROM clause.
For example:
SELECT *
FROM Database1.Table1 T1 INNER JOIN Database2.Table2 T2 ON T1.id = T2.id
The SourceBuilder application will provide an XML configuration file as output often called sources.xml. To use this in your own Java program or any software that supports JDBC the connection URL is: jdbc:unity://sources.xml You may specify an absolute or relative path to the sources.xml file.
There is documentation on their site at http://www.unityjdbc.com/support/ or contact them for free support.
Another way to get started quickly is to use the MultiSource SQL Plugin that comes with the open source query software SQuirreL SQL. The plugin will allow you to query any number of databases using SQL in SQuirreL and will generate the XML configuration files for you to use in other programs. The plugin is open source and free. The plugin also supports querying and joining NoSQL databases like MongoDB with relational databases such as MySQL and Postgres.
You don't need UnityJDBC for what you want to do, if you've already managed to load all the db-specific JDBC drivers.
Instead, you should look at doing each query in a separate thread. That way, you don't need to wait for one database to return its results before querying the next one.
I am doing a database migration work. I have to copy a database in MSSQL to MySql database. It was possible to come up with a small java utility to copy table stucture from MSSQL to MySql Database. Now i have to copy all data from MSSQL to MySql. I tried using resultset in java to obtain all data from a table but then it could only fetch a small part of data. Is there any alternate solution to get all data from table to resultset or to some other similar structure which i could possibly use, to insert the same data into mysql Db. There are more than 25,00,000 records for a table.
A JDBC result set should in principle allow you to iterate the entirity of a large query result.
However going via Java may not be the most efficient approach. Bulk export to a file and bulk import may be the way to go. It appears that MS has a bcp utility that may do the export.
The best way to achieve a database migration like you describe is to use and ETL Tool - there's a good overview of ETL here:
http://en.wikipedia.org/wiki/Extract,_transform,_load
There's no reason why you wouldn't be able to do this with JDBC and so if you are set on rolling your own please elaborate on 'could only fetch a small part of data':
what is the query you are running?
are you getting an exception?
which JDBC driver are you using to connect to MS-SQL?