in my application i need to copy a schema with its tables and store procedures from a base schemn to a new schema.
i am looking for a way to implement this.
i looked into exacting the mysqldump using cmd however it is not a good solution because i have a client side application and this requires an instillation of the server on the client side.
the other option is my own implantation using show query.
the problem here is that i need t implement it all from scratch and the must problematic part is that i will need to arrange the order of the tables according to there foreign key (because if there is a foreign key in the table, the table i am pointing to needs to be created first).
i also thought of creating a store procedure to do this but store procedures in my SQL cant access the disk.
perhaps someone has an idea on how this can be implemented in another way?
You can try using the Apache ddlutils. There is a way to export the ddls from a database to an xml file and re-import it back.
The api usage page has examples on how to export schema to an xml file, read from xml file and apply it to a new database. I have reproduced those functions below along with a small snippet on how to use it to accomplish what you are asking for. You can use this as starting point and optimize it further.
DataSource sourceDb;
DataSource targetDb;
writeDatabaseToXML(readDatabase(sourceDb), "database-dump.xml");
changeDatabase(targetDb,readDatabaseFromXML("database-dump.xml"));
public Database readDatabase(DataSource dataSource)
{
Platform platform = PlatformFactory.createNewPlatformInstance(dataSource);
return platform.readModelFromDatabase("model");
}
public void writeDatabaseToXML(Database db, String fileName)
{
new DatabaseIO().write(db, fileName);
}
public Database readDatabaseFromXML(String fileName)
{
return new DatabaseIO().read(fileName);
}
public void changeDatabase(DataSource dataSource,
Database targetModel)
{
Platform platform = PlatformFactory.createNewPlatformInstance(dataSource);
platform.createTables(targetModel, true, false);
}
You can use information_schema to fetch the foreign key information and build a dependency tree. Here is an example.
But I think you are trying to solve something that has been solved many times before. I'm not familiar with Java, but there are ORM tools (for Python at least) that can inspect your current database and create a complementing model in Java (or Python). Then you can deploy that model into another database.
Related
I'm looking to run a Java process on several machines, each of which will need to start a local OrientBD server, load a graph, perform our processes, then close. As such, I need to be able to embed the OServer start process from within Java.
There is plenty of advice about how to do so, including SA questions, however most seem to be out of date (so please don't mark this as a duplicate prematurely). The most directly relevant seems to be this, however it doesn't work - at least for me. With the below code, I get the subsequent error:
try {
final OServer server = OServerMain.create();
server.startup(server.getClass().getResourceAsStream("/orientdb-server-config.xml"));
server.activate();
} catch (Exception e) {
e.printStackTrace();
System.exit(-1);
}
2021-12-07 21:47:39:323 INFO Loading configuration from input stream [OServerConfigurationLoaderXml]
2021-12-07 21:47:39:633 INFO OrientDB Server v3.2.3 (build dc98198215aa57baf29b32adb657dc3733acdb55, branch develop) is starting up... [OServer]java.lang.NullPointerException
at com.orientechnologies.orient.core.Orient.onEmbeddedFactoryInit(Orient.java:957)
at com.orientechnologies.orient.core.db.OrientDBEmbedded.<init>(OrientDBEmbedded.java:97)
at com.orientechnologies.orient.core.db.OrientDBInternal.embedded(OrientDBInternal.java:119)
at com.orientechnologies.orient.server.OServer.startupFromConfiguration(OServer.java:388)
at com.orientechnologies.orient.server.OServer.startup(OServer.java:314)
at ems.definitions.instance.Graph.<init>(Graph.java:47)
I am using OrientDB version 3.2.3; the 'ALL' .jar downloaded from here. Note that this jar does not contain the parameters file orientdb-server-config.xml, so I have downloaded it directly from the source GitHub.
Is there an issue with my specific implementation, my approach in general or with the default config file I'm using? I look forward to hearing your thoughts.
The issue was three-fold:
I was using the 'ALL' .jar provided by the website. Instead I needed to use the libraries provided in the full source.
I did not account for the fact that when the code failed, it did not delete the database it half-created, thus could not execute the code I tried to remedy. I had to implement a temporary fail-safe to drop the database prior to initialisation to avoid this.
I was using the wrong(?) strategy in general.
My working method is as below.
orientDB = new OrientDB("embedded:/tmp/","admin","adminpwd", OrientDBConfig.defaultConfig());
/** THIS IS VERY MUCH ONLY FOR LOCAL TESTING **/
if(orientDB.exists(name))
orientDB.drop(name);
if(!orientDB.exists(name)) // if the database does not already exist, create it.
orientDB.execute("create database " + name + " PLOCAL users ( admin identified by 'adminpwd' role admin)");
db = orientDB.open(name, "admin", "adminpwd");
First of all, I want to clarify that my experience working with wikidata is very limited, so feel free to correct if any of my terminology is wrong.
I've been playing with wikidata toolkit, more specifically their wdtk-wikibaseapi. This allows you to get entity information and their different properties as such:
WikibaseDataFetcher wbdf = WikibaseDataFetcher.getWikidataDataFetcher();
EntityDocument q42 = wbdf.getEntityDocument("Q42");
List<StatementGroup> groups = ((ItemDocument) q42).getStatementGroups();
for(StatementGroup g : groups) {
List<Statement> statements = g.getStatements();
for(Statement s : statements) {
System.out.println(s.getMainSnak().getPropertyId().getId());
System.out.println(s.getValue());
}
}
The above would get me the entity Douglas Adams and all the properties under his site: https://www.wikidata.org/wiki/Q42
Now wikidata toolkit has the ability to load and process dump files, meaning you can download a dump to your local and process it using their DumpProcessingController class under the wdtk-dumpfiles library. I'm just not sure what is meant by processing.
Can anyone explain me what does processing mean in this context?
Can you do something similar to what was done using wdtk-wikibaseapi in the example above but using a local dump file and wdtk-dumpfiles i.e. get an entity and it's respective properties? I don't want to get the info from online source, only from the dump (offline).
If this is not possible using wikidata-toolkit, could you point me to somewhere that can get me started on getting entities and their properties from a dump file for wikidata please? I am using Java.
Recently I have found the article about Room db and there was first tip for pre-populating the data to database.
Is there currently some elegant way to pre-populate the data when the database is created?
I am using Dagger2, so the actual creation of the database is done quite easy.
#Module
class DatabaseModule{
#Provides
#Singleton
fun provideObjectBox(context: Context): BoxStore =
MyObjectBox.builder()
.androidContext(context)
.build()
}
And the way I am doing it now with the SharedPreferences. So I am just checking if it is the first set up of the database and than populating the database.
Also i guess when the question was created it was not possible there is a function added to the builder so you can simply call:
MyObjectBox.builder()
.initialDbFile(file)
.androidContext(context)
.build()
This will use the given file must contain all data in mdb format. I am using this feature for backuping user data so i dont have to create the file on my own.
As far as i know there is no easy posibility to create this file instead of creating objects and putting them in the boxStore.
I am copying the file with already existing data like this (not the pretty way though)
val dbFile = File(File(File(activity.getFilesDir(), "objectbox"), BoxStoreBuilder.DEFAULT_NAME), "data.mdb")
Just found the main contributer answered the same: https://stackoverflow.com/a/51765399/8524651
I am developing a web app using Spring MVC. Simply put, a user uploads a file which can be of different types (.csv, .xls, .txt, .xml) and the application parses this file and extracts data for further processing. The problem is that I format of the file can change frequently. So there must be some way for quick and easy customization. Being a bit familiar with Talend, I decided to give it a shot and use it as ETL tool for my app. This short tutorial shows how to run Talend job from within Java app - http://www.talendforge.org/forum/viewtopic.php?id=2901
However, jobs created using Talend can read from/write to physical files, directories or databases. Is it possible to modify Talend job so that it can be given some Java object as a parameter and then return Java object just as usual Java methods?
For example something like:
String[] param = new String[]{"John Doe"};
String talendJobOutput = teaPot.myjob_0_1.myJob.main(param);
where teaPot.myjob_0_1.myJob is the talend job integrated into my app
I did something similar I guess. I created a mapping in tallend using tMap and exported this as talend job (java se programm). If you include the libraries of that job, you can run the talend job as described by others.
To pass arbitrary java objects you can use the following methods which are present in every talend job:
public Object getValueObject() {
return this.valueObject;
}
public void setValueObject(Object valueObject) {
this.valueObject = valueObject;
}
In your job you have to cast this object. e.g. you can put in a List of HashMaps and use Java reflection to populate rows. Use tJavaFlex or a custom component for that.
Using this method I can adjust the mapping of my data visually in Talend, but still use the generated code as library in my java application.
Now I better understand your willing, I think this is NOT possible because Talend's architecture is made like a standalone app, with a "main" entry point merely as does the Java main() method :
public String[][] runJob(String[] args) {
int exitCode = runJobInTOS(args);
String[][] bufferValue = new String[][] { { Integer.toString(exitCode) } };
return bufferValue;
}
That is to say : the Talend execution entry point only accepts a String array as input and doesn't returns anything as output (except as a system return code).
So, you won't be able link to Talend (generated) code as a library but as an isolated tool that you can only parameterize (using context vars, see my other response) before launching.
You can see that in Talend help center or forum the only integration described is as an "external" job execution ... :
Talend knowledge base "Calling a Talend Job from an external Java application" article
Talend Community Forum "Java Object to Talend" topic
May be you have to rethink the architecture of your application if you want to use Talend as the ETL tool for your purpose.
Now from Talend ETL point of view : if you want to parameter the execution environment of your Jobs (for exemple the physical directory of the uploaded files), you should use context variables that can be loaded at execution time from a configuration file as mentioned here :
https://help.talend.com/display/TalendOpenStudioforDataIntegrationUserGuide53EN/2.6.6+Context+settings
I am currently using hibernate-sqlite.
To access dynamically created sqlite databases, this works perfectly. Now I want to secure the sqlite files. After some research I found out the way to go is (AES) encryption. Can anyone explain to me if this is possible using hibernate? And If so, how? If this doesn't work, is there any other solution for securing the data in the files?
You know about SQLite's Encryption Extension, right?
You can use the built-in encryption of the sqlite (System.Data.SQLite). See more details at http://sqlite.phxsoftware.com/forums/t/130.aspx
You can also Use SQLCipher, it's an opensource extension for SQLite that provides transparent 256-bit AES encryption of database files. http://sqlcipher.net
and as you need hibernate
you can use FluentNHibernate, you can use following configuration code:
private ISessionFactory createSessionFactory()
{
return Fluently.Configure()
.Database(SQLiteConfiguration.Standard.UsingFileWithPassword(filename, password))
.Mappings(m => m.FluentMappings.AddFromAssemblyOf<DBManager>())
.ExposeConfiguration(this.buildSchema)
.BuildSessionFactory();
}
private void buildSchema(Configuration config)
{
if (filename_not_exists == true)
{
new SchemaExport(config).Create(false, true);
}
}
Method UsingFileWithPassword(filename, password) encrypts a database file and sets password.
It runs only if the new database file is created. The old one not encrypted fails when is opened with this method.
EDIT :
more options for you
SEE - The official implementation.
wxSQLite - A wxWidgets style c++ wrapper that also implements SQLite's encryption.
SQLCipher - Uses openSSL's libcrypto to implement.
SQLiteCrypt - Custom implementation, modified API.
botansqlite3 - botansqlite3 is an encryption codec for SQLite3 that can use any algorithms in Botan for encryption.
The SEE and SQLiteCrypt require the purchase of a license.