Performance wise, is it smart to do this?
An example would be with Gson
Gson gson = new Gson();
String json = gson.toJson(myObject);
// store json string in sql with primary key
// retrieve json string in sql with primary key
I want to simplify the way i store and retrieve objects, instead of building and separating them into pieces and individual columns each time i store/retrieve from a database.
But my concern with using a json string is that the length may impact performance when the database fills up? Im not sure, this is why im asking.
There is not an issue with 'space' used or performance of such: MySQL will deal with that just fine.
That is, while the entire JSON chunk must be pulled/pushed for processing and changes MySQL will continue to handle such as best it ever did, even 'as the database fills up'.
However, there are problems with normalization and opaqueness of information when following this design. Databases are about information, but a blob of JSON is just .. a blob of JSON to an SQL database.
Because of this none of the data in the JSON can be used for relationships or queries nor can it participate in indices or constraints. So much for the "relational" part of the database..
Unless the JSON truly is opaque data, like the contents of a binary file, consider working on relevant normalization .. or switch to a (JSON) document-oriented database (eg. Raven, Redis, Couch, Mongo).
There is no space or performance issues with storing JSON strings. MySQL is capable to handle large blobs, if you need so.
The decision whether to store your data serialized into JSON or not should be based on how do you need to process this data. Relational databases, such as MySQL, suppose that you normalize the data and establish relationships between records. That said, in many cases it can be practical to do otherwise (i.e. store as JSON).
If you will store your data as JSON strings, you will not be able to effectively process this data using MySQL features, e.g. you cannot filter out or sort records by values stored as JSON. However, if you need only to store this data, while the processing is going to be done by the application code, it can be reasonable to use JSON.
As document-oriented databases like MongoDB become more popular, some of the traditional relational databases, such as PostgreSQL and MariaDB, recently also implemented native JSON support.
Related
I am trying to get value like key and value pair but i am doing it from json file and now there is another approach suggested lets do it from db tables. because if in future value change then only update the DB is Needed.
I think using json file is more good as value hardly going to change in future(rarest of rare).. although advantage of db approach is just change the db value and done...
So My point is json will be faster then DB and Using Json will reduce load on DB..as clicking UI it invoke extra call of DB..
What do you Think .. Please let me know..
This very much depends on how you are going to use these data.
Do you need to update it often?
Do you need to update by just one specific field?
Do you need to fetch records based on some specific field?
Do you need to fetch whole json or just some specific fields?
Do some parts of json reference any other tables?
Also, consider the size of those data, e.g. if the json files together may become more in size than the whole other tables, you may break db cache. From the other hand, you can always create separate database for your json files if you still need some relational database features.
So, I would anyway start with answering first 5 questions.
I have using maria db to store dynamic columns.In sql Query it is there in the maria db documentation. https://mariadb.com/kb/en/library/dynamic-columns/ .
But i couldn't find a spring boot jpa implementation.I have tried jpa native query.I am storing a Json in this dynamic column which is data type of Blob in maria db.But it is very tough because i couldn't find a way to store when json is nesting another objects or arrays.Is there any way to accomplish this task
JSON is just a string. It could be stored in a TEXT or BLOB column.
The issue is that it is a structured string and it is tempting to reach into it to inspect/modify fields. Don't. It won't be efficient.
Copy values out of the JSON that you need for WHERE or ORDER BY (etc) so that MySQL/MariaDB can efficiently access them. Sure, keep copies inside the JSON text/blob, if you like.
Please describe your use for JSON, there may be more tips on "proper" usage inside a database.
That being said, MariaDB 10.2 has picked up much of Oracle's JSON. But there is not a JSON datatype. And there are a few function differences.
And this does not explain how far behind the 3rd party software (spring-boot, etc) is.
I have here a bunch of XML-files which I like to store in a Cassandra database. Is there any possiblity out there to manage that or do I have to parse and reform the XML-files?
You can certainly store them as a blob or text but you will not be able to query the individual fields within the XML files. One other thing you'd want to be cautious of is payload size and partition size. Cassandra in general isn't really designed as an object store but depending on payload size and desired query functionality, you may either have to parse/chunk them out or look for an alternative solution.
I need to save permanently a big vocabulary and associate to each word some information (and use it to search words efficiently).
Is it better to store it in a DB (in a simply table and let the DBMS make the work of structuring data based on the key) or is it better to create a
trie data structure and then serialize it to a file and deserialize once the program is started, or maybe instead of serialization use a XML file?
Edit: the vocabulary would be in the order of 5 thousend to 10 thousend words in size, and for each word the metadata are structured in array of 10 Integer. The access to the word is very frequent (this is why I thought to trie data structure that have a search time ~O(1) instead of DB that use B-tree or something like that where the search is ~O(logn)).
p.s. using java.
Thanks!
using DB is better.
many companies are merged to DB, like the erp divalto was using serializations and now merged to DB to get performance
you have many choices between DBMS, if you want to see all data in one file the simple way is to use SQLITE. his advantage it not need any server DBMS running.
I am using Cassandra 1.2.2. I am finding it so easy to use Jackson to map my objects to and fro json and java for storing in database. I am actually tempted to do this to all of my data. My question is, Is this a good idea? What are the disadvantages of doing this to my application. My first guess is probably more processing overheads but is the juice worth the squeeze? and are there any other disadvantages that i need to know about?
One disadvantage is that to modify the data you have to read in the original, deserialize, make your change, serialize and write out the whole object. In Cassandra, writes are much more efficient than reads so it is beneficial to avoid reads before writes if possible.
The alternative is to use separate columns for each field in your JSON. You can use composite columns for multi-dimensional data.
So if you had the data:
{
name: "fred"
address: "some town"
age: 42
}
and you wanted to change the address, if you had these as separate Cassandra columns you'd just insert a column called address. If you had the JSON serialized you'd have to do much more work. This doesn't apply if your data is write-once.
Even if your data is write-once, if you just wanted to read one field from the data you can just read that column if stored separately rather than reading the whole thing and deserializing. This only applies if you want to read parts of your data.
In conclusion, there could be significant performance advantages to using separate columns if you have to update your data or if you only want to read parts at once.