I have the following problem for which I’m trying to find an elegant solution.
The front-end shows a search input field. The user is allowed to enter any string. I want to send a request to the backend and get all entities which contain the entered value in any column. It is unknown in which table and which column the entered string can be found.
Example:
Entities: Human, Building, Planet
All of them have 15+ attributes.
Now, the user enters the number 12 into the search field. I want the backend to find any entries in the database which contain the number 12 which could be a person’s shoe size or the street number the building is in and so on.
As technologies for the backend, I’m using JPA (Hibernate) and a Postgres database.
I’d like to implement that as generic as possible so I don’t have to modify anything if new tables or new attributes come along.
What I already tried is getting all table names like that:
"select col.table_name from information_schema.columns col join information_schema.tables tab on tab.table_schema = col.table_schema and tab.table_name = col.table_name and tab.table_type = 'BASE TABLE' where col.table_schema not in ('information_schema', 'pg_catalog')"
+ "group by col.table_name"
Since jpa querys use the Java class name instead of the table name, it’s no use tough. Same problem is for attributes. Using native querys to avoid this problem would make things even more complicated.
Is there any good solution to search the whole database for a specific value?
Related
I'm looking to have a GUI where when I click an Invoice it displays the information from both Customer and Product also, such as name, brand etc all in one row.
Do I have to put Name, brand, etc into Invoice too and inner join everything?
Invoice Table Customer Table Product Table
EDIT:
No, no need to modify the tables you're referring to. They all contain a unique primary key column which are referenced from the invoice table. Based on them the INNER JOIN can be formulated.
Maybe also worth mentioning: Don't confuse the INNER JOIN with the SELF JOIN which also exists.
The difference is that the INNER JOIN is still joining two different tables based on specific columns (e.g. id) whereby the SELF JOIN is joining a single table with itself.
Yes what you'll need is the INNER JOIN combining the information from your invoice table with the one from the customer table as well as the product table - all based on your given invoice id (column: idInvoice).
To obtain the needed information you don't need to add - and therefore repeat - it in the invoice table. Due to the join they'll be available for selection in one single query.
Your query should look like:
SELECT *
FROM invoice inv, customer cust, product prod
WHERE
inv.idCustomer = cust.idCostumer
AND
inv.idProduct = prod.idProduct
AND
inv.idInvoice = ${theIdOfTheInvoiceTheUserClickedOn}
Note: If you don't need all the information (columns) from the three tables (what the "*" stands for) you can replace the "*" with an enumeration explicitly stating only the columns you want to show. E.g. inv.id, cust.FirstName, cust.LastName.
Depending on the database technology/ dialect you're exactly using. The example above would be suitable for an Oracle database and should also suite most other databases, since only basic SQL features are being used in the query.
I'm assuming you're not using any ORM framework like Hibernate and that you'll need to write the query yourself (since you didn't provide any more detail on your technology stack). In case youre using an ORM framework the approach would need to look different, but the final generated query should look similar.
In the query above the first two conditions in the WHERE clause are forming the INNER JOIN implicitly, whereby the last third one is specifying which exact entry you're looking for.
Although you've asked only if an INNER JOIN is needed, I've provided the query here to you since your question implied you're not sure how to write one.
You might take it as an working example you can compare your solution with. You should try to understand how it's working and how it can be written and also research more on the SQL basics so that you can write it on your own as well.
Tip: PreparedStatements are the way to go to execute such queries to a database from Java in a safe way
In my opinion, based on your application, you can use a flat table that includes what you need and doesn't need to join tables. This solution is applicable when you are in a situation that you have small datasets (E.g. in banking, relationships between Terminal table and ATMTerminal, POSTerminal and CashlessTerminal tables).
And for situations that you have a relationship that one side is static data (like the above example) and another side is transactional data (like Banking Transactions), you should use the foreign key from the transaction table to the static data table.
I recommend you to use the second solution for your problem and handle relationships using foreign keys and get the columns you need using the join strategy.
I am using reactjs as a front-end and spring boot as back-end. I am using spring data jpa.
Here is my dilemma.. and I know the way I have written the logic is inefficient and would like to know and understand a better approach to address the issue.
Lets say there are thousands of records in the db and I need to retrieve them constantly and instantly (if possible)
I have 2 fields in UI. One is String and the other is Integer.
Name (String) SomeKindOfId (int)
------------- -------------------
I have to filter the records only after 3 characters are entered in Name field and/or in SomeKindOfId field.
How can I achieve this efficiently?
currently, I have
findAllByNameAndSomeKindOfId("%"+name+"%","%"+String.valueOf(SomeKindOfId)+"%")
in service
and something like this in repository class
Query("Select u from SomeTable u where u.name = :name and CAST(u.someKindOfId As string) like :someKindOfId")
List<CdoDimStPortfolio> findAllByEntityStatusAndDealName(#Param("name") String name,
#Param("someKindOfId") String someKindOfId);
The reason I am using casting is because I have someKindOfId as int in database
The best way to make efficient database queries is to create indexes by the values you are going to search.
You could create a index "name" and "someKindOfId" and the search for those values would be immediate, because it's like in a book, if you search a chapter page by page you will take much longer than if you go to the index and search for it.
For example for postgresql:
https://www.postgresql.org/docs/9.1/sql-createindex.html
CREATE INDEX constructs an index on the specified column(s) of the specified table. Indexes are primarily used to enhance database performance
I'm dealing with up to a billion records in oracle and I really need efficiency.
The first table is notification. I need to obtain the following data.
src_data_id | match_data_id
The second table is person_info. id is same as src_data_id and match_data_id from the notification table.
id | name
The third table is sample_info, in which self_object_id is the foreign key for person_info.
id | self_object_id
The forth table is sample_dna_gene where sample_id is same as id in sample_id.
sample_id | gene_info
I am writing a program in Java and I want to encalsulate a list of objects. Each object contains the name (from person_info) and gene_info (from gene_info).
Originally, I did it in 2 steps. I joined notification and person_info to obtain the ids. Then I joined person_info, sample_info and gene_info to obtain the names and their corresponding gene_info.
This would be fine for a smaller database, but dealing with up to a billion records, I need to worry about speed. I should not join the three tables like I did, but use simple sqls for each table, and join the pieces in Java instead.
It was easy to get ids from person_info with separate sqls, but I'm having trouble with obtaining their corresponding gene_info. I can get sample_info.id with a simple sql using in(id1,id2,id3...). I can then find gene_info with another simple sql using in(id1,id2,id3...).
I can obtain all these lists in java, but how do I put them together? I'm using spring and mybatis. Originally I could make one big messy sql and encapsulates all elements in the mapper. I'm not sure what to do now.
edit: The messy sql I have right now is
select to_char(sdg.gene_info), max(aa.pid), max(aa.sid), max(aa.id_card_no)
from (select max(pi.person_name),
max(pi.id) pid,
si.id sid,
max(pi.id_card_no),
max(pi.race)
from person_info pi
join sample_info si
on pi.id = si.self_object_id
group by si.id) aa
join sample_dna_gene sdg
on sdg.sample_id = aa.sid
group by to_char(sdg.gene_info)
where aa.pid in ('...')
It's a little more complicated than the orginal question. I need to group by id in sample_id first, then group by gene_info in sample_data_gene. I had to use a lot of max() so group by would work, and even then, I still could not get the gene_info group by to work properly. I'm not sure how ineffcient the max() is and how much it will slow down the query, but you can clearly see the point why I wanted to avoid such a messy sql now.
I had similar case. It was delt with 4 separate readers one for each table and merging was done on java side. Unfortunately prerequisite for that was sorting income streams on database side.
You read single record from stream one then you read records from stream 2 until key changes (as you sorted by that key and key is common for all tabs) then same for following streams. In my case that made sense as first table was very wide and next 3 had many rows for single key in table 1. If in your case there are no 1:n (where n is big) relations I don't see why such approach can be better than join.
I'm relatively new to working with JDBC and SQL. I have two tables, CustomerDetails and Cakes. I want to create a third table, called Transactions, which uses the 'Names' column from CustomerDetails, 'Description' column from Cakes, as well as two new columns of 'Cost' and 'Price'. I'm aware this is achievable through the use of relational databases, but I'm not exactly sure about how to go about it. One website I saw said this can be done using ResultSet, and another said using the metadata of the column. However, I have no idea how to go about either.
What you're probably looking to do is to create a 'SQL View' (to simplify - a virtual table), see this documentation
CREATE VIEW view_transactions AS
SELECT Name from customerdetails, Description from cakes... etc.
FROM customerdetails;
Or something along those lines
That way you can then query the View view_transactions for example as if it was a proper table.
Also why have you tagged this as mysql when you are using sqlite.
You should create the new table manually, i.e. outside of your program. Use the commandline 'client' sqlite3 for example.
If you need to, you can use the command .schema CustomerDetails in that tool to show the DDL ("metadata" if you want) of the table.
Then you can write your new CREATE TABLE Transactions (...) defining your new columns, plus those from the old tables as they're shown by the .schema command before.
Note that the .schema is only used here to show you the exact column definitions of the existing tables, so you can create matching columns in your new table. If you already know the present column definitions, because you created those tables yourself, you can of course skip that step.
Also note that SELECT Name from CUSTOMERDETAILS will always return the data from that table, but never the structure, i.e. the column definition. That data is useless when trying to derive a column definition from it.
If you really want/have to access the DB's metadata programatically, the documented way is to do so by querying the sqlite_master system table. See also SQLite Schema Information Metadata for example.
You should read up on the concept of data modelling and how relational databases can help you with it, then your transaction table might look just like this:
CREATE TABLE transactions (
id int not null primary key
, customer_id int not null references customerdetails( id )
, cake_id int not null references cakes( id )
, price numeric( 8, 2 ) not null
, quantity int not null
);
This way, you can ensure, that for each transaction (which is in this case would be just a single position of an invoice), the cake and customer exist.
And I agree with #hanno-binder, that it's not the best idea to create all this in plain JDBC.
Im using google app engine data-store built in eclipse using my model for the table. The id is just the date and time from android.
I can query by a row like this and it does work!
select from Quotes as Quotes ORDER BY votes DESC
I want to get my results back by my entities id however this query does not work
select from Quotes as Quotes ORDER BY Id DESC
Here is my table. How can I query by my id/Name and trust me ive tried
select from Quotes as Quotes ORDER BY ID/Name DESC
edit: you probably notice i have a dummyid. I do not want to use that row because I made it in a very hacky way and requires extra loading on the users side.
Oh, dear. I see the problem, now. You have a column named ID/Name. It's usually wise to keep identifiers limited to alphanumeric characters.
Can you rename the column? That would be the best step forward.
If that's not an option, you can wrap it in backticks so that it's treated as an identifier:
SELECT * FROM Quotes ORDER BY `ID/Name` DESC;
See SQL Fiddle, which almost certainly won't match your schema but should get the point across.
That Id/Name is the key field, imagine it is similar to primary key. to refer to that field in query, use
__key__
Example: select * from EntityTable where __key__ = Key('EntityTable', ....)
In your example, using date/time as key name is not really helpful, maybe you can find another info to be used as key.