Develop Reference

Partial search through a SQL database efficiently

What are some examples of efficiently searching through a directory as you're typing a person's name?
Say for example, we have a database with 1 million users. We start typing in the search box: "sea", it will display every user's name on a scroll-able window that has "sea" on it (kind of like searching through a Skype directory). After changing a letter, the window should update immediately. All of this is coming from a SQL database. What are few efficient libraries, algorithms that can do this without much delay?

First consider changing the task from "name contains substring" to "name starts with substring". If this is possible, then add index on your name column in database table and use the query:
select name from table where name like :1 || '%'
Limit the number of returned rows using DBMS-specific syntax, for example, for Oracle add
and rownum < 20
This query should return your rows pretty fast.
If you really need "contains substring", then decide whether you want the search to be handled by database or by an external text indexing solution.
For database-contained solution you'll have to use a different approach depending on DBMS. Every one of these solutions requires configuration steps not described here.
For Oracle you can use Oracle Text, see
http://www.oracle.com/technetwork/documentation/index-098492.html
The query will look like
select name from table where contains(name, :1) > 0
For Postgres you can use Full Text Search.
You can also use a solution that is not dependent on the database, for example, see Apache Solr:
http://lucene.apache.org/solr/

for example
SELECT name
FROM Table
WHERE name LIKE '%sea%'

Selecting individual data_types across mutliple tables using SQL and HQL from Oracle

Hello World, Here is the situation,
Basically instead of generating values from a table like like so
34.324, 09/13/2011, thankyou,
I would like to generate the type of that specific value
e.g
VarChar2, Date, varchar2(char 30)
Just to add another layer of difficulty a Java application pulls data from the multiple Oracle database tables using HQL. Thus I would need an equivalent HQL statement to the SQL statement (if it exists)..
I understand the DESC keyword lists the column data types for an entire table however as mentioned above I require specificity.
In Summary
I would like to reverse engineer this application to generate a report of the data_types of the data instead of the actual data itself. I could easily just manually walk through the code however they are over 200 entries and this will be a real pain.
Any help is truly appreciated. If this question is unclear please let me know and I will provide more details and examples.

How can i fetch a row from a database table that is not in my EOModel?

I am using WOLips and Eclipse, I'm working on a project and i just need to know about some Entity that is in other project but i don't want to reference it (Build path).
I need to execute a sql statement and look if some record of this table exists, no more.
I'm trying with EOUtilities and other classes of the framework but i can't find a way to do it.
Please, excuse my english.
Any help will be apreciated.

I imagine you could use EOUtilities.rawRowsForSQL(). You need a model name, but presumably you have that—you shouldn't need to reference the model's containing project from where you're trying to do this.

You can get information about any table in the database, even if it does not appear in your model. The only reason you need the model is that one needs the connection dictionary in order to connect to the database. But you could have 0 entities in your EOModel and it will still work.
You just need to provide the SQL. For example, with MySQL, one uses "desc tables". For Oracle, one needs to use the longer command that I always have to look up that does the same thing. Then look at the resulting array of dictionaries. The table name will be tied to some key, depending on the database and version of the database and/or JDBC drives, but the table name will be there. For MySQL, the key is either "FIELD" or "NAME", I think.

Java MySQL Programming

I am accessing a MySQL table that has over 1 million or more Records. I am using My SQL query browser which is unable to grab all the records and it break the connection in the middle.
Now I have to write a Java Program which access that particular table without being broken in the middle as this table will be modified and accessed frequently.
Can you experts suggest me how should do I go over this problem
either I create an Index on the table and how do I create index

There are different reasons why a MySQL connection might break during a query. Can you give the exact error message you receive?
A simplified explanation on how to add an index to the table for a simple query
Look at the field(s) in the WHERE
clause of the query
Add an index on the field(s) using
ALTER TABLE ADD INDEX
Use EXPLAIN on the query and check
if the query is actually using the
index.
IF you want more specific help, Post the SHOW CREATE TABLE and the EXPLAIN of your query.

MySQL query browser limits the number of records to be displayed for performance reasons, because it is an interactive program and nobody like to wait for half an hour before the program crashes with an out-of-memory error. You can change these limits in the settings.
Your Java program will face similar problems.
When using large datasets it is important to plan how you are going to access that dataset and create the necessary indexes.
It would be useful to edit the question to show the structure of the data. Generqlly it looks like this :
CREATE INDEX idx_customer_name ON customer (name);
Here are more details
If you just want to dump the data to work on the data using Excel you can try this on the commandline
mysqldump -u [username] -p -t -T/path/to/directory [database] --fields-enclosed-by=\" --fields-terminated-by=,
In my experience this is a very painful exercise as Excel really is not made to deal with this amount of rows, and the dump format usually is slightly, but infuriatingly incompatible.
Your best bet is to invest an hour of your time to go through a SQL tutorial like sql fundamentals and play with MySQL query browser to get a feel of what you can do with SQL. I guarantee your investment paid itself back by tomorrow.

I am not very well used to MySQL programming, but generally indexes are used to arrange the values of one or more columns in a database table in specific order.
SYNTAX
CREATE INDEX IndexName ON tableName (column);
Just go through this tutorial for more information,
http://dev.mysql.com/doc/refman/5.0/en/create-index.html

Do I need to normalize this MySQL db?

I have a classifieds website which uses SOLR to search for whatever ads the user wants to search for... SOLR then returns the ID:s of all the matches found. I then use the ID:s to fetch and display the ads from a MySQL table.
currently I have one huge table containing everything in MySQL.
Sometimes some of the fields are empty because for instance an apartment has no "model" but a car does.
Is this a problem for me if I use SOLR like I do?
Thanks

Ask yourself these questions:
Is your current implementation slow or prone to error?
Are you adding a lot of "hacks" in order to display content or fetch data correctly due to the de-normalization of your database?
In the long run, will you benefit from normalizing the table?
Hope that helps. It all depends on your situation! Personally, I build databases normalized and then de-normalize as needed to keep things speedy.

If you are using SOLR, why don't you just serve complete ad from solr instead of MySQL to save DB time?
One huge table usually is not goog option at all.