I have written a application that parses the html code of some web pages. My problem is with inserting that data into my mysq database. So for example i want to insert ľščťžýáíé and when i look into the table i get ?š??žýáíé.
I guess the problem could be that the html pages i'm downloading are encoded in cp1250. but the database is utf8.
BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream(),"cp1250"));
and this is how i download the data.
Do you have some ideas how to fix this problem? Because i allready ran out.
Edit: oh and when i write the data out to the console (with System.out, i know i shouldn't use it... :) ) then every character is showing up correctly.
issue a set names CP1251; just after your connect to mysql and before any inserts
So i found out what works.
As i'm connecting to via JDBC to MySQL i used the following connection string
conString = "jdbc:mysql://"+host+"/"+database+"?useUnicode=true&characterEncoding=utf8";
And this did the trick. I had to force JDBC to use utf8 for the connection using ?useUnicode=true&characterEncoding=utf8
Related
I am using Apache CouchDB 1.6.1 as my database. We are creating quiz and saving the question data in couchdb. Now, there are users who want to create question in hindi. Data gets saved in couchdb easily but on retrieval of the data from couchdb its get converted into some absurd font. May be there is some issue with the font family and all I am unaware of all this.
We are using Java with Gwt in our project.
public String getData(){
Session session = new Session("192.168.1.70", 5984);
String hindiresult=null;
try{
Database test = session.getDatabase("test");
Document testdoc = test.getDocument("testdoc");
hindiresult=testdoc.getString("hindifield");
}catch(Exception e){
e.printStackTrace();
}
return hindiresult;
}
}
This is our server side code fetching data here and returning it to client on alert. The Image I have already shared.
You need to use utf-8 encoding, to support hindi.
I just changed the encoding of the Eclipse Ide suggested in the following question from default to utf-8. HELPFUL LINK
Good afternoon,
I'm trying resolve the classic encoding error in java, but I don't know what to do...
I try:
add on jsp: <%#page contentType="text/html"pageEncoding="UTF-8"%>
use "SQL_Latin1_General_CP1_CI_AS" no select(sqlserver)
add "CharacterSet=UTF-8" on String conection of jdbc
add response.setContentType("application/json"); and response.setCharacterEncoding("utf-8"); on servlet
but nothing works!!!!
SGBD: SQL Server
Server: GlassFish
Exemple record of database "Está"
what can I do?
Seems that you have jtds parameter sendStringParametersAsUnicode=false
One solution is to change it to true. If not then:
SQL_Latin1_General_CP1_CI_AS is CP-1252 (Windows-1252) encoding, so to search in database you need to encode your Unicode string to Windows-1252:
new String(value.getBytes("UTF-8"), "Windows-1252")
Vice versa after read from database:
new String(value.getBytes("Windows-1252"), "UTF-8")
I have a very specific requirement of storing PDF data in Hbase columns. The source of Data is Mongo DB, from where the base64 encoded data is read and I will need to bulk upload it to Hbase table.
I realized that in base64 encoded string there are a lot of "\n" character which splits the entire string into parts. Not sure if it is because of this, but when I store the string as it is, using a put :
put.add(Bytes.toBytes(ColFamilyName), Bytes.toBytes(columnName), Bytes.toBytes(data.replaceAll("\n","").toString()));
It is storing only the first line from the entire encoded string. Eg :
If the actual content was something like this :
"JVBERi0xLjQKJaqrrK0KNCAwIG9iago8PAovQ3JlYXRvciAoQXBhY2hlIEZPUCBWZXJzaW9uIDEu
" +
"MSkKL1Byb2R1Y2VyIChBcGFjaGUgRk9QIFZlcnNpb24gMS4xKQovQ3JlYXRpb25EYXRlIChEOjIw\n" +
"MTUwODIyMTIxMjM1KzAzJzAwJykKPj4KZW5kb2JqCjUgMCBvYmoKPDwKICAvTiAzCiAgL0xlbmd0\n" +
It is storing only the first line which is :
JVBERi0xLjQKJaqrrK0KNCAwIG9iago8PAovQ3JlYXRvciAoQXBhY2hlIEZPUCBWZXJzaW9uIDEu
in the column. Even after trying to remove the "\n" manually it is the same output.
Could someone please guide me in the right direction here ?
Currently, I am also working on Base64 encoding. As per my understanding, you should try using
org.apache.hadoop.hbase.util.Base64.encodeBytes(byte[] source, int option)
method where DONT_BREAK_LINES can be used as an option.
Please let me know if this works fine.
Managed to solve it. The issue was when reading the Base64 encoded data from MongoDB Source. Read the data from Mongo DB document DBObject as:
jsonObj.get("receiptContent").toString().replaceAll("\n","")
And stored it as such in Hbase. Even from the Hue HBase UI Browser I can see the PDF content now.
I am reading encoded data from database. When I decode data I get something like
%PDF-1.3
23 Obj
xref
10000000 123n
.
.
.
.
%EOF
I am guessing it's a metadata of PDF file with data. My question is how do I create PDF file out of this with readable data only.
Thanks in advance.
First you need to know what the data in DB represents. I would suggest to connect to the database using a general client. For example, for Oracle - SqlDeveloper, for MS Sql Server - Visual Sql Server, etc
Display the table which has a column of type long, clob or blob and try to save it to a file. Name the file with an extension .pdf. Try to open the saved file and see if it gets open correctly by a pdf reader.
If this is a case, saving it from Java is trivial. For example: http://www.astral-consultancy.co.uk/cgi-bin/hunbug/doco.cgi?11120
I need to insert about 700 records(name,id) into sqlite permanently ,because app will get user's name from the database.
I think ,reading text file is a solution but not know this is the best.
Can you show me other options to insert about 700 records into database?
thanks
The best practice to add multiple inserts into database shown in this video tutorial, you can watch it from 10.15
[Android Sqlite3 video tutoridal][inserting multiple values into database using fast way]
https://www.youtube.com/watch?v=dBnOn17pI7c&list=PLGLfVvz_LVvQUjiCc8lUT9aO0GsWA4uNe&index=14
U have Sqlite browser in order to view sqlite database.Insert data using the browser and u can permanently use that database.
Or try adding data to database using webservices.
It really depends on what you want to do and why you want to do it. That being said, text files can work. I had a similar case where I stored a few thousand items into an SQLite database. I used a text file and a CSVReader to parse the text file.
InputStream is = new ByteArrayInputStream(theContent.getBytes());
BufferedReader br = new BufferedReader(new InputStreamReader(is));
CSVReader<String[]> csvReader = new CSVReaderBuilder<String[]>(br).strategy(new CSVStrategy('\t', '\b', '#', true, true)).entryParser(new EntryParser()).build();
while ((nextLine = csvReader.readNext()) != null) {
// Do Parsing work and Store to SQLite Database
}
If you know the data won't change and want the fastest solution, then a text file is sufficient. If the data will change frequently, then you're probably going to want to access a web service to update your data. The speed of this method will be affected by the internet speed of the user.