I am working on a small POC (which will be integrated into a bigger application) which consists of
Problem Context
reading an image in a simple java application program,
Convert image into a byte array byte[] imageEncodedBytes = baos.toByteArray()
storing it into a remote DB2 database (technology used was not imp, so I am using plain jdbc for now)
Reading it back from the DB and
converting it back to ensure that the re-creation of the image works. ( I am able to open the new re-created image in any image viewer)
Issues
The issues occur at step 5.
I read the image using a select query into a Result Set.
use the rs.getBlob("ColumnName") to get the blob value.
Fetch the byte array from the blob value using byte[] decodedArray = myBlob.getBytes(1, (int)myBlob.length())
Create the image from the obtained byte array.
At Step 3 the byte array decodedArray obtained from the blob differs from the byte array 'imageEncodedBytes' that I get when I read the image.
As a consequence, the following code to create the image from the byte array decodedArray fails.
ByteArrayInputStream bais = new ByteArrayInputStream(decodedArray);
//Writing to image
BufferedImage imag=ImageIO.read(bais); // Line of failure. No registered provider able to read bais
ImageIO.write(imag, "jpg", new File(dirName,"snap.jpg"));
References and Other data for issue investigation
I have referred the following links for verification
1. Inserting image in DB2
2. This Link here offers insight, but yet I was not able to determine, how to register the ImageReader.
4. When inserting the image to DB2 I am using - the following query
Statement st = conn.createStatement();
st.executeUpdate("INSERT INTO PHOTO (ID,PHOTO_NM,PHOTO_IM, THMBNL_IM) " + "VALUES (1,'blob("+bl+")',blob('"+bl+"')")
As an alternative to fetching blob value from the result set I have also used binaryStream = rs.getBinaryStream("PHOTO_IM") to get the binary stream and then get byte array from the binary stream. even in this case, the decodedArray is different from imageEncodedBytes
Please assist, I may be missing something extremely trivial here, but I am not able to figure out what. Any help/pointers will be greatly helpful. Thanks in advance.
The resolution is sort of a workaround that I have worked on.
I have used the jdbcTemplates to resolve the issue. The lobHandler Object of the jdbc template provides an easy way to manage the blobs.
Steps to resolve using the Spring Lob Handler
) Created a Data Source
) configured the Jdbc template to use the data source
) use the lobHandler code to execute the insert query
Code below
jdbcTemplate = new JdbcTemplate(dataSource);
jdbcTemplate.execute("INSERT INTO PHOTO (PHOTO_IM) VALUES (?)",
new AbstractLobCreatingPreparedStatementCallback(lobHandler) {
protected void setValues(PreparedStatement ps, LobCreator lobCreator) {
try {
lobCreator.setBlobAsBinaryStream(ps, 1, bean.getImageOrig(), bean.getImageLength());
} catch (java.sql.SQLException e) {
e.printStackTrace();
}
}
}
);
references
1. ) Stack Overflow Link - Spring JDBC Template Insert Blob
2. ) Stack Overflow Link - Close InputStream
3. ) Spring Doc for LobHandler
Related
I am creating Azure function using Java, My requirement I need to copy blob from one container to another container with encryption
so, for encrypting blob I am adding 4bites before and after the blob while uploading to sink container
now, I need to fetch blob content, for this I found one class in azure i.e,
#BlobInput(
name = "InputFileName",
dataType = "binary",
path = sourceContainerName+"/{InputFileName}")
byte[] content,
Here byte[] content, fetching content of blob
but I am facing some errors like, if I pass any file name as InputFileName parameter it is giving 200ok means returning successful. also it is difficult to mefor exception handling
so I am looking for other ways for fetching blob content.... please answer me if any methods or classes we have
If you are looking for more control, instead of using the bindings, you can use the Azure Storage SDK directly. Check out the quickstart doc for getting
setup.
This sample code has full end-to-end code that you could build upon. Here is the code that you are looking for in it for reference
String data = "Hello world!";
InputStream dataStream = new ByteArrayInputStream(data.getBytes(StandardCharsets.UTF_8));
/*
* Create the blob with string (plain text) content.
*/
blobClient.upload(dataStream, data.length());
dataStream.close();
/*
* Download the blob's content to output stream.
*/
int dataSize = (int) blobClient.getProperties().getBlobSize();
ByteArrayOutputStream outputStream = new ByteArrayOutputStream(dataSize);
blobClient.downloadStream(outputStream);
outputStream.close();
I have tried various different ways which i read from other questions but none of them seem working for me ...
I am using NetBeans 7.4 as front-end and Sqlite as Backend , iReport 5.5
I am making college ic-ards in which i want photo to be fetched from database ,In Database images are stored as blob , i am storing them as bytes..
My Requirement is getting student images from Database for ICard So i can print them..
Below is code i am using to insert image into Database
File image= new File(logPath2);
FileInputStream fis=new FileInputStream(image);
ByteArrayOutputStream bos=new ByteArrayOutputStream();
byte buf[]=new byte[1024];
for(int readNum;(readNum=fis.read(buf))!=-1; ) {
bos.write(buf,0,readNum);
}
person_image=bos.toByteArray();
I have already tried various image expressions and class expressions, but none seem working ..
Please note i am talking about sqlite, i know ways to do it by mysql but sqlite, its not rendering. But i don't wanna shift from sqlite to mysql.
Error i get is
Error filling print... Unable to get value for field 'Photo' of class 'java.io.InputStream' net.sf.jasperreports.engine.JRException: Unable to get value for field 'Photo' of class 'java.io.InputStream' at net.sf.jasperreports.engine.JRResultSetDataSource.getFieldValue(JRResultSetDataSource.java:319) at net.sf.jasperreports.engine.fill.JRFillDataset.setOldValues(JRFillDataset.java:1356) at net.sf.jasperreports.engine.fill.JRFillDataset.next(JRFillDataset.java:1257) at net.sf.jasperreports.engine.fill.JRFillDataset.next(JRFillDataset.java:1233) –
I am working around my project and got some Problem, I searched for it , but couldn't find meaning full learning resource, What I need is to Store Image in SQL Server Database using my Java program , and will need that Back to Retrieve, images are not of larger size they are ranging between 30 and 50 K,
I can load images from my Disk by using getImage() method in toolKit
Image imm = Toolkit.getDefaultToolkit().getImage("URl");
, but i don't know How to convert that image to Binary Format and to store in Database, and then retrieve Back From Database.
I want to Store that in VarBinary s by looking around several sites i found that the image type in SQL Server is soon going to Retire.
While it's not a good idea to store very large binary objects in the database, the Microsoft research paper "To BLOB or Not To BLOB: Large Object Storage in a Database or a Filesystem" indicates it's an efficient approach if object sizes are less than 256K. So it sounds like you've hit the sweet spot for database storage with 30K images.
You can load your image as a buffered image from a URL using:
BufferedImage imm = ImageIO.read(url);
Then convert it to a byte array with:
byte[] immAsBytes =
ByteArrayOutputStream baos = new ByteArrayOutputStream();
//use another encoding if JPG is innappropriate for you
ImageIO.write(imm, "jpg", baos );
baos.flush();
byte[] immAsBytes = baos.toByteArray();
baos.close();
Then store the byte array as recommended by this article as a VARBINARY in the database. In your JDBC code, you should wrap the byte array in a ByteArrayInputStream and set the PreparedStatement parameter as a BinaryStream:
PreparedStatement pstmt = commection.prepareStatement("INSERT INTO IMAGES (image) VALUES(?)");
ByteArrayInputStream bais = new ByteArrayInputStream(immAsBytes);
pstmt.setBinaryStream(1, bais, immAsBytes.length);
pstmt.executeUpdate();
pstmt.close();
To get the image out of the database, use ResultStatement.getBlob():
Blob immAsBlob = rs.getBlob();
byte[] immAsBytes = immAsBlob.getBytes(1, (int)immAsBlob.length()));
Finally, convert the byte array to a BufferedImage and do something with it:
InputStream in = new ByteArrayInputStream(immAsBytes);
BufferedImage imgFromDb = ImageIO.read(in);
It is generally not a very good idea to put into the database large resources like images, unless you really need ACID compliance when working with them. Database dumps and time required to produce them grow tremendously and you will need to use the more complex and fragile incremental backups.
Generally it may be more efficient to store the images on the filesystem, and only put the image path to the database after you are sure the image resource has been successfully created.
I have an sqback file respresenting an sqlite db file. I want to extract the data from this sqback file, ie, table names and contents, and convert it into csv file format. I want to do this using Java.
** The sqback file will have already been uploaded from android device to pc prior to processing. So I need a solution that is appropriate for taking place server side.
Does anyone have any leads on how to perform such a task?
If using Android you can take advantage of the built SQLiteDatabase and SQLiteOpenHelper. You'll find all the info you need here.
After parsing everything you can export to CSV the way you want by using File.
EDIT: So basically what you need to do is to parse the bytes by reading them and that way have access to the content. In some cases you don't even need to convert them to a String, since could be that you only need the value of the byte. (ie.: Offset: 44 Length:4 Description:The schema format number. Supported schema formats are 1, 2, 3, and 4.).
You can always check if your values are correct with any HEX editor, even opening the sqlite file with a text editor of any kind would help.
Let's start from scratch. First, reading the file. Two approaches
a. Read the whole file and parse it after
b. Read and parse the whole file in blocks (recommended, specially for bigger files)
Both approaches would share most of the following code:
File file = new File("YOUR FILE ROUTE");
int len = 1024 * 512; //512KB
try {
fis = new FileInputStream(file);
} catch (FileNotFoundException e1) {
fis = null;
e1.printStackTrace();
}
byte[] b = new byte[len];
int bread;
try {
while((bread = fis.read(b, 0, len))!=-1)
{
if(parseBlock(b, bread)==1)
return;
}
fis.close();
} catch (IOException e) {
e.printStackTrace();
}
The difference would be between getting partial blocks and parsing them on the fly (which I guess works for you) and getting the whole thing would be to just put:
fis.read(b, 0, fis.available);
Instead of the while loop.
Ultimately your approach is right, and that's the way to get bytes into a String. (new String(b)). Moreover the first characters are likely to represent weird symbols, if you have a look to the file format of SQL, these are reserved bytes for storing some metadata.
Open the sqlite file with any text editor and check that what you see there matches with what comes out of your code.
This website indicates which extra libraries to make use of, as well as provides examples of how to interact with the sqlite files (http://www.xerial.org/trac/Xerial/wiki/SQLiteJDBC#Usage)
important things to note:
1) make sure to include load the sqlite-JDBC driver using the current class loader. This is done with the line
2) the sqlite file IS a db, even if its not sitting on a server somewhere. So you still must create a connection to the file to interact with it. And you must open the connection and close the connection as well.
Connection connection = null;
connection = DriverManager.getConnection("jdbc:sqlite:" + path); // path is a String to the sqlite file (sqlite or sqback)
connection.close(); // after you are done with the file
3) Information can be extracted by using sql code to query the file. This returns a processable object of type ResultSet that holds your data pertaining to the query
Statement statement = connection.createStatement();
statement.setQueryTimeout(30);
ResultSet rs = statement.executeQuery("SELECT * FROM " + tblName);
4) from the ResultsSet you can grab data using the get commands with either the column index or the column header key
rs.getString("qo")
Hope that helps anyone having the same issue as I was having
I have been given a bit of a strange task, there are around 1500-2000 jpeg images all of around 1-50kb in size. They are currently stored in a simple database I made with Postgres. It's been a long time since I used Matlab and Postgres heavily so any help or suggestions is really appreciated!
I need to get the images that are stored in the database, out of the database into Java. The last step is retrieve the image from Java into Matlab so that the image is stored in the same way in which the imread function works in Matlab. The imread function reads an image in and creates a n by m by 3 matrix array of uint8 values which denote the pixel intensities of RGB.
Atm I have got the image in and out of the database in Java, currently storing the image in a bytea column data type. Is the best data type to use?
How can I get the data back out from the database, so that it is either the constructed jpeg image which I put in or is in the requested matrix array format?
Currently I do not understand the retrieved data. It is in a byte array of around 70,000 elements containing values between -128 to 128. Help!?!
Note: The database toolkit is unavailable to me
ANOTHER UPDATE: I have solved the problem related to the post regarding'UTF-8' encoding error.
If anyone stumbles upon this page, any answer posted will be tried as soon as I can! I really do appreciate your thoughts and answers. Thanks again.
When you say you have the image in a bytea column, how exactly is it stored? Is it storing the bytes of the JPEG file's contents, or an array of RGB pixel values, or something else? "Bytea" is just a binary string, which can store data in pretty much any format.
I'm assuming it's the JPEG contents. In that case what you can do is retrieve the jpeg contents via Java, save them to a temporary file, and call imread() on the temp file.
Those [-128,127] values are values of signed bytes Java. Even without the Database Toolbox, you can call regular JDBC or other Java code that uses it. The Java method you used to get those values - call it from Matlab (with your JAR on the classpath), and it should return that array as an int8 array, or something you can convert to one.
Given that in a Matlab variable named "bytes", you can write it out to a temp file with something like this.
file = [tempname() '.jpg'];
fid = fopen(file, 'wb');
fwrite(fid, bytes, 'int8');
fclose(fid);
By specifying 'int8' precision, I think you can skip the step of converting them to unsigned bytes, which is a more common convention. Writing int8s as 'int8' or uint8s as 'uint8' will produce the same file. If you do need to convert them to unsigned, use Matlab's typecast() function.
unsigned_bytes = typecast(bytes, 'uint8');
At that point you can call imread on the temp file and then delete it.
img = imread(file);
delete(file);
Problem solved :-)
I have managed to get the bytes stored within the bytea column within the database into a byte array. Then through creating a temp file ( using ByteArrayInputStream and Reader object to form a BufferedImage object which I write to a file) send this back into Matlab in an array.
Then process the data I retrieved and read from the temporary file in Matlab. Once the data is within Matlab all the temp files are deleted.
The code to process the ResultSet to create a temp image from a byte array received from the databases bytea column is shown below:
private static void processImageResultSet(ResultSet rs) throws SQLException, FileNotFoundException, IOException{
int i = 0; //used as a count and to name various temp files
while(rs.next()){ //loop through result sets
byte[] b = rs.getBytes(1); //the bytea column result
String location = getFileName(rs.getString(2)); //the name of the jpg file
ByteArrayInputStream bis = new ByteArrayInputStream(b); //creates stream storing byts
//To make individual names of temporary files unique the current time and date is stored
SimpleDateFormat df = new SimpleDateFormat("'Date 'yyyy-MM-dd HH'H'-mm'M'-ss'secs'-SS'ms'"); //formats date string
Calendar cal = Calendar.getInstance(); //gets instance of calendar time
String fileDate = df.format(cal.getTime()); //gets the time and date as a String
Iterator<?> readers = ImageIO.getImageReadersByFormatName("jpg"); //creates a reader object, that will read jpg codec compression format
Object source = bis; //object to store stream of bytes from database
ImageReader reader = (ImageReader) readers.next();
ImageInputStream iis = ImageIO.createImageInputStream(source); //creates image input stream from object source which stores byte stream
reader.setInput(iis, true); //sets the reader object to read image input stream
ImageReadParam param = reader.getDefaultReadParam();
Image image = reader.read(0, param);
BufferedImage bufferedImage = new BufferedImage(image.getWidth(null), image.getHeight(null), BufferedImage.TYPE_INT_RGB); //creates buffered image
Graphics2D g2 = bufferedImage.createGraphics();
g2.drawImage(image, null, null);
File imageFile = new File(location + " " + fileDate + " " + i + ".jpg"); //creates image file
ImageIO.write(bufferedImage, "jpg", imageFile); //writes buffered image object to created file
i++; //counts number of results from query within the ResultSet
}
}
Do you have access to the Database Toolbox in MATLAB? If so, you should be able to directly connect to a PostgreSQL database using the DATABASE function and then import and export data using the FETCH function or the QUERYBUILDER GUI. This may be easier than first going through Java.