I want to store some image descriptors on a server so that when image matching on an android phone I can fetch the precomputed image descriptors rather then doing it on the fly. I have successfully created an application that can take an input image and output the best match but I am having some trouble with placing the image descriptor matrix into a JSON file.
Below I have placed some code that I have tried to adapt to perform the function I want but I run into an error with these lines:
mat.get(0, 0, data);
The error it gives is:
Mat data type is not compatible: 5
The descriptor matrix is of type CV_32FC1 but it treats it as CV_8SC1. Full code is underneath, the idea is that I will pass in descriptor matrix to matToJson and then store the output on the server and then use matFromJson to retrieve the contents of the JSON file. I also cannot resolve Base64.DEFAULT as it shows an error. Any help would be greatly appreciated.
public static String matToJson(Mat mat){
JsonObject obj = new JsonObject();
if(mat.isContinuous()){
int cols = mat.cols();
int rows = mat.rows();
int elemSize = (int) mat.elemSize();
byte[] data = new byte[cols * rows * elemSize];
mat.get(0, 0, data);
obj.addProperty("rows", mat.rows());
obj.addProperty("cols", mat.cols());
obj.addProperty("type", mat.type());
// We cannot set binary data to a json object, so:
// Encoding data byte array to Base64.
String dataString = new String(Base64.encode(data, Base64.DEFAULT)); //Error here as well .default does not exist
obj.addProperty("data", dataString);
Gson gson = new Gson();
String json = gson.toJson(obj);
return json;
} else {
System.out.println("Mat not continuous.");
}
return "{}";
}
public static Mat matFromJson(String json){
JsonParser parser = new JsonParser();
JsonObject JsonObject = parser.parse(json).getAsJsonObject();
int rows = JsonObject.get("rows").getAsInt();
int cols = JsonObject.get("cols").getAsInt();
int type = JsonObject.get("type").getAsInt();
String dataString = JsonObject.get("data").getAsString();
byte[] data = Base64.decode(dataString.getBytes(), Base64.DEFAULT);
Mat mat = new Mat(rows, cols, type);
mat.put(0, 0, data);
return mat;
}
Found a solution to the issue here, the problem was being caused by using the incorrect data type in the array. rather than using byte it should have been float. however the solution I have linked above is much better in that it checks the data type prior to encoding the data.
Related
In Java, is there a way to retrieve a piece of information from a JSON object by index? I am accessing a financial data table on quandl and want to return the most recent mortgage rate posted in the table. The key is the date the rate was posted in string form. The key for the most recent data will change weekly but the data I want will always be the first key-value pair in the table.
I was able to do this in JavaScript in about 5 minutes. But it seems more cumbersome in Java. Below is the iteration of my code that seems to be closest to getting me where I want to go. I am able to return the first key-value pair set in the table as an object in Java, which is ... ["2017-12-14",3.93]. The final step is eluding me. How do I grab the 3.93 and return that? Or is there a better way to go about this?
double baseRate = 0.0;
default double getBaseRate() throws MalformedURLException {
try {
// make a GET request
URL url = new URL("https://www.quandl.com/api/v3/datasets/FMAC/30US.json?api_key=-c-s9zf8s1NdLbhVin1p");
HttpURLConnection request = (HttpURLConnection) url.openConnection();
request.connect();
InputStreamReader is = new InputStreamReader((InputStream) request.getContent());
// Convert response stream to a JSON object
JsonReader reader = Json.createReader(is);
JsonObject obj = reader.readObject();
reader.close();
request.disconnect();
// Drill down to the desired piece of data
JsonObject dataset = obj.getJsonObject("dataset");
JsonArray data = dataset.getJsonArray("data");
Object currentData = data.get(0);
System.out.println(currentData);
} catch (IOException e) {
e.printStackTrace();
}
return baseRate;
}
I think that you need to go down another level of array to access the value you required.
JsonArray data = dataset.getJsonArray("data");
JsonArray firstPieceOfData = data.get(0);
Object firstRate = firstPieceOfData.get(1);
If you use Jackson for reading your JSON you can use the .at() method which allows you to access the node's value via a JSON Pointer Expression which in your case would be "/dataset/data/0/1"
I've truncated your json for the purpose of this demo:
String jsonString = "{\"dataset\":{\"id\":4644596,\"dataset_code\":\"30US\",\"database_code\":\"FMAC\",\"name\":\"30-Year Fixed Rate Mortgage Average in the United States\",\"description\":\"Units: Percent\\u003cbr\\u003e\\u003cbr\\u003eNot Seasonally Adjusted\",\"refreshed_at\":\"2017-12-18T04:09:32.892Z\",\"newest_available_date\":\"2017-12-14\",\"oldest_available_date\":\"1971-04-02\",\"column_names\":[\"Date\",\"Value\"],\"frequency\":\"weekly\",\"type\":\"Time Series\",\"premium\":false,\"limit\":null,\"transform\":null,\"column_index\":null,\"start_date\":\"1971-04-02\",\"end_date\":\"2017-12-14\",\"data\":[[\"2017-12-14\",3.93],[\"2017-12-07\",3.94],[\"2017-11-30\",3.9]],\"collapse\":null,\"order\":null,\"database_id\":582}}";
ObjectMapper mapper = new ObjectMapper();
JsonNode jsonNodes = mapper.createObjectNode();
try {
jsonNodes = mapper.readTree(jsonString);
} catch (IOException e) {
//e.printStackTrace();
}
System.out.println(jsonNodes.at("/dataset/data/0/1").asDouble());// 3.93
I want to send my object to couchdb using the ektorp java client. But I couldn't write my bytearray value to couchdb properly. My java object as follow:
If I convert bytearray to String:
The metadata value is saved on couchdb as "AgIGZm9vBmJhegA=" (base64), This means that "foobaz". Why is my bytearray value changed?
My example code :
private CouchDbInstance dbInstance;
private CouchDbConnector db;
...
Map<String, Object> doc = new HashMap<>();
doc.put("_id", "foo.com:http/");
byte[] serilazeData = IOUtils.serialize(writer, fieldValue);
doc.put("metadata", serilazeData);
...
db.update(doc);
My main code block
public void put(K key, T obj) {
final Map<String, Object> doc = new HashMap<>();
doc.put("_id", key.toString());
Schema schema = obj.getSchema();
List<Field> fields = schema.getFields();
for (int i = 0; i < fields.size(); i++) {
if (!obj.isDirty(i)) {
continue;
}
Field field = fields.get(i);
Schema.Type type = field.schema().getType();
Object fieldValue = obj.get(field.pos());
Schema fieldSchema = field.schema();
fieldValue = serializeFieldValue(fieldSchema, fieldValue);
doc.put(field.name(), fieldValue);
}
db.update(doc);
}
private Object serializeFieldValue(Schema fieldSchema, Object fieldValue ){
...
byte[] data = null;
try {
SpecificDatumWriter writer = getDatumWriter(fieldSchema);
data = IOUtils.serialize(writer, fieldValue);
} catch (IOException e) {
LOG.error(e.getMessage(), e);
}
fieldValue = data;
...
return fieldValue;
}
The value is the base64 encoded string "foobaz". You should probably post your code as well to get any meaningful feedback regarding this issue.
edit: Now that you provided code, is it possible, that the object you are trying to update already exists in the database? If yes you need to get it first or provide the proper existing revision id for the update. Otherwise the update will be rejected.
CouchDB stores JSON documents, and JSON does not support byte arrays, so I guess Ektorp is applying its own Base64 conversion during conversion of your object to JSON, before sending to CouchDB, and maybe that is skipping some characters in the byte array.
You might prefer to sidestep the Ektorp behaviour by applying your own Base64 serialisation before calling Ektorp, and then deserialise yourself after fetching the document from CouchDB. Or you could use something like Jackson, which will handle the object/JSON conversion behind the scenes, including byte arrays.
Ektorp uses Jackson for json serialization, I think jackson defaults to base64 for byte arrays. As long as you read/write with Ektorp you should not have any problems.
But I see in your code that you have some kind of type system of your own so that complicates things. I suggest you use POJOS instead of rolling your own since you won't get much help from ektorp and jackson if you are doing it yourself.
I had two doubts:
1. When i am trying to print The byte array i am getting from java into client side.I am getting strange value kind. I am not sure whether it's a byte array or pixel data.If it's not byte array then what correction needed to do.Code reference are pasted below
2.currently i am reading image from the database and writing into one byte array in server side and i am adding this byte array into json object and sending.But image is not getting displayed Please see the code below:
//this line will fetch the image data from db and i am storing into byte array
byte[] imageData = smpResource.getValue();
//i am adding this byte array into json object and sending.
JSONObject result = new JSONObject();
result.put("image", imageData);
//Client side code looks like:
var temp = null;
var jqxhr = jQuery.post(
'$link.getContextPath()/setEmployee.do',
{
empID: getSelectedEmpID(), empName: getSelectedEmpName()
},
function (data) {
jQuery.each(data, function(field, value){
// here in value i will be getting that byte array and i am storing in the below img src
if( "image" == field ) {
temp = value;
// please check the attachments and please confirm whether it was printing byte array or pixel data
alert("temp" + temp);
}
});
selectedImage.innerHTML = "<img src='data:image/jpg;base64, temp'/>";
,
"json"
).error( function(){
// if there was an error, select the parent...
selectedTreeNode.remove();
});
}
Might be it got little complicated to make you understand guyz but i tried my best.But let me know i will try in some other way.
To show in image with a data/base64 url, you need to encode it in Base64 format (see http://en.wikipedia.org/wiki/Base64). You could modify your backend to write your image into base64 format (see Java BufferedImage to PNG format Base64 String), as a string rather than an array. It's also way more compact for your JSON!
In my application, I want to change the raw byte data to the audio. So, at first I receive the data and add it to a byte array list and than I want to change the whole array list to the audio. As the data is raw through audio jack, I would appreciate if you let me know which audio format is suitable (wav,3gp and so on)? Also, I have a problem with saving this data in a file. This is my code. in line of fileOuputStream.write( audiod.toArray()), it give an error and asked me to change the type of audiod to byte.
private static ArrayList<Byte> audiod = new ArrayList<Byte>();
audioFilePath = Environment.getExternalStorageDirectory().getAbsolutePath() + "/myaudio.wav";
if (dg == 1) {
//audiod.append(data[i]);
for (int i = 0; data != null && i < data.length; i++) {
audiod.add(data[i]);
}
if (stt.equalsIgnoreCase(("r"))) {
dg = 0;
recordButton.setEnabled(true);
File file = new File(audioFilePath);
try {
FileOutputStream fileOuputStream = new FileOutputStream(audioFilePath);
fileOuputStream.write( audiod.toArray());
fileOuputStream.close();
System.out.println("Done");
}
catch(Exception e) {
e.printStackTrace();
}
}
}
audiod.toArray() will give you a Byte[], however you need a byte[]. In Java there are so called wrapper classes for the primitive data types. A Byte is an object that holds byte values.
Why do you use a list of Byte at all? It needs a lot more memory than a simple byte[] you seem to have anyhow (data).
The data format (wav,3gp and so on) depends on how you got your data.
I fail to understand why my code is giving me HDF5 Library Exceptions. It points at the createScalarDS method as the source of the error. But I believe this method does exist. Can anyone tell me why this code is unable to create an opaque dataset? What should the modification(s) be? Thanks.
public static void createFile(Message message) throws Exception {
// retrieve an instance of H5File
FileFormat fileFormat = FileFormat
.getFileFormat(FileFormat.FILE_TYPE_HDF5);
if (fileFormat == null) {
System.err.println("Cannot find HDF5 FileFormat.");
return;
}
// create a new file with a given file name.
H5File testFile = (H5File) fileFormat.create(fname);
if (testFile == null) {
System.err.println("Failed to create file:" + fname);
return;
}
// open the file and retrieve the root group
testFile.open();
Group root = (Group) ((javax.swing.tree.DefaultMutableTreeNode) testFile
.getRootNode()).getUserObject();
Group g1 = testFile.createGroup("byte arrays", root);
// obtaining the serialized object
byte[] b = serializer.serialize(message);
int len = b.length;
byte[] dset_data = new byte[len + 1];
// Initialize data.
int indx = 0;
for (int jndx = 0; jndx < len; jndx++)
dset_data[jndx] = b[jndx];
dset_data[len] = (byte) (indx);
// create opaque dataset ---- error here…
Datatype dtype = testFile.createDatatype(Datatype.CLASS_OPAQUE,
(len * 4), Datatype.NATIVE, Datatype.NATIVE);
Dataset dataset = testFile.createScalarDS("byte array", g1, dtype,
dims1D, null, null, 0, dset_data);// error shown in this line
// close file resource
testFile.close();
}
I don't have grip on HDF5.
but, you can not directly use CLASS_OPAQUE
An opaque data type is a user-defined data type that can be used in the same way as a built-in data type. To create an opaque type check links:
http://idlastro.gsfc.nasa.gov/idl_html_help/Opaque_Datatypes.html
To create array datatype object:
Result = H5T_ARRAY_CREATE(Datatype_id, Dimensions)
example:
http://idlastro.gsfc.nasa.gov/idl_html_help/H5F_CREATE.html