It's been a few days that I'm struggling with WKB strings. I need to be able to parse it in order to get the equivalent geometry and extract points coordinates (X,Y,Z). I can't use PostGIS functions.
The only java library that I found was the JTS Topology Suite, that i use as follow :
String wkb = "01ea030000020000009b4d3899fe95154153d97e8f43875941000000000000454003085bc23f9615411b4dc406578759410000000000004740"
byte[] aux = WKBReader.hexToBytes(wkb);
try {
Geometry geom = new WKBReader().read(aux);
} catch (ParseException e) {
e.printStackTrace();
System.err.println("Bad WKB string.");
}
But the it gives me the following error :
com.vividsolutions.jts.io.ParseException: Unknown WKB type 234
234 is the decimal value of the hexadecimal string 'ea'. It's like the JTS Library was only looking at the frist 2 bytes instead of looking at the 4 bytes 'ea03', that correspond to 1002 in little endian (so a LineStringZ).
My question is then : does the JTS Topology Suite handle LineStringZ ? If not, why points can have a Z value ? And how can I parse it correctly ?
Thanks you for reading !
JTs only supports 2D geometries, it does not support a Z value. Points only have X and Y in it.
JTS does support 3D geometries at least in reading and writing. Support may be rather 2.5D than a real 3D but Z values are still carried on in the operations. The problem is that there are two ways for presenting XYZ, XYM, and XYZM geometries in WKB. JTS support the PostGIS EWKB variant as can be seem from the comment in the source code file https://sourceforge.net/p/jts-topo-suite/code/HEAD/tree/trunk/jts/java/src/com/vividsolutions/jts/io/WKBWriter.java
This implementation also supports the Extended WKB
standard. Extended WKB allows writing 3-dimensional coordinates
and including the geometry SRID value.
The presence of 3D coordinates is signified
by setting the high bit of the wkbType word.
The presence of an SRID is signified
by setting the third bit of the wkbType word.
EWKB format is upward compatible with the original SFS WKB format.
Your WKB is of the OGC variant which is defined in the OGC document http://portal.opengeospatial.org/files/?artifact_id=25355
JTS don't understand the four-digit geometry type codes. This mail thread gives some more information https://lists.osgeo.org/pipermail/geos-devel/2013-December/006757.html.
JTS Topology suite supports 3D data but in EWKB format. ISO WKB is not supported. If just so happens that you are using postgis, it supports EWKB. ST_GeomFromEWKB <-> ST_AsEWKB
Also if you are writing your EWKB with WKBWriter, don't forget to specify output dimmensions :
WKBWriter wkbw = new WKBWriter(3);
https://trac.osgeo.org/geos/ticket/707
https://postgis.net/docs/reference.html
Related
I need to I/O large images in TIFF format, so I need to use the BigTIFF format. I tried to use BioFormats (bioformats_package.jar) last version to read an image, using:
ImagePlus[] images = images = BF.openImagePlus(io) ;
Then, I wanted to access the Processor images[0].getProcessor() to modify/access the pixel values, but the result appears to not be any of the classical ImageJ processor: ByteProcessor, ColorProcessor, FloatProcessor, and ShortProcessor. I tested ALL of them using instanceof.
Any idea what type of Processor it could be?
Or how I would access the pixel values?
[EDIT] I still don't know what type of ImageProcessor the result is, but I found out that the saved pixels are encoded using Float, so 32 bits.
This is not ideal as I saved an image using Integer encoding, so there is a loss of precision when values are bigger than 20 millions.
According to the JavaDocs on ImagePlus, getProcessor() returns an ImageProcessor on which you can perform a convertToByteProcessor() to get a ByteProcessor, you have other converters on ImageProcessor for other subclasses.
I've download code from here https://github.com/vikrantlabde/iso8583-Java and after some modifications I'm parsing almost fine my fields....
I defined the schema like this:
ISOSCHEMA.put("1","BITMAP");
ISOSCHEMA.put("2","NUM-2-19-0_0");
ISOSCHEMA.put("3","NUMERIC-0-6-0_0");
ISOSCHEMA.put("4","NUMERIC-0-12-0_0");
ISOSCHEMA.put("7","NUMERIC-0-10-0_0");
ISOSCHEMA.put("11","NUMERIC-0-6-0_0");
ISOSCHEMA.put("12","NUMERIC-0-6-0_0");
ISOSCHEMA.put("13","NUMERIC-0-4-0_0");
ISOSCHEMA.put("22","NUMERIC-0-3-0_0");
ISOSCHEMA.put("23","NUMERIC-0-3-0_0");
ISOSCHEMA.put("35","NUM-2-37-0_0");
ISOSCHEMA.put("41","FCHAR-0-8-0_0");
ISOSCHEMA.put("49","FCHAR-0-3-0_0");
ISOSCHEMA.put("55","NUM-3-999-0_0");
The problem is the field 55 that is a binary field. The standard documentation says it:
55 Reserved ISO B 255 LLLVAR (ISO DOCUMENTATION)
I'm having an error parsing a string that has the bitmap turned on for the field 55.
I'm having from the output:
820200409F36020004950500000000009A031409039C01005F2A0209789F02060000000005009F03060000000000009F10201F430200200000000000000000045895000000000000000000000000000000009F260840D26C4BA5577CFB9F2701809F370443DD7E879F1A0202509F3303E0B0C8
But I expect:
820200409F36020004950500000000009A031409039C01005F2A0201249F02060000000005009F03060000000000009F10201F430200200000000000000000045895000000000000000000000000000000009F260840D26C4BA5577CFB9F2701809F370443DD7E879F1A0202509F3303E0B0C8
The length of the iso payload converted is highly different too...
The program output is:
303130307238060020C280C28200313636353433323131313232333334343535303030303030303030303030303030303031313031363138333432363030323339343133333433303130313630373130303133373635343332313131323233333434353564333131303232303030393238333030313031303238343031373430393132343233303832303230303430394633363032303030343935303530303030303030303030394130333134303930333943303130303546324130323039373839463032303630303030303030303035303039463033303630303030303030303030303039463130323031463433303230303230303030303030303030303030303030303034353839353030303030303030303030303030303030303030303030303030303030303030394632363038343044323643344241353537374346423946323730313830394633373034343344443745383739463141303230323530394633333033453042304338
What I expect is:
30313030723806002080820031363635343332313131323233333434353530303030303030303030303030303030303131303136313833343236303032333934313333343330313031363037313030313337363534333231313132323333343435353D33313130323230303039323833303031303130323834303137343039313234313135820200409F36020004950500000000009A031409039C01005F2A0201249F02060000000005009F03060000000000009F10201F430200200000000000000000045895000000000000000000000000000000009F260840D26C4BA5577CFB9F2701809F370443DD7E879F1A0202509F3303E0B0C8
One advice is:
I have to make the explicit conversion to hex from the resultant byte[] and viceversa.
It is:
String isoMessage = ISOUtil.hexString(packIsoMsg("0100", isofields).getBytes());
And:
unpackIsoMsg(new String(ISOUtil.hex2byte(isoMessage), "UTF-8"));
What about the definition of this kind of fields in this class? I'm really a newbie with the standard but I arrived here because jpos doesn't work in an Android environment. Also I'm confused with the last mentioned conversion to hex.
Any help is really appreciated...
Kind regards.
DE55 is defined as a Tag-Lag-Value (TLV) field that is not in the normal Binary / text / or numeric packed format you see the rest of ISO-8583 messages typically but is in ASN.1 BER-TLV / X.690-0207 format.
Unless you account for the BER-TLV you will not successfully unpack DE55 unless it is for non-EMV/Tokenization purposes. It threw me at first as well as I was thinking something more straight forward. Be aware that sometimes the field transport format is actually longer in this format than the original plain text or other binary data output so it is not the most efficient.
There are a couple of other fields depending on the ISO specification may also use BER-TLV but DE55 is the industry standard field to use BER-TLV for EMV functionality replacing DE55's previous use as a generic and rarely used 'fee field'.
The ISO-7816 specification its ISO-8583 use in detail for EMV and Tokenization, as well there are other references and quick guides out there if you are just looking for something not so in-depth. All volumes of the ISO-7816 specification can be found openly on the internet for free, or can be purchased directly (spendy) from the ISO organization if you want them in the plain ISO format.
I am not familiar with the specific JAVA Git you referenced but this one has a help page on how to use BER-TLV. Oracle also has a page on dealing with BER-TLV here. BinaryFoo has a Git available as well.
For the purposes of initial testing, if your data is just test data (DO NOT USE PRODUCTION DATA!) you can use http://www.emvlab.org/tlvutils/ to verify your results. Which when I input your inputs it kicks out your expected output.
what is the field defined by sender in field 55.Assign the same in unpacking.If they are sending string it should be LLLVAR.
when sending ISO message header must be in hex format.hence they convert bytes to hex.
From the looks of it, you consider 5F2A020978 as wrong whereas you expect 5F2A020124. The EMV tag's 5F2A data (with length 02) is the transaction currency code. This means your transaction is performed in EURO currency instead of Canadian dollar as you expect. You can find a currency code list here.
Hope this helps.
I have a string representation of a point in a given CRS (namely, EPSG:3035), such as "N10160E21266". I'd like to parse it using GeoTools to create a DirectPosition, or any another X/Y representation (in order to transform it from a CRS to another CRS; I have a MathTransform at hand).
Is there a parse method already implemented somewhere in GeoTools?
It's probably not difficult to create a custom one, but this may be error-prone and I'm not a GIS specialist (how to properly handle north/easting etc).
That is not a well known representation so there will be no standard parser. It shouldn't be too hard to write one that returns a DirectPosition or a JTS.Point.
The only geographic issue that you need to bear in mind is that epsg:3035 stores east as X and north as Y (which in this case is what you'd expect). You'll need to check the axis order of your target projection.
I am new to GIS area and I need to validate a geometry in WKT format in java, to check whether a simple polygon is a closed loop, i.e the start and end points of the vertices should be the same. I am currently using jGeometry class of oracle spatial(com.oracle.sdoapi), get the first and last vertices and comparing them. also, i am using getType() method to check whether it is a simple polygon or not. The following is the piece of code that am using:
WKT wkt = new WKT();
JGeometry geometry = wkt.toJGeometry(wkt.getBytes());
double[] d1 = geometry.getFirstPoint();
double[] d2 = geometry.getLastPoint();
if(!jGeometry.getType() == jGeometry.GTYPE_POLYGON){
//error message for other geometries
}
Is there any simple way of doing this or is there any API available? I dont want to reinvent the wheel, if it is already done and simple to use. Thanks!
The Java Topology Suite contains a WKTReader class that will suit your purposes. See http://tsusiatsoftware.net/jts/javadoc/com/vividsolutions/jts/io/WKTReader.html. You can use WKTReader to parse the WKT, and look for ParseExceptions, which indicate an invalid WKT.
If the WKT parses, you can then use the instanceof operator or WKTReader.getGeometryType() to determine the type of parsed Geometry class, and see if it's one of the Geometry types (Polygon or Multipolygon) with closed shells like Polygon or Multipolygon.
I have written a kernel density estimator in Java that takes input in the form of ESRI shapefiles and outputs a GeoTIFF image of the estimated surface. To test this module I need an example shapefile, and for whatever reason I have been told to retrieve one from the sample data included in R. Problem is that none of the sample data is a shapefile...
So I'm trying to use the shapefiles package's funciton convert.to.shapefile(4) to convert the bei dataset included in the spatstat package in R to a shapefile. Unfortunately this is proving to be harder than I thought. Does anyone have any experience in doing this? If you'd be so kind as to lend me a hand here I'd greatly appreciate it.
Thanks,
Ryan
References:
spatstat,
shapefiles
There are converter functions for Spatial objects in the spatstat and maptools packages that can be used for this. A shapefile consists of at least points (or lines or polygons) and attributes for each object.
library(spatstat)
library(sp)
library(maptools)
data(bei)
Coerce bei to a Spatial object, here just points without attributes since there are no "marks" on the ppp object.
spPoints <- as(bei, "SpatialPoints")
A shapefile requires at least one column of attribute data, so create a dummy.
dummyData <- data.frame(dummy = rep(0, npoints(bei)))
Using the SpatialPoints object and the dummy data, generate a SpatialPointsDataFrame.
spDF <- SpatialPointsDataFrame(spPoints, dummyData)
At this point you should definitely consider what the coordinate system used by bei is and whether you can represent that with a WKT CRS (well-known text coordinate reference system). You can assign that to the Spatial object as another argument to SpatialPointsDataFrame, or after create with proj4string(spDF) <- CRS("+proj=etc...") (but this is an entire problem all on its own that we could write pages on).
Load the rgdal package (this is the most general option as it supports many formats and uses the GDAL library, but may not be available because of system dependencies.
library(rgdal)
(Use writePolyShape in the maptools package if rgdal is not available).
The syntax is the object, then the "data source name" (here the current directory, this can be a full path to a .shp, or a folder), then the layer (for shapefiles the file name without the extension), and then the name of the output driver.
writeOGR(obj = spDF, dsn = ".", layer = "bei", driver = "ESRI Shapefile")
Note that the write would fail if the "bei.shp" already existed and so would have to be deleted first unlink("bei.shp").
List any files that start with "bei":
list.files(pattern = "^bei")
[1] "bei.dbf" "bei.shp" "bei.shx"
Note that there is no general "as.Spatial" converter for ppp objects, since decisions must be made as to whether this is a point patter with marks and so on - it might be interesting to try writing one, that reports on whether dummy data was required and so on.
See the following vignettes for further information and details on the differences between these data representations:
library(sp); vignette("sp")
library(spatstat); vignette("spatstat")
A general solution is:
convert the "ppp" or "owin" classed objects to appropriate classed objects from the sp package
use the writeOGR() function from package rgdal to write the Shapefile out
For example, if we consider the hamster data set from spatstat:
require(spatstat)
require(maptools)
require(sp)
require(rgdal)
data(hamster)
first convert this object to a SpatialPointsDataFrame object:
ham.sp <- as.SpatialPointsDataFrame.ppp(hamster)
This gives us a sp object to work from:
> str(ham.sp, max = 2)
Formal class 'SpatialPointsDataFrame' [package "sp"] with 5 slots
..# data :'data.frame': 303 obs. of 1 variable:
..# coords.nrs : num(0)
..# coords : num [1:303, 1:2] 6 10.8 25.8 26.8 32.5 ...
.. ..- attr(*, "dimnames")=List of 2
..# bbox : num [1:2, 1:2] 0 0 250 250
.. ..- attr(*, "dimnames")=List of 2
..# proj4string:Formal class 'CRS' [package "sp"] with 1 slots
This object has a single variable in the #data slot:
> head(ham.sp#data)
marks
1 dividing
2 dividing
3 dividing
4 dividing
5 dividing
6 dividing
So say we now want to write out this variable as an ESRI Shapefile, we use writeOGR()
writeOGR(ham.sp, "hamster", "marks", driver = "ESRI Shapefile")
This will create several marks.xxx files in directory hamster created in the current working directory. That set of files is the ShapeFile.
One of the reasons why I didn't do the above with the bei data set is that it doesn't contain any data and thus we can't coerce it to a SpatialPointsDataFrame object. There are data we could use, in bei.extra (loaded at same time as bei), but these extra data or on a regular grid. So we'd have to
convert bei.extra to a SpatialGridDataFrame object (say bei.spg)
convert bei to a SpatialPoints object (say bei.sp)
overlay() the bei.sp points on to the bei.spg grid, yielding values from the grid for each of the points in bei
that should give us a SpatialPointsDataFrame that can be written out using writeOGR() as above
As you see, that is a bit more involved just to give you a Shapefile. Will the hamster data example I show suffice? If not, I can hunt out my Bivand et al tomorrow and run through the steps for bei.