I am trying to serialize hashmaps and collections of two custom classes (containing more hashmaps and collections).
Class1: NodeStorage.java
#NotNull
private final String id;
#Nullable
private String type;
#Nullable
private HashMap<String, String> properties;
Class2: RelationshipStorage.java
#NotNull
private final String id;
#Nullable
private String type;
#Nullable
private HashMap<String, String> properties;
#NotNull
private final NodeStorage startNode;
#NotNull
private final NodeStorage endNode;
The collections to serialize:
private HashMap<NodeStorage, NodeStorage> readsSetNode;
private HashMap<NodeStorage, NodeStorage> updateSetNode;
private ArrayList<NodeStorage> deleteSetNode;
private ArrayList<NodeStorage> createSetNode;
private HashMap<RelationshipStorage, RelationshipStorage> readsSetRelationship;
private HashMap<RelationshipStorage, RelationshipStorage> updateSetRelationship;
private ArrayList<RelationshipStorage> deleteSetRelationship;
private ArrayList<RelationshipStorage> createSetRelationship;
What I tried until now:
kryo.register(NodeStorage.class, 1);
kryo.register(RelationshipStorage.class, 2);
kryo.register(HashMap.class, mapSerializer);
mapSerializer.setKeyClass(NodeStorage.class, kryo.getSerializer(NodeStorage.class));
mapSerializer.setKeyClass(RelationshipStorage.class, kryo.getSerializer(RelationshipStorage.class));
mapSerializer.setValuesCanBeNull(false);
mapSerializer.setKeysCanBeNull(false);
listSerializer.setElementClass(NodeStorage.class, kryo.getSerializer(NodeStorage.class));
listSerializer.setElementClass(RelationshipStorage.class, kryo.getSerializer(RelationshipStorage.class));
listSerializer.setElementsCanBeNull(false);
public byte[] serialize()
{
ByteArrayOutputStream stream = new ByteArrayOutputStream();
Output output = new Output(stream);
mapSerializer.write(kryo, output, readsSetNode);
byte[] bytes = output.toBytes();
output.close();
return bytes;
}
I tried it with kryo.writeclassandobject but it didn't work as well. I get:
> > #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> # SIGSEGV (0xb) at pc=0x00007f92f7f6efe0, pid=4637, tid=0x00007f92f94fd700
> #
> # JRE version: OpenJDK Runtime Environment (8.0_102-b14) (build 1.8.0_102-b14)
> # Java VM: OpenJDK 64-Bit Server VM (25.102-b14 mixed mode linux-amd64 compressed oops)
> # Problematic frame:
> # V [libjvm.so+0x787fe0]
> #
> # Core dump written
Full code at: https://github.com/Raycoms/thesis
Declaration:
private Kryo kryo = new Kryo();
MapSerializer mapSerializer = new MapSerializer();
CollectionSerializer listSerializer = new CollectionSerializer();
Kryo is not thread-safe, and practically any Java program has multiple threads going. For example, your log file shows that you have 25 threads running at the time of the crash. Even though your Kryo instance is private, the error is down deep in the bowels of Kryo, and there may be thread or JVM interactions there that are not under your control.
Try the thread-pool approach, quoted below, detailed in the Kryo readme:
import com.esotericsoftware.kryo.Kryo;
import com.esotericsoftware.kryo.pool.*;
KryoFactory factory = new KryoFactory() {
public Kryo create () {
Kryo kryo = new Kryo();
// configure kryo instance, customize settings
return kryo;
}
};
// Build pool with SoftReferences enabled (optional)
KryoPool pool = new KryoPool.Builder(factory).softReferences().build();
Then, in your serialize() function:
Kryo kryo = pool.borrow();
<... the rest of your code in serialize() before the return...>
pool.release(kryo);
return bytes;
As the README points out, you can alternatively use a callback:
// or use a callback to work with kryo - no need to borrow/release,
// that's done by `run`.
String value = pool.run(new KryoCallback() {
public String execute(Kryo kryo) {
return kryo.readObject(input, String.class);
}
});
OP's final code
Posted here from the OP's comment so that others can read it more easily. The OP was not able to use mapSerializer in the end, but was able to serialize using this code:
private byte[] serialize() {
KryoPool pool = new KryoPool.Builder(factory).softReferences().build();
Kryo kryo = pool.borrow();
Output output = new Output(0, 1024);
kryo.writeClassAndObject(output, readsSetNode);
byte[] bytes = output.toBytes();
output.close();
pool.release(kryo);
return bytes;
}
Note As #MartinGrotzke pointed out, per this issue, if you use kryo.register(class, id), make sure id>=10.
Related
Using BouncyCastle library and the code below it is possible to access the MS Windows certificate template information extension of a SSL certificate stored in MS Windows' trust/certificate store.
final KeyStore keyStore = KeyStore.getInstance("Windows-My", "SunMSCAPI");
keyStore.load(null, null);
final X509Certificate certificate = (X509Certificate)keyStore.aliases().nextElement();
// see https://learn.microsoft.com/en-us/windows/win32/seccertenroll/supported-extensions#template
final String XCN_OID_CERTIFICATE_TEMPLATE = "1.3.6.1.4.1.311.21.7";
final byte[] extensionValue = certificate.getExtensionValue(XCN_OID_CERTIFICATE_TEMPLATE);
final ASN1InputStream aIn = new ASN1InputStream(extensionValue);
ASN1Primitive asn1obj = aIn.readObject();
if (asn1obj instanceof DEROctetString)
{
final DEROctetString octets = (DEROctetString) asn1obj;
asn1obj = ASN1Primitive.fromByteArray(octets.getOctets());
}
final ASN1Sequence asn1seq = ASN1Sequence.getInstance(asn1obj);
final ASN1Encodable obj1 = asn1seq.getObjectAt(0);
final ASN1Primitive certificateTemplateOID = obj1.toASN1Primitive();
System.out.println(certificateTemplateOID.toString());
It will print out something like 1.3.6.1.4.1.311.21.8.... which is the OID of the certificate template.
Now I would like to know if there is a Java method or Java library which provides a mapping of this OID to the certificate template name (as a user-friendly string) - like it is done when you click on details of a certificate in Windows certificiate store:
Certificate template information in MS Windows
Additional links:
Windows certificate extension for certificate templates
Windows certificate extension for certificate templates - data structure
ObjectIdentifiers (OID) in PKI
the only way to get mapping for this OID value is to call CryptEnumOIDInfo function on a domain-joined machine. These OIDs are generated dynamically and stored in Active Directory. This function can read OIDs from local OID database and from Active Directory. Note, that you must call this function in AD forest where that certificate template is installed.
17.02.2021 update:
I specified wrong function. Actual function is CryptFindOIDInfo. The function accepts three parameters with the following values (for your given case):
dwKeyType -- must be 1 (CRYPT_OID_INFO_OID_KEY)
*pvKey -- must be a pointer to an ANSI string.
dwGroupId -- must be 9 (CRYPT_TEMPLATE_OID_GROUP_ID)
the function returns a pointer to a CRYPT_OID_INFO structure. pwszName will store friendly name of the requested OID. In a given case, union value of CRYPT_OID_INFO structure is DWORD dwValue. I have no idea how easy this is in Java.
As another option, if you can access .NET API from Java, you can try instantiating an Oid class using Oid(String) constructor:
var oid = new System.Security.Cryptography.Oid("1.3.6.1.4.1.311.21.8...");
Console.WriteLine(oid.FriendlyName); // this will output template display name.
As pointed out by #Crypt32, the function CryptFindOIDInfo has to be called via Java NativeAccess (JNA):
Here is the code for JNA Version 4.1.0:
final NativeLibrary crypt32 = NativeLibrary.getInstance("Crypt32");
final int CRYPT_OID_INFO_OID_KEY = 1;
final int XCN_CRYPT_TEMPLATE_OID_GROUP_ID = 9;
final Function functionCryptFindOIDInfo = crypt32.getFunction("CryptFindOIDInfo");
final String ptrPvKey = certificateTemplateOID.toString();
final Object[] argsCryptFindOIDInfo = new Object[] { CRYPT_OID_INFO_OID_KEY, ptrPvKey,
XCN_CRYPT_TEMPLATE_OID_GROUP_ID };
final CRYPT_OID_INFO result = (CRYPT_OID_INFO) functionCryptFindOIDInfo.invoke(CRYPT_OID_INFO.class,
argsCryptFindOIDInfo);
System.out.println(result.pwszName);
class CRYPT_OID_INFO.java
public class CRYPT_OID_INFO extends Structure
{
public int cbSize;
public String pszOID;
public WString pwszName;
public int dwGroupId;
public DUMMYUNION DUMMYUNIONNAME;
public CRYPT_DATA_BLOB ExtraInfo;
public String pwszCNGAlgid;
public String pwszCNGExtraAlgid;
#Override
public List<String> getFieldOrder()
{
final List<String> result = new ArrayList<>();
result.add("cbSize");
result.add("pszOID");
result.add("pwszName");
result.add("dwGroupId");
result.add("DUMMYUNIONNAME");
result.add("ExtraInfo");
result.add("pwszCNGAlgid");
result.add("pwszCNGExtraAlgid");
return result;
}
}
class DUMMYUNION .java
public class DUMMYUNION extends Union
{
public int dwValue;
public int Algid;
public int dwLength;
}
class CRYPT_DATA_BLOB
public class CRYPT_DATA_BLOB extends Structure
{
public int cbData;
public Pointer pbData;
#Override
public List<String> getFieldOrder()
{
final String[] array = new String[] { "cbData", "pbData" };
return Arrays.asList(array);
}
}
I am trying to deserialize some Kafka messages that were serialized by Nifi, using Hortonworks Schema Registry
Processor used on the Nifi Side as RecordWritter: AvroRecordSetWriter
Schema write strategy: HWX COntent-Encoded Schema Reference
I am able to deserialize these messsages in other Nifi kafka consumer. However I am trying to deserialize them from my Flink application using Kafka code.
I have the following inside the Kafka deserializer Handler of my Flink Application:
final String SCHEMA_REGISTRY_CACHE_SIZE_KEY = SchemaRegistryClient.Configuration.CLASSLOADER_CACHE_SIZE.name();
final String SCHEMA_REGISTRY_CACHE_EXPIRY_INTERVAL_SECS_KEY = SchemaRegistryClient.Configuration.CLASSLOADER_CACHE_EXPIRY_INTERVAL_SECS.name();
final String SCHEMA_REGISTRY_SCHEMA_VERSION_CACHE_SIZE_KEY = SchemaRegistryClient.Configuration.SCHEMA_VERSION_CACHE_SIZE.name();
final String SCHEMA_REGISTRY_SCHEMA_VERSION_CACHE_EXPIRY_INTERVAL_SECS_KEY = SchemaRegistryClient.Configuration.SCHEMA_VERSION_CACHE_EXPIRY_INTERVAL_SECS.name();
final String SCHEMA_REGISTRY_URL_KEY = SchemaRegistryClient.Configuration.SCHEMA_REGISTRY_URL.name();
Properties schemaRegistryProperties = new Properties();
schemaRegistryProperties.put(SCHEMA_REGISTRY_CACHE_SIZE_KEY, 10L);
schemaRegistryProperties.put(SCHEMA_REGISTRY_CACHE_EXPIRY_INTERVAL_SECS_KEY, 5000L);
schemaRegistryProperties.put(SCHEMA_REGISTRY_SCHEMA_VERSION_CACHE_SIZE_KEY, 1000L);
schemaRegistryProperties.put(SCHEMA_REGISTRY_SCHEMA_VERSION_CACHE_EXPIRY_INTERVAL_SECS_KEY, 60 * 60 * 1000L);
schemaRegistryProperties.put(SCHEMA_REGISTRY_URL_KEY, "http://schema_registry_server:7788/api/v1");
return (Map<String, Object>) HWXSchemaRegistry.getInstance(schemaRegistryProperties).deserialize(message);
And here is the HWXSchemaRegistryCode to deserialize the message:
import com.hortonworks.registries.schemaregistry.avro.AvroSchemaProvider;
import com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient;
import com.hortonworks.registries.schemaregistry.errors.SchemaNotFoundException;
import com.hortonworks.registries.schemaregistry.serdes.avro.AvroSnapshotDeserializer;
public class HWXSchemaRegistry {
private SchemaRegistryClient client;
private Map<String,Object> config;
private AvroSnapshotDeserializer deserializer;
private static HWXSchemaRegistry hwxSRInstance = null;
public static HWXSchemaRegistry getInstance(Properties schemaRegistryConfig) {
if(hwxSRInstance == null)
hwxSRInstance = new HWXSchemaRegistry(schemaRegistryConfig);
return hwxSRInstance;
}
public Object deserialize(byte[] message) throws IOException {
Object o = hwxSRInstance.deserializer.deserialize(new ByteArrayInputStream(message), null);
return o;
}
private static Map<String,Object> properties2Map(Properties config) {
Enumeration<Object> keys = config.keys();
Map<String, Object> configMap = new HashMap<String,Object>();
while (keys.hasMoreElements()) {
Object key = (Object) keys.nextElement();
configMap.put(key.toString(), config.get(key));
}
return configMap;
}
private HWXSchemaRegistry(Properties schemaRegistryConfig) {
_log.debug("Init SchemaRegistry Client");
this.config = HWXSchemaRegistry.properties2Map(schemaRegistryConfig);
this.client = new SchemaRegistryClient(this.config);
this.deserializer = this.client.getDefaultDeserializer(AvroSchemaProvider.TYPE);
this.deserializer.init(this.config);
}
}
But I am getting a 404 HTTP Error code(schema not found). I think this is due to incompatible "protocols" between Nifi configuration and HWX Schema Registry Client implementation, so schema identifier bytes that the client is looking for does not exist on the server, or something like this.
Can someone help on this?
Thank you.
Caused by: javax.ws.rs.NotFoundException: HTTP 404 Not Found
at org.glassfish.jersey.client.JerseyInvocation.convertToException(JerseyInvocation.java:1069)
at org.glassfish.jersey.client.JerseyInvocation.translate(JerseyInvocation.java:866)
at org.glassfish.jersey.client.JerseyInvocation.lambda$invoke$1(JerseyInvocation.java:750)
at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
at org.glassfish.jersey.internal.Errors.process(Errors.java:205)
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:390)
at org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:748)
at org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:404)
at org.glassfish.jersey.client.JerseyInvocation$Builder.get(JerseyInvocation.java:300)
at com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient$14.run(SchemaRegistryClient.java:1054)
at com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient$14.run(SchemaRegistryClient.java:1051)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:360)
at com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient.getEntities(SchemaRegistryClient.java:1051)
at com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient.getAllVersions(SchemaRegistryClient.java:872)
at com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient.getAllVersions(SchemaRegistryClient.java:676)
at HWXSchemaRegistry.(HWXSchemaRegistry.java:56)
at HWXSchemaRegistry.getInstance(HWXSchemaRegistry.java:26)
at SchemaService.deserialize(SchemaService.java:70)
at SchemaService.deserialize(SchemaService.java:26)
at org.apache.flink.streaming.connectors.kafka.internals.KafkaDeserializationSchemaWrapper.deserialize(KafkaDeserializationSchemaWrapper.java:45)
at org.apache.flink.streaming.connectors.kafka.internal.KafkaFetcher.runFetchLoop(KafkaFetcher.java:140)
at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase.run(FlinkKafkaConsumerBase.java:712)
at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:93)
at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:57)
at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:97)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:302)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711)
at java.lang.Thread.run(Thread.java:745)
I found a workaround. Since I wasn't able to get this working. I take the first bytes of the byte array to make several calls to schema registry and get the avro schema to deserialize later the rest of the byte array.
First byte (0) is protocol version (I figured out this is a Nifi-specific byte, since I didn't need it).
Next 8 bytes are the schema Id
Next 4 bytes are the schema version
The rest of the bytes are the message itself:
import com.hortonworks.registries.schemaregistry.SchemaMetadataInfo;
import com.hortonworks.registries.schemaregistry.SchemaVersionInfo;
import com.hortonworks.registries.schemaregistry.SchemaVersionKey;
import com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient;
try(SchemaRegistryClient client = new SchemaRegistryClient(this.schemaRegistryConfig)) {
try {
Long schemaId = ByteBuffer.wrap(Arrays.copyOfRange(message, 1, 9)).getLong();
Integer schemaVersion = ByteBuffer.wrap(Arrays.copyOfRange(message, 9, 13)).getInt();
SchemaMetadataInfo schemaInfo = client.getSchemaMetadataInfo(schemaId);
String schemaName = schemaInfo.getSchemaMetadata().getName();
SchemaVersionInfo schemaVersionInfo = client.getSchemaVersionInfo(
new SchemaVersionKey(schemaName, schemaVersion));
String avroSchema = schemaVersionInfo.getSchemaText();
byte[] message= Arrays.copyOfRange(message, 13, message.length);
// Deserialize [...]
}
catch (Exception e)
{
throw new IOException(e.getMessage());
}
}
I also thought that maybe I had to remove the first byte before calling the hwxSRInstance.deserializer.deserialize in my question code, since this byte seems to be a Nifi specific byte to communicate between Nifi processors, but it didn't work.
Next step is to build a cache with the schema texts to avoid calling multiple times the schema registry API.
New info: I will extend my answer to include the avro deserialization part, since it was some troubleshooting for me and I had to inspect Nifi Avro Reader source code to figure out this part (I was getting not valid Avro data exception when trying to use the basic avro deserialization code):
import org.apache.avro.Schema;
import org.apache.avro.file.SeekableByteArrayInput;
import org.apache.avro.generic.GenericDatumReader;
import org.apache.avro.generic.GenericRecord;
import org.apache.avro.io.BinaryDecoder;
import org.apache.avro.io.DatumReader;
import org.apache.avro.io.DecoderFactory;
private static GenericRecord deserializeMessage(byte[] message, String schemaText) throws IOException {
InputStream in = new SeekableByteArrayInput(message);
Schema schema = new Schema.Parser().parse(schemaText);
DatumReader<GenericRecord> datumReader = new GenericDatumReader<>(schema);
BinaryDecoder decoder = DecoderFactory.get().binaryDecoder(in, null);
GenericRecord genericRecord = null;
genericRecord = datumReader.read(genericRecord, decoder);
in.close();
return genericRecord;
}
If you want to convert GenericRecord to map, note that String values are not Strings objects, you need to cast the Keys and values of types string:
private static Map<String, Object> avroGenericRecordToMap(GenericRecord record)
{
Map<String, Object> map = new HashMap<>();
record.getSchema().getFields().forEach(field ->
map.put(String.valueOf(field.name()), record.get(field.name())));
// Strings are maped to Utf8 class, so they need to be casted (all the keys of records and those values which are typed as string)
if(map.get("value").getClass() == org.apache.avro.util.Utf8.class)
map.put("value", String.valueOf(map.get("value")));
return map;
}
I am trying to interface with a C library libzbc using jnr-ffi.
The first function call zbc_open works and returns a pointer to a open device. Then the next call to zbc_get_device_info causes a JVM crash.
What is the cause? How to solve it?
I think the error is somewhere in the interface of zbc or my passing parameters but searching for any documentation for JNR yielded no helpful results in google.The crash happens too if i omit the array part in the structure.
I am using the project of jnr-fuse as a starting point because my goal is to write a FUSE filesystem.
Crash log at pastebin
C functions:
void zbc_set_log_level( char *log_level )
int zbc_open (const char *filename, int flags, struct zbc_device **dev)
void zbc_get_device_info( struct zbc_device *dev, struct zbc_device_info * info)
C structure zbc_device_info:
enum zbc_dev_type zbd_type
enum zbc_dev_model zbd_model
char zbd_vendor_id [ZBC_DEVICE_INFO_LENGTH]
uint32_t zbd_flags
uint64_t zbd_sectors
uint32_t zbd_lblock_size
uint64_t zbd_lblocks
uint32_t zbd_pblock_size
uint64_t zbd_pblocks
uint64_t zbd_max_rw_sectors
uint32_t zbd_opt_nr_open_seq_pref
uint32_t zbd_opt_nr_non_seq_write_seq_pref
uint32_t zbd_max_nr_open_seq_req
JNR interface:
public interface zbc{
public void zbc_set_log_level(String level);
public int zbc_open(String filename,int flags, PointerByReference p);
public void zbc_get_device_info(Pointer dev,zbc_device_info info);
}
public static class zbc_device_info extends Struct {
int zbd_type;
int zbd_model;
byte zbd_vendor_id[]=new byte[32];
u_int32_t zbd_flags;
u_int64_t zbd_sectors;
u_int32_t zbd_lblock_size;
u_int64_t zbd_lblocks;
u_int32_t zbd_pblock_size;
u_int64_t zbd_pblocks;
u_int64_t zbd_max_rw_sectors;
u_int32_t zbd_opt_nr_open_seq_pref;
u_int32_t zbd_opt_nr_non_seq_write_seq_pref;
u_int32_t zbd_max_nr_open_seq_pref;
protected zbc_device_info(Runtime runtime) {
super(runtime);
}
}
The main function:
public static void main(String[] args) {
zbc z = LibraryLoader.create(zbc.class).load("zbc");
Runtime runtime=Runtime.getRuntime(z);
z.zbc_set_log_level("debug");
PointerByReference pr = new PointerByReference();
int ret=z.zbc_open("/temp/test.img", IO_RDWR,pr);
zbc_device_info info=new zbc_device_info(runtime);
z.zbc_get_device_info(pr.getValue(),info);
}
JVM crash:
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007fa794300c70, pid=29106, tid=0x00007fa7bd4a4700
#
# JRE version: OpenJDK Runtime Environment (8.0_131-b11) (build 1.8.0_131-8u131-b11-2ubuntu1.16.04.3-b11)
# Java VM: OpenJDK 64-Bit Server VM (25.131-b11 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C [jffi6917022509387828651.so+0x5c70] jffi_releaseArrays+0x60
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /home/mmmm/jnr-fuse/hs_err_pid29106.log
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
The declaration of Struct is incorrect. The fields are not declared. Try to use the following declaration:
package com.github.goto1134.libzbc;
import jnr.ffi.Runtime;
import jnr.ffi.Struct;
import jnr.ffi.util.EnumMapper.IntegerEnum;
public class zbc_device_info
extends Struct {
public static final int ZBC_DEVICE_INFO_LENGTH = 32;
public final Enum<zbc_dev_type> zbd_type = new Enum<>(zbc_dev_type.class);
public final Enum<zbc_dev_model> zbd_model = new Enum<>(zbc_dev_model.class);
public final UTF8String zbd_vendor_id = new UTF8String(ZBC_DEVICE_INFO_LENGTH);
public final Unsigned32 zbd_flags = new Unsigned32();
public final Unsigned64 zbd_sectors = new Unsigned64();
public final Unsigned32 zbd_lblock_size = new Unsigned32();
public final Unsigned64 zbd_lblocks = new Unsigned64();
public final Unsigned32 zbd_pblock_size = new Unsigned32();
public final Unsigned64 zbd_pblocks = new Unsigned64();
public final Unsigned64 zbd_max_rw_sectors = new Unsigned64();
public final Unsigned32 zbd_opt_nr_open_seq_pref = new Unsigned32();
public final Unsigned32 zbd_opt_nr_non_seq_write_seq_pref = new Unsigned32();
public final Unsigned32 zbd_max_nr_open_seq_req = new Unsigned32();
protected zbc_device_info(Runtime runtime) {
super(runtime);
}
public enum zbc_dev_type
implements IntegerEnum {
/**
* Unknown drive type.
*/
ZBC_DT_UNKNOWN(0x00),
/**
* Zoned block device (for kernels supporting ZBC/ZAC).
*/
ZBC_DT_BLOCK(0x01),
/**
* SCSI device.
*/
ZBC_DT_SCSI(0x02),
/**
* ATA device.
*/
ZBC_DT_ATA(0x03),
/**
* Fake device (emulation mode).
*/
ZBC_DT_FAKE(0x04);
private final int value;
zbc_dev_type(int value) {
this.value = value;
}
#Override
public int intValue() {
return value;
}
}
public enum zbc_dev_model
implements IntegerEnum {
/**
* Unknown drive model.
*/
ZBC_DM_DRIVE_UNKNOWN(0x00),
/**
* Host-aware drive model: the device type/signature is 0x00
* and the ZONED field of the block device characteristics VPD
* page B1h is 01b.
*/
ZBC_DM_HOST_AWARE(0x01),
/**
* Host-managed drive model: the device type/signature is 0x14/0xabcd.
*/
ZBC_DM_HOST_MANAGED(0x02),
/**
* Drive-managed drive model: the device type/signature is 0x00
* and the ZONED field of the block device characteristics VPD
* page B1h is 10b.
*/
ZBC_DM_DEVICE_MANAGED(0x03),
/**
* Standard block device: the device type/signature is 0x00
* and the ZONED field of the block device characteristics VPD
* page B1h is 00b.
*/
ZBC_DM_STANDARD(0x04);
private final int value;
zbc_dev_model(int value) {
this.value = value;
}
#Override
public int intValue() {
return value;
}
}
}
In hs logs there is a call stack that made the program crash. Can you provide it?
I am serialising an already trained classifier on my Computer and I am trying to deserialize it on Android. However, when I try to deserialize it on my Android Device I see this log in the console:
W/System.err: java.io.InvalidClassException: weka.classifiers.meta.LogitBoost;
Incompatible class (SUID): weka.classifiers.meta.LogitBoost: static final long serialVersionUID =-1105660358715833753L;
but expected weka.classifiers.meta.LogitBoost: static final long
serialVersionUID =-3905660358715833753L;
W/System.err: at java.io.ObjectInputStream.verifyAndInit(ObjectInputStream.java:2336)
Not sure what to do. I have to mention that I am using a stripped version of WEKA available here.
This is my classifier class used to train and serialize the classifier on my computer:
public class ClassifierSerializer {
//variables//
public ClassifierSerializer(String classifierName,
String classifierParameters,
String dataSourceFile,
String outputFileName) {
this.classifierName = classifierName;
this.classifierParameters = classifierParameters;
this.dataSourceFile = dataSourceFile;
this.outputFileName = outputFileName;
}
/**
* Output the trained classifier to the specified file
*/
public void serialize() {
readFile();
train();
writeToFile();
}
private void train() {
classifier = AbstractClassifier.forName(classifierName, Utils.splitOptions(classifierParameters));
classifier.buildClassifier(dataset);
}
private void writeToFile() {
// Serialize classifier
FileOutputStream fileStream;
fileStream = new FileOutputStream(outputFileName);
ObjectOutputStream objectStream = new ObjectOutputStream(fileStream);
objectStream.writeObject(classifier);
objectStream.close();
fileStream.close();
}
}
This is how I am using it in my main method:
public static void main(String[] args) throws Exception {
String datasource = "C:\\Users\\Georgi\\Desktop\\HAR_training.arff";
ClassifierSerializer classifierSerializer = new ClassifierSerializer(
"weka.classifiers.meta.LogitBoost",
"-P 100 -L -1.7976931348623157E308 -H 1.0 -S 1 -I 10 -W weka.classifiers.trees.DecisionStump",
datasource,
"logit.data");
classifierSerializer.serialize();
}
And this is how I am deserializing it on Android (I have the fine in my assets folder):
private Classifier getClassifier() {
// Add logic to make a network call to download the trained offline model/classifier
String filename = "logit.data";
ObjectInputStream objectStream = new ObjectInputStream(getAssets().open(filename));
Object obj =objectStream.readObject();
if (obj instanceof Classifier) {
return (Classifier) obj;
} else {
return null;
}
}
I found out what was going on (kind of). I decided to replace the full weka.jar library in my InteliJ project on my PC with the same stripped version of the weka library that I am using on my Android Project. For some reason that solved the problem. I assume, the versions were a bit different and some change differences were causing the problem.
When you input a URL with this format into your browser:
steam://connect/[IP]
Steam automatically starts up and connects to said IP.
How can this be achieved in a Java program, or how can I reverse-engineer this?
This is not a Java question necessarily.
This is more of a "How do I configure a URI Protocol Handler" which is system and OS dependent.
In Windows, you would do something similar to this:
HKEY_CLASSES_ROOT
alert
(Default) = "URL:Alert Protocol"
URL Protocol = ""
DefaultIcon
(Default) = "alert.exe,1"
shell
open
command
(Default) = "C:\Program Files\Alert\alert.exe" "%1"
This will allow you to open a Java application similar to this:
java -jar MyJar.jar arg1 arg2
See more here from Windows MSDN.
To do this pragmatically you would need to use JNA to connect to a native library.
Although Java SDK does have a way to access the registry, it is declared private. You could use java.lang.Reflect to circumvent this and access these private methods, however it could break at any time.
public class RegEdit {
private static Method regAddKey = null;
private static Method regSetVal = null;
private static Method regDeleteKey = null;
private static Method regDeleteValue = null;
private static final Preferences userRoot = Preferences.userRoot();
private static final Preferences systemRoot = Preferences.systemRoot();
private static final Class<? extends Preferences> userClass = userRoot.getClass();
public RegEdit() {
try {
regAddKey = userClass.getDeclaredMethod("WindowsRegCreateKeyEx", new Class[]{int.class, byte[].class});
regAddKey.setAccessible(true);
regSetVal = userClass.getDeclaredMethod("WindowsRegSetValueEx", new Class[]{int.class, byte[].class, byte[].class});
regSetVal.setAccessible(true);
regDeleteValue = userClass.getDeclaredMethod("WindowsRegDeleteValue", new Class[]{int.class, byte[].class});
regDeleteValue.setAccessible(true);
regDeleteKey = userClass.getDeclaredMethod("WindowsRegDeleteKey", new Class[]{int.class, byte[].class});
regDeleteKey.setAccessible(true);
} catch (NoSuchMethodException | SecurityException e) {
}
}
}
Hope this solves your question slightly.