Develop Reference

Unpack Proto3 Any in Java

I want to use a Protocol Buffer Message in a Java project, but I don't know what type will be used during runtime. It can be a simple datatype or another message object.
Example:
proto file:
...
message MyMessage {
repeated google.protobuf.Any params = 1;
}
...
to serialize my values, I use the following code (type of payload is unknown):
Any.Builder anyBuilder = Any.newBuilder();
ByteString byteString;
if (ClassUtils.isPrimitiveOrWrapper(payload.getClass())) {
byte[] serialize = SerializationUtils.serialize((Serializable) payload);
byteString = ByteString.copyFrom(serialize);
} else {
//serialize custom message
}
anyBuilder.setValue(byteString);
myMessage.addParams(Any.pack(anyBuilder.build()));
to deserialize my values, I use the following code:
List<Any> paramsList = myMessage.getParamsList();
Object[] objects = new Object[paramsList.size()];
int i = 0;
for (Any any : paramsList) {
ByteString value = any.getValue();
objects[i++] = SerializationUtils.deserialize(value.toByteArray());
}
The code throws org.apache.commons.lang3.SerializationException: java.io.EOFException
Can I use SerializationUtils#serialize for a independent communication (e.g. Java to Python; Java to C++) at all or do I have to use something else?
Can I use any in Proto3 for a simple datatype or is it restricted to proto messages?
Thanks!

Java. Getting URL from package

I'm making an application that captures the network trafic. I need to get the URL that the browser displays to the users and I thought I could use InetAddress to get the domain from the IP I had captured but It's not the same. For example, the domain I get from facebook.com is xx-fbcdn-shv-01-dft4.fbcdn.net and this is not always the same. Is there any other way to do it? Is it even possible?
I'm using JNetPCap library and the example code of it's page just to learn how to implement it in my proyect.
import java.net.InetAddress;
import java.net.UnknownHostException;
import java.util.ArrayList;
import java.util.Date;
import java.util.List;
import java.util.logging.Level;
import java.util.logging.Logger;
import org.jnetpcap.*;
import org.jnetpcap.packet.PcapPacket;
import org.jnetpcap.packet.PcapPacketHandler;
import org.jnetpcap.protocol.network.Ip4;
/**
*
* #author User
*/
public class Sniffer {
static private String device;
public static void main(String[] args) {
List<PcapIf> alldevs = new ArrayList<>(); // Will be filled with NICs
StringBuilder errbuf = new StringBuilder(); // For any error msgs
/**
* *************************************************************************
* First get a list of devices on this system
*************************************************************************
*/
int r = Pcap.findAllDevs(alldevs, errbuf);
if (r == Pcap.NOT_OK || alldevs.isEmpty()) {
System.err.printf("Can't read list of devices, error is %s", errbuf
.toString());
return;
}
System.out.println("Network devices found:");
int i = 0;
for (PcapIf device : alldevs) {
String description
= (device.getDescription() != null) ? device.getDescription()
: "No description available";
System.out.printf("#%d: %s [%s]\n", i++, device.getName(), description);
System.out.println(device.toString());
}
PcapIf device = alldevs.get(2);
/***************************************************************************
* Second we open up the selected device
**************************************************************************/
int snaplen = 64 * 1024; // Capture all packets, no trucation
int flags = Pcap.MODE_PROMISCUOUS; // capture all packets
int timeout = 10 * 1000; // 10 seconds in millis
Pcap pcap
= Pcap.openLive(device.getName(), snaplen, flags, timeout, errbuf);
if (pcap == null) {
System.err.printf("Error while opening device for capture: "
+ errbuf.toString());
return;
}
///*-------------------------------------------------------------------- ------------------------------
PcapBpfProgram program = new PcapBpfProgram();
String expression = "port 80";
int optimize = 0; // false
int netmask = 0xFFFFFF00; // 255.255.255.0
if (pcap.compile(program, expression, optimize, netmask) != Pcap.OK) {
System.err.println(pcap.getErr());
return;
}
if (pcap.setFilter(program) != Pcap.OK) {
System.err.println(pcap.getErr());
return;
}
///*
/**
* *************************************************************************
* Third we create a packet handler which will receive packets from the
* libpcap loop.
*************************************************************************
*/
PcapPacketHandler<String> jpacketHandler = (PcapPacket packet, String user) -> {
System.out.printf("Received packet at %s caplen=%-4d len=%-4d %s\n",
new Date(packet.getCaptureHeader().timestampInMillis()),
packet.getCaptureHeader().caplen(), // Length actually captured
packet.getCaptureHeader().wirelen(), // Original length
user // User supplied object
);
Ip4 ip = new Ip4();
//Tcp tcp= new Tcp();
byte[] sIP;
if (packet.hasHeader(ip)) {
try {
sIP = packet.getHeader(ip).source();
String sourceIP = org.jnetpcap.packet.format.FormatUtils.ip(sIP);
String myIp = InetAddress.getLocalHost().getHostAddress();
if (!myIp.equals(sourceIP)) {
System.out.println("source= "+sourceIP);
String domain;
domain = InetAddress.getByName(sourceIP).getHostName();
System.out.println("--------------------------"+domain);
}
} catch (UnknownHostException ex) {
Logger.getLogger(Sniffer.class.getName()).log(Level.SEVERE, null, ex);
}
}
};
/**
* *************************************************************************
* Fourth we enter the loop and tell it to capture 10 packets. The loop
* method does a mapping of pcap.datalink() DLT value to JProtocol ID,
* which is needed by JScanner. The scanner scans the packet buffer and
* decodes the headers. The mapping is done automatically, although a
* variation on the loop method exists that allows the programmer to
* sepecify exactly which protocol ID to use as the data link type for
* this pcap interface.
*************************************************************************
*/
pcap.loop(-1, jpacketHandler, "jNetPcap rocks!");
/**
* *************************************************************************
* Last thing to do is close the pcap handle
*************************************************************************
*/
pcap.close();
}
}

You can't. Big sites got several addresses, so Facebook.com resolves to many addresses but reverse lookup of those addresses does not resolve to Facebook.com.
If you can capture the conversation you can read http headers, there you can find the url you are interested in.

I found a way to do it reading the tcp packets and using it's destination port. Thanks you for your help minus, I wouldn't find the solution without your help. This is the handler method.
static String dominios[] = {"com", "org", "net", "info", "biz", "edu", "gob"};
PcapPacketHandler<String> jpacketHandler = (PcapPacket packet, String user) -> {
Tcp tcp= new Tcp();
Http http= new Http();
if (packet.hasHeader(tcp)) {
if (tcp.destination() == 443) {
int payloadstart = tcp.getOffset() + tcp.size();
JBuffer buffer = new JBuffer(64 * 1024);
buffer.peer(packet, payloadstart, packet.size() - payloadstart);
String payload = buffer.toHexdump(packet.size(), false, true, true);
for (String b : dominios) {
if (payload.contains(b)) {
procesarTcp(payload, b);
}
}
}
else if(packet.hasHeader(http)){
System.out.println(http.fieldValue(Http.Request.Host));
}
}
};
public static void procesarTcp(String payload, String dominio) {
payload= payload.substring(0,payload.indexOf(dominio)+dominio.length());
StringTokenizer token= new StringTokenizer(payload,"\n");
String pagina;
String aux;
payload="";
while(token.hasMoreTokens()){
aux=token.nextToken();
payload+=aux.substring(aux.indexOf(" ")+4, aux.length());
}
pagina= payload.substring(payload.lastIndexOf("..")+2,payload.length());
System.err.println(pagina);
}

Sending Several Queries in RxTx

I'm trying to send several queries at once to a device (car's ECU, specifically the ELM327) via RxTx. Here's the class that utilize RxTx:
import collection.JavaConversions._
import gnu.io._
import java.io._
import java.util.TooManyListenersException
class Serial private (portName: String,
baudRate: Int,
dataBits: Int,
stopBits: Int,
parity: Int,
flowControl: Int) {
private val portId = CommPortIdentifier.getPortIdentifier(portName)
private val serial = portId.open("Serial Connection from OBDScan",
5000).asInstanceOf[SerialPort]
setPortParameters(baudRate, dataBits, stopBits, parity, flowControl)
private val istream = serial.getInputStream
private val ostream = serial.getOutputStream
def this(portName: String, baudRate: Int = 115200) = this(portName, baudRate,
SerialPort.DATABITS_8,
SerialPort.STOPBITS_1,
SerialPort.PARITY_NONE,
SerialPort.FLOWCONTROL_NONE)
def close = {
try {
istream.close
ostream.close
} catch {
case ioe: IOException => // don't care, it's ended already
}
serial.close
}
def query(command: String) = {
ostream.write(command.getBytes)
ostream.write("\r\n".getBytes) // neccessary for Serial port.
}
def queryResult: String = {
try {
val availableBytes = istream.available
val buffer = new Array[Byte](availableBytes)
if (availableBytes > 0) {
istream.read(buffer, 0, availableBytes)
}
new String(buffer, 0, availableBytes)
} catch {
case ioe: IOException => "Something wrong! Please try again."
}
}
def addListener(listener: SerialPortEventListener) = {
try {
serial.addEventListener(listener)
serial.notifyOnDataAvailable(true)
} catch {
case tm: TooManyListenersException => println("Too many listener")
}
}
def removeListener = {
serial.removeEventListener
}
private def setPortParameters(br: Int,
db: Int,
sb: Int,
p: Int,
fc: Int) = {
serial.setSerialPortParams(baudRate, dataBits, stopBits, parity)
serial.setFlowControlMode(flowControl)
}
}
object Serial {
def connect(portName: String, baudRate: Int = 115200): Serial = {
try {
val ret = new Serial(portName, baudRate)
ret
} catch {
// exception handling omitted.
}
}
}
Now querying works fine and gives me the correct result. The problems comes when I sent several queries at once:
val serial = Serial.connect("COM34")
serial.query("AT Z")
serial.query("10 00")
The device recieved both of the queries, but returns only one result. If I want to get the next result, I have to send another query, which will results in the program get late by one query. If I call Thread.sleep after each query:
val serial = Serial.connect("COM34")
serial.query("AT Z")
Thread.sleep(500)
serial.query("10 00")
The problem's solved, but of course the entire application stops when Thread.sleep is called. I don't want this since the application will do this query all the time (which means it will hangs all the time if I do this).
Since I'm on Scala, I'm thinking on using Actors or something similar, but I think it's an overkill for a desktop application. Is there any way to do this without Actors? Maybe I'm reading the response from serial port wrong?
TL;DR: I want to do several queries to serial device via RxTx without locking the whole application (current solution achieved via Thread.sleep, which blocks the whole application). How can I do that?

how to check for word in text file store in amazon s3 cloud storage against user input?

How do i match words in text file that is store in amazon s3 cloud storage, against user input in search bar, and then download the specific file?

Your files in Amazon S3 can be made public and then accessed using HttpURLConnection, as in:
import java.io.*;
import java.net.HttpURLConnection;
import java.net.URL;
class Download
{public static void main (String args[])
{String u = "http://s3.amazonaws.com/Bucket/path/file.data";
String d = download(u);
System.err.println("Download "+u+"=\n"+d);
}
public static String download(String u)
{final int B = 1024*1024;
try
{final URL url = new URL(u);
final HttpURLConnection c = (HttpURLConnection)url.openConnection();
c.connect();
final int r = c.getResponseCode();
if (r != 200)
{System.err.println("Got response code "+r+", expected 200, cannot download "+u);
return null;
}
System.err.println("Content length="+c.getContentLength());
final InputStream i = new BufferedInputStream(url.openStream(), B);
final byte data[] = new byte[B];
String o = "";
for(int n = i.read(data); n != -1; n = i.read(data))
{o += new String(data, 0, n);
}
i.close();
return o;
}
catch (Exception e)
{System.err.println("Cannot download "+u+" due to "+e);
}
return null;
}
}
If you are unable to make these files public you will have to go the more complex route of using Amazon IAM to create a user, assign the user privileges, generate a key/secret combination and then use the Amazon AWS SDK for Java documented at http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/index.html. Please post back here if you need help with that route.

checkout cloudbash.sh - not just grep, any linux tools for your data on S3.

What is the root cause of the "USB Host support is unavailable" message?

I am trying to send data over a USB to serial cable using the jUSB library. I'm coding in the NetBeans IDE on Windows.
What is the problem behind the message: "USB Host support is unavailable" in the following code:
package usb.core;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.List;
import usb.core.*;
public class Main {
public static void main(String[] args) throws IOException {
try {
// Bootstrap by getting the USB Host from the HostFactory.
Host host = HostFactory.getHost();
// Obtain a list of the USB buses available on the Host.
Bus[] bus = host.getBusses();
int total_bus = bus.length;
System.out.println(total_bus);
// Traverse through all the USB buses.
for (int i = 0; i < total_bus; i++) {
// Access the root hub on the USB bus and obtain the
// number of USB ports available on the root hub.
Device root = bus[i].getRootHub();
int total_port = root.getNumPorts();
// Traverse through all the USB ports available on the
// root hub. It should be mentioned that the numbering
// starts from 1, not 0.
for (int j = 1; j <= total_port; j++) {
// Obtain the Device connected to the port.
Device device = root.getChild(j);
if (device != null) {
// USB device available, do something here.
// Obtain the current Configuration of the device
// and the number of interfaces available under the
// current Configuration.
Configuration config = device.getConfiguration();
int total_interface = config.getNumInterfaces();
// Traverse through the Interfaces
for (int k = 0; k < total_interface; k++) {
// Access the current Interface and obtain the
// number of endpoints available on the Interface
Interface itf = config.getInterface(k, 0);
int total_ep = itf.getNumEndpoints();
// Traverse through all the endpoints.
for (int l = 0; l < total_ep; l++) {
// Access the endpoint and
// obtain its I/O type
Endpoint ep = itf.getEndpoint(l);
String io_type = ep.getType();
boolean input = ep.isInput();
// If the endpoint is an input endpoint,
// obtain its InputStream and read in data.
if (input) {
InputStream in;
in = ep.getInputStream();
// Read in data here
in.close();
}
else {
// If the Endpoint is an output Endpoint,
// obtain its OutputStream and
// write out data.
OutputStream out;
out = ep.getOutputStream();
// Write out data here.
out.close();
}
}
}
}
}
}
}
catch (Exception e) {
System.out.println(e.getMessage());
}
}
}

I Googled that error message and I found:
A forum post from 2005 that said that on Linux this can be due to something else having grabbed exclusive use of the USB controller: http://bytes.com/topic/java/answers/16736-jusb
An online copy of the source code, which indicates that this happens if getHost's attempt to create a (platform specific) HostFactory fails. Unfortunately, the code eats unexpected exceptions (*), so it you'll need to use a Java debugger to figure out what the real code is.
(* The code catches Exception in maybeGetHost and other places and throws away the diagnostics! This is a major no-no, and a big red flag on overall code quality of the library. If I were you, I'd be looking for a better quality library to use.)