Develop Reference

Java. Getting URL from package

I'm making an application that captures the network trafic. I need to get the URL that the browser displays to the users and I thought I could use InetAddress to get the domain from the IP I had captured but It's not the same. For example, the domain I get from facebook.com is xx-fbcdn-shv-01-dft4.fbcdn.net and this is not always the same. Is there any other way to do it? Is it even possible?
I'm using JNetPCap library and the example code of it's page just to learn how to implement it in my proyect.
import java.net.InetAddress;
import java.net.UnknownHostException;
import java.util.ArrayList;
import java.util.Date;
import java.util.List;
import java.util.logging.Level;
import java.util.logging.Logger;
import org.jnetpcap.*;
import org.jnetpcap.packet.PcapPacket;
import org.jnetpcap.packet.PcapPacketHandler;
import org.jnetpcap.protocol.network.Ip4;
/**
*
* #author User
*/
public class Sniffer {
static private String device;
public static void main(String[] args) {
List<PcapIf> alldevs = new ArrayList<>(); // Will be filled with NICs
StringBuilder errbuf = new StringBuilder(); // For any error msgs
/**
* *************************************************************************
* First get a list of devices on this system
*************************************************************************
*/
int r = Pcap.findAllDevs(alldevs, errbuf);
if (r == Pcap.NOT_OK || alldevs.isEmpty()) {
System.err.printf("Can't read list of devices, error is %s", errbuf
.toString());
return;
}
System.out.println("Network devices found:");
int i = 0;
for (PcapIf device : alldevs) {
String description
= (device.getDescription() != null) ? device.getDescription()
: "No description available";
System.out.printf("#%d: %s [%s]\n", i++, device.getName(), description);
System.out.println(device.toString());
}
PcapIf device = alldevs.get(2);
/***************************************************************************
* Second we open up the selected device
**************************************************************************/
int snaplen = 64 * 1024; // Capture all packets, no trucation
int flags = Pcap.MODE_PROMISCUOUS; // capture all packets
int timeout = 10 * 1000; // 10 seconds in millis
Pcap pcap
= Pcap.openLive(device.getName(), snaplen, flags, timeout, errbuf);
if (pcap == null) {
System.err.printf("Error while opening device for capture: "
+ errbuf.toString());
return;
}
///*-------------------------------------------------------------------- ------------------------------
PcapBpfProgram program = new PcapBpfProgram();
String expression = "port 80";
int optimize = 0; // false
int netmask = 0xFFFFFF00; // 255.255.255.0
if (pcap.compile(program, expression, optimize, netmask) != Pcap.OK) {
System.err.println(pcap.getErr());
return;
}
if (pcap.setFilter(program) != Pcap.OK) {
System.err.println(pcap.getErr());
return;
}
///*
/**
* *************************************************************************
* Third we create a packet handler which will receive packets from the
* libpcap loop.
*************************************************************************
*/
PcapPacketHandler<String> jpacketHandler = (PcapPacket packet, String user) -> {
System.out.printf("Received packet at %s caplen=%-4d len=%-4d %s\n",
new Date(packet.getCaptureHeader().timestampInMillis()),
packet.getCaptureHeader().caplen(), // Length actually captured
packet.getCaptureHeader().wirelen(), // Original length
user // User supplied object
);
Ip4 ip = new Ip4();
//Tcp tcp= new Tcp();
byte[] sIP;
if (packet.hasHeader(ip)) {
try {
sIP = packet.getHeader(ip).source();
String sourceIP = org.jnetpcap.packet.format.FormatUtils.ip(sIP);
String myIp = InetAddress.getLocalHost().getHostAddress();
if (!myIp.equals(sourceIP)) {
System.out.println("source= "+sourceIP);
String domain;
domain = InetAddress.getByName(sourceIP).getHostName();
System.out.println("--------------------------"+domain);
}
} catch (UnknownHostException ex) {
Logger.getLogger(Sniffer.class.getName()).log(Level.SEVERE, null, ex);
}
}
};
/**
* *************************************************************************
* Fourth we enter the loop and tell it to capture 10 packets. The loop
* method does a mapping of pcap.datalink() DLT value to JProtocol ID,
* which is needed by JScanner. The scanner scans the packet buffer and
* decodes the headers. The mapping is done automatically, although a
* variation on the loop method exists that allows the programmer to
* sepecify exactly which protocol ID to use as the data link type for
* this pcap interface.
*************************************************************************
*/
pcap.loop(-1, jpacketHandler, "jNetPcap rocks!");
/**
* *************************************************************************
* Last thing to do is close the pcap handle
*************************************************************************
*/
pcap.close();
}
}

You can't. Big sites got several addresses, so Facebook.com resolves to many addresses but reverse lookup of those addresses does not resolve to Facebook.com.
If you can capture the conversation you can read http headers, there you can find the url you are interested in.

I found a way to do it reading the tcp packets and using it's destination port. Thanks you for your help minus, I wouldn't find the solution without your help. This is the handler method.
static String dominios[] = {"com", "org", "net", "info", "biz", "edu", "gob"};
PcapPacketHandler<String> jpacketHandler = (PcapPacket packet, String user) -> {
Tcp tcp= new Tcp();
Http http= new Http();
if (packet.hasHeader(tcp)) {
if (tcp.destination() == 443) {
int payloadstart = tcp.getOffset() + tcp.size();
JBuffer buffer = new JBuffer(64 * 1024);
buffer.peer(packet, payloadstart, packet.size() - payloadstart);
String payload = buffer.toHexdump(packet.size(), false, true, true);
for (String b : dominios) {
if (payload.contains(b)) {
procesarTcp(payload, b);
}
}
}
else if(packet.hasHeader(http)){
System.out.println(http.fieldValue(Http.Request.Host));
}
}
};
public static void procesarTcp(String payload, String dominio) {
payload= payload.substring(0,payload.indexOf(dominio)+dominio.length());
StringTokenizer token= new StringTokenizer(payload,"\n");
String pagina;
String aux;
payload="";
while(token.hasMoreTokens()){
aux=token.nextToken();
payload+=aux.substring(aux.indexOf(" ")+4, aux.length());
}
pagina= payload.substring(payload.lastIndexOf("..")+2,payload.length());
System.err.println(pagina);
}

SNMP response is null [SNMP4j]

I've hit kind of a brick wall. I got a small system that communicate with DSLAM's by SNMP.
Everything has worked fine for a couple of months, but when I recently added a new DSLAM to the system, I couldn't get an answer from it. Tried the other IP's and didn't have a problem.
After an hour or so, suddenly on of the other DSLAM's stopped answering me too. So now I've got two units without any communication, which kinda sucks. So of course I checked the units, and didn't find a problem. By my MIB-browser, I can reach all of the units... But not via my software. So the error lies somewhere in my software. So I checked Wireshark, and see that the getNext requests are going out, but I don't seem to get an answer. When I do it via the MIB browser, there comes an answer. But the funny thing is: the two requests are identical. So I must not be listening - and yes, it is listening.
Why in the world is this specific to some IP's, and dear Lord why do they contaminate eachother?
Let's look at some code:
public String GetNextValue(String OID, Dslam dslam) throws IOException {
Snmp snmp = new Snmp(new DefaultUdpTransportMapping());
snmp.listen();
CommunityTarget target = initializeTarget(dslam);
PDU request = new PDU();
request.setType(PDU.GETNEXT);
OID oid= new OID(OID);
request.add(new VariableBinding(oid));
PDU responsePDU=null;
ResponseEvent responseEvent;
responseEvent = snmp.send(request, target);
if (responseEvent != null){
System.out.println("resonse event not null..");
responsePDU = responseEvent.getResponse();
if ( responsePDU != null){
System.out.println("pdu not null..");
#SuppressWarnings("unchecked")
Vector <VariableBinding> tmpv = (Vector<VariableBinding>) responsePDU.getVariableBindings();
if(tmpv != null){
System.out.println("tmpv not null..");
VariableBinding vb = (VariableBinding) tmpv.get(0);
if(!vb.isException()){
return vb.getVariable().toString()
}
}
}
}
_errorHandler.criticalError("Response error in DSLAM communication");
return null;
}
And the initializer:
private CommunityTarget initializeTarget(Dslam dslam){
Address addr = new UdpAddress(dslam.getAddress() + "/" + dslam.getManagementPort() );
System.out.println("IP: " + dslam.getAddress() + " port: " + dslam.getManagementPort());
CommunityTarget target = new CommunityTarget(addr, new OctetString("public"));
target.setVersion(SnmpConstants.version2c);
target.setTimeout(3000);
target.setRetries(3);
return target;
}
And if we run a test upon a working DSLAM:
#Test
public void Lowtest() throws IOException{
SnmpController snmpController = SnmpController.GetInstance();
DslamGrabber dslamGrabber = new DslamGrabber();
Dslam dslam = dslamGrabber.getByDslamId("test5xda5");
String result = snmpController.GetNextValue(".1.3.6.1.4.1.637.61.1.39.3.3.1.1.2", dslam);
System.out.println(result);
}
Result:
IP: 195.215.96.135 port: 161
resonse event not null..
pdu not null..
tmpv not null..
OID: 1.3.6.1.4.1.637.61.1.39.3.3.1.1.2.1
BF512_2048
The we try against test5xda9 (the second one to succumb to this hideous disease-like error)
We get 3 retries in Wireshark, and the following output:
IP: 192.215.96.139 port: 161
resonse event not null..
Response error in DSLAM communication
null
I really hope somebody here can help me. I'm a few hours away to either break down in tears or break a DSLAM..
Best regards
Ben

Well, as a friend pointed out 192 does not equal 195.

Get ClusterName of MQ Queue using Java

I'm building a java application that connects to a MQQueueManager and extracts information about queues. I'm able to get data like QueueType, MaximumMessageLength and more. However, I also want the name of the cluster the queue might be in. There is no function that comes with the MQQueue that gives me this information. After searching the internet I found several things pointing in this direction, but no examples.
A part of my function that gives me the MaximumDepth is:
queueManager = makeConnection(host, portNo, qMgr, channelName);
queue = queueManager.accessQueue(queueName, CMQC.MQOO_INQUIRE);
maxQueueDepth = queue.getMaximumDepth();
(makeConnection is not shown here, it is the function that makes the actual connection to the QueueManager; I also left out the try/catch/finally for less clutter)
How do I get ClusterName and perhaps other data, that doesn't have a function like queue.getMaximumDepth()?

There are two ways to get information about a queue.
The API Inquire call gets operational status of a queue. This includes things like the name the MQOpen call resolved to or the depth if the queue is local. Much of the q.inquire functionality has been superseded with getter and setter functions on the queue. If you are not using the v8.0 client with the latest functionality, you are highly advised to upgrade. It can access all versions of QMgr.
The following code is from Getting and setting attribute values in WebSphere MQ classes for Java
// inquire on a queue
final static int MQIA_DEF_PRIORITY = 6;
final static int MQCA_Q_DESC = 2013;
final static int MQ_Q_DESC_LENGTH = 64;
int[] selectors = new int[2];
int[] intAttrs = new int[1];
byte[] charAttrs = new byte[MQ_Q_DESC_LENGTH]
selectors[0] = MQIA_DEF_PRIORITY;
selectors[1] = MQCA_Q_DESC;
queue.inquire(selectors,intAttrs,charAttrs);
System.out.println("Default Priority = " + intAttrs[0]);
System.out.println("Description : " + new String(charAttrs,0));
For things that are not part of the API Inquire call, a PCF command is needed. Programmable Command Format, commonly abbreviated as PCF, is a message format used to pass messages to the command queue and for reading messages from the command queue, event queues and others.
To use a PCF command the calling application must be authorized with +put on SYSTEM.ADMIN.COMMAND.QUEUE and for +dsp on the object being inquired upon.
IBM provides sample code.
On Windows, please see: %MQ_FILE_PATH%\Tools\pcf\samples
In UNIX flavors, please see: /opt/mqm/samp/pcf/samples
The locations may vary depending on where MQ was installed.
Please see: Handling PCF messages with IBM MQ classes for Java. The following snippet is from the PCF_DisplayActiveLocalQueues.java sample program.
public static void DisplayActiveLocalQueues(PCF_CommonMethods pcfCM) throws PCFException,
MQDataException, IOException {
// Create the PCF message type for the inquire.
PCFMessage pcfCmd = new PCFMessage(MQConstants.MQCMD_INQUIRE_Q);
// Add the inquire rules.
// Queue name = wildcard.
pcfCmd.addParameter(MQConstants.MQCA_Q_NAME, "*");
// Queue type = LOCAL.
pcfCmd.addParameter(MQConstants.MQIA_Q_TYPE, MQConstants.MQQT_LOCAL);
// Queue depth filter = "WHERE depth > 0".
pcfCmd.addFilterParameter(MQConstants.MQIA_CURRENT_Q_DEPTH, MQConstants.MQCFOP_GREATER, 0);
// Execute the command. The returned object is an array of PCF messages.
PCFMessage[] pcfResponse = pcfCM.agent.send(pcfCmd);
// For each returned message, extract the message from the array and display the
// required information.
System.out.println("+-----+------------------------------------------------+-----+");
System.out.println("|Index| Queue Name |Depth|");
System.out.println("+-----+------------------------------------------------+-----+");
for (int index = 0; index < pcfResponse.length; index++) {
PCFMessage response = pcfResponse[index];
System.out.println("|"
+ (index + pcfCM.padding).substring(0, 5)
+ "|"
+ (response.getParameterValue(MQConstants.MQCA_Q_NAME) + pcfCM.padding).substring(0, 48)
+ "|"
+ (response.getParameterValue(MQConstants.MQIA_CURRENT_Q_DEPTH) + pcfCM.padding)
.substring(0, 5) + "|");
}
System.out.println("+-----+------------------------------------------------+-----+");
return;
}
}

After more research I finally found what I was looking for.
This example of IBM: Getting and setting attribute values in WebSphere MQ classes helped me to set up the inquiry.
The necessary values I found in this list: Constant Field Values.
I also needed to expand the openOptionsArg of accessQueue(), else cluster queues cannot be inquired.
Final result:
(without makeConnection())
public class QueueManagerServices {
final static int MQOO_INQUIRE_TOTAL = CMQC.MQOO_FAIL_IF_QUIESCING | CMQC.MQOO_INPUT_SHARED | CMQC.MQOO_INQUIRE;
MQQueueManager queueManager = null;
String cluster = null;
MQQueue queue = null;
public String getcluster(String host, int portNo, String qMgr, String channelName){
try{
queueManager = makeConnection(host, portNo, qMgr, channelName);
queue = queueManager.accessQueue(queueName, MQOO_INQUIRE_TOTAL);
int MQCA_CLUSTER_NAME = 2029;
int MQ_CLUSTER_NAME_LENGTH = 48;
int[] selectors = new int[1];
int[] intAttrs = new int[1];
byte[] charAttrs = new byte[MQ_CLUSTER_NAME_LENGTH];
selectors[0] = MQCA_CLUSTER_NAME;
queue.inquire(selectors, intAttrs, charAttrs);
cluster = new String (charAttrs);
} catch (MQException e) {
System.out.println(e);
} finally {
if (queue != null){
queue.close();
}
if (queueManager != null){
queueManager.disconnect();
}
}
return cluster;
}
}

SNMP4J send never times out

I'm having a problem while using the SNMP4J library. My code works great when the SNMP server I'm sending the request to is running, but when it isn't running, I want the SNMP request to eventually time-out. However, it never times out. For example, the code below, even when dcadb2 does not exist (i.e. can not ping this host name), the program will never print "Timed out". It will attempt to reach the device for infinity.
private ResponseEvent getSnmpResponse() {
PDU pdu = createPdu();
Target target = getTarget();
try {
ResponseEvent event = snmp.send(pdu, target, null);
return event;
}
catch (IOException e ){
System.out.println("Timed out");
}
}
protected PDU createPdu() {
PDU pdu = new PDU();
pdu.add(new VariableBinding(new OID(DISK_TOTAL_OID)));
pdu.add(new VariableBinding(new OID(DISK_AVAIL_OID)));
pdu.add(new VariableBinding(new OID(DISK_USED_OID)));
pdu.add(new VariableBinding(new OID(DISK_PERCENT_USED_OID)));
pdu.setType( PDU.GET );
return pdu;
}
private Target getTarget() {
CommunityTarget target = new CommunityTarget();
target.setCommunity( new OctetString("public") );
target.setAddress( GenericAddress.parse("dcadb2/161") );
target.setRetries( 1 );
target.setTimeout( 1000L );
target.setVersion( SnmpConstants.version2c );
return target;
}

I found the solution. It turns out SNMP4J (or at least the version I'm using, 1.11.2) will time out if you give it an IP that doesn't exist, but not if you give it a hostname that doesn't exist. In the latter case, the thread will continue for infinity. So the solution is to use IP addresses.

How do I keep SNMP4J GETBULK from incrementing rightmost digit before OID?

I'm using this SNMP4J code to do some SNMP walks. But, when I run it on say, 1.3.6.1.2.1.31.1.1.1.1 which is ifName, it gets all the interfaces which are represented by 1.3.6.1.2.1.31.1.1.1.1.x, but then it also grabs 1.3.6.1.2.1.31.1.1.1.2 which are ifInMulticastPkts and then sometimes 1.3.6.1.2.1.31.1.1.1.3 which are ifInBroadcastPkts. I'm only interested in ifName.
How do keep the GETBULK from incrementing the last digit before traversing the MIB?
public ArrayList<String> walk(String oid) throws IOException, InterruptedException {
Snmp snmp = new Snmp(new DefaultUdpTransportMapping());
snmp.listen();
Address targetAddress = GenericAddress.parse(address);
CommunityTarget target = new CommunityTarget();
target.setCommunity(new OctetString(community));
target.setVersion(SnmpConstants.version2c);
target.setAddress(targetAddress);
target.setTimeout(3000); //3s
target.setRetries(1);
PDU pdu = new PDU();
pdu.setType(PDU.GETBULK);
pdu.setMaxRepetitions(200);
pdu.setNonRepeaters(0);
pdu.add(new VariableBinding(new OID(oid)));
ResponseEvent responseEvent = snmp.send(pdu, target);
PDU response = responseEvent.getResponse();
ArrayList<String> responsePieces = new ArrayList<String>();
if (response == null) {
System.out.println("TimeOut...");
}
else
{
if (response.getErrorStatus() == PDU.noError)
{
Vector<? extends VariableBinding> vbs = response.getVariableBindings();
for (VariableBinding vb : vbs) {
responsePieces.add(vb.toString());
}
}
else
{
System.out.println("Error:" + response.getErrorStatusText());
}
}
return responsePieces;
}

You should probably be using the getTable method of the TableUtils class to fetch the column(s) you need. That way, you don't have to worry about the nitty-gritty details of how the protocol behaves when walking a table.
That said, if you want to do it yourself... What you've discovered is the normal behavior of GetBulk. The Agent is only responsible for returning as many rows as you specified (200), if it has them. The response will also indicate what OID you should request if you want more rows.
It is only you as a manager who can tell when you've got all the data you want, and stop sending new GetBulk requests. You should simply discard the unwanted data in the last response. In your case, detect whether an OID is no longer a child of the column you requested.
You can read more about how SNMP walking and GetBulk work, in RFC 1905, particularly sections 4.2.3 and 4.2.3.1.
But do use the method provided by the API, it will save you some gray hairs, and is guaranteed to be correct.