Develop Reference

Twitter Java project : failed to establish connection properly

I am trying to consume Twitter streams with the help of a Java Kafka application.
I have created a Twitter developer account and a Twitter application and generated all the 4 keys required.
Please find the code below:
package com.github.simpleanand.kafkabeginner.tutorial2;
import java.util.List;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.TimeUnit;
import org.apache.http.HttpHost;
import org.apache.http.conn.params.ConnRoutePNames;
import org.apache.http.impl.client.DefaultHttpClient;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.google.common.collect.Lists;
import com.twitter.hbc.ClientBuilder;
import com.twitter.hbc.core.Client;
import com.twitter.hbc.core.Constants;
import com.twitter.hbc.core.Hosts;
import com.twitter.hbc.core.HttpHosts;
import com.twitter.hbc.core.endpoint.StatusesFilterEndpoint;
import com.twitter.hbc.core.processor.StringDelimitedProcessor;
import com.twitter.hbc.httpclient.auth.Authentication;
import com.twitter.hbc.httpclient.auth.OAuth1;
public class TwitterProducer {
private Logger logger = LoggerFactory.getLogger(TwitterProducer.class);
private String consumerKey = "<consumer-key>";
private String consumerSecret = "<consumerSecret>";
private String token = "<token value>";
private String secret = "<secret>";
public TwitterProducer() {
}
public static void main(String[] args) {
new TwitterProducer().run();
}
public void run() {
logger.info("inside run........");
// create a twitter client
/**
* Set up your blocking queues: Be sure to size these properly based on
* expected TPS of your stream
*/
BlockingQueue<String> msgQueue = new LinkedBlockingQueue<String>(100000);
Client client = createTwitterClient(msgQueue);
client.connect();
// create a kafka producer
// loop to send tweets to kafka
// on a different thread, or multiple different threads....
while (!client.isDone()) {
String msg = null;
try {
msg = msgQueue.poll(5, TimeUnit.SECONDS);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
client.stop();
}
if (null != msg) {
logger.info("Msg -> " + msg);
}
}
logger.info("end of application........");
}
public Client createTwitterClient(BlockingQueue<String> msgQueue) {
HttpHost proxy = new HttpHost("<compayny proxy value>", 8080);
DefaultHttpClient httpClient = new DefaultHttpClient();
httpClient.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, proxy);
/**
* Declare the host you want to connect to, the endpoint, and
* authentication (basic auth or oauth)
*/
Hosts hosebirdHosts = new HttpHosts(Constants.STREAM_HOST);
StatusesFilterEndpoint hosebirdEndpoint = new StatusesFilterEndpoint();
// Optional: set up some followings and track terms
// List<Long> followings = Lists.newArrayList(1234L, 566788L);
List<String> terms = Lists.newArrayList("bitcoin");
// hosebirdEndpoint.followings(followings);
hosebirdEndpoint.trackTerms(terms);
// These secrets should be read from a config file
Authentication hosebirdAuth = new OAuth1(consumerKey, consumerSecret, token, secret);
hosebirdAuth.setupConnection(httpClient);
// Creating a client:
ClientBuilder builder = new ClientBuilder().name("Hosebird-Client-01") // optional:
// mainly
// for
// the
// logs
.hosts(hosebirdHosts).authentication(hosebirdAuth).endpoint(hosebirdEndpoint)
.processor(new StringDelimitedProcessor(msgQueue));
Client hosebirdClient = builder.build();
// Attempts to establish a connection.
return hosebirdClient;
}
}
I'm getting the following error:
[hosebird-client-io-thread-0] INFO com.twitter.hbc.httpclient.ClientBase - Hosebird-Client-01 Establishing a connection
[hosebird-client-io-thread-0] WARN com.twitter.hbc.httpclient.ClientBase - Hosebird-Client-01 Unknown host - stream.twitter.com
[hosebird-client-io-thread-0] WARN com.twitter.hbc.httpclient.ClientBase - Hosebird-Client-01 failed to establish connection properly
[hosebird-client-io-thread-0] INFO com.twitter.hbc.httpclient.ClientBase - Hosebird-Client-01 Done processing, preparing to close connection
Please advise on how to solve this error.
It seems there is an issue of proxy object being sent.It that correct?

How to retrieve data from cloud (iothub)using mqtt in java

I am new to IoTHub. I have successfully sent messages to IOT hub (D2C) using python.The protocol we used is mqtt.We are trying to retrieve data from cloud(IOT hub) using java,but could not able to find out a proper way to recieve message from the cloud..My doubt is whether we can read messages from IOT Hub directly or we need to redirect the incoming messages to an event hub to retrieve the message.
Also I tried to read messages from iothub in java simultaneously while sending data to cloud,but I got the error as follows..(Lost connection to the server. Reconnecting 0 time.)
I use this code to read data from iothub,
import com.microsoft.azure.sdk.iot.device.DeviceClient;
import com.microsoft.azure.sdk.iot.device.IotHubMessageResult;
import com.microsoft.azure.sdk.iot.device.Message;
import com.microsoft.azure.sdk.iot.device.MessageCallback;
import com.microsoft.azure.sdk.iot.device.IotHubClientProtocol;
import com.microsoft.azure.sdk.iot.service.sdk.IotHubServiceClientProtocol;
import java.io.IOException;
import java.net.URISyntaxException;
import java.io.IOException;
import java.util.logging.Level;
import java.util.logging.Logger;
public class Kafkareception {
public static void main(String[] args) throws IOException {
try {
String connString = "HostName=";
IotHubClientProtocol protocol = IotHubClientProtocol.MQTT;
DeviceClient client = new DeviceClient(connString, protocol);
MessageCallback callback = new AppMessageCallback();
client.setMessageCallback(callback, null);
client.open();
} catch (URISyntaxException ex) {
Logger.getLogger(Kafkareception.class.getName()).log(Level.SEVERE, null, ex);
}
}
private static class AppMessageCallback implements MessageCallback {
public IotHubMessageResult execute(Message msg, Object context) {
System.out.println(new String(msg.getBytes(), Message.DEFAULT_IOTHUB_MESSAGE_CHARSET) + "Received message from hub: ");
return IotHubMessageResult.COMPLETE;
}
}
}

From information you provided, you may tried to setup two active connections of one device to Azure IoT Hub using DeviceClient: one is sending D2C messages and one is "reading data from iothub". You get the error maybe because:
IoT Hub only supports one active MQTT connection per device. Any new
MQTT connection on behalf of the same device ID causes IoT Hub to drop
the existing connection.
Ref:Communicate with your IoT hub using the MQTT protocol.
If you want to receive D2C message that sent to Azure IoT Hub you can use Event Hub-compatible endpoint(Java). No need to redirect the incoming messages to an event hub by yourself.
IoT Hub exposes the messages/events built-in endpoint for your
back-end services to read the device-to-cloud messages received by
your hub. This endpoint is Event Hub-compatible, which enables you to
use any of the mechanisms the Event Hubs service supports for reading
messages.
Ref: Understand Azure IoT Hub messaging and IoT Hub endpoints.

I read the data from iothub.We can use the code
import java.io.IOException;
import com.microsoft.azure.eventhubs.*;
import com.microsoft.azure.servicebus.*;
import java.nio.charset.Charset;
import java.time.*;
import java.util.function.*;
public class Datafetch {
public static void main(String[] args) throws IOException {
EventHubClient client0 = receiveMessages("0");
EventHubClient client1 = receiveMessages("1");
System.out.println("Press ENTER to exit.");
System.in.read();
try {
client0.closeSync();
client1.closeSync();
System.exit(0);
} catch (ServiceBusException sbe) {
System.exit(1);
}
}
private static EventHubClient receiveMessages(final String partitionId) {
String connStr = "Endpoint={youreventhubcompatibleendpoint};EntityPath={youreventhubcompatiblename};SharedAccessKeyName=iothubowner;SharedAccessKey={youriothubkey}";
EventHubClient client = null;
try {
client = EventHubClient.createFromConnectionStringSync(connStr);
} catch (Exception e) {
System.out.println("Failed to create client: " + e.getMessage());
System.exit(1);
}
try {
client.createReceiver(
EventHubClient.DEFAULT_CONSUMER_GROUP_NAME,
partitionId,
Instant.now()).thenAccept(new Consumer<PartitionReceiver>() {
public void accept(PartitionReceiver receiver) {
System.out.println("** Created receiver on partition " + partitionId);
try {
while (true) {
Iterable<EventData> receivedEvents = receiver.receive(100).get();
System.out.println(receivedEvents);
int batchSize = 0;
if (receivedEvents != null) {
for (EventData receivedEvent : receivedEvents) {
System.out.println(String.format("Offset: %s, SeqNo: %s, EnqueueTime: %s",
receivedEvent.getSystemProperties().getOffset(),
receivedEvent.getSystemProperties().getSequenceNumber(),
receivedEvent.getSystemProperties().getEnqueuedTime()));
System.out.println(String.format("| Device ID: %s", receivedEvent.getSystemProperties().getClass()));
System.out.println(String.format("| Message Payload: %s", new String(receivedEvent.getBody(),
Charset.defaultCharset())));
batchSize++;
}
}
System.out.println(String.format("Partition: %s, ReceivedBatch Size: %s", partitionId, batchSize));
}
} catch (Exception e) {
System.out.println("Failed to receive messages: " + e.getMessage());
}
}
});
} catch (Exception e) {
System.out.println("Failed to create receiver: " + e.getMessage());
}
return client;
}
}

SignalR java cannot connect to server

I'm having trouble connecting my java application to my SignalR Server.
The server is very simple and can be found here:
https://code.msdn.microsoft.com/windowsdesktop/Using-SignalR-in-WinForms-f1ec847b
I can connect web clients(javascript) and windows clients (C#) but I'm having trouble with my java client.(https://github.com/SignalR/java-client)
Here is my code so far:
package javaapplication2;
import java.io.IOException;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeoutException;
import microsoft.aspnet.signalr.client.SignalRFuture;
import microsoft.aspnet.signalr.client.hubs.HubConnection;
import microsoft.aspnet.signalr.client.hubs.HubProxy;
public class JavaApplication2 {
public static void main(String[] args) throws IOException, InterruptedException, ExecutionException, TimeoutException
{
String ServerURI = "http://localhost:8080/signalr";
HubConnection Connection = new HubConnection(ServerURI);
HubProxy HubProxy = Connection.createHubProxy("MyHub");
HubProxy.on("AddMessage", () -> { System.out.println("Some message"); });
Connection.error(new ErrorCallback() {
#Override
public void onError(Throwable error) {
error.printStackTrace(); //<==SocketException
}
});
SignalRFuture<Void> con =Connection.start();
con.get();
}
}
Update
When I run it I get a "ExecutionException: java.net.SocketException: Connection reset"
Exception in thread "main" java.util.concurrent.ExecutionException: java.net.SocketException: Connection reset
at microsoft.aspnet.signalr.client.SignalRFuture.get(SignalRFuture.java:112)
at microsoft.aspnet.signalr.client.SignalRFuture.get(SignalRFuture.java:102)
at javaapplication2.JavaApplication2.main(JavaApplication2.java:27)
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:209)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:704)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:675)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1535)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1440)
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
at microsoft.aspnet.signalr.client.http.java.NetworkRunnable.run(NetworkRunnable.java:72)
at java.lang.Thread.run(Thread.java:745)
-If I change "localhost" to something that does not exist ( e.g locahostX) I get a java.net.UnknownHostException
-If If change "localhost" to my IP I don't event get an exception...
-All the other apps work with both (localhost or IP)
At first I thought it was a firewall issue but it wasn't that...
Obviously I'm missing something...
Any ideas?
Thanks

It turns out that I had to use an overload of start,the one that takes as a parameter a ClientTransport object
public SignalRFuture<Void> start(ClientTransport transport)
If anyone has an explanation why the parameterless start method fails ,please post it as an answer and I will mark it as the solution.
Here is a full example that works:
package javaapplication2;
import java.io.IOException;
import java.util.Scanner;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeoutException;
import microsoft.aspnet.signalr.client.SignalRFuture;
import microsoft.aspnet.signalr.client.hubs.HubConnection;
import microsoft.aspnet.signalr.client.hubs.HubProxy;
import microsoft.aspnet.signalr.client.transport.ServerSentEventsTransport;
import microsoft.aspnet.signalr.client.hubs.SubscriptionHandler2;
public class JavaApplication2 {
public static void main(String[] args) throws IOException, InterruptedException, ExecutionException, TimeoutException
{
String ServerURI = "http://localhost:8080/signalr";
HubConnection Connection = new HubConnection(ServerURI);
HubProxy HubProxy = Connection.createHubProxy("MyHub");
HubProxy.on("AddMessage", new SubscriptionHandler2<String, String>() {
#Override
public void run(String e1, String e2) {
System.out.println(e1.toString()+ " -> " +e2.toString());
}
}, String.class, String.class);
SignalRFuture<Void> con =Connection.start(new ServerSentEventsTransport(Connection.getLogger())); //Or LongPollingTransport
con.get();
Scanner inputReader = new Scanner(System.in);
String line = inputReader.nextLine();
while (!"exit".equals(line)) {
HubProxy.invoke("send", "Console", line);
line = inputReader.next();
}
inputReader.close();
Connection.stop();
}
}

If I change "localhost" to something that does not exist ( e.g
locahostX) I get a java.net.UnknownHostException
Are you sure about this?
On the server command prompt run "ipconfig" to get the IP address of the server.
From the client command prompt type "ping " + IP address of the server.
If the ping sends packages, then try to put the IP in the string "ServerURI" to be something like "http://"+ServerIP+":8080/signalr".

Android Phone as Realtime MJPEG Video Server

I'm trying to use my phone as a realtime MJPEG video source. So far, capturing frames and converting them into JPEGs is no big deal. My real issue is sending the multipart response properly. There's tons of documentation about sending multipart responses out there, but the issue with them is that they all expect that all of the images are available at the time the HTTP request comes in (such as would be used for a multi-image upload). In order to stream in realtime, of course, I need to be able to begin to send the multipart response while continually adding jpegs in the body. I'm by no means a HTTP buff, so it's not desirable for me be required to roll my own HTTP response and write directly to a socket. Is there a library out there that supports this kind of behavior? I've scoured the internet for solutions, but I really don't see anything useful out there.
Any ideas? Worst case scenario, I'd be willing to look at human-readable documentation of how to write a multipart response by hand, but I'd really just rather use a library if that's possible.
Thanks in advance.
edit: got it working using the orielly servlet library as per sigmavirus' suggestion. Note that the MJPEG stream is more or less implicitly inferred from the fact that I'm sending a multipart/x-mixed-replace that only has image/jpeg's in it. Check out the comment in my code for a tutorial that shows what jetty libraries you'll need to get this running. Of course, you'll additionally need cos.jar, the Orielly servlet library. The code follows:
package edu.stevens.arpac.webclient;
import java.io.IOException;
import java.net.InetAddress;
import java.net.NetworkInterface;
import java.util.Collections;
import java.util.List;
import javax.servlet.ServletException;
import javax.servlet.ServletOutputStream;
import javax.servlet.http.HttpServletResponse;
import javax.servlet.http.HttpServletRequest;
import org.apache.http.conn.util.InetAddressUtils;
import org.eclipse.jetty.server.Handler;
import org.eclipse.jetty.server.Server;
import org.eclipse.jetty.server.handler.AbstractHandler;
import org.eclipse.jetty.server.Request;
import com.oreilly.servlet.MultipartResponse;
import com.oreilly.servlet.ServletUtils;
import android.os.Environment;
import android.util.Log;
// holla at http://puregeekjoy.blogspot.com/2011/06/running-embedded-jetty-in-android-app.html
public class JettyServer extends Thread
{
private static final String TAG = "JettyServer";
private Server webServer;
private Boolean isStarted = false;
public JettyServer()
{
super();
Log.i(TAG, "Initializing server to port 8080");
webServer = new Server(8080);
Handler handler = new AbstractHandler() {
public void handle(String target, Request request, HttpServletRequest servletRequest,
HttpServletResponse servletResponse) throws IOException, ServletException {
ServletOutputStream out = servletResponse.getOutputStream();
MultipartResponse multi = new MultipartResponse(servletResponse);
Boolean go = true;
while( go )
{
try
{
multi.startResponse("image/jpeg");
ServletUtils.returnFile(Environment.getExternalStorageDirectory().getPath() + "/ARPac/twi.jpg", out);
multi.endResponse();
}
catch(IOException ex)
{
go = false;
Log.i(TAG, "IO Failed with exception " + ex.getMessage());
}
}
request.setHandled(true);
}
};
webServer.setHandler(handler);
try {
webServer.start();
Log.d(TAG, "started Web server # " + getIPAddress());
isStarted = true;
}
catch (Exception e) {
Log.d(TAG, "unexpected exception starting Web server: " + e);
}
}
/**
* Get IP address from first non-localhost interface
* #return address or empty string
*/
private String getIPAddress()
{
try
{
List<NetworkInterface> interfaces = Collections.list(NetworkInterface.getNetworkInterfaces());
for (NetworkInterface intf : interfaces)
{
List<InetAddress> addrs = Collections.list(intf.getInetAddresses());
for (InetAddress addr : addrs)
{
if (!addr.isLoopbackAddress())
{
String sAddr = addr.getHostAddress().toUpperCase();
if (InetAddressUtils.isIPv4Address(sAddr))
{
//Log.d(TAG, "IP address is: " + sAddr);
return sAddr;
}
}
}
}
}
catch (Exception ex)
{
Log.e(TAG, "could not get IP address: " + ex.getMessage());
} // for now eat exceptions
Log.e(TAG, "Could not find a non-loopback IPv4 address!");
return "";
}
public void teardown()
{
if( isStarted )
{
try {
webServer.stop();
isStarted = false;
} catch (Exception e) {
Log.e(TAG, "Couldn't stop server. Probably was called when server already stopped.");
}
}
}
public void run()
{
}
}

Have you seen this? http://www.servlets.com/cos/javadoc/com/oreilly/servlet/MultipartResponse.html It looks like the example sends each part individually and waits a specified time limit before sending the next or receiving an interrupt.

java. Skype - send message

I'm trying to understand how to work with Skype using java (JSkype lib)
i use example (official site):
package testproj;
import net.lamot.java.jskype.general.AbstractMessenger;
import net.lamot.java.jskype.general.MessageListenerInterface;
import net.lamot.java.jskype.windows.Messenger;
import java.lang.Thread;
import java.lang.Exception;
import java.util.Date;
public class JSkype implements MessageListenerInterface {
private AbstractMessenger msgr = null;
public JSkype() {
msgr = new Messenger();
msgr.addListener(this);
msgr.initialize();
try {
Thread.sleep(5000);
msgr.sendMessage("MESSAGE echo123 test message");
} catch (Exception e) {
e.printStackTrace();
}
}
public static void main(String[] args) {
new JSkype();
}
public void onMessageReceived(String str) {
System.out.println("RESULT: "+str);
}
}
after run, in console i have many information, but for me more intresting information, that I receive after send message:
RESULT: MESSAGE 21129 STATUS SENDING
RESULT: MESSAGE 21129 STATUS SENDING
RESULT: CHAT #my.name/$echo123;9797238991f90d78 ACTIVITY_TIMESTAMP 1294574640
and now I'm trying to understand, how to determine the success of sending a message?
yep, we need parsind result string.. but what is a number 21129? 9797238991f90d78? how i can know this number before start parsing?