Develop Reference

Twitter Java project : failed to establish connection properly

I am trying to consume Twitter streams with the help of a Java Kafka application.
I have created a Twitter developer account and a Twitter application and generated all the 4 keys required.
Please find the code below:
package com.github.simpleanand.kafkabeginner.tutorial2;
import java.util.List;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.TimeUnit;
import org.apache.http.HttpHost;
import org.apache.http.conn.params.ConnRoutePNames;
import org.apache.http.impl.client.DefaultHttpClient;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.google.common.collect.Lists;
import com.twitter.hbc.ClientBuilder;
import com.twitter.hbc.core.Client;
import com.twitter.hbc.core.Constants;
import com.twitter.hbc.core.Hosts;
import com.twitter.hbc.core.HttpHosts;
import com.twitter.hbc.core.endpoint.StatusesFilterEndpoint;
import com.twitter.hbc.core.processor.StringDelimitedProcessor;
import com.twitter.hbc.httpclient.auth.Authentication;
import com.twitter.hbc.httpclient.auth.OAuth1;
public class TwitterProducer {
private Logger logger = LoggerFactory.getLogger(TwitterProducer.class);
private String consumerKey = "<consumer-key>";
private String consumerSecret = "<consumerSecret>";
private String token = "<token value>";
private String secret = "<secret>";
public TwitterProducer() {
}
public static void main(String[] args) {
new TwitterProducer().run();
}
public void run() {
logger.info("inside run........");
// create a twitter client
/**
* Set up your blocking queues: Be sure to size these properly based on
* expected TPS of your stream
*/
BlockingQueue<String> msgQueue = new LinkedBlockingQueue<String>(100000);
Client client = createTwitterClient(msgQueue);
client.connect();
// create a kafka producer
// loop to send tweets to kafka
// on a different thread, or multiple different threads....
while (!client.isDone()) {
String msg = null;
try {
msg = msgQueue.poll(5, TimeUnit.SECONDS);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
client.stop();
}
if (null != msg) {
logger.info("Msg -> " + msg);
}
}
logger.info("end of application........");
}
public Client createTwitterClient(BlockingQueue<String> msgQueue) {
HttpHost proxy = new HttpHost("<compayny proxy value>", 8080);
DefaultHttpClient httpClient = new DefaultHttpClient();
httpClient.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, proxy);
/**
* Declare the host you want to connect to, the endpoint, and
* authentication (basic auth or oauth)
*/
Hosts hosebirdHosts = new HttpHosts(Constants.STREAM_HOST);
StatusesFilterEndpoint hosebirdEndpoint = new StatusesFilterEndpoint();
// Optional: set up some followings and track terms
// List<Long> followings = Lists.newArrayList(1234L, 566788L);
List<String> terms = Lists.newArrayList("bitcoin");
// hosebirdEndpoint.followings(followings);
hosebirdEndpoint.trackTerms(terms);
// These secrets should be read from a config file
Authentication hosebirdAuth = new OAuth1(consumerKey, consumerSecret, token, secret);
hosebirdAuth.setupConnection(httpClient);
// Creating a client:
ClientBuilder builder = new ClientBuilder().name("Hosebird-Client-01") // optional:
// mainly
// for
// the
// logs
.hosts(hosebirdHosts).authentication(hosebirdAuth).endpoint(hosebirdEndpoint)
.processor(new StringDelimitedProcessor(msgQueue));
Client hosebirdClient = builder.build();
// Attempts to establish a connection.
return hosebirdClient;
}
}
I'm getting the following error:
[hosebird-client-io-thread-0] INFO com.twitter.hbc.httpclient.ClientBase - Hosebird-Client-01 Establishing a connection
[hosebird-client-io-thread-0] WARN com.twitter.hbc.httpclient.ClientBase - Hosebird-Client-01 Unknown host - stream.twitter.com
[hosebird-client-io-thread-0] WARN com.twitter.hbc.httpclient.ClientBase - Hosebird-Client-01 failed to establish connection properly
[hosebird-client-io-thread-0] INFO com.twitter.hbc.httpclient.ClientBase - Hosebird-Client-01 Done processing, preparing to close connection
Please advise on how to solve this error.
It seems there is an issue of proxy object being sent.It that correct?

AccessControlException while trying to create a directory on Azure

I'm trying this to Create Directory in azureDataLake using azure data lake dependency
<dependency>
<groupId>com.microsoft.azure</groupId>
<artifactId>azure-data-lake-store-sdk</artifactId>
<version>2.1.5</version>
</dependency>
Using the following method:
private ADLStoreClient client;
public boolean createDirectory(String path) {
try {
// create directory
client.createDirectory(path);
} catch (ADLException ex) {
printExceptionDetails(ex);
return false;
} catch (Exception ex) {
log.error(" Exception in createDirectory : {}", ex);
return false;
}
return true;
}
and I got this exception:
Error creating directory /gx-zweappdhd004/home/azhdipaasssh2/ADH/Compta/1458/1533632735200/RAPPORTS/
Operation MKDIRS failed with HTTP403 : AccessControlException
I checked the permission and I have all of them, so it's not related to the permissions.
Update:
To be more specefic the problem happening inside the method isSuccessfulResponse(), and exactlly in this line HttpTransport.java#L137 because the httpResponseCode equals to 403, can anybody explain this.
Update2:
I found that this line is returning the 403 status : HttpTransport.java#L288, I also tried to evaluate conn.getErrorStream().read() and I got this stream is closed, FYI this is bug occures sometimes and not always.

I didn't reproduce your issue,you could refer to my working code:
import com.microsoft.aad.adal4j.AuthenticationContext;
import com.microsoft.aad.adal4j.AuthenticationResult;
import com.microsoft.aad.adal4j.ClientCredential;
import com.microsoft.azure.datalake.store.ADLStoreClient;
import java.io.IOException;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
public class CreateDirectory {
static ADLStoreClient client;
public static void main(String[] args) throws InterruptedException, ExecutionException, IOException {
setup();
}
public static void setup() throws IOException, ExecutionException, InterruptedException {
String APP_ID = "<your app id>";
String APP_SECRET = "<your app secret>";
String dirName = "/jay";
String StoreAcct = "jaygong";
String authority = "https://login.microsoftonline.com/<your tenant id>";
String resourcUrl = "https://management.core.windows.net/";
ExecutorService service = Executors.newFixedThreadPool(1);
AuthenticationContext context = new AuthenticationContext(authority, true, service);
// Acquire Token
Future<AuthenticationResult> result = context.acquireToken(
resourcUrl,
new ClientCredential(APP_ID, APP_SECRET),
null
);
String token = result.get().getAccessToken();
System.out.println(token);
String account = StoreAcct + ".azuredatalakestore.net";
client = ADLStoreClient.createClient(account, token);
client.createDirectory(dirName);
System.out.println("finish.....");
}
}
Don't forget grant access ADL permissons to your client.
Hope it helps you.

Azure Service Bus IQueueClient.registerSessionHandler: Unable to consume messages continuously from Queue/Topic

Requirement: I need to get all the messages from all the sessions from the queue. registerSessionHandler should consume messages as soon as they appear in the queue.
Issue: the code accepts the messages that are pertaining to one of the session id (xyz) and then it just waits. And even when more messages are pushed with that session id (xyz), it fails to consume it.
Any suggestions - what obvious thing I am missing here.
I have registerSessionHandler for receiving session messages continuously from the queue. Whenever i start new session, i am getting message of only one sessionID.
Test Queue: In this queue, there are 20 messages available with 4 different sessionIDs. Whenever i run Java application (QueueSessionReceiveTest.java), i get only 5 messages which is associated with single session ID.
MAVEN - azure-servicebus - 1.1.1
RECEIVE CODE:
import java.time.Duration;
import com.microsoft.azure.servicebus.IMessage;
import com.microsoft.azure.servicebus.IQueueClient;
import com.microsoft.azure.servicebus.QueueClient;
import com.microsoft.azure.servicebus.ReceiveMode;
import com.microsoft.azure.servicebus.SessionHandlerOptions;
import com.microsoft.azure.servicebus.primitives.ConnectionStringBuilder;
public class QueueSessionReceiveTest {
private static final String connectionString = "Endpoint=sb://XXXXXX";
private static final String queueName = "test";
private static IQueueClient queueClient;
public static void main(String[] args) throws Exception {
queueClient = new QueueClient(new ConnectionStringBuilder(connectionString, queueName), ReceiveMode.PEEKLOCK);
queueClient.registerSessionHandler(new QueueMessageSessionHandler(), new SessionHandlerOptions(1, false, Duration.ofMinutes(1)));
}}
SESSION HANDLER CODE:
import java.util.concurrent.CompletableFuture;
import com.microsoft.azure.servicebus.ExceptionPhase;
import com.microsoft.azure.servicebus.IMessage;
import com.microsoft.azure.servicebus.IMessageSession;
import com.microsoft.azure.servicebus.IQueueClient;
import com.microsoft.azure.servicebus.ISessionHandler;
public class QueueMessageSessionHandler implements ISessionHandler {
#Override
public CompletableFuture<Void> onMessageAsync(IMessageSession session, IMessage iMessage) {
return session.completeAsync(iMessage.getLockToken()).thenRunAsync(() -> Logger.debug("some log") );
}
#Override
public CompletableFuture<Void> OnCloseSessionAsync(IMessageSession session) {
return null;
}
#Override
public void notifyException(Throwable exception, ExceptionPhase exceptionPhase) {
// Do nothing
}}

Please refer below link for solution.
https://github.com/Azure/azure-service-bus-java/issues/187

Voice call using twilo in web application

we are trying to use voice call for our web application.
we tried using below code:
public class MakeCall {
public static final String ACCOUNT_SID = "ACbXXXXXXXXXXXXXXXXXXXXXXXX";
public static final String AUTH_TOKEN = "545XXXXXXXXXXXXXXXXXXXXXXXX";
public static final String TWILIO_NUMBER = "+185XXXXXXXXX";
public static void main(String[] args) throws TwilioRestException {
TwilioRestClient client = new TwilioRestClient(ACCOUNT_SID, AUTH_TOKEN);
Account mainAccount = client.getAccount();
CallFactory callFactory = mainAccount.getCallFactory();
Map<String, String> callParams = new HashMap<String, String>();
callParams.put("From",TWILIO_NUMBER);
callParams.put("To", "+919014512394");
callParams.put("Url", "http://ahoy.twilio.com/voice/api/demo");
Call call = callFactory.create(callParams);
System.out.println(call.getSid());
}
}
From above code,we are able to hear twilio customer voice i.e,Welcome to the Twilio voice demo app. Press 1 to hear the weather forecast tomorrow. Press 2 to hear a song. Press 3 to create or join a conference bridge. Press 4 to record your voice for 5 seconds.
Actually we want to speak with other mobile number by calling from twilio number
Basically we are new to twilio API.Plz guide us
Advance thanks

Twilio developer evangelist here.
The code you've written will initiate a new call when you call that endpoint. If you want to initiate a new call and then connect to another number so the two of you can talk, you need to change that slightly do it Dials the number for you.
Here's afull working example that shows you how to pick who you want to speak with, and then dials that person.
package com.twilio;
import com.twilio.twiml.Gather;
import com.twilio.twiml.Method;
import com.twilio.twiml.Play;
import com.twilio.twiml.Say;
import com.twilio.twiml.TwiMLException;
import com.twilio.twiml.VoiceResponse;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.util.HashMap;
public class TwilioServlet extends HttpServlet {
public void service(HttpServletRequest request, HttpServletResponse response) throws IOException {
// Create a dict of people we know.
HashMap<String, String> callers = new HashMap<String, String>();
callers.put("+14158675309", "Curious George");
callers.put("+14158675310", "Boots");
callers.put("+14158675311", "Virgil");
String fromNumber = request.getParameter("From");
String knownCaller = callers.get(fromNumber);
String message;
if (knownCaller == null) {
// Use a generic message
message = "Hello Monkey";
} else {
// Use the caller's name
message = "Hello " + knownCaller;
}
// Create a TwiML response and add our friendly message.
VoiceResponse twiml = new VoiceResponse.Builder()
.say(new Say.Builder(message).build())
// Play an MP3 for incoming callers.
.play(new Play.Builder("http://demo.twilio.com/hellomonkey/monkey.mp3").build())
.gather(new Gather.Builder()
.action("/handle-key")
.method(Method.POST)
.numDigits(1)
.say(new Say
.Builder("To speak to a real monkey, press 1. Press any other key to start over.")
.build())
.build()
)
.build();
response.setContentType("application/xml");
try {
response.getWriter().print(twiml.toXml());
} catch (TwiMLException e) {
e.printStackTrace();
}
}
}
You start off with a HashMap containing all your telephone numbers
Upon pressing a number, the /handle-key endpoint is called. This is where the logic for dialing another number happens
package com.twilio;
import com.twilio.twiml.Dial;
import com.twilio.twiml.Number;
import com.twilio.twiml.Say;
import com.twilio.twiml.TwiMLException;
import com.twilio.twiml.VoiceResponse;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
public class TwilioHandleKeyServlet extends HttpServlet {
#Override
public void service(HttpServletRequest request, HttpServletResponse response) throws IOException {
String digits = request.getParameter("Digits");
VoiceResponse twiml;
// Check if the user pressed "1" on their phone
if (digits != null && digits.equals("1")) {
// Connect 310 555 1212 to the incoming caller.
Number number = new Number.Builder("+13105551212").build();
Dial dial = new Dial.Builder().number(number).build();
// If the above dial failed, say an error message.
Say say = new Say.Builder("The call failed, or the remote party hung up. Goodbye.").build();
twiml = new VoiceResponse.Builder().dial(dial).say(say).build();
} else {
// If they didn't press 1, redirect them to the TwilioServlet
response.sendRedirect("/twiml");
return;
}
response.setContentType("application/xml");
try {
response.getWriter().print(twiml.toXml());
} catch (TwiMLException e) {
e.printStackTrace();
}
}
}
You can read a full explanation of this and find other examples in this quickstart.
Hope this helps you out.

SignalR java cannot connect to server

I'm having trouble connecting my java application to my SignalR Server.
The server is very simple and can be found here:
https://code.msdn.microsoft.com/windowsdesktop/Using-SignalR-in-WinForms-f1ec847b
I can connect web clients(javascript) and windows clients (C#) but I'm having trouble with my java client.(https://github.com/SignalR/java-client)
Here is my code so far:
package javaapplication2;
import java.io.IOException;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeoutException;
import microsoft.aspnet.signalr.client.SignalRFuture;
import microsoft.aspnet.signalr.client.hubs.HubConnection;
import microsoft.aspnet.signalr.client.hubs.HubProxy;
public class JavaApplication2 {
public static void main(String[] args) throws IOException, InterruptedException, ExecutionException, TimeoutException
{
String ServerURI = "http://localhost:8080/signalr";
HubConnection Connection = new HubConnection(ServerURI);
HubProxy HubProxy = Connection.createHubProxy("MyHub");
HubProxy.on("AddMessage", () -> { System.out.println("Some message"); });
Connection.error(new ErrorCallback() {
#Override
public void onError(Throwable error) {
error.printStackTrace(); //<==SocketException
}
});
SignalRFuture<Void> con =Connection.start();
con.get();
}
}
Update
When I run it I get a "ExecutionException: java.net.SocketException: Connection reset"
Exception in thread "main" java.util.concurrent.ExecutionException: java.net.SocketException: Connection reset
at microsoft.aspnet.signalr.client.SignalRFuture.get(SignalRFuture.java:112)
at microsoft.aspnet.signalr.client.SignalRFuture.get(SignalRFuture.java:102)
at javaapplication2.JavaApplication2.main(JavaApplication2.java:27)
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:209)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:704)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:675)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1535)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1440)
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
at microsoft.aspnet.signalr.client.http.java.NetworkRunnable.run(NetworkRunnable.java:72)
at java.lang.Thread.run(Thread.java:745)
-If I change "localhost" to something that does not exist ( e.g locahostX) I get a java.net.UnknownHostException
-If If change "localhost" to my IP I don't event get an exception...
-All the other apps work with both (localhost or IP)
At first I thought it was a firewall issue but it wasn't that...
Obviously I'm missing something...
Any ideas?
Thanks

It turns out that I had to use an overload of start,the one that takes as a parameter a ClientTransport object
public SignalRFuture<Void> start(ClientTransport transport)
If anyone has an explanation why the parameterless start method fails ,please post it as an answer and I will mark it as the solution.
Here is a full example that works:
package javaapplication2;
import java.io.IOException;
import java.util.Scanner;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeoutException;
import microsoft.aspnet.signalr.client.SignalRFuture;
import microsoft.aspnet.signalr.client.hubs.HubConnection;
import microsoft.aspnet.signalr.client.hubs.HubProxy;
import microsoft.aspnet.signalr.client.transport.ServerSentEventsTransport;
import microsoft.aspnet.signalr.client.hubs.SubscriptionHandler2;
public class JavaApplication2 {
public static void main(String[] args) throws IOException, InterruptedException, ExecutionException, TimeoutException
{
String ServerURI = "http://localhost:8080/signalr";
HubConnection Connection = new HubConnection(ServerURI);
HubProxy HubProxy = Connection.createHubProxy("MyHub");
HubProxy.on("AddMessage", new SubscriptionHandler2<String, String>() {
#Override
public void run(String e1, String e2) {
System.out.println(e1.toString()+ " -> " +e2.toString());
}
}, String.class, String.class);
SignalRFuture<Void> con =Connection.start(new ServerSentEventsTransport(Connection.getLogger())); //Or LongPollingTransport
con.get();
Scanner inputReader = new Scanner(System.in);
String line = inputReader.nextLine();
while (!"exit".equals(line)) {
HubProxy.invoke("send", "Console", line);
line = inputReader.next();
}
inputReader.close();
Connection.stop();
}
}

If I change "localhost" to something that does not exist ( e.g
locahostX) I get a java.net.UnknownHostException
Are you sure about this?
On the server command prompt run "ipconfig" to get the IP address of the server.
From the client command prompt type "ping " + IP address of the server.
If the ping sends packages, then try to put the IP in the string "ServerURI" to be something like "http://"+ServerIP+":8080/signalr".