How do I keep an API Gateway Websocket API with a Lambda Java back end from timing out after 30 seconds?
The AWS API Gateway and re:invent videos mention using pings or heartbeats to keep the Websocket connection alive but I haven't found a straight-forward working example (in Java). The front end is using HTML 5 Websockets (vanilla javascript).
I'm including my Lambda test driver in Java. It causes the API Gateway WS API to timeout after 30 seconds as documented. The driver will return successfully if you remove the delay. Any help is appreciated...
import java.io.InputStream;
import java.io.OutputStream;
import java.util.Map;
import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.RequestStreamHandler;
import com.fasterxml.jackson.databind.ObjectMapper;
public class TestWSClient implements RequestStreamHandler {
public void handleRequest(InputStream inputStream, OutputStream outputStream, Context context) {
try {
//
// How do we keep API Gateway from timing out after 30 seconds?
// Is something like this even needed in the lambda?
//
new Thread(() -> {
while(true) {
try {
// Ping client every 15 seconds
Thread.sleep(15000);
//outputStream.write(); // What to write -- 0x89 0x00?
outputStream.flush();
} catch(Exception e) { e.printStackTrace(); }
}
}).start();
//
// Simulate long processing time or streaming
//
// NOTE: commenting sleep enables service to return w/o a timeout
// connection from API Gateway
//
try { Thread.sleep(60000); } catch(Exception e) {}
var response = Map.of(
"statusCode", 200,
"headers", Map.of("Content-Type", "text/csv"),
"body", "Hello,World"
);
ObjectMapper om = new ObjectMapper();
outputStream.write(om.writeValueAsBytes(response));
outputStream.flush();
} catch(Exception e) { e.printStackTrace(); }
finally { try { outputStream.close(); } catch(Exception e) {} }
}
}
I do not think I understand your problem correctly but here is how WebSocket API work in my experience.
client(s) <-(1)-> API Gateway <-(2)-> Lambda
1) is the web socket connection which stays open for a maximum of 2 hours, with idle timeout of 10 minutes as mentioned here. https://docs.aws.amazon.com/apigateway/latest/developerguide/limits.html
2 )communication is managed using #connection https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-how-to-call-websocket-api-connections.html
I believe you want to use #connection for talking to your API Gateway from lambda.
It looks like a NAT Gateway is required for a Lambda to call the postToConnection function provided by the AWS ApiGatewayManagementApi SDK.
Its a shame this requirement isn't specified anywhere regarding Websockets API or API Gateway.
Search for 'NAT' on this page, and read that sentence:
Related
I'm having trouble rewriting java test cases in robot framework.
in order to do this, i need to create new java keywords, but the way tests are implemented, don't make it easy !
this is an example of script that i need to rewrite in RF :
try
{
ServerSocket server = Utils.startSocketServer;
while(true)
{
Socket socket = server.accept();
ObjectInputStream ois = new ObjectInputStream(socket.getInputStream());
RequestX request = (RequestX) ois.readObject();
if(request.getSource().equals(String.INFO)
{
/** do something **/
}
else if(request.getSource().equals(String.X)
{
/** do something **/
}
else
{
/** do something **/
}
/** break on condition **/
}
Utils.closeSocketServer(server);
}catch(Exception e)
{
/** do something **/
}
Any suggestion on how i can make this into a RF test case !
Make the whole script into a single keyword is not an option because somewhere in that loop, in the do something comment, i also need to call keywords.
The main idea is to fragment this script into functions so that i can use them as java keywords in RF but i still can't figure this out!
So, i did further researches and this is what i came up with :
Split this code into functions so that i can call and use them as keywords in robot framework.
So code became like this :
public static String SendTask(String taskFile)
{
ServerSocket server = null;
try
{
server = startSocketServer();
if (taskFile != null)
{
Utils.sendJMSWakeUp();
while(true)
{
Socket socket = server.accept();
ObjectInputStream ois = getInputStream(socket);
RequestX request = (cast)ois.readObject();
if (getSource(request,Strings.INFO)
{
/** log info **/
}
/** if the current jms queue is Scheduler then send task !*/
else if (getSource(request,Strings.SCHEDULER))
{
/** send task **/
break;
}
}
}
else
{
assertion(false, "Illegal Argument Value null");
}
}catch (Exception e)
{
/** log errors **/
}finally
{
/** close socket server & return a task id **/
}
}
the same goes for every JMS queue that I am listening to
public static String getTaskAck(String taskId);
public static String getTaskresult(String taskId);
it did work in my case for synchronous task execution. But this is very incovenient for asynchronous task execution. Because each time i'll have to wait for response on keyword, so the next keyword may fail because the response that he is supposed to read was already sent !
i could look into process BuiltIn library or RobotFramework-Async library for parallel keyword execution but it will be harder to process for many asynchronous jms messages.
After further investigation, i think i will look into robotframework-jmsLibrary. some developpment enhancement has to be done like adding activeMq.
This way, i can send and consume many asynchronous messages via activeMq then process every message via robotframework-jmsLibrary
Example :
RF-jmsLibrary <==> synchronous <==> activeMq <==> asynchronous <==> system
i want to use the poloniex API. https://poloniex.com/support/api/
So far i made Jawampa ( https://github.com/Matthias247/jawampa ) running with IntelliJ.
My first Question is, how to login successfuly? (The Docu of Jawampa doesnt help)
I got a API Key and a Secret. Which functions i have to use in the builder of Jawampa:
withRealm
withRoles
withConnectorProvider
withConnectionConfiguration
withSerializations
withStrictUriValidation
withAuthId
withAuthMethod
withObjectMapper
I have so far this code
try {
WampClientBuilder builder = new WampClientBuilder();
builder.withConnectorProvider(connectorProvider)
.withUri("wss://api.poloniex.com")
.withAuthId("APIKEY")
.withRealm("realm2")
.withInfiniteReconnects()
.withReconnectInterval(1, TimeUnit.SECONDS);
client1 = builder.build();
} catch (Exception e) {
e.printStackTrace();
return;
}
Is wss://api.poloniex.com correct or should i use wss://api.poloniex.com/returnTicker for that client?
Do I have to make always a new client for every URI?
Thank you so much in advance.
My first Question is, how to login successfuly?
You don't have to authenticate to access Poloniex Push API via WAMP protocol. Push API methods are public, so you don't have to supply the API key and secret. Just connect to wss://api.poloniex.com and subscribe to a desired feed (Ticker, Order Book and Trades, Trollbox).
Btw, you need to supply the API Key only with Trading API methods. And the Secret is used to sign a POST data.
Which functions i have to use in the builder of Jawampa:
This is how you connect to the Push API:
WampClient client;
try {
WampClientBuilder builder = new WampClientBuilder();
IWampConnectorProvider connectorProvider = new NettyWampClientConnectorProvider();
builder.withConnectorProvider(connectorProvider)
.withUri("wss://api.poloniex.com")
.withRealm("realm1")
.withInfiniteReconnects()
.withReconnectInterval(5, TimeUnit.SECONDS);
client = builder.build();
} catch (Exception e) {
return;
}
Once your client is connected, you subscribe to a feed like this:
client.statusChanged().subscribe(new Action1<WampClient.State>() {
#Override
public void call(WampClient.State t1) {
if (t1 instanceof WampClient.ConnectedState) {
subscription = client.makeSubscription("trollbox")
.subscribe((s) -> { System.out.println(s.arguments()); }
}
}
});
client.open();
Is wss://api.poloniex.com correct or should i use
wss://api.poloniex.com/returnTicker for that client?
wss://api.poloniex.com is correct. Besides, returnTicker belongs to the Public API and is accessed via HTTP GET requests.
Do I have to make always a new client for every URI?
In respect to the Push API, once you connected a client to wss://api.poloniex.com, you can use this client to make subscriptions to multiple feeds. For example:
client.statusChanged().subscribe(new Action1<WampClient.State>() {
#Override
public void call(WampClient.State t1) {
if (t1 instanceof WampClient.ConnectedState) {
client.makeSubscription("trollbox")
.subscribe((s) -> { System.out.println(s.arguments()); });
client.makeSubscription("ticker")
.subscribe((s) -> { System.out.println(s.arguments()); });
}
}
});
However, according to Jawampa Docs:
After a WampClient was closed it can not be reopened again. Instead of this a new instance of the WampClient should be created if necessary.
I'll try to be brief.
I wish to create communication between 2 java apps (that will later be transported to android) without passing through a server. As such, I have spent weeks looking around, and after a lot of work I found stun and ice4j. The best explanation of how to use ice4j I found here, and it pretty much showed me what I need to do to add stun servers to an agent (I don't really know what an agent is, just that it manages my communications with STUN and TURN), through this code:
import java.io.IOException;
import java.net.InetAddress;
import java.net.UnknownHostException;
import java.util.logging.Level;
import java.util.logging.Logger;
import org.ice4j.Transport;
import org.ice4j.TransportAddress;
import org.ice4j.ice.Agent;
import org.ice4j.ice.IceMediaStream;
import org.ice4j.ice.harvest.StunCandidateHarvester;
public class ice4jTesting {
public static void main(String[] args) {
Agent agent = new Agent();
String[] hostnames = new String[] {"jitsi.org", "numb.viagenie.ca", "stun.ekiga.net"};
for(String hostname: hostnames) {
try {
TransportAddress address;
address = new TransportAddress(InetAddress.getByName(hostname), 3478, Transport.UDP);
agent.addCandidateHarvester(new StunCandidateHarvester(address));
} catch (UnknownHostException ex) {
Logger.getLogger(SimpleStun.class.getName()).log(Level.SEVERE, null, ex);
}
}
IceMediaStream stream = agent.createMediaStream("audio");
int port = 5000;
try {
agent.createComponent(stream, Transport.UDP, port, port, port+100);
// The three last arguments are: preferredPort, minPort, maxPort
} catch (IllegalArgumentException | IOException ex) {
Logger.getLogger(SimpleStun.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
however, after this the tutorial utilizes SDPUtils, a class that is in the source code of ice4j I found on github, to recieve the SDP information from the agent. However I got ice4j.jar from the central maven repository, and added it to my netbeans regular project (I did this because I am not very familiar with maven, and just wanted a regular library on my regular project). This jar library does not have the SDPUtils class, and since I don't really understand enough of this code to fix it myself, I was wondering if any of you could help me either fix the code above, or show me an example of how to answer the question on the title.
However, unless you can either do what I said in the last sentence, or point me to some sample code, your help will most likely not be useful, since I am mentally incapable of understanding the theory behind this completely because of the many concepts I do not know.
I have until the end of this week to figure this out, and if I don't I'm pretty screwed. So please, if you can or know someone that can help, I would extremely appreciate it.
Thanks for reading it so far and trying to help :)
There you go
SdpUtils.java
Actually i'm also working on the same as my University project. From last week i'm digging web for p2p connection establish over nat.
I know that form where you got above Code snipet, i would like to inform you that there is errors in that code Here is the one that i corrected
import java.io.IOException;
import java.net.BindException;
import java.net.InetAddress;
import org.ice4j.Transport;
import org.ice4j.TransportAddress;
import org.ice4j.ice.Agent;
import org.ice4j.ice.IceMediaStream;
import org.ice4j.ice.harvest.StunCandidateHarvester;
public class IceTest {
public static void main(String[] args) throws Exception {
Agent agent = new Agent(); // A simple ICE Agent
/*** Setup the STUN servers: ***/
String[] hostnames = new String[] { "jitsi.org", "numb.viagenie.ca", "stun.ekiga.net" };
// Look online for actively working public STUN Servers. You can find
// free servers.
// Now add these URLS as Stun Servers with standard 3478 port for STUN
// servrs.
for (String hostname : hostnames) {
try {
// InetAddress qualifies a url to an IP Address, if you have an
// error here, make sure the url is reachable and correct
TransportAddress ta = new TransportAddress(InetAddress.getByName(hostname), 3478, Transport.UDP);
// Currently Ice4J only supports UDP and will throw an Error
// otherwise
agent.addCandidateHarvester(new StunCandidateHarvester(ta));
} catch (Exception e) {
e.printStackTrace();
}
}
/*
* Now you have your Agent setup. The agent will now be able to know its
* IP Address and Port once you attempt to connect. You do need to setup
* Streams on the Agent to open a flow of information on a specific
* port.
*/
IceMediaStream stream = agent.createMediaStream("audio");
int port = 5000; // Choose any port
try {
agent.createComponent(stream, Transport.UDP, port, port, port + 100);
// The three last arguments are: preferredPort, minPort, maxPort
} catch (BindException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IllegalArgumentException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
/*
* Now we have our port and we have our stream to allow for information
* to flow. The issue is that once we have all the information we need
* each computer to get the remote computer's information. Of course how
* do you get that information if you can't connect? There might be a
* few ways, but the easiest with just ICE4J is to POST the information
* to your public sever and retrieve the information. I even use a
* simple PHP server I wrote to store and spit out information.
*/
String toSend = null;
try {
toSend = SdpUtils.createSDPDescription(agent);
// Each computersends this information
// This information describes all the possible IP addresses and
// ports
} catch (Throwable e) {
e.printStackTrace();
}
/*The String "toSend" should be sent to a server. You need to write a PHP, Java or any server.
* It should be able to have this String posted to a database.
* Each program checks to see if another program is requesting a call.
* If it is, they can both post this "toSend" information and then read eachother's "toSend" SDP string.
* After you get this information about the remote computer do the following for ice4j to build the connection:*/
String remoteReceived = ""; // This information was grabbed from the server, and shouldn't be empty.
SdpUtils.parseSDP(agent, remoteReceived); // This will add the remote information to the agent.
//Hopefully now your Agent is totally setup. Now we need to start the connections:
agent.addStateChangeListener(new StateListener()); // We will define this class soon
// You need to listen for state change so that once connected you can then use the socket.
agent.startConnectivityEstablishment(); // This will do all the work for you to connect
}
}
This code Requires SIP server to be setup and the one on ice4j test is saying something else just have a look at Ice.java
My application sends message to Amazon Simple Notification Service (SNS) topic but sometime (6/10) I get java.net.UnknownHostException:sqs.ap-southeast-1.amazonaws.com. The reason of exception is described in the amazon web services discussion forums, please look: https://forums.aws.amazon.com/thread.jspa?messageID=499290񹹚.
My problem is similar to what described in forums of amazon but my rate of publishing messages to topic is very dynamic. It can be 1 message/second or 1 message/minute or no message in an hour. I am looking for a cleaner, better and safe approach, which guaranties sending of message to SNS topic.
Description of problem in detail:
Topic_Arn= arn of SNS topic where application wants to publish message
msg = Message to send in topic
// Just a sample example which publish message to Amazon SNS topic
class SimpleNotificationService {
AmazonSNSClient mSnsClient = null;
static {
createSnsClient()
}
private void static createSnsClient() {
Region region = Region.getRegion(Regions.AP_SOUTHEAST_1);
AWSCredentials credentials = new
BasicAWSCredentials(AwsPropertyLoader.getInstance().getAccessKey(),
AwsPropertyLoader.getInstance().getSecretKey());
mSqsClient = new AmazonSQSClient(credentials);
mSqsClient.setRegion(region);
}
public void static publishMessage(String Topic_Arn, String msg) {
PublishRequest req = new PublishRequest(Topic_Arn, msg);
mSnsClient.publish(req);
}
}
class which calls SimpleNotificationService
class MessagingManager {
public void sendMessage(String message) {
String topic_arn = "arn:of:amazon:sns:topic";
SimpleNotificationService.publishMessage(topic_arn, message);
}
}
Please note that this is a sample code, not my actual code. Here can be class design issue but please ignore those if they are not related to problem.
My thought process says to have try-catch block inside sendMessage, so when we catch UnknownHostException then again retry but I am not sure how to write this in safer, cleaner and better way.
So MessagingManager class will look something like this:
class MessagingManager {
public void sendMessage(String message) {
String topic_arn = "arn:of:amazon:sns:topic";
try {
SimpleNotificationService.publishMessage(topic_arn, message);
} catch (UnknownHostException uhe) {
// I need to catch AmazonClientException as aws throws
//AmazonClientException when sees UnknownHostException.
// I am mentioning UnknownHostException for non-aws user to understand
// my problem in better way.
sendMessage(message); // Isn't unsafe? - may falls into infinite loop
}
}
}
I am open for answers like this: java.net.UnknownHostException: Invalid hostname for server: local but my concern is to dependent on solution at application code-level and less dependent on changes to machine. As my server application is going to run in many boxes (developer boxes, testing boxes or production boxes). If changes in machine host-files or etc is only guaranted solution then I prefer that to include with code level changes.
Each AWS SDK implements automatic retry logic. The AWS SDK for Java automatically retries requests, and you can configure the retry settings using the ClientConfiguration class.
Below is the sample example to create SNS client. It retries for 25 times if encounters UnKnownHostException. It uses default BackOff and retry strategy. If you want to have your own then you need to implement these two interfaces: http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/retry/RetryPolicy.html
private void static createSnsClient() {
Region region = Region.getRegion(Regions.AP_SOUTHEAST_1);
AWSCredentials credentials = new
BasicAWSCredentials(AwsPropertyLoader.getInstance().getAccessKey(),
AwsPropertyLoader.getInstance().getSecretKey());
ClientConfiguration clientConfiguration = new ClientConfiguration();
clientConfiguration.setMaxErrorRetry(25);
clientConfiguration.setRetryPolicy(new RetryPolicy(null, null, 25, true));
mSnsClient = new AmazonSNSClient(credentials, clientConfiguration);
mSnsClient.setRegion(region);
}
Have you considering looking into the JVM TTL for the DNS Cache?
http://docs.aws.amazon.com/AWSSdkDocsJava/latest//DeveloperGuide/java-dg-jvm-ttl.html
I'm trying to use BlazeDS's AMFConnection class to connect to pyamf, but when I call AMFConnection.call(), I get HTTP status 400 (Bad Request - "The request body was unable to be successfully decoded."). I'm more or less following this example: (pyamf.org/wiki/ClientHowTo ... sorry, I'm a new user so I guess I can't use hyperlinks. append a "http://" to those if you want to follow them)
Here's my code:
package amfconnectiontest;
import flex.messaging.io.amf.client.AMFConnection;
import flex.messaging.io.amf.client.exceptions.*;
public class Main {
public static void main(String[] args) {
AMFConnection amfConnection = new AMFConnection();
String url = "http://demo.pyamf.org/gateway/recordset";
String service = "service.getLanguages";
try
{
amfConnection.connect(url);
}
catch (ClientStatusException cse)
{
System.out.println(cse);
return;
}
// Make a remoting call and retrieve the result.
try
{
Object result = amfConnection.call(service);
System.out.println("results: " + result.toString());
}
catch (ClientStatusException cse)
{
System.out.println(cse);
}
catch (ServerStatusException sse)
{
System.out.println(sse);
}
// Close the connection.
amfConnection.close();
}
}
Any ideas?
The ability to en/decode BlazeDS specific messages (implementing ISmallMessage) has landed on the PyAMF trunk (r2726 and up). See the related ticket - http://pyamf.org/ticket/581
This version or one very similar is likely to become 0.5. If you need to connect to a BlazeDS service I would suggest checking out the trunk.