Develop Reference

How to mock response of an external API called from within an internal API integration test

I am writing integration tests for an application that uses an internal REST API to perform an action; all Java.
In the API, there is a POST method that calls upon an external API. In the test, I need to send a request to my API to perform an action.
The problem is, I do not want to send real requests to the external API when running the integration tests.
How can I mock the response of the external API call?
Is there a way I can still send a POST request to my API (mocked or otherwise) within my test, but use a mocked response for the external call performed in the Java POST method?

I've done this as follows: create a service layer that makes the API calls to external services. I built mine on Spring's RestTemplate but you could use whatever library to make calls. So it'll have methods like get() or post().
Then I use Postman to perform some requests against the external API and save the responses in files and add these files to my project.
Finally, in my tests, I mock out the calls to my little API service layer so that instead of going to the external API, it reads from the test files I saved previously. This runs the code under test with known payloads that came from the external API, but without requiring a connection to it during the test, and which won't change until I update the responses in the files myself.
I use EasyMock but any mocking library will work. Here's an example of what a test looks like.
#Test
public void test_addPhoneToExternalService() {
// Mock my real API service.
ApiService mockApiService = EasyMock.createMock(ApiService.class);
// Construct my service under test using my mock instead of the real one.
ServiceUnderTest serviceUnderTest = new ServiceUnderTest(mockApiService);
// Expect the body of the POST request to look like this.
Map<String, Object> requestBody = new HashMap<String, Object>() {{
put("lists", 0);
put("phone", "800-555-1212");
put("firstName", "firstName");
put("lastName", "lastName");
}};
// Read the response that I manually saved as json.
JsonNode expectedResponse = Utils.readJson("response.json");
expect(mockApiService.post(anyObject(),
eq("https://rest.someservice.com/api/contacts"), eq(serialize(requestBody))))
.andReturn(expectedResponse);
EasyMock.replay(mockApiService);
// Call the code under test. It ingests the response
// provided by the API service, which is now mocked,
// and performs some operations, then returns a value
// based on what the response contained (for example,
// "{\"id\":42}").
long id = serviceUnderTest.addPhone("firstName", "lastName", "800-555-1212");
Assert.assertThat(id, is(42L));
EasyMock.verify(mockApiService);
}
Hope that helps!

Is there a way to get a two-way connection from client-browser to server application simulating a shell?

Problem
Very short: I want to create Spring Shell, but as a web application.
I want to create a web-application (preferably using Spring Boot), where the frontend (ReactJS) looks like a terminal (or shell), and the backend processes inputted commands. Look at https://codepen.io/AndrewBarfield/pen/qEqWMq. I want to build a full web app for something that looks like that.
I want to build a framework, so that I can develop backend commands without knowing anything about the frontend/web application structure. I basically want to instantiate a "Terminal" object, where I give some kind of input-stream and output-stream. This way I can program this Terminal based on my given interfaces and structure, without the need of setting up all kind of front-end stuff.
A good summary of the question would be: how to send all keyboard inputs to the backend, and how to send all output to the frontend?
The reason I want to create a web application, is because I want it to be available online.
What I tried
I think the way of reaching this is using websockets. I have created a small web application using this (https://developer.okta.com/blog/2018/09/25/spring-webflux-websockets-react) tutorial, without the security part. The websocket part is almost suitable, I just cannot get an "input" and "output" stream-like object.
#Controller
public class WebSocketController {
private SimpMessagingTemplate simpMessagingTemplate;
#Autowired
public WebSocketController(SimpMessagingTemplate simpMessagingTemplate) {
this.simpMessagingTemplate = simpMessagingTemplate;
}
#MessageMapping("/queue")
#SendToUser("/topic/greetings")
public Greeting greeting(HelloMessage message, #Header(name = "simpSessionId") String sessionId) throws Exception {
System.out.println(sessionId);
// Do some command parsing or whatever.
String output = "You inputted:" + HtmlUtils.htmlEscape(message.getName());
return new Greeting(output);
}
private MessageHeaders createHeaders(String sessionId) {
SimpMessageHeaderAccessor headerAccessor = SimpMessageHeaderAccessor.create(SimpMessageType.MESSAGE);
headerAccessor.setSessionId(sessionId);
return headerAccessor.getMessageHeaders();
}
Now with this code, you can parse a command. However, it doesn't keep any "state". I don't know how it works with states and websockets.
I saw you had this Spring Sessions + WebSockets (https://docs.spring.io/spring-session/docs/current/reference/html5/guides/boot-websocket.html), but this is not really what I want.
I can send a message from the backend to the frontend by using this code:
simpMessagingTemplate.convertAndSendToUser(sessionId, "/topic/greetings", "hey", createHeaders(sessionId));
However, I want my terminal to be able to wait for input commands from the user. Seems like a stretch, but does anybody know how to achieve this?
What I sort of want
I basically want other people to program to this interface:
public interface ITerminal {
void setInputStream(Object someKindOfWrapperForTheInput);
void setOutputStream(Object someWrapperOfSimpMessagingTemplate);
void start();
}
When somebody opens the web application, they get a dedicated terminal object (so a single connection per user). Whever somebody enters a command in the frontend application, I want it to be received by the terminal object, processed, and response outputted to the frontend.
Reasons for doing this
I really like creating command-line applications, and I don't like building frontend stuff. I work as a software engineer for a company where we build a web application, where I mostly program backend stuff. All the frontend part is done by other people (lucky for me!). However, I like doing some projects at home, and this seemed cool.
If you have any thoughts or ideas on how to approach this, just give an answer! I am interested in the solution, using the SpringBoot framework is not a requirement. I ask this question using Spring Boot and ReactJS, because I have already built applications with that. A lot has been figured out already, and I think this probably exists as well.
The only requirement is that I can achieve this with Java on a tomcat-server. The rest is optional :)
Unclear?
I tried my best to make my story clear, but I am not sure if my purpose of what I want to achieve is clear. However, I don't know how to formulate it in such a way you understand. If you have any suggestions or questions, dont hesitate to comment!

If the only thing you want is a Live Spring shell that shows up in the browser it's fairly simple, all you need is to expose a standard WebSocket via the WebSocketConfigurer, then add a WebSocketHandler that executes the command and then returns the resulting String as a TextMessage.
Firstly the Socket configuration that allows clients to connect to the 'cli' endpoint
#Configuration
#EnableWebSocket
public class WebSocketConfiguration implements WebSocketConfigurer {
#Override
public void registerWebSocketHandlers(WebSocketHandlerRegistry registry) {
registry.addHandler(cliHandler(), "/cli").setAllowedOrigins("*");
}
#Bean
public CLIWebSocketHandler cliHandler() {
return new CLIWebSocketHandler();
}
}
Then the WebSocketHandler that executes the command. I recommend that for every #ShellMethod you specify the return type as String, don't use logging or System writes as they won't be returned during the evaluation.
#Component
public class CLIWebSocketHandler extends TextWebSocketHandler {
#Autowired
private Shell shell;
#Override
protected void handleTextMessage(WebSocketSession session, TextMessage message) throws Exception {
String result = shell.evaluate(() -> message.getPayload()).toString();
session.sendMessage(new TextMessage(result));
}
}
You can use an extension like Simple WebSocket Client to test it, by going to ws://localhost:port/cli
This is the most basic solution, adding features like security should be easy after this. Notice that I don't use STOMP, because you probably want to isolate users. But it can work alongside STOMP based endpoints, so you can have pub-sub functionality for other parts of the project.
From the question I sense that answer you'd like is something that involved Input and OutputStreams. You could possibly look into redirecting the output of Spring Shell to a different stream then have them forwarded to the sessions but it's probably much more complicated and has other trade-offs. It's simpler to just return a String as the result, it looks better in print outs anyway.

Java commons-httpclient: Testing timeout values

Abstract: how do devs Integration TEST timeouts for http requests?
Backstory: My team is having issues related to unusually long lasting HTTP web requests. We use the commons-httpclient version 3 by Apache. The code looks similar to this:
PostMethod post = new PostMethod(endpoint);
post.getParams().setSoTimeout(someInt);
httpClient.executeMethod(post);
The time to complete this request is usually acceptable (2 seconds or so), but occasionally, we will see 50-60 second requests despite having our SO timeout set to 4 seconds. This prompted me to do some research and found that most people are setting Connection Timeouts ANNNNND SO timeouts. It appears that SO timeouts should be set lower (as they simply time the distance between bytes in transit) and the the connection timeout is what we originally planned to use (i.e. initial delay between request and 1st byte returned).
Here is the code we scraped and plan on using:
httpClient.getHttpConnectionManager().getParams()
.setConnectionTimeout(someInt);
httpClient.getHttpConnectionManager().getParams()
.setSoTimeout(someInt);
The main pain here is that we are unable to integration test this change. More precisely, we are confused on how to integration test the delays coming from a socket connection to a foreign server. After digging through the commons-httpclient, I see protected and private classes that we will have to reproduce (because they are unextendable and unusable from outside the class), mock and string together the classes to ultimately get down to the socket class in java (which relies on a java native method -- which we would also need to reproduce and inject via mocks -- something I dont see frequently at that level).
The reason I am reaching out to Stack Overflow is to see how others are testing this/not testing this. I want to avoid testing this functionality in a performance environment at all costs.
Another thought of mine was to set up a mockserver to respond to the httpclient with a programmable delay time. I haven't seen an example of that yet.

First of all, there is no such thing as unit testing http requests - that would be integration testing.
Secondly, you can use a tool like JMeter to send http requests and test whether the response is being received in a certain amount of time as shown here in JMeter.

Taking the mock server route, I managed to set up a small web server with an API endpoint that I could test against. Here is the related code:
Lightweight server setup:
TJWSEmbeddedJaxrsServer server = new TJWSEmbeddedJaxrsServer();
server.setPort(SERVER_PORT);
server.getDeployment().getResources().add(new TestResource());
server.start();
API Endpoint:
/**
* In order to test the timeout, the resource will be injected into an embedded server.
* Each endpoint should have a unique use case.
*/
#Path("tests")
public class TestResource {
#POST
#Produces({MediaType.APPLICATION_XML})
#Path("socket-timeout")
public Response testSocketTimeout() throws InterruptedException {
Thread.sleep(SOCKET_TIMEOUT_SLEEP);
return Response.ok().build();
}
}
Within the api endpoint related class, I can control the sleep timeout which then triggers a socket timeout within the httpclient class. Its a bit hacky, but it works to test the functionality in the way I wanted to (simple, lightweight and effective).

Jersey client get sent data to String

I am building some JUnit tests for a REST client using Jersey, I therefore need to have a copy of the data sent to the server to run some tests in JUnit.
Currently my clients invokes:
Invocation invocation = serviceWebTarget.request(MediaType.APPLICATION_JSON).
buildPut(Entity.json((QARecord) valuesList.get(0)));
Response response = invocation.invoke();
In between the two calls the QARecord object is serialized to JSON and sent to the server but I cannot find a way to access it.
By debugging the code I found no variable in either invocation or response which contains the converted JSON text.
How can get the sent data into a String or a File for my JUnit test to check what has been sent?

As i understood you want to check what exactly client will sent to server as a request, am i right ?
If yes how exactly does your Unit test look like ?
For instance Jersey provides JerseyTest class which is base for testing of client code.
In few words such test will run special testcontainer which is able to execute your handlers inside.
By combining it with Mockito / or creating your own handlers by yourself, you can verify what is "captured" by them as a client request at the end of the test (when response is received by client). Among others it'll give you possibility to check not only what your client code is sending to server but also check behaviour of client by emulating various responses (successful or exceptional).
If you just want to get content of what client is really send to the server you can write jersey client filter and get body of request from there.
Filters and Interceptors

How do I unit test code which calls the Jersey Client API?

I wrote code which calls the Jersey client API which in turn calls a web service which is out of my control. I do not want my unit test to call the actual web service.
What is the best approach for writing a unit test for code which calls the Jersey client API? Should I use the Jersey server API to write a JAX-RS web service and then use the Jersey Test Framework for the unit test? Or should I mock out the Jersey web service calls? I have access to JMock. Or should I try another approach?
During my research, I found this discussion describing various options, but I did find a complete solution. Are there any code examples available showing a suggested JUnit approach? I could not find any in the Jersey documentation.
Here is the relevant source code:
public String getResult(URI uri) throws Exception {
// error handling code removed for clarity
ClientConfig clientConfig = new DefaultClientConfig();
Client client = Client.create(clientConfig);
WebResource service = client.resource(uri);
String result = service.accept(accept).get(String.class);
return result;
}
Here are examples of test code I would like to pass. I would like to test (1) passing in a valid URI and getting a valid string back and (2) passing in an invalid (for whatever reason -- unreachable or unauthorized) URI and getting an exception back.
#Test
public void testGetResult_ValidUri() throws Exception {
String xml = retriever.getResult(VALID_URI);
Assert.assertFalse(StringUtils.isBlank(xml));
}
#Test(expected = IllegalArgumentException.class)
public void testGetResult_InvalidUri() throws Exception {
retriever.getResult(INVALID_URI);
}
Everything above is the simple description of what my code does. In reality, there is a layer on top of that that accepts two URIs, first tries calling the first URI, and if that URI fails then it tries calling the second URI. I would like to have unit tests covering (1) the first URI succeeds, (2) the first URI fails and the second URI succeeds, and (3) both URIs fail. This code is sufficiently complex that I want to test these different scenarios using JUnit, but to do this I either need to run actual stand-in web services or mock out the Jersey client API calls.

Try to use Mockito or Easymock for mocking service calls. You need to mock only these methods which are actually used - no need to mock every method. You can creat mock object for WebResource class, then mock accept method call.
In #BeforeClass/#Before JUnit test method write something like (Mockito example)
WebResource res = mock(WebResource.class);
when(res.accept(something)).thenReturn(thatWhatYouWant);
Then in your tests you can use res object as if it was real object and call mock method on it. Instead of returning value you can also throw exceptions. Mockito is pretty cool.

Typically what you are really after is "does the way I use the Jersey Client DSL produce a request to the correct URL with the correct payload and URL parameters". Testing this with Mockito is really verbose and the setup code will usually end up looking something like this:
when(authentication.queryParam(eq("sa"), anyBoolean())).thenReturn(testAuthentication);
when(testAuthentication.resolveTemplate("channel", "smf")).thenReturn(testAuthentication);
when(testAuthentication.request(
MediaType.APPLICATION_JSON_TYPE)).thenReturn(mockRequestBuilder);
when(mockRequestBuilder.post(any(Entity.class))).thenReturn(mockResponse);
when(mockResponse.readEntity(ResponseWrapper.class)).thenReturn(successfulAuthResponse());
And this is basically just for a single REST request. It's overly verbose, and instead of testing the hoped outcome you are just replicating the steps you think are correct in using the Jersey Client DSL.
Instead of the above, I would aim for mocking a simple service. For this I've used WireMock which starts a Jetty server and where I can stub things like "expect a request to this URL, respond with this message and verify that the payload is this".
I know this is edging on an integration test and it is a bit slower than just using Mockito but I value testing the real outcome and I value the readability of the tests way more in this case.
Setup for a WireMock based Jersey Client test looks something like this:
#Test
public void exactUrlOnly() {
stubFor(get(urlEqualTo("/some/thing"))
.willReturn(aResponse()
.withHeader("Content-Type", "text/plain")
.withBody("Hello world!")));
assertThat(testClient.get("/some/thing").statusCode(), is(200));
assertThat(testClient.get("/some/thing/else").statusCode(), is(404));
}

Just implement a work-alike service and in your unit test setup start the service using HttpServerFactory.