I have an AWS Lambda implemented in java. The lambda generates a file, then writes it to the output, using the Base64 encoder. I'm trying to write a unit test for it, but it enters an infinite loop when the file is written.
What I'd like to do is capture what is written to the encodedStream in the unit test, write it to the temporary folder, and then compare the contents to the expected contents, but the test hangs until eventually an out of memory exception is thrown.
Lambda code
public class MyLambda implements RequestStreamHandler {
private static final Logger LOGGER = LogManager.getLogger(MyLambda.class);
#Override
public void handleRequest(#Nonnull InputStream inputStream, #Nonnull OutputStream outputStream, #Nonnull Context context) {
try (OutputStream encodedStream = Base64.getEncoder().wrap(outputStream);){
encodedStream.write("This is written to file".getBytes());
} catch (IOException e) {
LOGGER.info("IOException occurred ", e);
}
}
}
Unit test
public class MyLambdaTest {
#Rule
public TemporaryFolder temporaryFolder = new TemporaryFolder();
#Test
public void testRequest() throws IOException {
MyLambda myLambda = new MyLambda();
InputStream inputStream = mock(InputStream.class);
OutputStream mockOutputStream = mock(OutputStream.class);
Context mockContext = mock(Context.class);
doNothing().when(mockOutputStream).write(anyInt());
doNothing().when(mockOutputStream).write(any(byte[].class));
doNothing().when(mockOutputStream).write(any(byte[].class), anyInt(), anyInt());
myLambda.handleRequest(inputStream, mockOutputStream, mockContext);
FileUtils.writeByteArrayToFile(temporaryFolder.newFile(), <captured bytes>);
}
}
I have deployed the code to AWS, so I know it works, but I'd like to have a proper unit test written for it for future builds
Instead of mocking the OutputStream, you can create a ByteArrayOutputStream. It's basically just an array of bytes that implements OutputStream. And then you can verify the correct content was written with ByteArrayOutputSteam#toBytes(), or ByteArrayOutputStream#toString()
FileOutputStream fout =
new FileOutputStream(temporaryFolder.newFile("testout.txt"));
MyLambda myLambda = new MyLambda();
myLambda.handleRequest(null, fout, null);
fout.close();
Hi Joseph,
Please find my attempt above. I have used a real FileOutputStream.
Related
/*
this method will create the required manifest file in compatible format such that
quicksight can import data from specified s3 bucket
*/
private JSONObject CreateManifestFileJSONObject(JSONObject ManifestFile){
JSONArray URIPrefixArray= new JSONArray();
URIPrefixArray.put(PrefixLocation);
JSONObject URIPrefixJSONObject= new JSONObject();
URIPrefixJSONObject.put("URIPrefixes",URIPrefixArray);
JSONArray FileLocationsArray= new JSONArray();
FileLocationsArray.put(URIPrefixJSONObject);
JSONObject globalUploadSettings= new JSONObject();
globalUploadSettings.put("format","JSON");
ManifestFile.put("globalUploadSettings",globalUploadSettings);
ManifestFile.put("fileLocations",FileLocationsArray);
return(ManifestFile);
}
/*
this method will upload the ManifestFile to same S3 Bucket in which data files is stored
*/
private void UploadManifestFileJSONObjectToS3(JSONObject ManifestFile){
try {
AmazonS3 S3Client = new Utility().SetUpS3Client();
byte[] fileContentBytes = (ManifestFile.toString()).getBytes();
InputStream fileInputStream = new ByteArrayInputStream(fileContentBytes);
ObjectMetadata objectMetadata = new ObjectMetadata();
objectMetadata.setContentLength(fileContentBytes.length);
S3Client.putObject(new PutObjectRequest(Bucket_Name, ManifestFileName, fileInputStream, objectMetadata).withCannedAcl(CannedAccessControlList.PublicRead));
}
catch(Exception exception){
exception.printStackTrace();
}
}
public void handler() {
System.out.println("inside the manifest file");
try {
JSONObject ManifestFile = new JSONObject();
ManifestFile = CreateManifestFileJSONObject(ManifestFile);
UploadManifestFileJSONObjectToS3(ManifestFile);
}
catch(Exception exception){
exception.printStackTrace();
}
I want to test handler method but handlor method is calling private methods so I do not know how to write the test class for this code.
i want to write unit test for this class please help
this is the test class I am able to create up to this point but it will not surely mock s3 behaviour
#Test
public void handler() {
ManifestFileHandler manifestFileHandler=new ManifestFileHandler();
manifestFileHandler.handler();
}
You can test private methods with the help of PowerMock provides utilities which uses reflection to do certain things.
check the below example,
https://examples.javacodegeeks.com/core-java/mockito/mockito-mock-private-method-example-with-powermock/
There are two schools of thought on unit testing private functions. The first is that you make them public (or protected or package accessible) and test them as you would a public function. The second is that if they are private they are part of the encapsulated implementation detail and you only need to test them through the public functions.
My personal view is that complicated private functions are often a sign that you are breaking the single responsibility principle and it's likely you should have logic in private functions that should be split into a separate class that can then be tested through its public methods.
With respect to the code you've posted you have a larger problem than how to test the private functions: your class depends on other classes that you don't have control over. You have no way of mocking the behaviour of those classes to test various scenarios or to verify that they have been called correctly. I suspect it is this problem that is really behind your question.
As an example, I would suggest you inject a S3Client into your class rather than create it internally through new Utility().SetUpS3Client(). That way you can mock its behaviour and verify it is called correctly by your code. Attempting to do that with the real version of this class will be challenging.
So using this model, your code might look something like:
public class ManifestFileHandler {
private final S3Client client;
public ManifestFileHandler(S3Client client) {
this.client = client;
}
private void upload(JSONObject manifestFile) {
...
client.putObject(...);
}
public void handleManifest() {
...
upload(manifestFile);
...
}
}
And your test code (using mockito):
#Test
void testManifestUpload() {
S3Client client = mock(S3Client.class);
ManifestFileHandler handler = new ManifestFileHandler(client);
handler.handleManifest();
verify(client).putObject(expectedObject);
}
If you need to capture the argument passed to putObject and assert various aspects of it then that is possible with most mocking tools (including mockito) but is beyond the scope of your question.
How do you mock file reading/writing via JUnit?
Here is my scenario
MyHandler.java
public abstract class MyHandler {
private String path = //..path/to/file/here
public synchronized void writeToFile(String infoText) {
// Some processing
// Writing to File Here
File file = FileUtils.getFile(filepath);
file.createNewFile();
// file can't be written, throw FileWriteException
if (file.canWrite()) {
FileUtils.writeByteArrayToFile(file, infoText.getBytes(Charsets.UTF_8));
} else {
throw new FileWriteException();
}
}
public String readFromFile() {
// Reading from File here
String infoText = "";
File file = new File(path);
// file can't be read, throw FileReadException
if (file.canRead()) {
infoText = FileUtils.readFileToString(file, Charsets.UTF_8);
} else {
throw FileReadException();
}
return infoText
}
}
MyHandlerTest.java
#RunWith(PowerMockRunner.class)
#PrepareForTest({
MyHandler.class
})
public class MyHandlerTest {
private static MyHandler handler = null;
// Some Initialization for JUnit (i.e #Before, #BeforeClass, #After, etc)
#Test(expected = FileWriteException.class)
public void writeFileTest() throws Exception {
handler.writeToFile("Test Write!");
}
#Test(expected = FileReadException.class)
public void readFileTest() throws Exception {
handler.readFromFile();
}
}
Given above source, Scenario when file is not writable (write permission not allowed) is OK, However, when i try to do scenario wherein file is not readable (read permission not allowed). It always read the file, i have already tried to modify the file permission on the test code via below
File f = new File("..path/to/file/here");
f.setReadable(false);
However, I did some reading, setReadable() always returns false (failed) when run on Windows machine.
Is there a way to modify the file permission of the target file programmatically in relation to JUnit?
Note
Target source code to test cannot be modified, meaning
Myhandler.class is a legacy code which is not to be modified.
Instead of relying on the operating system file permissions, use PowerMock to mock FileUtils.getFile(...) and make it return an instance of File (e.g. anonymous sub class) that returns a specific value for canWrite()/canRead().
Mocking static methods with Mockito
Since Mockito cannot mock static methods, use a File factory instead (or refactor your FileUtils to be a factory), then you can mock it and return a mocked File instance as well, where you can also mock any File methods you want.
So instead of FileUtils.getFile(filepath) you will now have something like FileFactory.getInstance().getFile(filepath) for example, where you can mock getFile(String) method easily.
In jUnit there's a handy rule for scenarios like yours.
public class MyHandlerTest {
#Rule
// creates a temp folder that will be removed after each test
public org.junit.rules.TemporaryFolder folder = new org.junit.rules.TemporaryFolder();
private MyHandler handler;
#Before
public void setUp() throws Exception {
File file = folder.newFile("myFile.txt");
// do whatever you need with it - fill with test content and so on.
handler = new MyHandler(file.getAbsolutePath()); // use the real thing
}
// Test whatever behaviour you need with a real file and predefined dataset.
}
I have got an OutputStream which can be initialized as a chain of OutputStreams. There could be any level of chaining .Only thing guaranteed is that at the end of the chain is a FileOutputStream.
I need to recreate this chained outputStream with a modified Filename in FileOutputStream. This would have been possible if out variable (which stores the underlying chained outputStream) was accessible ; as shown below.
public OutputStream recreateChainedOutputStream(OutputStream os) throws IOException {
if(os instanceof FileOutputStream) {
return new FileOutputStream("somemodified.filename");
} else if (os instanceof FilterOutputStream) {
return recreateChainedOutputStream(os.out);
}
}
Is there any other way of achieving the same?
You can use reflection to access the os.out field of the FilterOutputStream, this has however some drawbacks:
If the other OutputStream is also a kind of RolloverOutputStream, you can have a hard time reconstructing it,
If the other OutputStream has custom settings, like GZip compression parameter, you cannot reliable read this
If there is a
A quick and dirty implementation of recreateChainedOutputStream( might be:
private final static Field out;
{
try {
out = FilterInputStream.class.getField("out");
out.setAccessible(true);
} catch(Exception e) {
throw new RuntimeException(e);
}
}
public OutputStream recreateChainedOutputStream(OutputStream out) throws IOException {
if (out instanceof FilterOutputStream) {
Class<?> c = ou.getClass();
COnstructor<?> con = c.getConstructor(OutputStream.class);
return con.invoke(this.out.get(out));
} else {
// Other output streams...
}
}
While this may be ok in your current application, this is a big no-no in the production world because the large amount of different kind of OutputStreams your application may recieve.
A better way to solve would be a kind of Function<String, OutputStream> that works as a factory to create OutputStreams for the named file. This way the external api keeps its control over the OutputStreams while your api can adress multiple file names. An example of this would be:
public class MyApi {
private final Function<String, OutputStream> fileProvider;
private OutputStream current;
public MyApi (Function<String, OutputStream> fileProvider, String defaultFile) {
this.fileProvider = fileProvider;
selectNewOutputFile(defaultFile);
}
public void selectNewOutputFile(String name) {
OutputStream current = this.current;
this.current = fileProvider.apply(name);
if(current != null) current.close();
}
}
This can then be used in other applications as:
MyApi api = new MyApi(name->new FileOutputStream(name));
For simple FileOutputStreams, or be used as:
MyApi api = new MyApi(name->
new GZIPOutputStream(
new CipherOutputStream(
new CheckedOutputStream(
new FileOutputStream(name),
new CRC32()),
chipper),
1024,
true)
);
For a file stream that stored checksummed using new CRC32(), chipped using chipper, gzip according to a 1024 buffer with sync write mode.
I am trying to store an OutputStream as a member of a class so that I can write to it from multiple methods. I put together this jUnit test to demonstrate the problem I have.
public class XmlStreamWriterTest {
private OutputStream outputStream;
#Before
public void setUp() throws Exception {
File file = new File("xmltester.xml");
this.outputStream = new FileOutputStream(file);
}
#After
public void tearDown() throws Exception {
this.outputStream.close();
}
//This doesn't work.
#Test
public void testOutputStream() throws Exception {
XMLOutputFactory xmlOutputFactory = XMLOutputFactory.newInstance();
XMLStreamWriter xmlStreamWriter = xmlOutputFactory.createXMLStreamWriter(this.outputStream);
xmlStreamWriter.writeStartElement("test");
xmlStreamWriter.writeCharacters("This is a test.");
xmlStreamWriter.writeEndElement();
xmlStreamWriter.flush();
}
//This works
#Test
public void testOutputStreamLocal() throws Exception {
File file = new File("xmltester2.xml");
OutputStream outputStreamLocal = new FileOutputStream(file);
XMLOutputFactory xmlOutputFactory = XMLOutputFactory.newInstance();
XMLStreamWriter xmlStreamWriter = xmlOutputFactory.createXMLStreamWriter(outputStreamLocal);
xmlStreamWriter.writeStartElement("test");
xmlStreamWriter.writeCharacters("This is a test.");
xmlStreamWriter.writeEndElement();
xmlStreamWriter.close();
}
}
Of the resulting files, only the second method pushes any values to the file. Do I have to pass the OutputStream to every method directly? Why doesn't the testOutputStream() method work?
I'm using the jrockit jdk 1.6.0_29, but I tried running on JDK 8 and it worked the same.
If you step through this test in a debugger and put a breakpoint on the "flush/close" (i.e. last) line in each test, when you step over it you can see that the file is written in both cases.
The problem is your setup method.
This is what is happening...
Setup is called, outputStream created (this will overwrite any existing file!!)
testOutputStream DOES work and outputs the file
Setup is called again, as an outputstream is created here, the file from the first test will be overwritten
The second test sets up another output stream
Basically, move the code out of your setup method into the first test case
I have class that gets GenericFile as input argument reads data and does some additional processing. I need to test it:
public class RealCardParser {
public static final Logger l = LoggerFactory.getLogger(RealCardParser.class);
#Handler
public ArrayList<String> handle(GenericFile genericFile) throws IOException {
ArrayList<String> strings = new ArrayList<String>();
FileInputStream fstream = new FileInputStream((File) genericFile.getFile());
DataInputStream in = new DataInputStream(fstream);
BufferedReader br = new BufferedReader(new InputStreamReader(in));
String strLine = br.readLine();//skip header
while ((strLine = br.readLine()) != null) {
l.info("handling in parser: {}", strLine);
strings.add(strLine);
}
br.close();
return strings;
}
}
The issue is with new FileInputStream. I can mock GenericFile but it is useless cause FileInputStream checks if file exists. I changed my class so:
public class RealCardParser {
public static final Logger l = LoggerFactory.getLogger(RealCardParser.class);
protected BufferedReader getBufferedReader(GenericFile genericFile) throws FileNotFoundException {
FileInputStream fstream = new FileInputStream((File) genericFile.getFile());
DataInputStream in = new DataInputStream(fstream);
return new BufferedReader(new InputStreamReader(in));
}
#Handler
public ArrayList<String> handle(GenericFile genericFile) throws IOException {
ArrayList<String> strings = new ArrayList<String>();
BufferedReader br = getBufferedReader(genericFile);
String strLine = br.readLine();//skip header
while ((strLine = br.readLine()) != null) {
l.info("handling in parser: {}", strLine);
strings.add(strLine);
}
br.close();
return strings;
}
}
So now I can override method getBufferedReader and test method handler:
#RunWith(MockitoJUnitRunner.class)
public class RealCardParserTest {
RealCardParser parser;
#Mock
GenericFile genericFile;
#Mock
BufferedReader bufferedReader;
#Mock
File file;
#Before
public void setUp() throws Exception {
parser = new RealCardParser() {
#Override
public BufferedReader getBufferedReader(GenericFile genericFile) throws FileNotFoundException {
return bufferedReader;
}
};
when(genericFile.getFile()).thenReturn(file);
when(bufferedReader.readLine()).thenReturn("header").thenReturn("1,2,3").thenReturn(null);
}
#Test
public void testParser() throws Exception {
parser.handle(genericFile);
//do some asserts
}
}
Handler method now is covered with tests, but I still have uncovered method getBufferedReader that leads to cobertura problems.
How to test method getBufferedReader or maybe there is another solution of the problem?
You can mock FileInputStream by using PowerMockRunner and PowerMockito. See the below code for mocking-
#RunWith(PowerMockRunner.class)
#PrepareForTest({
FileInputStream.class
})
public class A{
#Test
public void testFileInputStream ()
throws Exception
{
final FileInputStream fileInputStreamMock = PowerMockito.mock(FileInputStream.class);
PowerMockito.whenNew(FileInputStream.class).withArguments(Matchers.anyString())
.thenReturn(fileInputStreamMock);
//Call the actual method containing the new constructor of FileInputStream
}
}
Maybe this is a bad idea, but my first approach would have been creating an actual test-file rather than mocking the stream object.
One could argue that this would test the GenericFile class rather than the getBufferedReader method.
Maybe an acceptable way would be to return an actually existing test-file through the mocked GenericFile for testing the getBufferedReader?
I would first extract the creation of the Stream into a dependency. So your RealCardParser gets a StreamSource as a dependency.
Now you can take appart your problem:
for your current test provide a mock (or in this case I would prefer a fake) implementation returning a Stream constructed from a String.
Test the actual StreamSource with a real file, ensuring that it returns the correct content and what not.
I know this isn't the answer that you want.
The idea of unit testing is to make sure your logic is correct. Unit tests catch bugs where incorrect logic has been written. If a method contains no logic (that is, no branching, looping or exception handling), then it is uneconomical to unit test it. By that, I mean that a unit test costs money - time to write it, and time to maintain it. Most unit tests pay us back for that investment, either by finding bugs, or re-assuring us that there are no bugs in the domain of what is being tested.
But a unit test for your getBufferedReader method would not pay you back for our investment. It has a finite cost, but zero benefit, because there is no actual logic that can go wrong. Therefore, you should NOT write such a unit test. If your Cobertura settings or your organisational standards require the existence of such a unit test, then those settings or standards are WRONG and should be changed. Otherwise, your employer's money is being spent on something that has an infinite cost:benefit ratio.
I strongly recommend that your standards are changed so that you only write unit test for methods that contain branching, looping or exception handling.
When you are having this question. You are probably not following dependency inversion principle correctly. You should use InputStream whenever it's possible. If your write your FileInputStream adapter method like this:
class FileReader {
public InputStream readAsStream() {
return new FileInputStream("path/to/File.txt");
}
}
Then you can mock the method to return ByteArrayInputStream alternatively. This is much easier to deal with, because you only need to pass a string to the stream instead of dealing with the specific FileInputStream implementation.
If you are using mockito to mock, the sample goes like this:
FileReader fd = mock(FileReader());
String fileContent = ...;
ByteArrayInputStream bais = new ByteArrayInputStream(fileContent);
when(fd.readAsStream()).thenReturn(bais);