Develop Reference

How to create mock CsvExceptions to use with csvToBean.getCapturedExceptions()

I am trying to write some unit tests to see if a logging method gets called for csv exceptions. The flow goes something like this:
CsvToBean is used to parse some info and each bean that is produced has some work done on it.
After all this, CsvToBean.getCapturedExceptions().forEach() is used to processed the exceptions.
How to I create some of these exceptions for testing?
public void parseAndSaveReportToDB(Reader reader, String reportFileName,ItemizedActivityRepository iaRepo,
ICFailedRecordsRepository icFailedRepo,
String reportCols) throws Exception {
try {
CsvToBean<ItemizedActivity> csvToBean = new CsvToBeanBuilder<ItemizedActivity>(reader).withType(ItemizedActivity.class).withThrowExceptions(false).build();
csvToBean.parse().forEach(itmzActvty -> {
itmzActvty.setReportFileName(reportFileName);
String liteDesc = itmzActvty.getBalanceTransactionDescription();
if (liteDesc.contains(":")) {
liteDesc = liteDesc.substring(liteDesc.indexOf(":")+1).trim();
}
itmzActvty.setLiteDescription(liteDesc);
itmzActvty.setAmount(convertCentToDollar(itmzActvty.getAmount()));
iaRepo.save(itmzActvty);
});
log.info("Successfully saved report data in DB");
csvToBean.getCapturedExceptions().forEach(csvExceptionObj -> logFailedRecords(reportFileName, csvExceptionObj, icFailedRepo, reportCols));
reader.close();
} catch (Exception ex) {
log.error("Exception when saving report data to DB", ex);
throw ex;
}
}
In this code I need to trigger the logFailedRecords method. To do so I need to fill the captured exceptions queue with an exception. I don't know how to get an exception in there.
What I have is not much since I keep hitting walls
#Test
public void testParseAndSaveReportToDBWithExceptions() throws Exception {
// CsvException csvExceptionObject = new CsvException("testException");
CsvToBean<ItemizedActivity> csvToBean = mock(CsvToBean.class);//<ItemizedActivity>(reader).withType(ItemizedActivity.class).withThrowExceptions(false).build().class);
BufferedReader reader = mock(BufferedReader.class);
ReportingMetadata rmd = this.getReportingMetadata();
verify(this.reportsUtil).parseAndSaveReportToDB(reader,"test.csv",
this.iaRepo,this.icFailedRepo,rmd.getReportCols());
// System.out.println(csvToBean.getCapturedExceptions().toString());
}

Spring Boot file upload issue - Could not store the file error only occur after few days

I have a RESTful API created using Java Spring Boot 2.4.2.
1 of the main issue that I encountered recently is that, the Multipart file upload is working fine but the same code will not work after couple of days. It will work back after restarted the RESTFul JAR application.
The error that been display in Postman:
Could not store the file. Error
The relevant code to this is here:
try {
FileUploadUtil.saveFile(uploadPath, file.getOriginalFilename(), file);
} catch (IOException e) {
throw new RuntimeException("Could not store the file. Error: " + e.getMessage());
}
And the FileUploadUtil class:
public class FileUploadUtil {
public static void saveFile(String uploadDir, String fileName, MultipartFile multipartFile) throws IOException {
Path uploadPath = Paths.get(uploadDir);
if (!Files.exists(uploadPath)) {
Files.createDirectories(uploadPath);
}
try (InputStream inputStream = multipartFile.getInputStream()) {
Path filePath = uploadPath.resolve(fileName);
Files.copy(inputStream, filePath, StandardCopyOption.REPLACE_EXISTING);
} catch (IOException ioe) {
throw new IOException("Could not save uploaded file: " + fileName, ioe);
}
}
public static File fileFor(String uploadDir, String id) {
return new File(uploadDir, id);
}}
And the main POST API method head that called the first part of the code above is:
#PostMapping(value = "/clients/details/{clientDetailsId}/files/{department}", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
#PreAuthorize("hasAuthority('PERSONNEL') or hasAuthority('CUSTODIAN') or hasAuthority('ADMIN')")
public ResponseEntity<ClientDetails> createClientDetailsFiles(#PathVariable("clientDetailsId") long clientDetailsId,
#PathVariable("department") String department,
#RequestPart(value = "FORM_SEC_58", required = false) MultipartFile[] FORM_SEC_58_file,#RequestPart(value = "FORM_SEC_78", required = false) MultipartFile[] FORM_SEC_78_file,
#RequestPart(value = "FORM_SEC_105", required = false) MultipartFile[] FORM_SEC_105_file,
#RequestPart(value = "FORM_SEC_51", required = false) MultipartFile[] FORM_SEC_51_file,
#RequestPart(value = "FORM_SEC_76", required = false) MultipartFile[] FORM_SEC_76_file)
And the application.properties side:
spring.servlet.multipart.enabled=true
spring.servlet.multipart.max-file-size=90MB
spring.servlet.multipart.max-request-size=90MB
Can anyone advise what is the issue ya?

I had the same issue, it was working file once, but after some time, it stopped working. I am almost certain your issue is this, because, we have the same code and our scenario is the same. The way I figured it out was:
Debugging
I added a statement to print the error to see what was actually going on, what you are currently doing is only taking message of error, not the whole error. So change it to:
try {
FileUploadUtil.saveFile(uploadPath, file.getOriginalFilename(), file);
} catch (IOException e) {
e.printStackTrace();
throw new RuntimeException("Could not store the file. Error: " + e.getMessage());
}
Actual Problem
And the error was FileAlreadyExistsException FileAlreadyExistsException
Basically what that means is that you are trying to upload a file with same name twice.
Solution
To fix this issue you can use different approaches. One of them is to generate UUID for file and store it also in database, to access later.

Download files with netty

I am creating a very basic webserver using netty and java. I will have basic functionality. It's main responsibilities would be to serve responses for API calls done from a client (e.g a browser, or a console app I am building) in JSON form or send a zip file. For that reason I have created the HttpServerHanddler class which is responsible for getting the request, parsing it to find the command and call the appropriate api call.It extends SimpleChannelInboundHandler
and overrides the following functions;
#Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
LOG.debug("channelActive");
}
#Override
public void channelReadComplete(ChannelHandlerContext ctx) {
LOG.debug("In channelComplete()");
ctx.flush();
}
#Override
public void channelRead0(ChannelHandlerContext ctx, Object msg)
throws IOException {
ctx = processMessage(ctx, msg);
if (!HttpHeaders.isKeepAlive(request)) {
// If keep-alive is off, close the connection once the content is
// fully written.
ctx.writeAndFlush(Unpooled.EMPTY_BUFFER).addListener(
ChannelFutureListener.CLOSE);
}
}
private ChannelHandlerContext processMessage(ChannelHandlerContext ctx, Object msg){
if (msg instanceof HttpRequest) {
HttpRequest request = this.request = (HttpRequest) msg;
if (HttpHeaders.is100ContinueExpected(request)) {
send100Continue(ctx);
}
//parse message to find command, parameters and cookies
ctx = executeCommand(command, parameters, cookies)
}
if (msg instanceof LastHttpContent) {
LOG.debug("msg is of LastHttpContent");
if (!HttpHeaders.isKeepAlive(request)) {
// If keep-alive is off, close the connection once the content is
// fully written.
ctx.writeAndFlush(Unpooled.EMPTY_BUFFER).addListener(
ChannelFutureListener.CLOSE);
}
}
return ctx;
}
private ChanndelHandlerContext executeCommand(String command, HashMap<String, List<String>>> parameters, Set<Cookie> cookies>){
//switch case to see which command has to be invoked
switch(command){
//many cases
case "/report":
ctx = myApi.getReport(parameters, cookies); //This is a member var of ServerHandler
break;
//many more cases
}
return ctx;
}
In my Api class that has the getReport function.
getReport
public ChannelHandlerContext getReportFile(Map<String, List<String>> parameters,
Set<Cookie> cookies) {
//some initiliazations. Actual file handing happens bellow
File file = new File(fixedReportPath);
RandomAccessFile raf = null;
long fileLength = 0L;
try {
raf = new RandomAccessFile(file, "r");
fileLength = raf.length();
LOG.debug("creating response for file");
this.response = Response.createFileResponse(fileLength);
this.ctx.write(response);
this.ctx.write(new HttpChunkedInput(new ChunkedFile(raf, 0,
fileLength,
8192)),
this.ctx.newProgressivePromise());
} catch (FileNotFoundException fnfe) {
LOG.debug("File was not found", fnfe);
this.response = Response.createStringResponse("failure");
this.ctx.write(response);
} catch (IOException ioe) {
LOG.debug("Error getting file size", ioe);
this.response = Response.createStringResponse("failure");
this.ctx.write(response);
} finally {
try {
raf.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
return this.ctx;
}
Response class is responsible for handling various types of response creations (JsonString JsonArray JsonInteger File, etc)
public static FullHttpResponse createFileResponse(long fileLength) {
FullHttpResponse response = new DefaultFullHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.OK);
HttpHeaders.setContentLength(response, fileLength);
response.headers().set(HttpHeaders.Names.CONTENT_TYPE, "application/octet-stream");
return response;
}
My Api works great for my Json responses(easier to achieve) but It won't work well with my json responses, but not with my file response. When making a request from e.g chrome it only hangs and does not download the file. Should I do something else when downloading a file using netty? I know its not the best wittern code, I still think I have some bits and pieces missing from totally understanding the code, but I would like your advice on how to handle download on my code. For my code I took under consideration this and this

First, some remarks on your code...
Instead of returning ctx, I would prefer to return the last Future for the last command, such that your last event (no keep alive on) could use it directly.
public void channelRead0(ChannelHandlerContext ctx, Object msg)
throws IOException {
ChannelFuture future = processMessage(ctx, msg);
if (future != null && !HttpHeaders.isKeepAlive(request)) {
// If keep-alive is off, close the connection once the content is
// fully written.
future.addListener(ChannelFutureListener.CLOSE);
}
}
Doing this way will allow to directly close without having any "pseudo" send, even empty.
Important: Note that in Http, the response is managed such that there are chunk send for all data after the first HttpResponse item, until the last one which is empty (LastHttpContent). Sending another empty one (Empty chunk but not LastHttpContent) could break the internal logic.
Moreover, you're doing the work twice (once in read0, once in processMessage), which could lead to some issues perhaps.
Also, since you check for KeepAlive, you should ensure to set it back in the response:
if (HttpHeaders.isKeepAlive(request)) {
response.headers().set(CONNECTION, HttpHeaders.Values.KEEP_ALIVE);
}
On your send, you have 2 choices (depending on the usage of SSL or not): you've selected only the second one, which is more general, so of course valid in all cases but less efficient.
// Write the content.
ChannelFuture sendFileFuture;
ChannelFuture lastContentFuture;
if (ctx.pipeline().get(SslHandler.class) == null) {
sendFileFuture =
ctx.write(new DefaultFileRegion(raf.getChannel(), 0, fileLength), ctx.newProgressivePromise());
// Write the end marker.
lastContentFuture = ctx.writeAndFlush(LastHttpContent.EMPTY_LAST_CONTENT); // <= last writeAndFlush
} else {
sendFileFuture =
ctx.writeAndFlush(new HttpChunkedInput(new ChunkedFile(raf, 0, fileLength, 8192)),
ctx.newProgressivePromise()); // <= last writeAndFlush
// HttpChunkedInput will write the end marker (LastHttpContent) for us.
lastContentFuture = sendFileFuture;
}
This is this lastContentFuture that you can get back to the caller to check the KeepAlive.
Note however that you didn't include a single flush there (except with your EMPTY_BUFFER but which can be the main reason of your issue there!), contrary to the example (from which I copied the source).
Note that both use a writeAndFlush for the last call (or the unique one).

Check client is alive with HttpServletRequest object

I'm writing a Spring web application and I'm mapping the "/do" URL path to the following Controller's method
#Controller
public class MyController
{
#RequestMapping(value="/do", method=RequestMethod.GET)
public String do()
{
File f = new File("otherMethodEnded.tmp");
while (!f.exists())
{
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
}
}
// ok, let's continue
}
}
The otherMethodEnded.tmp file is written by one another Controller's method, so when the client calls the second URL I expect the first method to exit the while loop.
Everything works, except when the client calls the "/do" URL and then closes the connection before the response was received. The problem is that the server remains in the while (!f.exists()) loop even though the client is down and cannot call the second URL to unlock the while loop.
I would try to retrieve the connection status of the "/do" URL and exit the loop when the connection is closed by the client, but I cannot find any way to do so.
I tried with the HttpServletRequest.getSession(false) method but the returned HttpSession object is always not null, so the HttpServletRequest object is not updated in case of connection close of the client.
How can I check whether the client is still waiting for the risponse or not?

The simplest way to verify something is not right is to define a timeout value and then during your loop test if your time spent waiting has exceeded the timeout.
something like:
#Controller
public class MyController
{
private static final long MAX_LOOP_TIME = 1000 * 60 * 5; // 5 minutes? choose a value
#RequestMapping(value="/do", method=RequestMethod.GET)
public String do()
{
File f = new File("otherMethodEnded.tmp");
long startedAt = System.currentTimeMillis()
boolean forcedExit = false;
while (!forcedExit && !f.exists())
{
try {
Thread.sleep(5000);
if (System.currentTimeMillis() - startedAt > MAX_LOOP_TIME) {
forcedExit = true;
}
} catch (InterruptedException e) {
forcedExit = true;
}
}
// ok, let's continue
// if forcedExit , handle error scenario?
}
}
Additionally: InterruptedException is not something to blindly catch and ignore. see this discussion
In your case I would really exit the while loop if you're interrupted.
You only know if the client is no longer waiting on your connection when you notice the output stream you write to (response.outputstream) is closed. But there isn't a way to detect it.
(see this question for details)
Seeing as you've indicated your client does occasional callbacks, you could on the clientside poll if the other call has been completed. If this other call has completed, do the operation, otherwise return directly and have the client do the call again. (assuming you are sending json, but adapt as you require)
something like
public class MyController
{
#RequestMapping(value="/do", method=RequestMethod.GET)
public String do()
{
File f = new File("otherMethodEnded.tmp");
if (f.exists()) {
// do what you set out to do
// ok, let's continue
// and return with a response that indicates the call did what it did
// view that returns json { "result" : "success" }
return "viewThatSIgnalsToClientThatOperationSucceeded";
} else {
// view that returns json: { "result" : "retry" }
return "viewThatSignalsToClientToRetryIn5Seconds";
}
}
}
Then the clientside would run something like: (pseudojavascript as it's been a while)
val callinterval = setInterval(function() { checkServer() }, 5000);
function checkServer() {
$.ajax({
// ...
success: successFunction
});
}
function successFunction(response) {
// connection succeeded
var json = $.parseJSON(response);
if (json.result === "retry") {
// interval will call this again
} else {
clearInterval(callinterval);
if (json.result === "success") {
// do the good stuff
} else if (json.result === "failure") {
// report that the server reported an error
}
}
}
Ofcourse this is just semi-serious code but it's roughly how i'd try it if I were to have the dependency. If this is regarding afile upload, keep in mind that this file may not contain all of the bytes yet. file exists != file = completely uploaded, unless you use move it. cp / scp / etc. is not atomic.

I want to delete file from server using http doDelete() method

if i want to delete welcome.html file how to delete it using http doDelete() methode how to do it i am new to java so plz help me
public void doDelete(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
boolean success = false;
File file = null;
try {
file = searchFile(request);
} catch (Exception ex) {
java.util.logging.Logger.getLogger(Request.class.getName()).
log(java.util.logging.Level.SEVERE, null, ex);
}
if (!file.exists()) {
response.sendError(HttpServletResponse.SC_NOT_FOUND);
return;
} else {
success = file.delete(); // actual delete operation
}
if (success) {
response.setStatus(HttpServletResponse.SC_NO_CONTENT);
}
}
private String searchFile(HttpServletRequest req) throws Exception {
String fileName = req.getPathInfo();
fileName = fileName.substring(1);
return fileName;
}

While we wait for you to explain how your code "doesn't work" ... I should point out that if this code did work, it would be extremely dangerous.
Your code makes no attempt to check that the user (i.e. the guy sending the request) should be allowed to delete the file, or what the user is attempting to delete. If some bad guy sent you a DELETE request with ".."'s in it, they could probably trick your web server into attempting to delete any file in the file system!!! (Hopefully you never run your web servers as "root" ...)
UPDATE: The answer to your Question is simple. Change
file = searchFile(request);
to
file = new File(searchFile(request));
But that is the least of your problems!