Unit testing using MockMultipartHttpServletRequest (throws NullPointerException in ItemInputStream.makeAvailable) - java

I've written a transformer class that takes an HttpServletRequest and transforms it into another type that holds a pointer to the InputStream from the servlet request. (The idea is to abstract the incoming transport protocol from the request handling, so I could also write a similar transformer from FTP, for instance.)
Now I'm trying to write a unit test for this, and I'm having problems. I've managed to figure out the correct boilerplate to create a valid Multipart HTTP request (using the Spring classes MockMultipartHttpServletRequest and MockMultipartFile), but now I get a NullPointerException in the initialize() method of my UploadRequest class. I'm guessing the problem is that somehow the stream inside the MockMultipartHttpServletRequest isn't being initialized correctly, but I can't figure out what I should do differently.
Any suggestions would be gratefully accepted!
This is the stack trace:
java.lang.NullPointerException
at org.apache.commons.fileupload.MultipartStream$ItemInputStream.makeAvailable(MultipartStream.java:976)
at org.apache.commons.fileupload.MultipartStream$ItemInputStream.read(MultipartStream.java:886)
at java.io.InputStream.read(InputStream.java:82)
at org.apache.commons.fileupload.util.Streams.copy(Streams.java:96)
at org.apache.commons.fileupload.util.Streams.copy(Streams.java:66)
at org.apache.commons.fileupload.MultipartStream.readBodyData(MultipartStream.java:592)
at org.apache.commons.fileupload.MultipartStream.discardBodyData(MultipartStream.java:618)
at org.apache.commons.fileupload.MultipartStream.skipPreamble(MultipartStream.java:637)
at org.apache.commons.fileupload.FileUploadBase$FileItemIteratorImpl.findNextItem(FileUploadBase.java:984)
at org.apache.commons.fileupload.FileUploadBase$FileItemIteratorImpl.<init>(FileUploadBase.java:965)
at org.apache.commons.fileupload.FileUploadBase.getItemIterator(FileUploadBase.java:331)
at org.apache.commons.fileupload.servlet.ServletFileUpload.getItemIterator(ServletFileUpload.java:148)
at com.ooyala.UploadRequest.initialize(UploadRequest.java:51)
at com.ooyala.UploadRequestTest.testCreateFromServletRequest(UploadRequestTest.java:57)
Here's an abbreviated version of my transformer class:
public class UploadRequest {
private Map<String, String> params;
private InputStream strIn;
private Logger Log = Logger.getLogger(UploadRequest.class.getName());
public UploadRequest()
{
params = new HashMap<String, String>();
}
public void initialize(HttpServletRequest sRequest,
ServletFileUpload upload)
throws IOException, FileUploadException
{
Enumeration<String> paramNames = sRequest.getParameterNames();
while (paramNames.hasMoreElements()) {
String pName = paramNames.nextElement();
params.put(pName, sRequest.getParameter(pName));
}
params.put("request_uri", sRequest.getRequestURI());
FileItemIterator iter = upload.getItemIterator(sRequest);
while (iter.hasNext()) {
FileItemStream item = iter.next();
try {
if (!item.isFormField()) {
// Skip form fields
params.put("original_file_name", item.getName());
strIn = item.openStream();
}
} catch (IOException ex) {
Log.severe("File uploading exception: " + ex.getMessage());
throw ex;
}
}
}
And here's the unit test:
import org.springframework.mock.web.MockMultipartHttpServletRequest;
import org.springframework.mock.web.MockMultipartFile;
import org.apache.commons.fileupload.servlet.ServletFileUpload;
// etc.... other imports
#RunWith(JMock.class)
public class UploadRequestTest {
private UploadRequest upRequest;
#Before
public void setUp()
{
context.setImposteriser(ClassImposteriser.INSTANCE);
upRequest = new UploadRequest();
}
#Test
public void testCreateFromServletRequest()
throws IOException, FileUploadException
{
String text_contents = "hello world";
MockMultipartHttpServletRequest sRequest =
new MockMultipartHttpServletRequest();
sRequest.setMethod("POST");
String boundary = generateBoundary();
String contentType = "multipart/form-data; boundary="+boundary;
sRequest.setContentType(contentType);
sRequest.setRequestURI("/foo");
sRequest.addParameter("test_param","test_value");
sRequest.addFile(
new MockMultipartFile("file1","test_upload.txt","text/plain",
text_contents.getBytes()));
ServletFileUpload upload = new ServletFileUpload();
assertTrue(upload.isMultipartContent(sRequest));
upRequest.initialize(sRequest, upload);
}
}

I have the same issue and I googled but no answer. I plugged in the source code from the library, You need to send content, whatever. The library might need to check if it is null in the skip method
MockMultipartHttpServletRequest request
request.setContent("whatever".getBytes());
Posted here for others

Add boundary condition
Generate contents as follows
MockMultipartHttpServletRequest request =
this.generateMockMultiPartHttpServletRequest(true);
MockMultipartFile mockMultipartFile = null;
try {
request.setContentType("multipart/form-data; boundary=-----1234");
request.setCharacterEncoding("text/plain");
String endline = "\r\n";
String bondary = "-----1234";
String textFile = this.encodeTextFile("-----1234", "\r\n", "file","test.csv",
"text/UTF-8", FileUtils.readFileToString((new File(csvFilePath)), "UTF-8"));
StringBuilder content = new StringBuilder(textFile.toString());
content.append(endline);
content.append(endline);
content.append(endline);
content.append("--");
content.append(bondary);
content.append("--");
content.append(endline);
request.setContent(content.toString().getBytes());
request.setMethod("POST");
mockMultipartFile = new MockMultipartFile("file",
FileUtils.readFileToByteArray(new File(csvFilePath)));
} catch (Exception e1) {
e1.printStackTrace();
}
request.addFile(mockMultipartFile);
Function to encode text
private String encodeTextFile(String bondary, String endline, String name,
String filename, String contentType, String content) {
final StringBuilder sb = new StringBuilder(64);
sb.append(endline);
sb.append("--");
sb.append(bondary);
sb.append(endline);
sb.append("Content-Disposition: form-data; name=\"");
sb.append(name);
sb.append("\"; filename=\"");
sb.append(filename);
sb.append("\"");
sb.append(endline);
sb.append("Content-Type: ");
sb.append(contentType);
sb.append(endline);
sb.append(endline);
sb.append(content);
return sb.toString();
}

I went through the same problem, after searching lot I got this post in which I answered with code that solved my problem.
The Shriprasad's solution works well for text file. But I had some problems with binary files.
https://stackoverflow.com/a/30541653/2762092

Related

How to append input param to string output?

I have a method which takes the string as input, filters for specific key and value, returns the value as String output. I have a requirement to append input param to output string. The input is array int id. Here is the method code snippet:
private static String headerstomap(String headers) {
String sHeaders = headers.replace("[", "");
sHeaders = sHeaders.replace("]", "");
String res = Arrays.stream(sHeaders.split(", "))
.filter(s->s.contains("Uniquename"))
.findFirst()
.map(name->name.split(":")[1])
.orElse("Not Present");
return res;
}
Input is: [DomainValue:MYSQL,Oracle,SAP, Uniquename:jvmErrors_v1]
There is a rest API which takes the input param, gets the relevant data. Calls the above method to create a filename. The REST resource is:
public void downloadRecords(#PathVariable Long[] ids, HttpServletResponse response) throws Exception {
I need the method to return: jvmErrors_v1_1
Essentially, add an underscore at the end and append the input param.
Here is the REST resource:
public void downloadRecords(#PathVariable Long[] ids, HttpServletResponse response) throws Exception {
List<IDZip> iDZip = messageRepository.findbyId(ids);
IDZip iDZip = iDZip.get(0);
String xml = new ObjectMapper().writeValueAsString(iDZip);
String fileName = "id.zip";
String xmlname = messageController.headerstomap(iDZip.getheaders());
byte[] data = xml.getBytes();
byte[] bytes;
try (ByteOutputStream bout = new ByteOutputStream(); ZipOutputStream zout = new ZipOutputStream(bout)) {
for (Long id : ids) {
zout.setLevel(1);
ZipEntry ze = new ZipEntry(xmlname);
ze.setSize(data.length);
ze.setTime(System.currentTimeMillis());
zout.putNextEntry(ze);
zout.write(data);
zout.closeEntry();
}
bytes = bout.getBytes();
}
response.setContentType("application/zip");
response.setContentLength(bytes.length);
response.setHeader("Content-Disposition", "attachment; " + String.format("filename=" + fileName));
ServletOutputStream outputStream = response.getOutputStream();
FileCopyUtils.copy(bytes, outputStream);
outputStream.close();
}
There is IDZip class which holds getters and setters...
public String getheaders() {
return headers;
}
public void setheaders(String headers) {
this.headers = headers;
}
I will give you a gift today only if you will accept it, first debugs the process into small section system.out.println("step 1"); in each of the line then i will take it from that point

addAttachment error in XML-RPC while using Java

My Java code
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.Vector;
import helma.xmlrpc.*;
public class test {
private final static String server_url =
"http://confluence.xyz.com:8080/rpc/xmlrpc";
public static void main (String [] args) {
try {
XmlRpcClient server = new XmlRpcClient(server_url);
Vector<Object> params = new Vector<Object>(2);
params.add("user");
params.add("pass");
String token = (String) server.execute("confluence2.login", params );
System.out.println(token);
Vector<Object> page = new Vector<Object>(3);
page.add(token);
page.add("~username");
page.add("test_page");
Object token1 = server.execute("confluence2.getPage", page );
System.out.println(token1.hashCode());
String fileName = "C:/New folder/a.jpeg";
String contentType = "image/jpeg";
Vector<Object> attachment = new Vector<Object>(2);
attachment.add("a.jpeg");
attachment.add(contentType);
System.out.println(attachment);
byte[] bytes = Files.readAllBytes(Paths.get(fileName));
System.out.println(bytes);
Vector<Object> attach = new Vector<Object>(4);
attach.add(token);
attach.add(token1.hashCode());
attach.add(attachment);
attach.add(bytes);
System.out.println(attach);
server.execute("confluence2.addAttachment", attach);
}
catch (Exception exception) {
System.err.println("JavaClient: " + exception.toString());
}
}
}
Everything works fine except on line where "addAttachment" is called,
error i get is
JavaClient: helma.xmlrpc.XmlRpcException: java.lang.NoSuchMethodException: com.sun.proxy.$Proxy2104.addAttachment(java.lang.String, int, java.util.Vector, [B)
can anyone help me with any other library which i should be using. it seems helma.xmlrpc doesn't have addAttachment method
I used org.apache.xmlrpc.client.XmlRpcClient and not helma but the concepts should be the same. It's not that "helma.xmlrpc doesn't have addAttachment method", it's just that you're calling addAttachment() with the wrong parameters. Try calling it with the proper parameters listed at https://developer.atlassian.com/confdev/confluence-rest-api/confluence-xml-rpc-and-soap-apis/remote-confluence-methods
addAttachment(String token, long contentId, Attachment attachment, byte[] attachmentData)
so for apache xmlrpc, my partial code looks like:
//add attachment to the page
byte[] bytes = FileUtils.readFileToByteArray(new File(FILE_TO_ATTACH));
Map<String, String> attachInfo = new HashMap<String, String>();
attachInfo.put("fileName", FILENAME);
attachInfo.put("contentType", CONTENT_TYPE);
attachInfo.put("comment", COMMENT);
//actually add it now
client.execute("confluence1.addAttachment", new Object[]{token, PAGEID, attachInfo, bytes});

Hide #ModelAttribute variables from appearing in URL? Spring MVC

I'm using Spring MVC 3 and I have the following Controller
#RequestMapping(value="FileUploadForm",method=RequestMethod.GET)
public String showForm(ModelMap model){
UploadForm form = new UploadForm();
model.addAttribute("FORM", form);
return "FileUploadForm";
}
#RequestMapping(value="FileUploadForm",method=RequestMethod.POST)
public ModelAndView processForm(#ModelAttribute(value="FORM") UploadForm form,BindingResult result){
if(!result.hasErrors()){
FileOutputStream outputStream = null;
String filePath = System.getProperty("java.io.tmpdir") + "/" + form.getFile().getOriginalFilename();
try {
outputStream = new FileOutputStream(new File(filePath));
outputStream.write(form.getFile().getFileItem().get());
outputStream.close();
System.out.println(form.getName());
return new ModelAndView(new RedirectView("success?Filepath="+filePath, true, true, false));
} catch (Exception e) {
System.out.println("Error while saving file");
return new ModelAndView("FileUploadForm");
}
}else{
return new ModelAndView("FileUploadForm");
}
}
This controller get the filepath and use to do a blast
#RequestMapping(value="success")
public String blasta(#ModelAttribute("Filepath") String filepath, Model model){
Blast sb = new Blast("somepath");
String[] blastIt = sb.blast("somepath", filepath);
String newLine = System.getProperty("line.separator");
ArrayList<Object> result = new ArrayList<>();
for (int i = 5; i < blastIt.length; i++) {
if(blastIt[i].startsWith("Lambda")){
break;
} else {
seila.add(blastIt[i]);
System.out.println(blastIt[i]);
}
model.addAttribute("RESULT", result);
}
File f1 = new File(filepath);
f1.delete();
return "success";
}
Everything works fine, but I still get the filepath in the url.
http://localhost:8081/Ambase/success?Filepath=filePath
And I want this way if it's possible
http://localhost:8081/Ambase/success
try adding this code to servlet-config.xml
<mvc:annotation-driven ignoreDefaultModelOnRedirect="true" />
To avoid this issue you should use RedirectAttributes. It will add the value of filePath to the redirect view params and you can get that in the controller blasta.
To achieve this you need to add one more parameter in the controller function processForm. At the end of all the parameters add RedirectAttributes attributes and then add following line just above the RedirectView statement.
attributes.addFlashAttribute("Filepath",filePath);
And then you can get this attribute in the ModelMap inside blasta controller function.
Hope this helps you. Cheers.

using dbpedia spotlight in java or scala

Does anyone know where to find a little how to on using dbpedia spotlight in java or scala? Or could anyone explain how it's done? I can't find any information on this...
The DBpedia Spotlight wiki pages would be a good place to start.
And I believe the installation page has listed the most popular ways (using a jar, or set up a web service) to use the application.
It includes instructions on using the Java/Scala API with your own installation, or calling the Web Service.
There are some additional data needed to be downloaded to run your own server for full service, good time to make a coffee for yourself.
you need download dbpedia spotlight (jar file) after that u can use next two classes ( author pablomendes ) i only make some change .
public class db extends AnnotationClient {
//private final static String API_URL = "http://jodaiber.dyndns.org:2222/";
private static String API_URL = "http://spotlight.dbpedia.org:80/";
private static double CONFIDENCE = 0.0;
private static int SUPPORT = 0;
private static String powered_by ="non";
private static String spotter ="CoOccurrenceBasedSelector";//"LingPipeSpotter"=Annotate all spots
//AtLeastOneNounSelector"=No verbs and adjs.
//"CoOccurrenceBasedSelector" =No 'common words'
//"NESpotter"=Only Per.,Org.,Loc.
private static String disambiguator ="Default";//Default ;Occurrences=Occurrence-centric;Document=Document-centric
private static String showScores ="yes";
#SuppressWarnings("static-access")
public void configiration(double CONFIDENCE,int SUPPORT,
String powered_by,String spotter,String disambiguator,String showScores){
this.CONFIDENCE=CONFIDENCE;
this.SUPPORT=SUPPORT;
this.powered_by=powered_by;
this.spotter=spotter;
this.disambiguator=disambiguator;
this.showScores=showScores;
}
public List<DBpediaResource> extract(Text text) throws AnnotationException {
LOG.info("Querying API.");
String spotlightResponse;
try {
String Query=API_URL + "rest/annotate/?" +
"confidence=" + CONFIDENCE
+ "&support=" + SUPPORT
+ "&spotter=" + spotter
+ "&disambiguator=" + disambiguator
+ "&showScores=" + showScores
+ "&powered_by=" + powered_by
+ "&text=" + URLEncoder.encode(text.text(), "utf-8");
LOG.info(Query);
GetMethod getMethod = new GetMethod(Query);
getMethod.addRequestHeader(new Header("Accept", "application/json"));
spotlightResponse = request(getMethod);
} catch (UnsupportedEncodingException e) {
throw new AnnotationException("Could not encode text.", e);
}
assert spotlightResponse != null;
JSONObject resultJSON = null;
JSONArray entities = null;
try {
resultJSON = new JSONObject(spotlightResponse);
entities = resultJSON.getJSONArray("Resources");
} catch (JSONException e) {
//throw new AnnotationException("Received invalid response from DBpedia Spotlight API.");
}
LinkedList<DBpediaResource> resources = new LinkedList<DBpediaResource>();
if(entities!=null)
for(int i = 0; i < entities.length(); i++) {
try {
JSONObject entity = entities.getJSONObject(i);
resources.add(
new DBpediaResource(entity.getString("#URI"),
Integer.parseInt(entity.getString("#support"))));
} catch (JSONException e) {
LOG.error("JSON exception "+e);
}
}
return resources;
}
}
second class
/**
* #author pablomendes
*/
public abstract class AnnotationClient {
public Logger LOG = Logger.getLogger(this.getClass());
private List<String> RES = new ArrayList<String>();
// Create an instance of HttpClient.
private static HttpClient client = new HttpClient();
public List<String> getResu(){
return RES;
}
public String request(HttpMethod method) throws AnnotationException {
String response = null;
// Provide custom retry handler is necessary
method.getParams().setParameter(HttpMethodParams.RETRY_HANDLER,
new DefaultHttpMethodRetryHandler(3, false));
try {
// Execute the method.
int statusCode = client.executeMethod(method);
if (statusCode != HttpStatus.SC_OK) {
LOG.error("Method failed: " + method.getStatusLine());
}
// Read the response body.
byte[] responseBody = method.getResponseBody(); //TODO Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommended.
// Deal with the response.
// Use caution: ensure correct character encoding and is not binary data
response = new String(responseBody);
} catch (HttpException e) {
LOG.error("Fatal protocol violation: " + e.getMessage());
throw new AnnotationException("Protocol error executing HTTP request.",e);
} catch (IOException e) {
LOG.error("Fatal transport error: " + e.getMessage());
LOG.error(method.getQueryString());
throw new AnnotationException("Transport error executing HTTP request.",e);
} finally {
// Release the connection.
method.releaseConnection();
}
return response;
}
protected static String readFileAsString(String filePath) throws java.io.IOException{
return readFileAsString(new File(filePath));
}
protected static String readFileAsString(File file) throws IOException {
byte[] buffer = new byte[(int) file.length()];
#SuppressWarnings("resource")
BufferedInputStream f = new BufferedInputStream(new FileInputStream(file));
f.read(buffer);
return new String(buffer);
}
static abstract class LineParser {
public abstract String parse(String s) throws ParseException;
static class ManualDatasetLineParser extends LineParser {
public String parse(String s) throws ParseException {
return s.trim();
}
}
static class OccTSVLineParser extends LineParser {
public String parse(String s) throws ParseException {
String result = s;
try {
result = s.trim().split("\t")[3];
} catch (ArrayIndexOutOfBoundsException e) {
throw new ParseException(e.getMessage(), 3);
}
return result;
}
}
}
public void saveExtractedEntitiesSet(String Question, LineParser parser, int restartFrom) throws Exception {
String text = Question;
int i=0;
//int correct =0 ; int error = 0;int sum = 0;
for (String snippet: text.split("\n")) {
String s = parser.parse(snippet);
if (s!= null && !s.equals("")) {
i++;
if (i<restartFrom) continue;
List<DBpediaResource> entities = new ArrayList<DBpediaResource>();
try {
entities = extract(new Text(snippet.replaceAll("\\s+"," ")));
System.out.println(entities.get(0).getFullUri());
} catch (AnnotationException e) {
// error++;
LOG.error(e);
e.printStackTrace();
}
for (DBpediaResource e: entities) {
RES.add(e.uri());
}
}
}
}
public abstract List<DBpediaResource> extract(Text text) throws AnnotationException;
public void evaluate(String Question) throws Exception {
evaluateManual(Question,0);
}
public void evaluateManual(String Question, int restartFrom) throws Exception {
saveExtractedEntitiesSet(Question,new LineParser.ManualDatasetLineParser(), restartFrom);
}
}
main()
public static void main(String[] args) throws Exception {
String Question ="Is the Amazon river longer than the Nile River?";
db c = new db ();
c.configiration(0.0, 0, "non", "CoOccurrenceBasedSelector", "Default", "yes");
System.out.println("resource : "+c.getResu());
}
I just add one little fix for your answer.
Your code is running, if you add the evaluate method call:
public static void main(String[] args) throws Exception {
String question = "Is the Amazon river longer than the Nile River?";
db c = new db ();
c.configiration(0.0, 0, "non", "CoOccurrenceBasedSelector", "Default", "yes");
c.evaluate(question);
System.out.println("resource : "+c.getResu());
}
Lamine
In the request method of the second class (AnnotationClient) in Adel's answer, the author Pablo Mendes hasn't finished
TODO Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommended.
which is an annoying warning that needs to be removed by replacing
byte[] responseBody = method.getResponseBody(); //TODO Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommended.
// Deal with the response.
// Use caution: ensure correct character encoding and is not binary data
response = new String(responseBody);
with
Reader in = new InputStreamReader(method.getResponseBodyAsStream(), "UTF-8");
StringWriter writer = new StringWriter();
org.apache.commons.io.IOUtils.copy(in, writer);
response = writer.toString();

apache.commons.fileupload throws MalformedStreamException

I have got this piece of code (I didn't write, just maintaining):
public class MyMultipartResolver extends CommonsMultipartResolver{
public List parseEmptyRequest(HttpServletRequest request) throws IOException, FileUploadException {
String contentType = request.getHeader(CONTENT_TYPE);
int boundaryIndex = contentType.indexOf("boundary=");
InputStream input = request.getInputStream();
byte[] boundary = contentType.substring(boundaryIndex + 9).getBytes();
MultipartStream multi = new MultipartStream(input, boundary);
multi.setHeaderEncoding(getHeaderEncoding());
ArrayList items = new ArrayList();
boolean nextPart = multi.skipPreamble();
while (nextPart) {
Map headers = parseHeaders(multi.readHeaders());
// String fieldName = getFieldName(headers);
String subContentType = getHeader(headers, CONTENT_TYPE);
if (subContentType == null) {
FileItem item = createItem(headers, true);
OutputStream os = item.getOutputStream();
try {
multi.readBodyData(os);
} finally {
os.close();
}
items.add(item);
} else {
multi.discardBodyData();
}
nextPart = multi.readBoundary();
}
return items;
}
}
I am using commons-fileupload.jar version 1.2.1 and obviously the code is using some deprecated methods...
Anyway, while trying to use this code to upload a very large file (780 MB) I get this:
org.apache.commons.fileupload.MultipartStream$MalformedStreamException: Stream ended unexpectedly
at org.apache.commons.fileupload.MultipartStream$ItemInputStream.makeAvailable(MultipartStream.java:983)
at org.apache.commons.fileupload.MultipartStream$ItemInputStream.read(MultipartStream.java:887)
at java.io.InputStream.read(InputStream.java:89)
at org.apache.commons.fileupload.util.Streams.copy(Streams.java:94)
at org.apache.commons.fileupload.util.Streams.copy(Streams.java:64)
at org.apache.commons.fileupload.MultipartStream.readBodyData(MultipartStream.java:593)
at org.apache.commons.fileupload.MultipartStream.discardBodyData(MultipartStream.java:619)
that is thrown from 'multi.discardBodyData();' line.
My question:
How can I avoid this error and be able to be able to succeed collecting the FileItems?
catch
(org.apache.commons.fileupload.MultipartStream.MalformedStreamException e)
{
e.printStackTrace();
return ERROR;
}
Catch the exception and handle it via ..either InputStream or Return Error use it in struts action tag

Categories

Resources