I'm using httpclient to download images from a webpage and I'm trying to save them to disk but not having much luck. I'm using the code below to fetch the image but not sure what needs to be done next to actually get it to disk, the fetch would be on a JPG or a PNG image path... thanks
HttpContext localContext = new BasicHttpContext();
localContext.setAttribute(ClientContext.COOKIE_STORE,HttpClientFetch.emptyCookieStore);
HttpGet httpget = new HttpGet(pPage.imageSrc);
HttpResponse response;
response = httpClient.execute(httpget, localContext);
Header[] headers = response.getAllHeaders();
for(Header h: headers) {
logger.info("HEADERS: "+h.getName()+ " value: "+h.getValue());
}
HttpEntity entity = response.getEntity();
Header contentType = response.getFirstHeader("Content-Type");
byte[] tmpFileData;
if (entity != null) {
InputStream instream = entity.getContent();
int l;
tmpFileData = new byte[2048];
while ((l = instream.read(tmpFileData)) != -1) {
}
}
tmpFileData should now hold the bytes of the jpg from the website.
if (entity != null) {
InputStream instream = entity.getContent();
OutputStream outstream = new FileOutputStream("YourFile");
org.apache.commons.io.IOUtils.copy(instream, outstream);
}
Better use Apache commons-io, then you can just copy one InputStream to one OutputStream (FileOutputStream in your case).
Have a look at FileOutputStream and its write method.
FileOutputStream out = new FileOutputStream("outputfilename");
out.write(tmpFileData);
Related
I need to send(upload) a jpeg image to a Servlet and rather than saving it on file, I want to turn it into a BufferedImage and do some processing on it.
This is my code for the client side:
HttpClient httpclient = new DefaultHttpClient();
httpclient.getParams().setParameter(CoreProtocolPNames.PROTOCOL_VERSION, HttpVersion.HTTP_1_1);
HttpPost httppost = new HttpPost("http://localhost:9000/upload");
File file = new File("/tmp/lena.jpg");
MultipartEntity mpEntity = new MultipartEntity();
ContentBody cbFile = new FileBody(file, "image/jpeg");
mpEntity.addPart("userfile", cbFile);
httppost.setEntity(mpEntity);
System.out.println("executing request " + httppost.getRequestLine());
HttpResponse response = httpclient.execute(httppost);
HttpEntity resEntity = response.getEntity();
System.out.println(response.getStatusLine());
if (resEntity != null) {
System.out.println(EntityUtils.toString(resEntity));
}
if (resEntity != null) {
resEntity.consumeContent();
}
httpclient.getConnectionManager().shutdown();
How can I receive the image in a Servlet and process it?
I tried this on my receiving side Servlet, but the image is null:
InputStream is = request.getInputStream();
BufferedImage bImageFromConvert = ImageIO.read(is);
Finally, I should not save anything on disk in the process.
try:
Part file = request.getPart("userfile");
InputStream is = file.getInputStream();
I want to read the content of a webpage with the following methods, but I only get 60-70 percent of it.
I've tried 2 different methods to read the webpage, both with the same result. I also tried different Urls. I get no errors or timeouts.
What I am doing wrong ?
URL url = new URL(uri.toString());
HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
try
{
InputStream in = new BufferedInputStream(urlConnection.getInputStream());
BufferedReader br = new BufferedReader(new InputStreamReader(in));
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = br.readLine()) != null)
{
sb.append(line + "\n");
}
br.close();
this.content = sb.toString();
}
finally
{
urlConnection.disconnect();
}
AND
HttpGet get = new HttpGet(uri);
HttpClient defaultHttp = new DefaultHttpClient(httpParameters);
HttpResponse response = defaultHttp.execute(get);
StatusLine status = response.getStatusLine();
if(status.getStatusCode() == HttpStatus.SC_OK)
{
HttpEntity entity = response.getEntity();
InputStream stream = entity.getContent();
String encoding = "utf-8";
//long length = entity.getContentLength();
//if(entity.getContentEncoding() != null)
//{
// encoding = entity.getContentEncoding().getValue();
//}
//if(length > 0)
//{
byte[] buffer = new byte[1024];
long read = 0;
do
{
read = stream.read(buffer);
if(read > 0)
{
this.content += new String(buffer, encoding);
}
}while(read > 0);
//}
}
#edit
I've tried it with C# and WinForms. I read the complete html source of that webpage.
With java-android it doesn't work.
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create("http://www.kicker.de");
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
StreamReader reader = new StreamReader(response.GetResponseStream());
string content = reader.ReadToEnd();
reader.Close();
response.Close();
the httpurlconnection in apache's util jar has limited the biggest bytes in a response, i couldn't remember the number of it.
But in most of time ,may you use the http conncetion in UI thread , so sometimes it's not safe,and maybe will be killed, you can choose to deal with the http request in a thread but not the UI thread. So I want to know if you do it in the UT thread
I have currently the same Problem. I tried my Code in a simple Java Application and I receive the whole content. But on Android, the Content is incomplete. This Question is now a year old. I guess you have solved it in the meantime. Can you please add your Solution?
Edit:
I wrote the content into a File on my Android Device. The Content was complete!
It seems logcat doesn´t show the complete Output you receive from the Devie.
I've written a download Servlet to return a file based on a messageID parameter. Below is the doGet method.
#Override
protected void doGet(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
// This messageID would be used to get the correct file eventually
long messageID = Long.parseLong(request.getParameter("messageID"));
String fileName = "C:\\Users\\Soto\\Desktop\\new_audio1.amr";
File returnFile = new File(fileName);
ServletOutputStream out = response.getOutputStream();
ServletContext context = getServletConfig().getServletContext();
String mimetype = context.getMimeType("C:\\Users\\Soto\\Desktop\\new_audio1.amr");
response.setContentType((mimetype != null) ? mimetype : "application/octet-stream");
response.setContentLength((int)returnFile.length());
response.setHeader("Content-Disposition", "attachment; filename=\"" + "new_audio.amr" + "\"");
FileInputStream in = new FileInputStream(returnFile);
byte[] buffer = new byte[4096];
int length;
while((length = in.read(buffer)) > 0) {
out.write(buffer, 0, length);
}
in.close();
out.flush();
}
I then wrote some code to retrieve the file.
String url = "http://localhost:8080/AudioFileUpload/DownloadServlet";
String charset = "UTF-8";
// The id of the audio message requested
String messageID = "1";
//URLConnection connection = null;
try {
String query = String.format("messageID=%s", URLEncoder.encode(messageID, charset));
//URLConnection connection;
//URL u = new URL(url + "?" + query);
//connection = u.openConnection();
//InputStream in = connection.getInputStream();
HttpClient httpClient = new DefaultHttpClient();
httpClient.getParams().setParameter(CoreProtocolPNames.PROTOCOL_VERSION, HttpVersion.HTTP_1_1);
HttpGet httpGet = new HttpGet(url + "?" + query);
HttpResponse response = httpClient.execute(httpGet);
System.out.println(response.getStatusLine());
InputStream in = response.getEntity().getContent();
FileOutputStream fos = new FileOutputStream(new File("C:\\Users\\Soto\\Desktop\\new_audio2.amr"));
byte[] buffer = new byte[4096];
int length;
while((length = in.read(buffer)) > 0) {
fos.write(buffer, 0, length);
}
//connection = new URL(url + "?" + query).openConnection();
//connection.setRequestProperty("Accept-Charset", charset);
//InputStream response = connection.getInputStream();
} catch (MalformedURLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Now this code works fine. I can download the audio file and it works correctly. What I want to know is how to, if possible, get the name of the file as it is downloaded instead of giving it my own name. Also, is it possible to get the file without having to read from the stream (maybe some library that does it for you)? I kind of want to hide the dirty stuff.
Thanks
For setting the download file name do the following on response object in Servlet code
response.setHeader("Content-disposition",
"attachment; filename=" +
"new_audio1.amr" );
EDIT:
I see you are already doing it. Just try removing the slashes you have added.
With attachment, the file will be served with the provided name properly. When inline, browsers seem to ignore filename, and usually give the servletname part of the URL as default name when saving the inline contents.
You could try mapping that URL to an appropriate filename, if that is suitable.
Here's a SO related question: Securly download file inside browser with correct filename
You may also find this link useful: Filename attribute for Inline Content-Disposition Meaningless?
I think you cannot download file without streaming. For I/O you must use stream.
i just wanted to ask about sending gzip for post requests using HttpClient in Android?
where to get that OutputStream to be passed in the GZIPOutputstream?
any snippets?
Hi UseHttpUriRequest as shown below
String urlval=" http"//www.sampleurl.com/";
HttpUriRequest req = new HttpGet(urlval);
req.addHeader("Accept-Encoding", "gzip");
httpClient.execute(req);
and then Check response for content encoding as shown below :
InputStream is = response.getEntity().getContent();
Header contentEncoding = response.getFirstHeader("Content-Encoding");
if (contentEncoding != null && contentEncoding.getValue().equalsIgnoreCase("gzip")) {
is = new GZIPInputStream(is);
}
If your data is not too large, you can do it like this:
HttpClient httpClient = new DefaultHttpClient();
HttpPost httpost = new HttpPost(POST_URL);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
GZIPOutputStream gos = new GZIPOutputStream(baos);
gos.write(data.getBytes());
gos.close();
ByteArrayEntity byteArrayEntity = new ByteArrayEntity(baos.toByteArray());
httpost.setEntity(byteArrayEntity);
The url: http://www.teamliquid.net/replay/download.php?replay=1830 is a download link to a .rep file.
My question is: how to download this content in java knowing the name of the original rep file in order to save it with a defined prefix, like path/_.rep
//I was trying to run wget from java but I don't see how to get the original file's name.
Get the redirected URL,
http://www.teamliquid.net/replay/upload/coco%20vs%20snssoflsekd.rep
You can get the filename from this URL.
It's tricky to get the redirected URL. See my answer to this question on how to do it with Apache HttpClient 4,
HttpClient 4 - how to capture last redirect URL
EDIT: Here is a sample using HttpClient 4.0,
String url = "http://www.teamliquid.net/replay/download.php?replay=1830";
HttpClient httpClient = new DefaultHttpClient();
HttpGet httpget = new HttpGet(url);
HttpContext context = new BasicHttpContext();
HttpResponse response = httpClient.execute(httpget, context);
if (response.getStatusLine().getStatusCode() != HttpStatus.SC_OK)
throw new IOException(response.getStatusLine().toString());
HttpUriRequest currentReq = (HttpUriRequest) context.getAttribute(
ExecutionContext.HTTP_REQUEST);
String currentUrl = URLDecoder.decode(currentReq.getURI().toString(), "UTF-8");
int i = currentUrl.lastIndexOf('/');
String fileName = null;
if (i < 0) {
fileName = currentUrl;
} else {
fileName = currentUrl.substring(i+1);
}
OutputStream os = new FileOutputStream("/tmp/" + fileName);
InputStream is = response.getEntity().getContent();
byte[] buf = new byte[4096];
int read;
while ((read = is.read(buf)) != -1) {
os.write(buf, 0, read);
}
os.close();
After running this code, I get this file,
/tmp/coco vs snssoflsekd.rep