i want to download a csv file from a internet url.
I have tried every code that i can find online and not able to download.
URL google = new URL("myurl");
ReadableByteChannel rbc = Channels.newChannel(google.openStream());
FileOutputStream fos = new FileOutputStream("C://excelsheet.csv");
fos.getChannel().transferFrom(rbc, 0, 1 << 24);
This code is not working
I used this function , it is not working too.
public stacvoid saveUrl(String filename, String urlString) throws MalformedURLException, IOException
{
BufferedInputStream in = null;
FileOutputStream fout = null;
try
{
in = new BufferedInputStream(new URL(urlString).openStream());
fout = new FileOutputStream(filename);
byte data[] = new byte[1024];
int count;
while ((count = in.read(data, 0, 1024)) != -1)
{
fout.write(data, 0, count);
}
}
finally
{
if (in != null)
in.close();
if (fout != null)
fout.close();
}
}
I tried a simple input stream on my url , that doesn't work too.
URL oracle = new URL("myurl");
URLConnection yc = oracle.openConnection();
BufferedReader br = new BufferedReader(new InputStreamReader(
yc.getInputStream()));
Now instead of myurl , if i type any other url , the bufferedreader does get data.
If I enter myurl in a browser i get a popup to save or download the csv file.
So the problem is not with an incorrrect "myurl"
Is it possible that the servelet/code running on "myurl" actually checks if the request is coming from a browser and only then sends the data.
Thanks everyone . Used httpclient and it works for me. Heres the code that downloads an csv file , saves it locally and then reads it and parses all the tokens.
HttpClient httpclient = new DefaultHttpClient();
HttpGet httpget = new HttpGet("your url to download");
HttpResponse response = httpclient.execute(httpget);
System.out.println(response.getProtocolVersion());
System.out.println(response.getStatusLine().getStatusCode());
System.out.println(response.getStatusLine().getReasonPhrase());
System.out.println(response.getStatusLine().toString());
HttpEntity entity = response.getEntity();
if (entity != null) {
FileOutputStream fos = new java.io.FileOutputStream("C://excelsheet.csv");
entity.writeTo(fos);
fos.close();
}
//create BufferedReader to read csv file
BufferedReader br = new BufferedReader( new FileReader("C://excelsheet.csv"));
String strLine = "";
StringTokenizer st = null;
int lineNumber = 0, tokenNumber = 0;
while( (strLine = br.readLine()) != null)
{
lineNumber++;
//break comma separated line using ","
st = new StringTokenizer(strLine, ",");
while(st.hasMoreTokens())
{
//display csv values
tokenNumber++;
System.out.println("Line # " + lineNumber +
", Token # " + tokenNumber
+ ", Token : "+ st.nextToken());
}
//reset token number
tokenNumber = 0;
}
}
catch (Exception e) {
System.out.println("insdie the catch part");
System.out.println(e.toString());
}
Have you tried the HttpClient lib?
I used it sometime ago to download a set of exams from a site automatically.
Related
I need to develop an API which can open connection to an URL which returns a ZIP file. This URL works perfectly fine when accessed from browser or Postman but when I try to access it from Java Code (tried HttpClient/RestTemplate etc) it returns an HTML file. I want to get the zip file and want to store it at particular directory.
HttpClient client = HttpClientBuilder.create().setDefaultCredentialsProvider(provider).build();
HttpGet request = new HttpGet(url);
logger.info("Request to Asset Store: URL " + request.getURI());
HttpResponse response = client.execute(request);
if (response != null) {
for (Header header : response.getAllHeaders()) {
System.out.println(header.getName() + " - " + header.getValue());
}
BufferedInputStream bis = new BufferedInputStream(response.getEntity().getContent());
final ZipInputStream is = new ZipInputStream(bis);
try {
ZipEntry entry;
while ((entry = is.getNextEntry()) != null) {
System.out.printf("File: %s Size %d Modified on %TD %n", entry.getName(),
entry.getSize(), new Date(entry.getTime()));
extractEntry(entry, is);
}
System.out.println("OUT");
} finally {
is.close();
}
}
private static void extractEntry(final ZipEntry entry, InputStream is) throws IOException {
String exractedFile = "D://" + entry.getName();
FileOutputStream fos = null;
try {
fos = new FileOutputStream(exractedFile);
final byte[] buf = new byte[2048];
int read = 0;
int length;
while ((length = is.read(buf, 0, buf.length)) >= 0) {
fos.write(buf, 0, length);
}
} catch (IOException ioex) {
fos.close();
I have a raw file in res/raw named "pack.dat".
I can open an InputStream with the following code:
InputStream fis = null;
try {
fis = context.getResources().openRawResource(R.raw.pack);
BufferedReader br = new BufferedReader(new InputStreamReader(fis));
String nextLine;
int i = 0, j = 0;
while ((nextLine = br.readLine()) != null) {
if (j == 5) {
j = 0;
i++;
}
}
} catch (Exception e) {
e.printStackTrace();
}
This is working, I can read from that file.
But unfortunately I need a FileInputStream. When I do this:
FileInputStream fs = null;
Uri url = Uri.parse("android.resource://" +
context.getPackageName() + "/" + R.raw.pack);
File file = new File(url.toString());
try {
fs = new FileInputStream(file);
fs.getChannel().position(0);
fs.read(bDatensatz, 0, indexlaenge);
} catch (Exception e) {
e.printStackTrace();
}
I get a "file not found" at
fs = new FileInputStream(file);
context.getResources().openRawResource(R.raw.pack) in the first example returns an InputStream.
What can I use to get a FileInputStream instead?
copied from another thread! may be this should help you
FileInputStream fis;
fis = openFileInput("test.txt");
StringBuffer fileContent = new StringBuffer("");
byte[] buffer = new byte[1024];
int n = 0;
while ((n = fis.read(buffer)) != -1)
{
fileContent.append(new String(buffer, 0, n));
}
But unfortunately I need a FileInputStream
Why?
I get a "file not found" at fs = new FileInputStream(file);
That is because you are trying to open something that is not a file.
What can I use to get a FileInputStream instead?
You would need to copy the resource to a local file (e.g., using openFileOutput() and Java I/O). Then, you can open the local file (e.g., using openFileInput()) and get a FileInputStream.
Or, just use the InputStream, fixing whatever code that you are using that is expecting a FileInputStream.
I'm trying to read a website but strangely it returns only part of it. It just ends in the middle of section.
I tried using the setChunkedStreamingMode method but it didn't change anything.
HttpURLConnection connection = ((HttpURLConnection) new URL(url).openConnection());
connection.setDoOutput(true);
connection.setDoInput(true);
connection.setInstanceFollowRedirects(false);
connection.setRequestMethod("POST");
// I write some data...
String content = readInputStream(connection.getInputStream());
ReadInputStream method:
private static String readInputStream(InputStream in) throws IOException {
int len = 0;
byte[] buffer = new byte[10000];
ByteArrayOutputStream byteArray = new ByteArrayOutputStream();
while((len = in.read(buffer)) != -1) {
byteArray.write(buffer,0, len);
}
in.close();
return new String(byteArray.toByteArray());
}
Is there some sort of limit of data?
there is no problem with HttpUrlConnection
try this:
StringBuilder stringBuilder = new StringBuilder();
BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream));
String line;
while ((line = reader.readLine()) != null) {
stringBuilder.append(line);
}
inputStream.close();
I am sending a mp3 file using the following code to a server.In server I want a php code to receive this byte array of mp3 and convert it into file and store it there.
try {
final ByteArrayOutputStream out = new ByteArrayOutputStream();
File file = new File(Environment.getExternalStorageDirectory()+ "/my.mp3");
final InputStream in = new FileInputStream(file);
final byte[] buf = new byte[2048];
int n;
while ((n = in.read(buf)) >= 0) {
out.write(buf, 0, n);
}
final byte[] data = out.toByteArray();
String urlString = "http://10.0.0.56/upload.php";
HttpPost postRequest = new HttpPost(urlString);
postRequest.setEntity(new ByteArrayEntity(data));
HttpClient client = new DefaultHttpClient();
HttpResponse response = client.execute(postRequest);
HttpEntity entity = response.getEntity();
InputStream ins = entity.getContent();
BufferedReader br = new BufferedReader(new InputStreamReader(ins));
String temp_str;
StringBuilder sb = new StringBuilder();
while((temp_str = br.readLine()) != null) {
sb.append(temp_str);
}
Log.e("response", sb.toString());
} catch (Exception e) {
// handle exception here
Log.e(e.getClass().getName(), e.getMessage());
return "exception";
}
Any one know how to read the mp3 file using php.
Your upload.php could look something like this:
$mp3_bin = file_get_contents('php://input');
file_put_contents( '/path/to/my/mp3', $mp3_bin );
Short and sweet ;)
Edit:
# A bit of golfing gives a one liner:
file_put_contents('/path/to/my.mp3', file_get_contents('php://input'));
public static String fileUploadFromPath(String url, String path) throws Throwable {
System.out.println("IN fileUploadFromPath ");
String responseData = "";
String NL = System.getProperty("line.separator");
try {
System.out.println("url ************ " + url);
File file = new File(path);
System.out.println("file ************ " + file.getAbsolutePath()
+ " : " + file.exists());
StringBuilder text = new StringBuilder();
if (file.exists()) {
BufferedReader br = new BufferedReader(new FileReader(file));
String line;
while ((line = br.readLine()) != null) {
text.append(line);
text.append(NL);
}
HttpClient httpClient = new DefaultHttpClient();
HttpPost postRequest = new HttpPost(url);
// System.out.println("postRequest ************ " +
// postRequest);
MultipartEntity multipartContent = new MultipartEntity();
ByteArrayBody key = new ByteArrayBody(text.toString()
.getBytes(), AgricultureUtils.getInstance()
.getTimeStamp() + ".3gp");
multipartContent.addPart(AgricultureUtils.getInstance()
.getTimeStamp() + ".3gp", key);
postRequest.setEntity(multipartContent);
HttpResponse response = httpClient.execute(postRequest);
BufferedReader in = new BufferedReader(new InputStreamReader(
response.getEntity().getContent()));
StringBuffer sb = new StringBuffer("");
String content = "";
while ((content = in.readLine()) != null) {
sb.append(content + NL);
}
in.close();
/*
* File myDir = new File(Constants.dirctory); if
* (!myDir.exists()) { myDir.mkdirs(); } File myFile = new
* File(myDir, fileName); FileOutputStream mFileOutStream = new
* FileOutputStream(myFile);
* mFileOutStream.write(sb.toString().getBytes());
* mFileOutStream.flush(); mFileOutStream.close();
*/
System.out.println("response " + sb);
}
} catch (Throwable e) {
System.out.println("Exception In Webservice ----- " + e);
throw e;
}
return responseData;
}
I want to upload an audio file into server.
I am able upload audio file to server through above code but the file is not working(not playing in system). If u have any idea please help me.
You should be using neither FileReader nor StringBuilder here as it treats the data as characters (encoded according to the default system character set). Really, you should not be using a Reader at all. Binary data should be handled via InputStream, e.g.
final ByteArrayOutputStream out = new ByteArrayOutputStream();
try (final InputStream in = new FileInputStream(file)) {
final byte[] buf = new byte[2048];
int n;
while ((n = in.read(buf)) >= 0) {
out.write(buf, 0, n);
}
}
final byte[] data = out.toByteArray();
Did you check checksum on server? Is it arriving unmodified?
If not, try other method to post files. This one helped me a lot:
/**
* Post request (upload files)
* #param sUrl
* #param params Form data
* #param files
* #return
*/
public static HttpData post(String sUrl, Hashtable<String, String> params, ArrayList<File> files) {
HttpData ret = new HttpData();
try {
String boundary = "*****************************************";
String newLine = "rn";
int bytesAvailable;
int bufferSize;
int maxBufferSize = 4096;
int bytesRead;
URL url = new URL(sUrl);
HttpURLConnection con = (HttpURLConnection) url.openConnection();
con.setDoInput(true);
con.setDoOutput(true);
con.setUseCaches(false);
con.setRequestMethod("POST");
con.setRequestProperty("Connection", "Keep-Alive");
con.setRequestProperty("Content-Type", "multipart/form-data;boundary="+boundary);
DataOutputStream dos = new DataOutputStream(con.getOutputStream());
//dos.writeChars(params);
//upload files
for (int i=0; i<files.size(); i++) {
Log.i("HREQ", i+"");
FileInputStream fis = new FileInputStream(files.get(i));
dos.writeBytes("--" + boundary + newLine);
dos.writeBytes("Content-Disposition: form-data; "
+ "name="file_"+i+"";filename=""
+ files.get(i).getPath() +""" + newLine + newLine);
bytesAvailable = fis.available();
bufferSize = Math.min(bytesAvailable, maxBufferSize);
byte[] buffer = new byte[bufferSize];
bytesRead = fis.read(buffer, 0, bufferSize);
while (bytesRead > 0) {
dos.write(buffer, 0, bufferSize);
bytesAvailable = fis.available();
bufferSize = Math.min(bytesAvailable, maxBufferSize);
bytesRead = fis.read(buffer, 0, bufferSize);
}
dos.writeBytes(newLine);
dos.writeBytes("--" + boundary + "--" + newLine);
fis.close();
}
// Now write the data
Enumeration keys = params.keys();
String key, val;
while (keys.hasMoreElements()) {
key = keys.nextElement().toString();
val = params.get(key);
dos.writeBytes("--" + boundary + newLine);
dos.writeBytes("Content-Disposition: form-data;name=""
+ key+""" + newLine + newLine + val);
dos.writeBytes(newLine);
dos.writeBytes("--" + boundary + "--" + newLine);
}
dos.flush();
BufferedReader rd = new BufferedReader(
new InputStreamReader(con.getInputStream()));
String line;
while ((line = rd.readLine()) != null) {
ret.content += line + "rn";
}
//get headers
Map<String, List<String>> headers = con.getHeaderFields();
Set<Entry<String, List<String>>> hKeys = headers.entrySet();
for (Iterator<Entry<String, List<String>>> i = hKeys.iterator(); i.hasNext();) {
Entry<String, List<String>> m = i.next();
Log.w("HEADER_KEY", m.getKey() + "");
ret.headers.put(m.getKey(), m.getValue().toString());
if (m.getKey().equals("set-cookie"))
ret.cookies.put(m.getKey(), m.getValue().toString());
}
dos.close();
rd.close();
} catch (MalformedURLException me) {
} catch (IOException ie) {
} catch (Exception e) {
Log.e("HREQ", "Exception: "+e.toString());
}
return ret;
}
This is taken from:
http://moazzam-khan.com/blog/?p=490
Check link for dependencies and usage.