Develop Reference

How to get spreadsheets from a specific Google Drive folder?

The code provided in this tutorial (snippet given below) retrieves a list of all the spreadsheets for the authenticated user.
public class MySpreadsheetIntegration {
public static void main(String[] args) throws AuthenticationException,
MalformedURLException, IOException, ServiceException {
SpreadsheetService service = new SpreadsheetService("MySpreadsheetIntegration-v1");
// TODO: Authorize the service object for a specific user (see other sections)
// Define the URL to request. This should never change.
URL SPREADSHEET_FEED_URL = new URL(
"https://spreadsheets.google.com/feeds/spreadsheets/private/full");
// Make a request to the API and get all spreadsheets.
SpreadsheetFeed feed = service.getFeed(SPREADSHEET_FEED_URL,
SpreadsheetFeed.class);
List<SpreadsheetEntry> spreadsheets = feed.getEntries();
// Iterate through all of the spreadsheets returned
for (SpreadsheetEntry spreadsheet : spreadsheets) {
// Print the title of this spreadsheet to the screen
System.out.println(spreadsheet.getTitle().getPlainText());
}
}
}
But I don't want to get all the spreadsheets. I only want to get those spreadsheets that are in a particular folder (if the folder exists, otherwise terminate the program). Is it possible using this API? If yes, how?
As far as my understanding goes, the SpreadsheetFeed has to be changed. But I didn't get any example snippet against it.

I worked out the solution as follows:
First, get the fileId of that particular folder. Use setQ() to pass query checking for folder and folder name. The following snippet will be useful:
result = driveService.files().list()
.setQ("mimeType='application/vnd.google-apps.folder'
AND title='" + folderName + "'")
.setPageToken(pageToken)
.execute();
Then, get the list of files in that particular folder. I found it from this tutorial. Snippet is as follows:
private static void printFilesInFolder(Drive service, String folderId) throws IOException {
Children.List request = service.children().list(folderId);
do {
try {
ChildList children = request.execute();
for (ChildReference child : children.getItems()) {
System.out.println("File Id: " + child.getId());
}
request.setPageToken(children.getNextPageToken());
} catch (IOException e) {
System.out.println("An error occurred: " + e);
request.setPageToken(null);
}
} while (request.getPageToken() != null &&
request.getPageToken().length() > 0);
}
Lastly, check for spreadsheets and get worksheet feeds for them. The following snippet might help.
URL WORKSHEET_FEED_URL = new URL("https://spreadsheets.google.com/feeds/worksheets/" + fileId + "/private/full");
WorksheetFeed feed = service.getFeed(WORKSHEET_FEED_URL, WorksheetFeed.class);
worksheets = feed.getEntries();

Batching multiple files to Amazon S3 using the Java SDK

I'm trying to upload multiple files to Amazon S3 all under the same key, by appending the files. I have a list of file names and want to upload/append the files in that order. I am pretty much exactly following this tutorial but I am looping through each file first and uploading that in part. Because the files are on hdfs (the Path is actually org.apache.hadoop.fs.Path), I am using the input stream to send the file data. Some pseudocode is below (I am commenting the blocks that are word for word from the tutorial):
// Create a list of UploadPartResponse objects. You get one of these for
// each part upload.
List<PartETag> partETags = new ArrayList<PartETag>();
// Step 1: Initialize.
InitiateMultipartUploadRequest initRequest = new InitiateMultipartUploadRequest(
bk.getBucket(), bk.getKey());
InitiateMultipartUploadResult initResponse =
s3Client.initiateMultipartUpload(initRequest);
try {
int i = 1; // part number
for (String file : files) {
Path filePath = new Path(file);
// Get the input stream and content length
long contentLength = fss.get(branch).getFileStatus(filePath).getLen();
InputStream is = fss.get(branch).open(filePath);
long filePosition = 0;
while (filePosition < contentLength) {
// create request
//upload part and add response to our list
i++;
}
}
// Step 3: Complete.
CompleteMultipartUploadRequest compRequest = new
CompleteMultipartUploadRequest(bk.getBucket(),
bk.getKey(),
initResponse.getUploadId(),
partETags);
s3Client.completeMultipartUpload(compRequest);
} catch (Exception e) {
//...
}
However, I am getting the following error:
com.amazonaws.services.s3.model.AmazonS3Exception: The XML you provided was not well-formed or did not validate against our published schema (Service: Amazon S3; Status Code: 400; Error Code: MalformedXML; Request ID: 2C1126E838F65BB9), S3 Extended Request ID: QmpybmrqepaNtTVxWRM1g2w/fYW+8DPrDwUEK1XeorNKtnUKbnJeVM6qmeNcrPwc
at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1109)
at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:741)
at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:461)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:296)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3743)
at com.amazonaws.services.s3.AmazonS3Client.completeMultipartUpload(AmazonS3Client.java:2617)
If anyone knows what the cause of this error might be, that would be greatly appreciated. Alternatively, if there is a better way to concatenate a bunch of files into one s3 key, that would be great as well. I tried using java's builtin SequenceInputStream but that did not work. Any help would be greatly appreciated. For reference, the total size of all the files could be as large as 10-15 gb.

I know it's probably a bit late but worth giving my contribution.
I've managed to solve a similar problem using the SequenceInputStream.
The tricks is in being able to calculate the total size of the result file and then feeding the SequenceInputStream with an Enumeration<InputStream>.
Here's some example code that might help:
public void combineFiles() {
List<String> files = getFiles();
long totalFileSize = files.stream()
.map(this::getContentLength)
.reduce(0L, (f, s) -> f + s);
try {
try (InputStream partialFile = new SequenceInputStream(getInputStreamEnumeration(files))) {
ObjectMetadata resultFileMetadata = new ObjectMetadata();
resultFileMetadata.setContentLength(totalFileSize);
s3Client.putObject("bucketName", "resultFilePath", partialFile, resultFileMetadata);
}
} catch (IOException e) {
LOG.error("An error occurred while combining files. {}", e);
}
}
private Enumeration<? extends InputStream> getInputStreamEnumeration(List<String> files) {
return new Enumeration<InputStream>() {
private Iterator<String> fileNamesIterator = files.iterator();
#Override
public boolean hasMoreElements() {
return fileNamesIterator.hasNext();
}
#Override
public InputStream nextElement() {
try {
return new FileInputStream(Paths.get(fileNamesIterator.next()).toFile());
} catch (FileNotFoundException e) {
System.err.println(e.getMessage());
throw new RuntimeException(e);
}
}
};
}
Hope this helps!

Save file from a website with java

I'm trying to build a jsoup based java app to automatically download English subtitles for films (I'm lazy, I know. It was inspired from a similar python based app). It's supposed to ask you the name of the film and then download an English subtitle for it from subscene.
I can make it reach the download link but I get an Unhandled content type error when I try to 'go' to that link. Here's my code
public static void main(String[] args) {
try {
String videoName = JOptionPane.showInputDialog("Title: ");
subscene(videoName);
}
catch (Exception e) {
System.out.println(e.getMessage());
}
}
public static void subscene(String videoName){
try {
String siteName = "http://www.subscene.com";
String[] splits = videoName.split("\\s+");
String codeName = "";
String text = "";
if(splits.length>1){
for(int i=0;i<splits.length;i++){
codeName = codeName+splits[i]+"-";
}
videoName = codeName.substring(0, videoName.length());
}
System.out.println("videoName is "+videoName);
// String url = "http://www.subscene.com/subtitles/"+videoName+"/english";
String url = "http://www.subscene.com/subtitles/title?q="+videoName+"&l=";
System.out.println("url is "+url);
Document doc = Jsoup.connect(url).get();
Element exact = doc.select("h2.exact").first();
Element yuel = exact.nextElementSibling();
Elements lis = yuel.children();
System.out.println(lis.first().children().text());
String hRef = lis.select("div.title > a").attr("href");
hRef = siteName+hRef+"/english";
System.out.println("hRef is "+hRef);
doc = Jsoup.connect(hRef).get();
Element nonHI = doc.select("td.a40").first();
Element papa = nonHI.parent();
Element link = papa.select("a").first();
text = link.text();
System.out.println("Subtitle is "+text);
hRef = link.attr("href");
hRef = siteName+hRef;
Document subDownloadPage = Jsoup.connect(hRef).get();
hRef = siteName+subDownloadPage.select("a#downloadButton").attr("href");
Jsoup.connect(hRef).get(); //<-- Here's where the problem lies
}
catch (java.io.IOException e) {
System.out.println(e.getMessage());
}
}
Can someone please help me so I don't have to manually download subs?
I just found out that using
java.awt.Desktop.getDesktop().browse(java.net.URI.create(hRef));
instead of
Jsoup.connect(hRef).get();
downloads the file after prompting me to save it. But I don't want to be prompted because this way I won't be able to read the name of the downloaded zip file (I want to unzip it after saving using java).

Assuming that your files are small, you can do it like this. Note that you can tell Jsoup to ignore the content type.
// get the file content
Connection connection = Jsoup.connect(path);
connection.timeout(5000);
Connection.Response resultImageResponse = connection.ignoreContentType(true).execute();
// save to file
FileOutputStream out = new FileOutputStream(localFile);
out.write(resultImageResponse.bodyAsBytes());
out.close();
I would recommend to verify the content before saving.
Because some servers will just return a HTML page when the file cannot be found, i.e. a broken hyperlink.
...
String body = resultImageResponse.body();
if (body == null || body.toLowerCase().contains("<body>"))
{
throw new IllegalStateException("invalid file content");
}
...

Here:
Document subDownloadPage = Jsoup.connect(hRef).get();
hRef = siteName+subDownloadPage.select("a#downloadButton").attr("href");
//specifically here
Jsoup.connect(hRef).get();
Looks like jsoup expects that the result of Jsoup.connect(hRef) should be an HTML or some text that it's able to parse, that's why the message states:
Unhandled content type. Must be text/*, application/xml, or application/xhtml+xml
I followed the execution of your code manually and the last URL you're trying to access returns a content type of application/x-zip-compressed, thus the cause of the exception.
In order to download this file, you should use a different approach. You could use the old but still useful URLConnection, URL or use a third party library like Apache HttpComponents to fire a GET request and retrieve the result as an InputStream, wrap it into a proper writer and write your file into your disk.
Here's an example about doing this using URL:
URL url = new URL(hRef);
InputStream in = url.openStream();
OutputStream out = new BufferedOutputStream(new FileOutputStream("D:\\foo.zip"));
final int BUFFER_SIZE = 1024 * 4;
byte[] buffer = new byte[BUFFER_SIZE];
BufferedInputStream bis = new BufferedInputStream(in);
int length;
while ( (length = bis.read(buffer)) > 0 ) {
out.write(buffer, 0, length);
}
out.close();
in.close();

Getting html Email Content in JavaMail

I've made an E- Mail Client for my Android- Phone using the JavaMail API. I dont knoq how to get the E- Mail Content if it's a html- Mail. I'm using the following Code to get the Content:
public void printMessage(int messageNo) throws Exception {
Log.i("MsgNo", "Getting message number: " + messageNo);
Message m = null;
try {
m = folder.getMessage(messageNo);
dumpPart(m);
} catch (IndexOutOfBoundsException iex) {
Log.i("Out of Range","Message number out of range");
}
}
public static void dumpPart(Part p) throws Exception {
if (p instanceof Message)
dumpEnvelope((Message)p);
Object content = p.getContent();
Log.i("dumpPart",(String) content);
String ct = p.getContentType();
try {
pr("CONTENT-TYPE: " + (new ContentType(ct)).toString());
Log.i("MsgNo", "Content Type");
} catch (ParseException pex) {
pr("BAD CONTENT-TYPE: " + ct);
Log.i("MsgNo", "Bad Content Type");
}
//* Using isMimeType to determine the content type avoids
// * fetching the actual content data until we need it.
if (p.isMimeType("text/plain")) {
pr("This is plain text");
pr("---------------------------");
Log.i("Text", (String)p.getContent());
} else {
Log.i("MsgNo", "Just a Separator");
// just a separator
pr("---------------------------");
}
}
In the Logcat, i get the return value of dumpenvelope((Message)p); , but after that nothing.
Does anybody know what to do?

Is any exception thrown?
Did you enable debugging and examine the protocol trace to see what might have failed?
Are you using IMAP?
It looks like your program was created out of pieces of the JavaMail sample program called msgshow.java, did you find the complete original sample program?
This JavaMail FAQ entry might help as well.

runtime error inputstream cannot be null

i have this runtime error in my logcat:
05-11 06:24:23.672: ERROR/AndroidRuntime(327): java.lang.RuntimeException:
Unable to create application net.osmand.activities.OsmandApplication:
java.lang.IllegalArgumentException: InputStream cannot be null
...
after debugin i found the exception comme how???
in this methode :
private BaseOsmandRender loadRenderer(String name, Set<String> loadedRenderers) throws IOException, SAXException {
InputStream is = null;
if(externalRenderers.containsKey(name)){
is = new FileInputStream(externalRenderers.get(name));
} else if(internalRenderers.containsKey(name)){
is = OsmandRenderingRulesParser.class.getResourceAsStream(internalRenderers.get(name));
} else {
throw new IllegalArgumentException("Not found " + name); //$NON-NLS-1$
}
BaseOsmandRender b = new BaseOsmandRender();
b.init(is);
loadedRenderers.add(name);
List<BaseOsmandRender> dependencies = new ArrayList<BaseOsmandRender>();
for (String s : b.getDepends()) {
if (loadedRenderers.contains(s)) {
the "is" varible is always null
any help please

If I remember correctly, getResourceAsStream returns null if it cannot find the resource. I suspect that's the cause of your problem.

There are some crazy restrictions for files in application resources.
If you trying to load resources, try to make *.mp3 extension for them. Its magic.