Develop Reference

When using service account impersonation, when calling export on Google Docs using v3 API, viewedByMeTime timestamp is updated

I am using a service account to access google doc files of users in my enterprise google account using impersonation.
See:
https://developers.google.com/drive/api/v3/about-auth#OAuth2Authorizing
So far so good.
Then, I need to download contents of Google Docs.
When calling Google Drive API to download the contents of a Google Doc, the documentation says to run the following:
https://developers.google.com/drive/api/v3/manage-downloads
Here is a java program that should reproduce the problem:
import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.api.client.http.HttpRequestInitializer;
import com.google.api.client.http.javanet.NetHttpTransport;
import com.google.api.client.json.jackson2.JacksonFactory;
import com.google.api.client.util.SecurityUtils;
import com.google.api.services.drive.Drive;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.security.GeneralSecurityException;
import java.util.Arrays;
import java.util.List;
public class FetchGoogleDocContentsWithServiceAccount {
static int readTimeout = 60000;
static int connectTimeout = 60000;
static String serviceAccountId = "";
static String serviceAccountEmail = "";
static String serviceAccountPrivateKeyFile = "";
static String serviceAccountPrivateKeyFilePassword = "";
static String fileId = "";
static JacksonFactory jacksonFactory = new JacksonFactory();
static NetHttpTransport httpTransport = new NetHttpTransport();
static List<String> googleScopeList = Arrays.asList("https://www.googleapis.com/auth/drive.readonly",
"https://www.googleapis.com/auth/admin.directory.group.readonly",
"https://www.googleapis.com/auth/admin.directory.user.alias.readonly",
"https://www.googleapis.com/auth/admin.directory.group", "https://www.googleapis.com/auth/admin.directory.user",
"https://www.googleapis.com/auth/drive");
public static void main(String[] args) throws Exception {
Drive drive = (new Drive.Builder(httpTransport,
jacksonFactory,
getRequestInitializer(getGoogleCredentials())))
.setApplicationName("Sample app").build();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
drive.files().export(fileId, "application/vnd.google-apps.document")
.executeMediaAndDownloadTo(baos);
System.out.println(baos.toString("UTF-8"));
}
public static HttpRequestInitializer getRequestInitializer(final GoogleCredential requestInitializer) {
return httpRequest -> {
requestInitializer.initialize(httpRequest);
httpRequest.setConnectTimeout(readTimeout);
httpRequest.setReadTimeout(connectTimeout);
};
}
public static GoogleCredential getGoogleCredentials() {
GoogleCredential credential;
try {
GoogleCredential.Builder b = new GoogleCredential.Builder().setTransport(httpTransport)
.setJsonFactory(jacksonFactory).setServiceAccountId(serviceAccountId)
.setServiceAccountPrivateKey(SecurityUtils.loadPrivateKeyFromKeyStore(SecurityUtils.getPkcs12KeyStore(),
new FileInputStream(new File(serviceAccountPrivateKeyFile)), serviceAccountPrivateKeyFilePassword,
"privatekey", serviceAccountPrivateKeyFilePassword))
.setServiceAccountScopes(googleScopeList);
if (serviceAccountEmail != null) {
b = b.setServiceAccountUser(serviceAccountEmail);
}
credential = b.build();
} catch (IOException | GeneralSecurityException e1) {
throw new RuntimeException("Could not build client secrets", e1);
}
return credential;
}
}
When I have performed this operation, we are seeing that the viewedByMeTime field is actually being updated as the impersonated user.
This is not good, because now people think someone might have stolen access to their account. They are going to open tickets with the security team.
Is this expected? How can I make this stop? Is there another method in the API I can call to download the google docs without updating this timestamp?
Also opened a ticket on the github for the google drive java sdk: https://github.com/googleapis/google-api-java-client-services/issues/3160

Updating the viewedByMeTime field upon calling the endpoint is indeed intended behaviour. Any action performed through the API is considered the same way as if the user did that action manually (i.e. that field would be updated too when the user visits the document through the UI).
By using domain-wise delegation (or "user impersonation"), you have no way to avoid this issue.
The only workaround would be to give the service account access to this file, and let it export the file without domain-wide delegation. The viewedByMeTime field will be updated only for the service account itself, but not for the original owner of that file (or any other user having access to it).

Not able to parse new york times article using boilerpipe

I am trying to get news article from 'new york times' url but it is not giving any output, but if I try for any other newspaper it gives output. I want to know if something is wrong with my code or boilerpipe is not able to fetch it. Plus sometimes the output is not in english language means it shows in unicode mainly for 'daily news', I want to know reason for that also.
import java.io.InputStream;
import java.net.URL;
import org.xml.sax.InputSource;
import de.l3s.boilerpipe.document.TextDocument;
import de.l3s.boilerpipe.extractors.ArticleExtractor;
import de.l3s.boilerpipe.extractors.DefaultExtractor;
import de.l3s.boilerpipe.sax.BoilerpipeSAXInput;
class ExtractData
{
public static void main(final String[] args) throws Exception
{
URL url;
url = new URL(
"http://www.nytimes.com/2013/03/02/nyregion/us-judges-offer-addicts-a-way-to-avoid-prison.html?hp&_r=0");
// NOTE We ignore HTTP-based character encoding in this demo...
final InputStream urlStream = url.openStream();
final InputSource is = new InputSource(urlStream);
final BoilerpipeSAXInput in = new BoilerpipeSAXInput(is);
final TextDocument doc = in.getTextDocument();
urlStream.close();
// You have the choice between different Extractors
//System.out.println(DefaultExtractor.INSTANCE.getText(doc));
System.out.println(ArticleExtractor.INSTANCE.getText(doc));
}
}

Nytimes.com has a paywall and it returns HTTP 303 for your request, you could try to handle the redirect and cookies. Trying other user-agent strings might also work.

Jackrabbit WebDAV Synchronization Examples?

I'm using the Jackrabbit library for communicating with a cloud storage using the webdav protocol. I need a way to list all files from a specific directory and get the last modified property but I can't seem to find any working examples on this.
I basically need code to synchronize files from the local directory with the webdav url.
import java.io.File;
import java.io.FileInputStream;
import org.apache.commons.httpclient.Credentials;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.UsernamePasswordCredentials;
import org.apache.commons.httpclient.auth.AuthScope;
import org.apache.commons.httpclient.methods.InputStreamRequestEntity;
import org.apache.commons.httpclient.methods.RequestEntity;
import org.apache.jackrabbit.webdav.client.methods.DavMethod;
import org.apache.jackrabbit.webdav.client.methods.MkColMethod;
import org.apache.jackrabbit.webdav.client.methods.PutMethod;
public class WebDavClient
{
private String resourceUrl;
private HttpClient client;
private Credentials credentials;
private DavMethod method;
public WebDavClient(String resourceUrl, String username, String password)
throws Exception
{
this.resourceUrl = resourceUrl;
client = new HttpClient();
credentials = new UsernamePasswordCredentials(username, password);
client.getState().setCredentials(AuthScope.ANY, credentials);
}
public int upload(String fileToUpload) throws Exception
{
method = new PutMethod(getUpdatedWebDavPath(fileToUpload));
RequestEntity requestEntity = new InputStreamRequestEntity(
new FileInputStream(fileToUpload));
((PutMethod) method).setRequestEntity(requestEntity);
client.executeMethod(method);
return method.getStatusCode();
}
public int createFolder(String folder) throws Exception
{
method = new MkColMethod(getUpdatedWebDavPath(folder));
client.executeMethod(method);
return method.getStatusCode();
}
private String getUpdatedWebDavPath(String file)
{
// Make sure file names do not contain spaces
return resourceUrl + "/" + new File(file).getName().replace(" ", "");
}
}
Usage example for uploading the file Test.txt to the Backup folder:
String myAccountName = "...";
String myPassword = "...";
WebDavClient webdavUploader = new WebDavClient("https:\\\\webdav.hidrive.strato.com\\users\\" + myAccountName + "\\Backup", myAccountName, myPassword);
webdavUploader.upload("C:\\Users\\Username\\Desktop\\Test.txt");
Here's a list of different DavMethods that could be helpful:
http://jackrabbit.apache.org/api/1.6/org/apache/jackrabbit/webdav/client/methods/package-summary.html
Please help, I'm been struggling on this for so long!

Take a look at the AMES WebDAV Client code from Krusche and Partner on the EU portal. It is licensed under GPL, so should it may fit your purpose.
https://joinup.ec.europa.eu/svn/ames-web-service/trunk/AMES-WebDAV/ames-webdav/src/de/kp/ames/webdav/WebDAVClient.java
It works for me, though to access e.g. Win32LastModifiedTime I need to get the custom namespace, e.g.
private static final Namespace WIN32_NAMESPACE = Namespace.getNamespace("Z2", "urn:schemas-microsoft-com:");
and retrieve the custom Property Win32LastModifiedTime from the properties.
/*
* Win32LastModifiedTime
*/
String win32lastmodifiedtime = null;
DavProperty<?> Win32LastModifiedTime = properties.get("Win32LastModifiedTime", WIN32_NAMESPACE);
if ((Win32LastModifiedTime != null) && (Win32LastModifiedTime.getValue() != null)) win32lastmodifiedtime = Win32LastModifiedTime.getValue().toString();

Cannot run public class in one .java from another

I have created a basic program that takes whatever is input into two textfields and exports them to a file. I would now like to encrypt that file, and alredy have the encryptor. The problem is that I cannot call it. Here is my code for the encryptor:
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.*;
import javax.crypto.Cipher;
import javax.crypto.CipherInputStream;
import javax.crypto.CipherOutputStream;
import javax.crypto.spec.SecretKeySpec;
public class FileEncryptor {
private String algo;
private File file;
public FileEncryptor(String algo,String path) {
this.algo=algo; //setting algo
this.file=new File(path); //settong file
}
public void encrypt() throws Exception{
//opening streams
FileInputStream fis =new FileInputStream(file);
file=new File(file.getAbsolutePath());
FileOutputStream fos =new FileOutputStream(file);
//generating key
byte k[] = "HignDlPs".getBytes();
SecretKeySpec key = new SecretKeySpec(k,algo.split("/")[0]);
//creating and initialising cipher and cipher streams
Cipher encrypt = Cipher.getInstance(algo);
encrypt.init(Cipher.ENCRYPT_MODE, key);
CipherOutputStream cout=new CipherOutputStream(fos, encrypt);
byte[] buf = new byte[1024];
int read;
while((read=fis.read(buf))!=-1) //reading data
cout.write(buf,0,read); //writing encrypted data
//closing streams
fis.close();
cout.flush();
cout.close();
}
public static void main (String[] args)throws Exception {
new FileEncryptor("DES/ECB/PKCS5Padding","C:\\Users\\*******\\Desktop\\newtext").encrypt();//encrypts the current file.
}
}
Here is the section of my file creator that is failing to call this:
FileWriter fWriter = null;
BufferedWriter writer = null;
try{
fWriter = new FileWriter("C:\\Users\\*******\\Desktop\\newtext");
writer = new BufferedWriter(fWriter);
writer.write(Data);
writer.close();
f.dispose();
FileEncryptor encr = new FileEncryptor(); //problem lies here.
encr.encrypt //public void that does the encryption.
new complete(); //different .java that is working fine.
Ok, I think I have it sussed. Thank you to all those who contributed.

You didn't pass anything into your constructor when using the new operator in your file creator:
FileEncryptor encr = new FileEncryptor(); //problem lies here.
However, you did when testing it in main in FileEncryptor:
new FileEncryptor("DES/ECB/PKCS5Padding","C:\\Users\\*******\\Desktop\\newtext").encrypt();//encrypts the current file.
Pass appropriate parameters.

When you try to create your new FileEncryptor object, you have to use one of the constructors you implemented in the FileEncryptor.java file. Like this:
String anAlgo = "something";
String aPath = "something"'
FileEncryptor encr = new FileEncryptor(anAlgo, aPath);
Hope this helps.

There is only one constructor in this class which takes 2 arguments. This means that the object creation mechanism requires 2 arguments. If you need to create an object without these arguments, you can also provide a no argument constructor in addition to the 2 argument constructor like public FileEncryptor(){//Default Constructor} but then it would not make any sense since algo and path are required to perform encryption

Your public FileEncryptor(String algo,String path) have constructor with two parameter. The compiler automatically provides a no-argument, default constructor for any class which doesn't have constructors. But as soon as you declare one, you won't be able call the empty constructor using new FileEncryptor(); unless you specify it in the class context:
public class FileEncryptor {
private String algo;
private File file;
public FileEncryptor(String algo,String path) {
this.algo=algo; //setting algo
this.file=new File(path); //settong file
}
public FileEncryptor()
{
// your code here
}

You are missing the arguments for the Constructor of your FileEncryptor class.
This is your Constructor.
public FileEncryptor(String algo,String path) {
this.algo=algo; //setting algo
this.file=new File(path); //settong file
}
But you are creating an object like this.
FileEncryptor encr = new FileEncryptor(); //problem lies here.
You need to pass it the encryption algorithm name which you want to use and the path for the file which you want to encrypt.
The Cipher class gets an instance of the algorithm which you want to use for encryption.
Cipher encrypt = Cipher.getInstance(algo);
Go through the documentation of Cipher class to see what types are supported. Here is the link.
http://docs.oracle.com/javase/7/docs/api/javax/crypto/Cipher.html#getInstance%28java.lang.String%29

Distributing a Java program with content

For the past couple of months I've been working on a game in java for a university project. It's coming up to the end of the project and I would like to compile the project into a single file which is easy to distribute. The game currently runs from inside the IDE and relies on the working directory being set somewhere specific (i.e. the content directory with sounds/textures etc). What's the best way to put all this together for portability? I'm hoping there is some way to compile the content into the jar file...
NB. I'm using NetBeans, so any solutions which are easy in netbeans get extra credit ;)
Addendum::
For future reference, I found a way of accessing things by directory, ythis may not be the best way but it works:
File directory = new File(ClassLoader.getSystemResource("fullprototypeone/Content/Levels/").toURI());
And now I can just use that file object as normal

You can embed resources in jar file - this is just a zip after all. The standard way to do that is to put resource files in some directory in your sources hierachy. Then you refer to them by Object.getClass().getResourceAsStream(). So you will need to change the way you retrieve them in your code.
You can read more here: Object.getClass().getResourceAsStream(). Of course instead of object you use some class from your package.
when you put those resource files in your src hierachy I believe Netbeans should jar them for you with standard build of the project.

Here is the manual for JNLP and Java Web Start. These technologies exist just for the task you've described.

Well one way is to access the resources via Class.getResourceAsStream (or Class.getResource). Then make sure that the files are in the JAR file.
Off the top of my head (and without trying it) you should be able to put the resources in with the source files which will get NetBeans to put them into the JAR file. Then change the File stuff to the getResource calls.
I would suggest making a simple program that plays a sound and trying it out before you try to convert the whole project over.
If you try and it doesn't work let me know and I'll see if I can dig into it a bit more (posting the code of the simple project would be good if it comes to that).

Here is the code I promised in another comment... it isn't quite what I remember but it might get you started.
Essentially you call: String fileName = FileUtils.getFileName(Main.class, "foo.txt");
and it goes and finds that file on disk or in a JAR file. If it is in the JAR file it extracts it to a temp directory. You can then use "new File(fileName)" to open the file which, no matter where it was before, will be on the disk.
What I would do is take a look at the getFile method and look at what you can do with the JAR file to iterate over the contents of it and find the files in a given directory.
Like I said, not exactly what you want, but does do a lot of the initial work for you.
import java.io.Closeable;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.UnsupportedEncodingException;
import java.net.URI;
import java.net.URISyntaxException;
import java.net.URL;
import java.net.URLDecoder;
import java.security.CodeSource;
import java.security.ProtectionDomain;
import java.util.zip.ZipEntry;
import java.util.zip.ZipException;
import java.util.zip.ZipFile;
public class FileUtils
{
public static String getFileName(final Class<?> owner,
final String name)
throws URISyntaxException,
ZipException,
IOException
{
String fileName;
final URI uri;
try
{
final String external;
final String decoded;
final int pos;
uri = getResourceAsURI(owner.getPackage().getName().replaceAll("\\.", "/") + "/" + name, owner);
external = uri.toURL().toExternalForm();
decoded = external; // URLDecoder.decode(external, "UTF-8");
pos = decoded.indexOf(":/");
fileName = decoded.substring(pos + 1);
}
catch(final FileNotFoundException ex)
{
fileName = null;
}
if(fileName == null || !(new File(fileName).exists()))
{
fileName = getFileNameX(owner, name);
}
return (fileName);
}
private static String getFileNameX(final Class<?> clazz, final String name)
throws UnsupportedEncodingException
{
final URL url;
final String fileName;
url = clazz.getResource(name);
if(url == null)
{
fileName = name;
}
else
{
final String decoded;
final int pos;
decoded = URLDecoder.decode(url.toExternalForm(), "UTF-8");
pos = decoded.indexOf(":/");
fileName = decoded.substring(pos + 1);
}
return (fileName);
}
private static URI getResourceAsURI(final String resourceName,
final Class<?> clazz)
throws URISyntaxException,
ZipException,
IOException
{
final URI uri;
final URI resourceURI;
uri = getJarURI(clazz);
resourceURI = getFile(uri, resourceName);
return (resourceURI);
}
private static URI getJarURI(final Class<?> clazz)
throws URISyntaxException
{
final ProtectionDomain domain;
final CodeSource source;
final URL url;
final URI uri;
domain = clazz.getProtectionDomain();
source = domain.getCodeSource();
url = source.getLocation();
uri = url.toURI();
return (uri);
}
private static URI getFile(final URI where,
final String fileName)
throws ZipException,
IOException
{
final File location;
final URI fileURI;
location = new File(where);
// not in a JAR, just return the path on disk
if(location.isDirectory())
{
fileURI = URI.create(where.toString() + fileName);
}
else
{
final ZipFile zipFile;
zipFile = new ZipFile(location);
try
{
fileURI = extract(zipFile, fileName);
}
finally
{
zipFile.close();
}
}
return (fileURI);
}
private static URI extract(final ZipFile zipFile,
final String fileName)
throws IOException
{
final File tempFile;
final ZipEntry entry;
final InputStream zipStream;
OutputStream fileStream;
tempFile = File.createTempFile(fileName.replace("/", ""), Long.toString(System.currentTimeMillis()));
tempFile.deleteOnExit();
entry = zipFile.getEntry(fileName);
if(entry == null)
{
throw new FileNotFoundException("cannot find file: " + fileName + " in archive: " + zipFile.getName());
}
zipStream = zipFile.getInputStream(entry);
fileStream = null;
try
{
final byte[] buf;
int i;
fileStream = new FileOutputStream(tempFile);
buf = new byte[1024];
i = 0;
while((i = zipStream.read(buf)) != -1)
{
fileStream.write(buf, 0, i);
}
}
finally
{
close(zipStream);
close(fileStream);
}
return (tempFile.toURI());
}
private static void close(final Closeable stream)
{
if(stream != null)
{
try
{
stream.close();
}
catch(final IOException ex)
{
ex.printStackTrace();
}
}
}
}
Edit:
Sorry, I don't have time to work on this right now (if I get some I'll do it and post the code here). This is what I would do though:
Look at the ZipFile class and use the entries() method to find all of the files/directories in the zip file.
the ZipEntry has an isDirectory() method that you can use to figure out what it is.
I think the code I posted in this answer will give you a way to pick a temporary directory to extract the contents to.
I think the code I posted in this answer could help with copying the ZipEntry contents to the file system.
once the items are on the file system the code you already have for iterating over the directory would still work. You would add a new method to the FileUtils class in the code above and be able to find all of the files as you are doing now.
There is probably a better way to do it, but off the top of my head I think that will work.

yes ! put your compiled .class files and your resources with folders in a jar file .. you ll have to build a manifest file as well .. you can find the tutorial about making a .jar file on
google. most probably you ll be referred to java.sun.com .