I know it is possible to change the permission mode of a file using:
Runtime.getRuntime().exec( "chmod 777 myfile" );.
This example sets the permission bits to 777. Is it possible to set the permission bits to 777 programmatically using Java? Can this be done to every file?
Using chmod in Android
Java doesn't have native support for platform dependent operations like chmod. However, Android provides utilities for some of these operations via android.os.FileUtils. The FileUtils class is not part of the public SDK and is therefore not supported. So, use this at your own risk:
public int chmod(File path, int mode) throws Exception {
Class fileUtils = Class.forName("android.os.FileUtils");
Method setPermissions =
fileUtils.getMethod("setPermissions", String.class, int.class, int.class, int.class);
return (Integer) setPermissions.invoke(null, path.getAbsolutePath(), mode, -1, -1);
}
...
chmod("/foo/bar/baz", 0755);
...
Reference : http://www.damonkohler.com/2010/05/using-chmod-in-android.html?showComment=1341900716400#c4186506545056003185
Android make it difficult to interact with other applications and their data except through Intents. Intents won't work for permissions because you're dependent on the application receiving the Intent to do/provide what you want; they probably weren't designed to tell anybody their files' permissions. There are ways around it, but only when the applications are desgned to operate in the same JVM.
So each application can only change it's file. for more detail in file permission see http://docs.oracle.com/javase/1.4.2/docs/guide/security/permissions.html
As commented previously, android.os.FileUtils has changed and the solution posted by Ashraf no longer works. The following method should work on all versions of Android (although it does use reflection and if a manufacturer made major changes this may not work).
public static void chmod(String path, int mode) throws Exception {
Class<?> libcore = Class.forName("libcore.io.Libcore");
Field field = libcore.getDeclaredField("os");
if (!field.isAccessible()) {
field.setAccessible(true);
}
Object os = field.get(field);
Method chmod = os.getClass().getMethod("chmod", String.class, int.class);
chmod.invoke(os, path, mode);
}
Obviously you will need to own the file to make any permission changes.
Here is a solution using Apache Commons.IO FileUtils, and the appropriate methods on File objects.
for (File f : FileUtils.listFilesAndDirs(new File('/some/path'), TrueFileFilter.TRUE, TrueFileFilter.TRUE)) {
if (!f.setReadable(true, false)) {
throw new IOException(String.format("Failed to setReadable for all on %s", f));
}
if (!f.setWritable(true, false)) {
throw new IOException(String.format("Failed to setWritable for all on %s", f));
}
if (!f.setExecutable(true, false)) {
throw new IOException(String.format("Failed to setExecutable for all on %s", f));
}
}
This is the equivalent of chmod -R 0777 /some/path. Adjust the set{Read,Writ,Execut}able calls to implement other modes. (I'll happily update this answer if someone posts appropriate code to do this.)
Related
I'm trying to understand a comment that a colleague made. We're using testcontainers to create a fixture:
import org.testcontainers.containers.GenericContainer;
import org.testcontainers.utility.DockerImageName;
public class SalesforceFixture extends GenericContainer<SalesforceFixture> {
private static final String APPLICATION_NAME = "salesforce-emulator";
public SalesforceFixture() {
// super(ImageResolver.resolve(APPLICATION_NAME));
super(DockerImageName.parse("gcr.io/ad-selfserve/salesforce-emulator:latest"));
...
}
...
The commented code is what it used to be. The next line is my colleague's suggestion. And on that line he commented:
This is the part I don't know. The [ImageResolver] gets the specific version of the emulator, rather than the latest. You need a docker-info file for that though, which jib doesn't automatically generate (but I think it can).
This is what I know or have figured so far:
SalesforceFixture is a class that will be used by other projects to write tests. It spins up a container in Docker, running a service that emulates the real service's API. It's like a local version of the service that behaves enough like the real thing that if one writes code and tests using the fixture, it should work the same in production. (This is where my knowledge ends.)
I looked into ImageResolver—it seems to be a class we wrote that searches a filesystem for something:
public static String resolve(String applicationName, File... roots) {
Stream<File> searchPaths = Arrays.stream(roots).flatMap((value) -> {
return Stream.of(new File(value, "../" + applicationName), new File(value, applicationName));
});
Optional<File> buildFile = searchPaths.flatMap((searchFile) -> {
if (searchFile.exists()) {
File imageFile = new File(searchFile + File.separator + "/target/docker/image-name");
if (imageFile.exists()) {
return Stream.of(imageFile);
}
}
return Stream.empty();
}).findAny();
InputStream build = (InputStream)buildFile.map(ImageResolver::fileStream).orElseGet(() -> {
return searchClasspath(applicationName);
});
if (build != null) {
try {
return IOUtils.toString(build, Charset.defaultCharset()).trim();
} catch (IOException var6) {
throw new RuntimeException("An exception has occurred while reading build file", var6);
}
} else {
throw new RuntimeException("Could not resolve target image for application: " + applicationName);
}
}
But I'm confused. What filesystem? Like, what is the present working directory? My local computer, wherever I ran the Java program from? Or is this from within some container? (I don't think so.) Or maybe the directory structure inside a .jar file? Or somewhere in gcr.io?
What does he mean about a "specific version number" vs. "latest"? I mean, when I build this project, whatever it built is all I have. Isn't that equivalent to "latest"? In what case would an older version of an image be present? (That's what made me think of gcr.io.)
Or, does he mean, that in the project using this project's image, one will not be able to specify a version via Maven/pom.xml—it will always spin up the latest.
Sorry this is long, just trying to "show my work." Any hints welcome. I'll keep looking.
I can't comment on specifics of your own internal implementations, but ImageResolver seems to work on your local filesystem, e.g. it looks into your target/ directory and also touches the classpath. I can imagine this code was just written for resolving an actual image name (not an image), since it also returns a String.
Regarding latest, using a latest tag for a Docker image is generally considered an anti-pattern, so likely your colleague is commenting about this. Here is a random article from the web explaining some of the issues with latest tag:
https://vsupalov.com/docker-latest-tag/
Besides, I don't understand why you ask these questions which are very specific to your project here on SO rather than asking your colleague.
What is the best way to find the user's home directory in Java?
The difficulty is that the solution should be cross-platform; it should work on Windows 2000, XP, Vista, OS X, Linux, and other Unix variants. I am looking for a snippet of code that can accomplish this for all platforms, and a way to detect the platform.
Per Java bug 4787931, system property user.home does not work correctly on Windows XP, so using this system property is not an acceptable solution as it is not cross-platform.
The bug you reference (bug 4787391) has been fixed in Java 8. Even if you are using an older version of Java, the System.getProperty("user.home") approach is probably still the best. The user.home approach seems to work in a very large number of cases. A 100% bulletproof solution on Windows is hard, because Windows has a shifting concept of what the home directory means.
If user.home isn't good enough for you I would suggest choosing a definition of home directory for windows and using it, getting the appropriate environment variable with System.getenv(String).
Actually with Java 8 the right way is to use:
System.getProperty("user.home");
The bug JDK-6519127 has been fixed and the "Incompatibilities between JDK 8 and JDK 7" section of the release notes states:
Area: Core Libs / java.lang
Synopsis
The steps used to determine the user's home directory on Windows have changed to follow the Microsoft recommended approach. This change
might be observable on older editions of Windows or where registry
settings or environment variables are set to other directories. Nature
of Incompatibility
behavioral RFE
6519127
Despite the question being old I leave this for future reference.
System.getProperty("user.home");
See the JavaDoc.
The concept of a HOME directory seems to be a bit vague when it comes to Windows. If the environment variables (HOMEDRIVE/HOMEPATH/USERPROFILE) aren't enough, you may have to resort to using native functions via JNI or JNA. SHGetFolderPath allows you to retrieve special folders, like My Documents (CSIDL_PERSONAL) or Local Settings\Application Data (CSIDL_LOCAL_APPDATA).
Sample JNA code:
public class PrintAppDataDir {
public static void main(String[] args) {
if (com.sun.jna.Platform.isWindows()) {
HWND hwndOwner = null;
int nFolder = Shell32.CSIDL_LOCAL_APPDATA;
HANDLE hToken = null;
int dwFlags = Shell32.SHGFP_TYPE_CURRENT;
char[] pszPath = new char[Shell32.MAX_PATH];
int hResult = Shell32.INSTANCE.SHGetFolderPath(hwndOwner, nFolder,
hToken, dwFlags, pszPath);
if (Shell32.S_OK == hResult) {
String path = new String(pszPath);
int len = path.indexOf('\0');
path = path.substring(0, len);
System.out.println(path);
} else {
System.err.println("Error: " + hResult);
}
}
}
private static Map<String, Object> OPTIONS = new HashMap<String, Object>();
static {
OPTIONS.put(Library.OPTION_TYPE_MAPPER, W32APITypeMapper.UNICODE);
OPTIONS.put(Library.OPTION_FUNCTION_MAPPER,
W32APIFunctionMapper.UNICODE);
}
static class HANDLE extends PointerType implements NativeMapped {
}
static class HWND extends HANDLE {
}
static interface Shell32 extends Library {
public static final int MAX_PATH = 260;
public static final int CSIDL_LOCAL_APPDATA = 0x001c;
public static final int SHGFP_TYPE_CURRENT = 0;
public static final int SHGFP_TYPE_DEFAULT = 1;
public static final int S_OK = 0;
static Shell32 INSTANCE = (Shell32) Native.loadLibrary("shell32",
Shell32.class, OPTIONS);
/**
* see http://msdn.microsoft.com/en-us/library/bb762181(VS.85).aspx
*
* HRESULT SHGetFolderPath( HWND hwndOwner, int nFolder, HANDLE hToken,
* DWORD dwFlags, LPTSTR pszPath);
*/
public int SHGetFolderPath(HWND hwndOwner, int nFolder, HANDLE hToken,
int dwFlags, char[] pszPath);
}
}
Others have answered the question before me but a useful program to print out all available properties is:
for (Map.Entry<?,?> e : System.getProperties().entrySet()) {
System.out.println(String.format("%s = %s", e.getKey(), e.getValue()));
}
Alternative would be to use Apache CommonsIO FileUtils.getUserDirectory() instead of System.getProperty("user.home"). It will get you the same result and there is no chance to introduce a typo when specifying system property.
There is a big chance you already have Apache CommonsIO library in your project. Don't introduce it if you plan to use it only for getting user home directory.
As I was searching for Scala version, all I could find was McDowell's JNA code above. I include my Scala port here, as there currently isn't anywhere more appropriate.
import com.sun.jna.platform.win32._
object jna {
def getHome: java.io.File = {
if (!com.sun.jna.Platform.isWindows()) {
new java.io.File(System.getProperty("user.home"))
}
else {
val pszPath: Array[Char] = new Array[Char](WinDef.MAX_PATH)
new java.io.File(Shell32.INSTANCE.SHGetSpecialFolderPath(null, pszPath, ShlObj.CSIDL_MYDOCUMENTS, false) match {
case true => new String(pszPath.takeWhile(c => c != '\0'))
case _ => System.getProperty("user.home")
})
}
}
}
As with the Java version, you will need to add Java Native Access, including both jar files, to your referenced libraries.
It's nice to see that JNA now makes this much easier than when the original code was posted.
I would use the algorithm detailed in the bug report using System.getenv(String), and fallback to using the user.dir property if none of the environment variables indicated a valid existing directory. This should work cross-platform.
I think, under Windows, what you are really after is the user's notional "documents" directory.
If you want something that works well on windows there is a package called WinFoldersJava which wraps the native call to get the 'special' directories on Windows. We use it frequently and it works well.
Hi I have test program which loads files into hdfs at this path user/user1/data/app/type/file.gz Now this test program runs multiple times by multiple users. So I want to set file permission to rwx so that anyone can delete this file. I have the following code
fs.setPermission(new Path("user/user1/data"),new FsPermission(FsAction.ALL,FsAction.ALL,FsAction.ALL))
Above line gives drwxrwxrwx to all dirs but for the file.gz it gives permission as -rw-r--r-- why so? Because of this reason another user apart from me not able to delete this file through test program. I can delete file through test program because I have full permssion.
Please guide. I am new to Hadoop. Thanks in advance.
Using FsShell APIs solved my dir permission problem. It may be not be optimal way but since I am solving it for test code it should be fine.
FsShell shell=new FsShell(conf);
try {
shell.run(new String[]{"-chmod","-R","777","user/usr1/data"});
}
catch ( Exception e) {
LOG.error("Couldnt change the file permissions ",e);
throw new IOException(e);
}
I think Hadoop doesn't provide a java API to provide permission recursively in the version you are using. The code is actually giving permission to the dir user/user1/data and nothing else. You should better use FileSystem.listStatus(Path f) method to list down all files in the directory and use Filsystem.setPermission to them individually. It's working currently on my 2.0.5-alpha-gphd-2.1.0.0 cluster.
However from Hadoop 2.2.0 onward a new constructor
FsPermission(FsAction u, FsAction g, FsAction o, boolean sb)
is being provided. The boolean field may be the recursive flag you wanted. But the documentation(including param name) is too poor to infer something concrete.
Also have a look at Why does "hadoop fs -mkdir" fail with Permission Denied? although your sitaution may be different. (in my case dfs.permissions.enabled is still true).
I wrote this in scala but I think you could easily adapt it:
def changeUserGroup(user:String,fs:FileSystem, path: Path): Boolean ={
val changedPermission = new FsPermission(FsAction.ALL,FsAction.ALL,FsAction.ALL, true)
val fileList = fs.listFiles(path, true)
while (fileList.hasNext()) {
fs.setPermission(fileList.next().getPath(),changedPermission)
}
return true
}
You will also have to add a logic for the error handling, I am always returning true.
I load an external library that is placed in ./lib. Are these two solutions to set the java.library.path equivalent?
Set path in console when executing jar:
java -Djava.library.path=./lib -jar myApplication.jar
Set path in the code before loading library:
System.setProperty("java.library.path", "./lib");
If they are equivalent, why in the second solution can Java not find the library while the first one is ok?
If not, is there a way the set the path in the code?
Although it is not well documented, the java.library.path system property is a "read-only" property as far as the System.loadLibrary() method is concerned. This is a reported bug but it was closed by Sun as opposed to getting fixed. The problem is that the JVM's ClassLoader reads this property once at startup and then caches it, not allowing us to change it programatically afterward. The line System.setProperty("java.library.path", anyVal); will have no effect except for System.getProperty() method calls.
Luckily, someone posted a workaround on the Sun forums. Unfortunately, that link no longer works but I did find the code on another source. Here is the code you can use to work around not being able to set the java.library.path system property:
public static void addDir(String s) throws IOException {
try {
// This enables the java.library.path to be modified at runtime
// From a Sun engineer at http://forums.sun.com/thread.jspa?threadID=707176
//
Field field = ClassLoader.class.getDeclaredField("usr_paths");
field.setAccessible(true);
String[] paths = (String[])field.get(null);
for (int i = 0; i < paths.length; i++) {
if (s.equals(paths[i])) {
return;
}
}
String[] tmp = new String[paths.length+1];
System.arraycopy(paths,0,tmp,0,paths.length);
tmp[paths.length] = s;
field.set(null,tmp);
System.setProperty("java.library.path", System.getProperty("java.library.path") + File.pathSeparator + s);
} catch (IllegalAccessException e) {
throw new IOException("Failed to get permissions to set library path");
} catch (NoSuchFieldException e) {
throw new IOException("Failed to get field handle to set library path");
}
}
WARNING: This may not work on all platforms and/or JVMs.
Generally speaking, both approaches have the same net effect in that the system property java.library.path is set to the value ./lib.
However, some system properties are only evaluated at specific points in time, such as the startup of the JVM. If java.library.path is among those properties (and your experiment seems to indicate that), then using the second approach will have no noticeable effect except for returning the new value on future invocations of getProperty().
As a rule of thumb, using the -D command line property works on all system properties, while System.setProperty() only works on properties that are not only checked during startup.
you can add three lines
System.setProperty("java.library.path", "/path/to/libs" );
Field fieldSysPath = ClassLoader.class.getDeclaredField( "sys_paths" );
fieldSysPath.setAccessible( true );
fieldSysPath.set( null, null );
and also import java.lang.reflect.Field
It's ok to solve the problem
This is an addendum to this answer to Jesse Webb's amazing answer above: https://stackoverflow.com/a/6408467/257299
For Java 17:
import jdk.internal.loader.NativeLibraries;
final Class<?>[] declClassArr = NativeLibraries.class.getDeclaredClasses();
final Class<?> libraryPaths =
Arrays.stream(declClassArr)
.filter(klass -> klass.getSimpleName().equals("LibraryPaths"))
.findFirst()
.get();
final Field field = libraryPaths.getDeclaredField("USER_PATHS");
final MethodHandles.Lookup lookup = MethodHandles.privateLookupIn(Field.class, MethodHandles.lookup());
final VarHandle varHandle = lookup.findVarHandle(Field.class, "modifiers", int.class);
varHandle.set(field, field.getModifiers() & ~Modifier.FINAL);
Since package jdk.internal.loader from module java.base is not normally accessible, you will need to add "exports" and "opens" to both the compiler and JVM runtime args.
--add-exports=java.base/jdk.internal.loader=ALL-UNNAMED
--add-opens=java.base/jdk.internal.loader=ALL-UNNAMED
--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
Read more here:
--add-exports: https://stackoverflow.com/a/53647605/257299
--add-opens: https://stackoverflow.com/a/61663667/257299
Remove final modifier on Java12+: https://stackoverflow.com/a/56043252/257299
What is the best way to find the user's home directory in Java?
The difficulty is that the solution should be cross-platform; it should work on Windows 2000, XP, Vista, OS X, Linux, and other Unix variants. I am looking for a snippet of code that can accomplish this for all platforms, and a way to detect the platform.
Per Java bug 4787931, system property user.home does not work correctly on Windows XP, so using this system property is not an acceptable solution as it is not cross-platform.
The bug you reference (bug 4787391) has been fixed in Java 8. Even if you are using an older version of Java, the System.getProperty("user.home") approach is probably still the best. The user.home approach seems to work in a very large number of cases. A 100% bulletproof solution on Windows is hard, because Windows has a shifting concept of what the home directory means.
If user.home isn't good enough for you I would suggest choosing a definition of home directory for windows and using it, getting the appropriate environment variable with System.getenv(String).
Actually with Java 8 the right way is to use:
System.getProperty("user.home");
The bug JDK-6519127 has been fixed and the "Incompatibilities between JDK 8 and JDK 7" section of the release notes states:
Area: Core Libs / java.lang
Synopsis
The steps used to determine the user's home directory on Windows have changed to follow the Microsoft recommended approach. This change
might be observable on older editions of Windows or where registry
settings or environment variables are set to other directories. Nature
of Incompatibility
behavioral RFE
6519127
Despite the question being old I leave this for future reference.
System.getProperty("user.home");
See the JavaDoc.
The concept of a HOME directory seems to be a bit vague when it comes to Windows. If the environment variables (HOMEDRIVE/HOMEPATH/USERPROFILE) aren't enough, you may have to resort to using native functions via JNI or JNA. SHGetFolderPath allows you to retrieve special folders, like My Documents (CSIDL_PERSONAL) or Local Settings\Application Data (CSIDL_LOCAL_APPDATA).
Sample JNA code:
public class PrintAppDataDir {
public static void main(String[] args) {
if (com.sun.jna.Platform.isWindows()) {
HWND hwndOwner = null;
int nFolder = Shell32.CSIDL_LOCAL_APPDATA;
HANDLE hToken = null;
int dwFlags = Shell32.SHGFP_TYPE_CURRENT;
char[] pszPath = new char[Shell32.MAX_PATH];
int hResult = Shell32.INSTANCE.SHGetFolderPath(hwndOwner, nFolder,
hToken, dwFlags, pszPath);
if (Shell32.S_OK == hResult) {
String path = new String(pszPath);
int len = path.indexOf('\0');
path = path.substring(0, len);
System.out.println(path);
} else {
System.err.println("Error: " + hResult);
}
}
}
private static Map<String, Object> OPTIONS = new HashMap<String, Object>();
static {
OPTIONS.put(Library.OPTION_TYPE_MAPPER, W32APITypeMapper.UNICODE);
OPTIONS.put(Library.OPTION_FUNCTION_MAPPER,
W32APIFunctionMapper.UNICODE);
}
static class HANDLE extends PointerType implements NativeMapped {
}
static class HWND extends HANDLE {
}
static interface Shell32 extends Library {
public static final int MAX_PATH = 260;
public static final int CSIDL_LOCAL_APPDATA = 0x001c;
public static final int SHGFP_TYPE_CURRENT = 0;
public static final int SHGFP_TYPE_DEFAULT = 1;
public static final int S_OK = 0;
static Shell32 INSTANCE = (Shell32) Native.loadLibrary("shell32",
Shell32.class, OPTIONS);
/**
* see http://msdn.microsoft.com/en-us/library/bb762181(VS.85).aspx
*
* HRESULT SHGetFolderPath( HWND hwndOwner, int nFolder, HANDLE hToken,
* DWORD dwFlags, LPTSTR pszPath);
*/
public int SHGetFolderPath(HWND hwndOwner, int nFolder, HANDLE hToken,
int dwFlags, char[] pszPath);
}
}
Others have answered the question before me but a useful program to print out all available properties is:
for (Map.Entry<?,?> e : System.getProperties().entrySet()) {
System.out.println(String.format("%s = %s", e.getKey(), e.getValue()));
}
Alternative would be to use Apache CommonsIO FileUtils.getUserDirectory() instead of System.getProperty("user.home"). It will get you the same result and there is no chance to introduce a typo when specifying system property.
There is a big chance you already have Apache CommonsIO library in your project. Don't introduce it if you plan to use it only for getting user home directory.
As I was searching for Scala version, all I could find was McDowell's JNA code above. I include my Scala port here, as there currently isn't anywhere more appropriate.
import com.sun.jna.platform.win32._
object jna {
def getHome: java.io.File = {
if (!com.sun.jna.Platform.isWindows()) {
new java.io.File(System.getProperty("user.home"))
}
else {
val pszPath: Array[Char] = new Array[Char](WinDef.MAX_PATH)
new java.io.File(Shell32.INSTANCE.SHGetSpecialFolderPath(null, pszPath, ShlObj.CSIDL_MYDOCUMENTS, false) match {
case true => new String(pszPath.takeWhile(c => c != '\0'))
case _ => System.getProperty("user.home")
})
}
}
}
As with the Java version, you will need to add Java Native Access, including both jar files, to your referenced libraries.
It's nice to see that JNA now makes this much easier than when the original code was posted.
I would use the algorithm detailed in the bug report using System.getenv(String), and fallback to using the user.dir property if none of the environment variables indicated a valid existing directory. This should work cross-platform.
I think, under Windows, what you are really after is the user's notional "documents" directory.
If you want something that works well on windows there is a package called WinFoldersJava which wraps the native call to get the 'special' directories on Windows. We use it frequently and it works well.