I have a requirement to read a file continously from a specific path.
Means flink job should continously poll the specified location and read a file that will arrive at this location at certains intervals .
Example: my location on windows machine is C:/inputfiles get a file file_1.txt at 2:00PM, file_2.txt at 2:30PM, file_3.txt at 3:00PM.
I experimented it with below code .
import org.apache.flink.api.common.functions.FlatMapFunction;
import org.apache.flink.api.common.io.FilePathFilter;
import org.apache.flink.api.java.io.TextInputFormat;
import org.apache.flink.core.fs.FileSystem;
import org.apache.flink.streaming.api.datastream.DataStream;
import org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.api.functions.source.FileProcessingMode;
import org.apache.flink.util.Collector;
import java.util.Arrays;
import java.util.List;
public class ContinuousFileProcessingTest {
public static void main(String[] args) throws Exception {
final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
env.enableCheckpointing(10);
String localFsURI = "D:\\FLink\\2021_01_01\\";
TextInputFormat format = new TextInputFormat(new org.apache.flink.core.fs.Path(localFsURI));
format.setFilesFilter(FilePathFilter.createDefaultFilter());
DataStream<String> inputStream =
env.readFile(format, localFsURI, FileProcessingMode.PROCESS_CONTINUOUSLY, 100);
SingleOutputStreamOperator<String> soso = inputStream.map(String::toUpperCase);
soso.print();
soso.writeAsText("D:\\FLink\\completed", FileSystem.WriteMode.OVERWRITE);
env.execute("read and write");
}
}
Now to test this on flink cluster i brought flink cluster up using flink's 1.9.2 version and i was able to achieve my goal of reading file continously at some intervals.
Note: Flink's 1.9.2 version can bring up cluster on windows machine.
But now i have to upgrade flink's version from 1.9.2 to 1.12 .And we used docker to bring cluster up on 1.12 (unlike 1.9.2).
Unlike windows path i changed the file location as per docker location but the same above program in not running there.
Moreover:
Accessing file is not the problem.Means if i put the file before starting the job then this job reads these files correctly but if i add any new file at runtime then it does not read this newly added files.
Need help to find the solution.
Thanks in advance.
Try to reduce directoryScanInterval from sample code to Duration.ofSeconds(50).toMillis() and checkout StreamExecutionEnvironment.setRuntimeMode(RuntimeExecutionMode.AUTOMATIC) mode.
For RuntimeExecutionMode referred from https://ci.apache.org/projects/flink/flink-docs-master/api/java/org/apache/flink/api/common/RuntimeExecutionMode.html
Working code as below:
public class ContinuousFileProcessingTest {
private static final Logger log = LoggerFactory.getLogger(ReadSpecificFilesFlinkBatch.class);
public static void main(String[] args) throws Exception {
final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
env.enableCheckpointing(10);
env.setRuntimeMode(RuntimeExecutionMode.AUTOMATIC);
String localFsURI = "file:///usr/test";
// create the monitoring source along with the necessary readers.
TextInputFormat format = new TextInputFormat(new org.apache.flink.core.fs.Path(localFsURI));
log.info("format : " + format.toString());
format.setFilesFilter(FilePathFilter.createDefaultFilter());
log.info("setFilesFilter : " + FilePathFilter.createDefaultFilter().toString());
log.info("getFilesFilter : " + format.getFilePath().toString());
DataStream<String> inputStream =
env.readFile(format, localFsURI, FileProcessingMode.PROCESS_CONTINUOUSLY, Duration.ofSeconds(50).toMillis());
SingleOutputStreamOperator<String> soso = inputStream.map(String::toUpperCase);
soso.writeAsText("file:///usr/test/completed.txt", FileSystem.WriteMode.OVERWRITE);
env.execute("read and write");
}
}
This code works on docker desktop with Flink 1.12 and container file path as file:///usr/test.Note Keep parallalism as minimum 2 so that parallelly files can be processed.
Related
In a cloned git repository, I want to pick only the files that are modified (i.e, files that are ready to commit or which are shown as 'modified' if I run the command 'git status'). I do not want to do it on date change comparison as files could have been modified on any day over a period of time.
I need the collection of file names with their absolute file paths.
Is there any such git utility in Java available? Or what will be the better approach?
import java.io.File;
import java.util.Set;
import org.eclipse.jgit.api.Git;
import org.eclipse.jgit.api.Status;
import org.eclipse.jgit.api.errors.GitAPIException;
public class GitModifiedFileExtractor {
public static void main(String[] args) throws IllegalStateException, GitAPIException {
Git myGitRepo = Git.init().setDirectory(new File("C:\\myClonedGitRepo")).call();
Status status = myGitRepo.status().call();
Set<String> modifiedFiles = status.getModified();
for (String modifiedFile : modifiedFiles) {
System.out.println("Modified File - " + modifiedFile);
}
}
// Similarly we can get files - added, missing, removed, untracked, etc.,
// from status object.
}
I am running a Storm Topology which is getting tweets from Kafka on AWS
Ubuntu Server 14.04 LTS instances with 4 nodes - Nimbus, a Supervisor, a Kafka-Zookeeper node, a Zookeeper (for Storm cluster). My Storm UI is up and running and I am able to submit topologies. I have two brokers, but I'm only using the broker.id=0 one. I have tweets in it under a topic. My kafka server is running fine too.
I created the kafka-topic in this way:
bin/kafka-topics.sh --create --zookeeper localhost:2181/kafka --replication-factor 1 --partitions 1 --topic twitter1
The thing I'm confused about is:
SpoutConfig kafkaConfig = new SpoutConfig(kafkaHosts, topicName+"-0", "/kafka", topicName+"-0");
I think my errors are sprouting from this point. Complete code is:
import org.apache.storm.tuple.Fields;
import org.apache.storm.kafka.BrokerHosts;
import org.apache.storm.kafka.KafkaSpout;
import org.apache.storm.kafka.SpoutConfig;
import org.apache.storm.kafka.ZkHosts;
import java.util.Arrays;
import org.apache.storm.Config;
import org.apache.storm.StormSubmitter;
import org.apache.storm.spout.SchemeAsMultiScheme;
import org.apache.storm.topology.TopologyBuilder;
import org.apache.storm.kafka.StringScheme;
public class TwitterTopology{
public static void main(String[] args) {
String topicName = "twitter1";
String topologyName = args[0];
String kafkaIp = "xxx.31.xxx.207"; //hiding the IPs here. This is the IP for my kafka-zk node. Is this ok?
String nimbusHost = "xxx.31.xxx.70";
String kafkaHost = kafkaIp + ":9092";
BrokerHosts kafkaHosts = new ZkHosts(kafkaHost);
SpoutConfig kafkaConfig = new SpoutConfig(kafkaHosts, topicName, "/kafka", topicName);
kafkaConfig.scheme = new SchemeAsMultiScheme(new StringScheme());
KafkaSpout kafkaSpout = new KafkaSpout(kafkaConfig);
TopologyBuilder builder = new TopologyBuilder();
builder.setSpout("twitter-spout", kafkaSpout, 8);
builder.setBolt("WordSplitterBolt", new JsonWordSplitterBolt(5)).shuffleGrouping("twitter-spout");
builder.setBolt("IgnoreWordsBolt", new IgnoreWordsBolt()).shuffleGrouping("WordSplitterBolt");
builder.setBolt("WordCounterBolt", new WordCounterBolt(5, 5 * 60, 50)).shuffleGrouping("IgnoreWordsBolt");
Config config = new Config();
config.setDebug(false);
config.setMaxTaskParallelism(5);
config.put(Config.NIMBUS_HOST, nimbusHost);
config.put(Config.NIMBUS_THRIFT_PORT, 6627);
config.put(Config.STORM_ZOOKEEPER_PORT, 2181);
config.put(Config.STORM_ZOOKEEPER_SERVERS, Arrays.asList(kafkaIp));
try {
config.setNumWorkers(20);
config.setMaxSpoutPending(5000);
StormSubmitter.submitTopology(topologyName, config, builder.createTopology());
} catch (Exception e) {
throw new IllegalStateException("Couldn't initialize the topology", e);
}
}
}
I am getting this exception in the Storm UI:
Unable to get offset lags for kafka. Reason: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /brokers/topics/twitter1/partitions at org.apache.zookeeper.KeeperException.create(KeeperException.java:99) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1590) at org.apache.curator.shaded.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:242) at org.apache.curator.shaded.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:231) at org.apache.curator.shaded.connection.StandardConnectionHandlingPolicy.callWithRetry(StandardConnectionHandlingPolicy.java:64) at org.apache.curator.shaded.RetryLoop.callWithRetry(RetryLoop.java:100) at org.apache.curator.shaded.framework.imps.GetChildrenBuilderImpl.pathInForeground(GetChildrenBuilderImpl.java:228) at org.apache.curator.shaded.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:219) at org.apache.curator.shaded.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:41) at org.apache.storm.kafka.monitor.KafkaOffsetLagUtil.getLeadersAndTopicPartitions(KafkaOffsetLagUtil.java:319) at org.apache.storm.kafka.monitor.KafkaOffsetLagUtil.getOffsetLags(KafkaOffsetLagUtil.java:256) at org.apache.storm.kafka.monitor.KafkaOffsetLagUtil.main(KafkaOffsetLagUtil.java:124)
The error Unable to get offset lags for kafka remains constant while the other part of the exception changes according to the zkroot path that I change (3rd argument in SpoutConfig). I don't know how exactly to fill these arguments up to have the Kafka pull in the tweets from my topic.
I used the tutorial present here to write the code for submitting the topology: http://stdatalabs.blogspot.ca/2016/10/real-time-stream-processing-using.html
I have made numerous changes for the maven dependencies. My pom.xml has all the dependencies for storm-core, kafka, etc. with the latest versions available in the maven repo.
The zkHosts() should contain the zookeeper's config instead of kafka. If your zookeeper and kafka are on the same server.
Try giving the correct port for zookeeper(2181)
Refer https://storm.apache.org/releases/1.2.3/storm-kafka.html
In Java, I'm dynamically creating a set of files and I'd like to change the file permissions on these files on a linux/unix file system. I'd like to be able to execute the Java equivalent of chmod. Is that possible Java 5? If so, how?
I know in Java 6 the File object has setReadable()/setWritable() methods. I also know I could make a system call to do this, but I'd like to avoid that if possible.
Full control over file attributes is available in Java 7, as part of the "new" New IO facility (NIO.2). For example, POSIX permissions can be set on an existing file with setPosixFilePermissions(), or atomically at file creation with methods like createFile() or newByteChannel().
You can create a set of permissions using EnumSet.of(), but the helper method PosixFilePermissions.fromString() will uses a conventional format that will be more readable to many developers. For APIs that accept a FileAttribute, you can wrap the set of permissions with with PosixFilePermissions.asFileAttribute().
Set<PosixFilePermission> ownerWritable = PosixFilePermissions.fromString("rw-r--r--");
FileAttribute<?> permissions = PosixFilePermissions.asFileAttribute(ownerWritable);
Files.createFile(path, permissions);
In earlier versions of Java, using native code of your own, or exec-ing command-line utilities are common approaches.
Prior to Java 6, there is no support of file permission update at Java level. You have to implement your own native method or call Runtime.exec() to execute OS level command such as chmod.
Starting from Java 6, you can useFile.setReadable()/File.setWritable()/File.setExecutable() to set file permissions. But it doesn't simulate the POSIX file system which allows to set permission for different users. File.setXXX() only allows to set permission for owner and everyone else.
Starting from Java 7, POSIX file permission is introduced. You can set file permissions like what you have done on *nix systems. The syntax is :
File file = new File("file4.txt");
file.createNewFile();
Set<PosixFilePermission> perms = new HashSet<>();
perms.add(PosixFilePermission.OWNER_READ);
perms.add(PosixFilePermission.OWNER_WRITE);
Files.setPosixFilePermissions(file.toPath(), perms);
This method can only be used on POSIX file system, this means you cannot call it on Windows system.
For details on file permission management, recommend you to read this post.
In addition to erickson's suggestions, there's also jna, which allows you to call native libraries without using jni. It's shockingly easy to use, and I've used it on a couple of projects with great success.
The only caveat is that it's slower than jni, so if you're doing this to a very large number of files that might be an issue for you.
(Editing to add example)
Here's a complete jna chmod example:
import com.sun.jna.Library;
import com.sun.jna.Native;
public class Main {
private static CLibrary libc = (CLibrary) Native.loadLibrary("c", CLibrary.class);
public static void main(String[] args) {
libc.chmod("/path/to/file", 0755);
}
}
interface CLibrary extends Library {
public int chmod(String path, int mode);
}
For Windows 7 with NIO 2:
public static void main(String[] args) throws IOException {
Path file = Paths.get("c:/touch.txt");
AclFileAttributeView aclAttr = Files.getFileAttributeView(file, AclFileAttributeView.class);
System.out.println(aclAttr.getOwner());
for (AclEntry aclEntry : aclAttr.getAcl()) {
System.out.println(aclEntry);
}
System.out.println();
UserPrincipalLookupService upls = file.getFileSystem().getUserPrincipalLookupService();
UserPrincipal user = upls.lookupPrincipalByName(System.getProperty("user.name"));
AclEntry.Builder builder = AclEntry.newBuilder();
builder.setPermissions( EnumSet.of(AclEntryPermission.READ_DATA, AclEntryPermission.EXECUTE,
AclEntryPermission.READ_ACL, AclEntryPermission.READ_ATTRIBUTES, AclEntryPermission.READ_NAMED_ATTRS,
AclEntryPermission.WRITE_ACL, AclEntryPermission.DELETE
));
builder.setPrincipal(user);
builder.setType(AclEntryType.ALLOW);
aclAttr.setAcl(Collections.singletonList(builder.build()));
}
Just to update this answer unless anyone comes across this later, since JDK 6 you can use
File file = new File('/directory/to/file');
file.setWritable(boolean);
file.setReadable(boolean);
file.setExecutable(boolean);
you can find the documentation on Oracle File(Java Platform SE 7). Bear in mind that these commands only work if the current working user has ownership or write access to that file. I am aware that OP wanted chmod type access for more intricate user configuration. these will set the option across the board for all users.
If you want to set 777 permission to your created file than you can use the following method:
public void setPermission(File file) throws IOException{
Set<PosixFilePermission> perms = new HashSet<>();
perms.add(PosixFilePermission.OWNER_READ);
perms.add(PosixFilePermission.OWNER_WRITE);
perms.add(PosixFilePermission.OWNER_EXECUTE);
perms.add(PosixFilePermission.OTHERS_READ);
perms.add(PosixFilePermission.OTHERS_WRITE);
perms.add(PosixFilePermission.OTHERS_EXECUTE);
perms.add(PosixFilePermission.GROUP_READ);
perms.add(PosixFilePermission.GROUP_WRITE);
perms.add(PosixFilePermission.GROUP_EXECUTE);
Files.setPosixFilePermissions(file.toPath(), perms);
}
You can use the methods of the File class:
http://docs.oracle.com/javase/7/docs/api/java/io/File.html
Apache ant chmod (not very elegant, adding it for completeness) credit shared with #msorsky
Chmod chmod = new Chmod();
chmod.setProject(new Project());
FileSet mySet = new FileSet();
mySet.setDir(new File("/my/path"));
mySet.setIncludes("**");
chmod.addFileset(mySet);
chmod.setPerm("+w");
chmod.setType(new FileDirBoth());
chmod.execute();
for Oralce Java 6:
private static int chmod(String filename, int mode) {
try {
Class<?> fspClass = Class.forName("java.util.prefs.FileSystemPreferences");
Method chmodMethod = fspClass.getDeclaredMethod("chmod", String.class, Integer.TYPE);
chmodMethod.setAccessible(true);
return (Integer)chmodMethod.invoke(null, filename, mode);
} catch (Throwable ex) {
return -1;
}
}
works under solaris/linux.
There is an example class on Oracle Docs which works very much similar to the UNIX chmod. It works with java se 7+ though.
import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.attribute.FileAttribute;
import java.nio.file.attribute.PosixFileAttributes;
import java.nio.file.attribute.PosixFilePermission;
import java.nio.file.attribute.PosixFilePermissions;
import java.util.Set;
public class FileAndDirectory1 {
public static void main(String[] args) {
File file = new File("fileTest1.txt");
System.out.println(file.getAbsoluteFile());
try {
//file.createNewFile();
if(!file.exists())
{
//PosixFilePermission is an enum class, PosixFilePermissions is a final class
//create file permissions from string
Set<PosixFilePermission> filePermissions = PosixFilePermissions.fromString("---------"/* "rwxrwxrwx" */);
FileAttribute<?> permissions = PosixFilePermissions.asFileAttribute(filePermissions);
Files.createFile(file.toPath(), permissions);
// printing the permissions associated with the file
System.out.println("Executable: " + file.canExecute());
System.out.println("Readable: " + file.canRead());
System.out.println("Writable: "+ file.canWrite());
file.setExecutable(true);
file.setReadable(true);
file.setWritable(true);
}
else
{
//modify permissions
//get the permission using file attributes
Set<PosixFilePermission> perms = Files.readAttributes(file.toPath(), PosixFileAttributes.class).permissions();
perms.remove(PosixFilePermission.OWNER_WRITE);
perms.add(PosixFilePermission.OWNER_READ);
perms.add(PosixFilePermission.OWNER_EXECUTE);
perms.add(PosixFilePermission.GROUP_WRITE);
perms.add(PosixFilePermission.GROUP_READ);
perms.add(PosixFilePermission.GROUP_EXECUTE);
perms.add(PosixFilePermission.OTHERS_WRITE);
perms.add(PosixFilePermission.OTHERS_READ);
perms.add(PosixFilePermission.OTHERS_EXECUTE);
Files.setPosixFilePermissions(file.toPath(), perms);
System.out.println("Executable: " + file.canExecute());
System.out.println("Readable: " + file.canRead());
System.out.println("Writable: "+ file.canWrite());
file.delete();
}
} catch (IOException e) {
e.printStackTrace();
}
Path path = Paths.get(String.valueOf(file));
System.out.println(path);
}
}
Permission 777 is the same as rwxrwxrwx which you can set as follows:
Files.setPosixFilePermissions(path, PosixFilePermissions.fromString("rwxrwxrwx"))
I ran a sample pig script in mapreduce mode and it ran successfully.
My pigscript:
allsales = load 'sales' as (name,price,country);
bigsales = filter allsales by price >999;
sortedbigsales = order bigsales by price desc;
store sortedbigsales into 'topsales';
Now, I am trying to implement that in eclipse (currently I am running using libraries).
One doubt: Pig Local mode means that we need hadoop installation as default?
IdLocal.java:
public class IdLocal {
public static void main(String[] args) {
try {
PigServer pigServer = new PigServer("local");
runIdQuery(pigServer, "/home/sreeveni/myfiles/pig/data/sales");
} catch (Exception e) {
}
}
public static void runIdQuery(PigServer pigServer, String inputFile)
throws IOException {
pigServer.registerQuery("allsales = load '" + inputFile+ "' as (name,price,country);");
pigServer.registerQuery("bigsales = filter allsales by price >999;");
pigServer.registerQuery("sortedbigsales = order bigsales by price desc;");
pigServer.store("sortedbigsales","/home/sreeveni/myfiles/OUT/topsalesjava");
}
}
The console is showing success for me, but my part file is empty.
Why is it so?
1) local mode pig does not mean that you have to have hadoop installed. you can run it without hadoop and hdfs. Everything will be performed single threaded on your machine and it should read/write from your local filesystem by default.
2) Regarding your empty output, ensure that your input file exists on your local filesystem and that it has records in the 'price' field greater than 999. You could be filtering them all out otherwise. Also, pig defaults to tab separated files. Is your inputFile tab separated? if not, then your schema definition will have the 'name' field hold the entire row in the file, and 'price' and 'country' will always be null.
hope that helps
In Java, I'm dynamically creating a set of files and I'd like to change the file permissions on these files on a linux/unix file system. I'd like to be able to execute the Java equivalent of chmod. Is that possible Java 5? If so, how?
I know in Java 6 the File object has setReadable()/setWritable() methods. I also know I could make a system call to do this, but I'd like to avoid that if possible.
Full control over file attributes is available in Java 7, as part of the "new" New IO facility (NIO.2). For example, POSIX permissions can be set on an existing file with setPosixFilePermissions(), or atomically at file creation with methods like createFile() or newByteChannel().
You can create a set of permissions using EnumSet.of(), but the helper method PosixFilePermissions.fromString() will uses a conventional format that will be more readable to many developers. For APIs that accept a FileAttribute, you can wrap the set of permissions with with PosixFilePermissions.asFileAttribute().
Set<PosixFilePermission> ownerWritable = PosixFilePermissions.fromString("rw-r--r--");
FileAttribute<?> permissions = PosixFilePermissions.asFileAttribute(ownerWritable);
Files.createFile(path, permissions);
In earlier versions of Java, using native code of your own, or exec-ing command-line utilities are common approaches.
Prior to Java 6, there is no support of file permission update at Java level. You have to implement your own native method or call Runtime.exec() to execute OS level command such as chmod.
Starting from Java 6, you can useFile.setReadable()/File.setWritable()/File.setExecutable() to set file permissions. But it doesn't simulate the POSIX file system which allows to set permission for different users. File.setXXX() only allows to set permission for owner and everyone else.
Starting from Java 7, POSIX file permission is introduced. You can set file permissions like what you have done on *nix systems. The syntax is :
File file = new File("file4.txt");
file.createNewFile();
Set<PosixFilePermission> perms = new HashSet<>();
perms.add(PosixFilePermission.OWNER_READ);
perms.add(PosixFilePermission.OWNER_WRITE);
Files.setPosixFilePermissions(file.toPath(), perms);
This method can only be used on POSIX file system, this means you cannot call it on Windows system.
For details on file permission management, recommend you to read this post.
In addition to erickson's suggestions, there's also jna, which allows you to call native libraries without using jni. It's shockingly easy to use, and I've used it on a couple of projects with great success.
The only caveat is that it's slower than jni, so if you're doing this to a very large number of files that might be an issue for you.
(Editing to add example)
Here's a complete jna chmod example:
import com.sun.jna.Library;
import com.sun.jna.Native;
public class Main {
private static CLibrary libc = (CLibrary) Native.loadLibrary("c", CLibrary.class);
public static void main(String[] args) {
libc.chmod("/path/to/file", 0755);
}
}
interface CLibrary extends Library {
public int chmod(String path, int mode);
}
For Windows 7 with NIO 2:
public static void main(String[] args) throws IOException {
Path file = Paths.get("c:/touch.txt");
AclFileAttributeView aclAttr = Files.getFileAttributeView(file, AclFileAttributeView.class);
System.out.println(aclAttr.getOwner());
for (AclEntry aclEntry : aclAttr.getAcl()) {
System.out.println(aclEntry);
}
System.out.println();
UserPrincipalLookupService upls = file.getFileSystem().getUserPrincipalLookupService();
UserPrincipal user = upls.lookupPrincipalByName(System.getProperty("user.name"));
AclEntry.Builder builder = AclEntry.newBuilder();
builder.setPermissions( EnumSet.of(AclEntryPermission.READ_DATA, AclEntryPermission.EXECUTE,
AclEntryPermission.READ_ACL, AclEntryPermission.READ_ATTRIBUTES, AclEntryPermission.READ_NAMED_ATTRS,
AclEntryPermission.WRITE_ACL, AclEntryPermission.DELETE
));
builder.setPrincipal(user);
builder.setType(AclEntryType.ALLOW);
aclAttr.setAcl(Collections.singletonList(builder.build()));
}
Just to update this answer unless anyone comes across this later, since JDK 6 you can use
File file = new File('/directory/to/file');
file.setWritable(boolean);
file.setReadable(boolean);
file.setExecutable(boolean);
you can find the documentation on Oracle File(Java Platform SE 7). Bear in mind that these commands only work if the current working user has ownership or write access to that file. I am aware that OP wanted chmod type access for more intricate user configuration. these will set the option across the board for all users.
If you want to set 777 permission to your created file than you can use the following method:
public void setPermission(File file) throws IOException{
Set<PosixFilePermission> perms = new HashSet<>();
perms.add(PosixFilePermission.OWNER_READ);
perms.add(PosixFilePermission.OWNER_WRITE);
perms.add(PosixFilePermission.OWNER_EXECUTE);
perms.add(PosixFilePermission.OTHERS_READ);
perms.add(PosixFilePermission.OTHERS_WRITE);
perms.add(PosixFilePermission.OTHERS_EXECUTE);
perms.add(PosixFilePermission.GROUP_READ);
perms.add(PosixFilePermission.GROUP_WRITE);
perms.add(PosixFilePermission.GROUP_EXECUTE);
Files.setPosixFilePermissions(file.toPath(), perms);
}
You can use the methods of the File class:
http://docs.oracle.com/javase/7/docs/api/java/io/File.html
Apache ant chmod (not very elegant, adding it for completeness) credit shared with #msorsky
Chmod chmod = new Chmod();
chmod.setProject(new Project());
FileSet mySet = new FileSet();
mySet.setDir(new File("/my/path"));
mySet.setIncludes("**");
chmod.addFileset(mySet);
chmod.setPerm("+w");
chmod.setType(new FileDirBoth());
chmod.execute();
import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.attribute.FileAttribute;
import java.nio.file.attribute.PosixFileAttributes;
import java.nio.file.attribute.PosixFilePermission;
import java.nio.file.attribute.PosixFilePermissions;
import java.util.Set;
public class FileAndDirectory1 {
public static void main(String[] args) {
File file = new File("fileTest1.txt");
System.out.println(file.getAbsoluteFile());
try {
//file.createNewFile();
if(!file.exists())
{
//PosixFilePermission is an enum class, PosixFilePermissions is a final class
//create file permissions from string
Set<PosixFilePermission> filePermissions = PosixFilePermissions.fromString("---------"/* "rwxrwxrwx" */);
FileAttribute<?> permissions = PosixFilePermissions.asFileAttribute(filePermissions);
Files.createFile(file.toPath(), permissions);
// printing the permissions associated with the file
System.out.println("Executable: " + file.canExecute());
System.out.println("Readable: " + file.canRead());
System.out.println("Writable: "+ file.canWrite());
file.setExecutable(true);
file.setReadable(true);
file.setWritable(true);
}
else
{
//modify permissions
//get the permission using file attributes
Set<PosixFilePermission> perms = Files.readAttributes(file.toPath(), PosixFileAttributes.class).permissions();
perms.remove(PosixFilePermission.OWNER_WRITE);
perms.add(PosixFilePermission.OWNER_READ);
perms.add(PosixFilePermission.OWNER_EXECUTE);
perms.add(PosixFilePermission.GROUP_WRITE);
perms.add(PosixFilePermission.GROUP_READ);
perms.add(PosixFilePermission.GROUP_EXECUTE);
perms.add(PosixFilePermission.OTHERS_WRITE);
perms.add(PosixFilePermission.OTHERS_READ);
perms.add(PosixFilePermission.OTHERS_EXECUTE);
Files.setPosixFilePermissions(file.toPath(), perms);
System.out.println("Executable: " + file.canExecute());
System.out.println("Readable: " + file.canRead());
System.out.println("Writable: "+ file.canWrite());
file.delete();
}
} catch (IOException e) {
e.printStackTrace();
}
Path path = Paths.get(String.valueOf(file));
System.out.println(path);
}
}
for Oralce Java 6:
private static int chmod(String filename, int mode) {
try {
Class<?> fspClass = Class.forName("java.util.prefs.FileSystemPreferences");
Method chmodMethod = fspClass.getDeclaredMethod("chmod", String.class, Integer.TYPE);
chmodMethod.setAccessible(true);
return (Integer)chmodMethod.invoke(null, filename, mode);
} catch (Throwable ex) {
return -1;
}
}
works under solaris/linux.
There is an example class on Oracle Docs which works very much similar to the UNIX chmod. It works with java se 7+ though.
Permission 777 is the same as rwxrwxrwx which you can set as follows:
Files.setPosixFilePermissions(path, PosixFilePermissions.fromString("rwxrwxrwx"))