SVN server is accessible over https. So I need to read a file that is located there. I followed the snippet from svnkit wiki (http://svn.svnkit.com/repos/svnkit/tags/1.3.5/doc/examples/src/org/tmatesoft/svn/examples/repository/DisplayFile.java), but my SVNKindNode is NONE and as a result no file is read. Nevertheless there's no exceptions during connection. So I can assume that I do connect correctly to SVN server, but then something goes wrong.
Here is the code:
public class SVNRepoConnector {
private String username = "user";
private String password = "pwd";
private String baseUrl = "https://mysvnserver.com/svn/project/trunk";
private String filePath = "/myproject/src/main/webapp/file.html";
public void downloadSchema() {
DAVRepositoryFactory.setup();
SVNRepository repository = null;
try {
repository = SVNRepositoryFactory.create(SVNURL.parseURIEncoded(baseUrl));
ISVNAuthenticationManager authManager = SVNWCUtil.createDefaultAuthenticationManager(username, password);
repository.setAuthenticationManager(authManager);
SVNNodeKind nodeKind = repository.checkPath(filePath, -1);
if(nodeKind == SVNNodeKind.NONE) {
System.err.println("There is file at: " + baseUrl + filePath);
System.exit(1);
} else if (nodeKind == SVNNodeKind.DIR) {
System.err.println("The entry at " + baseUrl + filePath + " is a directory while a file was expected.");
System.exit(1);
}
SVNProperties properties = new SVNProperties();
ByteArrayOutputStream out = new ByteArrayOutputStream();
repository.getFile(filePath, -1, properties, out);
System.out.println("Content:\n");
try {
out.writeTo(System.out);
} catch (IOException e) {
e.printStackTrace();
}
} catch (SVNException e) {
e.printStackTrace();
}
}
public static void main(String[] args) {
SVNRepoConnector connector = new SVNRepoConnector();
connector.downloadSchema();
}
}
I receive "There is file at..." due to SVNNodeKind equals NONE. I cannot understand what is wrong here. How to read file from SVN over https?
Btw, my svnkit is 1.8.5.
Specify a relative path (unless baseUrl is the repository root):
private String filePath = "myproject/src/main/webapp/file.html";
instead of
private String filePath = "/myproject/src/main/webapp/file.html";
I found the solution after thorough debugging of the sources.
In short, the problem is in the second argument of repository.checkPath(filePath, -1); and repository.getFile(filePath, -1, properties, out);. filePath must be file name and path to it must be in the baseUrl field. After these changes everything started working correctly.
Regarding the snippet, in case of www/license.html, one should pass 1 as a second arg.
Related
I've been trying to read a properties file and want it to be dynamic, I'm doing this in aws-cdk.
My project layout:
Main Project
resources
config.properties
src
main/java/com/myorg
xxxstage.java
The class xxxstage.java has following code:
public class xxxstage extends Stage {
public xxxstage(final Construct scope, final String id) {
this(scope, id, null);
}
public xxxstage(final Construct scope, final String id, final StageProps props) {
super(scope, id, props);
String account = null;
InputStream inputStream = null;
try {
Properties prop = new Properties();
String propFileName = "resources/config.properties";
inputStream = this.getClass().getClassLoader().getResourceAsStream(propFileName);
System.out.println("inputStream is -> "+inputStream);
if (inputStream != null) {
prop.load(inputStream);
} else {
throw new FileNotFoundException("property file '" + propFileName + "' not found in the classpath");
}
// get the property value and print it out
account = prop.getProperty("account.id");
System.out.println("account id -> "+account);
} catch (Exception e) {
System.out.println("Exception: " + e);
} finally {
try{
inputStream.close();
}
catch (Exception e){
System.out.println("Exception: " + e);
}
}
new xxxStack(this, "xxxStack", StackProps.builder()
.env(new Environment.Builder()
.account(account)
.region("us-east-1")
.build())
.build());
}
}
The line where I'm trying to print System.out.println("inputStream is -> "+inputStream); is showing null, and hence the FileNotFoundException.
It worth noting that this is working fine when I run it on local java-project, its however failing during build phase of the aws-codepipeline.
In the pipeline build phase I'm getting:
inputStream is -> null
Exception: java.io.FileNotFoundException: property file 'resources/config.properties' not found in the classpath
Exception: java.lang.NullPointerException
Can someone please help?
EDIT 1 - Adding environment variables:
Environment Vars
PATH=/root/.npm/_npx/230/bin:/usr/local/bin/sbt/bin:/root/.phpenv/shims:/root/.phpenv/bin:/root/.goenv/shims:/root/.goenv/bin:/go/bin:/root/.phpenv/shims:/root/.phpenv/bin:/root/.pyenv/shims:/root/.pyenv/bin:/root/.rbenv/shims:/usr/local/rbenv/bin:/usr/local/rbenv/shims:/root/.dotnet/:/root/.dotnet/tools/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/tools:/codebuild/user/bin
AWS_CONTAINER_CREDENTIALS_RELATIVE_URI=/v2/credentials/xxx-xxx-xxx
CODEBUILD_CI=true
CODEBUILD_AUTH_TOKEN=xxx-xxx-xxx
JAVA_8_HOME=/usr/lib/jvm/java-1.8.0-amazon-corretto
JDK_8_HOME=/usr/lib/jvm/java-1.8.0-amazon-corretto
CODEBUILD_BUILD_ARN=arn:aws:codebuild:us-east-1:821518525729:build/pipelinePipelinexxx-C-xxx-xxx-xxx
CODEBUILD_GOPATH=/codebuild/output/src123245
GOLANG_15_VERSION=1.15.12
CODEBUILD_BUILD_SUCCEEDING=1
GOENV_DISABLE_GOPATH=1
JRE_HOME=/usr/lib/jvm/java-11-amazon-corretto
CDK_DEFAULT_REGION=us-east-1
JAVA_11_HOME=/usr/lib/jvm/java-11-amazon-corretto
PHP_74_VERSION=7.4.13
CODEBUILD_SOURCE_VERSION=arn:aws:s3:::pipelinexxxstack-pipelinexxxartifacts-1cacuj92rramf/ServiceDeploymentPip/(user)/c1BGqPX
RUBY_BUILD_SRC_DIR=/usr/local/rbenv/plugins/ruby-build
JDK_HOME=/usr/lib/jvm/java-11-amazon-corretto
PWD=/codebuild/output/src12345/src
CODEBUILD_CONTAINER_NAME=default
PYTHON_37_VERSION=3.7.10
CODEBUILD_START_TIME=1642556874326
CDK_DEFAULT_ACCOUNT=12345
AWS_REGION=us-east-1
PYTHON_38_VERSION=3.8.10
CODEBUILD_BUILD_URL=https://us-east-1.console.aws.amazon.com/codebuild/home?region=us-east-1#/builds/pipelinePipelinexxxSynthC-ofdfmXGrWl5m:xxx-xxx-xxx/view/new
CDK_OUTDIR=cdk.out
DOTNET_31_SDK_VERSION=3.1.404
CODEBUILD_BUILD_ID=pipelinePipelinexxxSynthC-ofdfmXGrWl5m:xxx-xxx-xxx
GOPATH=/go:/codebuild/output/src12345
CODEBUILD_RESOLVED_SOURCE_VERSION=xxx-xxx-xxx
OLDPWD=/codebuild/output/src12345/src
RUBY_26_VERSION=2.6.6
AWS_STS_REGIONAL_ENDPOINTS=regional
DOTNET_ROOT=/root/.dotnet
_PROJECT_CONFIG_HASH=xxx-xxx-xxx
CODEBUILD_AGENT_ENDPOINT=http://127.0.0.1:port
LC_CTYPE=C.UTF-8
JRE_8_HOME=/usr/lib/jvm/java-1.8.0-amazon-corretto/jre
CODEBUILD_BUILD_IMAGE=aws/codebuild/standard:5.0
PYYAML_VERSION=5.4.1
FORCE_COLOR=0
CODEBUILD_BMR_URL=https://CODEBUILD_AGENT:port
JAVA_HOME=/usr/lib/jvm/java-11-amazon-corretto
CODEBUILD_SRC_DIR=/codebuild/output/src12345/src
AWS_DEFAULT_REGION=us-east-1
AWS_EXECUTION_ENV=AWS_ECS_EC2
ECS_CONTAINER_METADATA_URI=http://169.254.170.2/v3/xxx-xxx-xxx
ECS_CONTAINER_METADATA_URI_V4=http://169.254.170.2/v4/xxx-xxx-xxx
CODEBUILD_INITIATOR=codepipeline/ServicexxxPipeline
MAVEN_OPTS= -Dmaven.wagon.httpconnectionManager.maxPerRoute=2
CDK_CONTEXT_JSON={"#aws-cdk/aws-apigateway:usagePlanKeyOrderInsensitiveId":true,"#aws-cdk/core:enableStackNameDuplicates":true,"aws-cdk:enableDiffNoFail":true,"#aws-cdk/core:stackRelativeExports":true,"#aws-cdk/aws-ecr-assets:dockerIgnoreSupport":true,"#aws-cdk/aws-secretsmanager:parseOwnedSecretName":true,"#aws-cdk/aws-kms:defaultKeyPolicies":true,"#aws-cdk/aws-s3:grantWriteWithoutAcl":true,"#aws-cdk/aws-ecs-patterns:removeDefaultDesiredCount":true,"#aws-cdk/aws-rds:lowercaseDbIdentifier":true,"#aws-cdk/aws-efs:defaultEncryptionAtRest":true,"#aws-cdk/aws-lambda:recognizeVersionProps":true,"#aws-cdk/aws-cloudfront:defaultSecurityPolicyTLSv1.2_2021":true,"#aws-cdk/core:newStyleStackSynthesis":true,"aws:cdk:enable-path-metadata":true,"aws:cdk:enable-asset-metadata":true,"aws:cdk:version-reporting":true,"aws:cdk:bundling-stacks":[]}
CODEBUILD_LOG_PATH=xxx-xxx-xxx
CODEBUILD_EXECUTION_ROLE_BUILD=
CODEBUILD_BUILD_NUMBER=31
GOLANG_16_VERSION=1.16.4
PHP_73_VERSION=7.3.25
CODEBUILD_FE_REPORT_ENDPOINT=https://codebuild.us-east-1.amazonaws.com/
CODEBUILD_LAST_EXIT=0
AWS_NODEJS_CONNECTION_REUSE_ENABLED=1
MAVEN_CMD_LINE_ARGS= -e -q compile exec:java
NUGET_XMLDOC_MODE=skip
DOTNET_5_SDK_VERSION=5.0.202
NODE_12_VERSION=12.22.2
PYTHON_39_VERSION=3.9.5
CDK_CLI_VERSION=2.8.0
NODE_14_VERSION=14.17.2
MAVEN_PROJECTBASEDIR=/codebuild/output/src12345/src
CDK_CLI_ASM_VERSION=16.0.0
JRE_11_HOME=/usr/lib/jvm/java-11-amazon-corretto
RUBY_27_VERSION=2.7.2
HOSTNAME=12345
JDK_11_HOME=/usr/lib/jvm/java-11-amazon-corretto
CODEBUILD_PROJECT_UUID=xxx-xxx-xxx
PHP_80_VERSION=8.0.0
CODEBUILD_KMS_KEY_ID=arn:aws:kms:us-east-1:(account-num):alias/aws/s3
HOME=/root
I found an answer here and it worked for me. The location of properties file matters.
As I am very new to Apache-Camel, I stuck with a use case.
I want to achieve moving of all files to a specific directory without sub-directories in target using camel route,
for example-
SourceDirectory/file1.xml
SourceDirectory/subDir1/file2.xml
SourceDirectory/subDir2/file3.xml
SourceDirectory/subDir3/subDir4/file4.xml
should be moved to a destination Directory
destDir/file1.xml
destDir/file2.xml
destDir/file3.xml
destDir/file4.xml
The code below copies file including all sub-directories to destination
String src="ftp://username:password#host/srcDir/";
String destDir="ftp://username:password#host/destDir/";
fromUri = src+"?recursive=true&delete=true";
from(fromUri)
.to(destDir);
To achieve this currently I am using ftp client
private void moveOverFTP(String from, String to) {
FTPClient ftpClient = new FTPClient();
try {
URL url = new URL(from);
String[] info = url.getUserInfo().split(":");
ftpClient.connect(url.getHost());
ftpClient.login(info[0], info[1]);
String srcFolderPath = url.getPath();
String targetFolder = new URL(to).getPath();
move(srcFolderPath, targetFolder, ftpClient);
ftpClient.logout();
} catch (IOException e) {
logger.error(e.getMessage(), e);
}
}
private void move(String srcFolderPath, String targetFolder, FTPClient ftpClient) throws IOException {
FTPFile[] files = ftpClient.listFiles(srcFolderPath);
for (FTPFile file : files) {
String fileName = file.getName();
if (file.isDirectory()) {
String tempSrcPath = srcFolderPath + fileName + "/";
move(tempSrcPath, targetFolder, ftpClient);
// delete empty directory
ftpClient.removeDirectory(tempSrcPath);
} else {
System.out.println("Moving "+srcFolderPath + fileName +" to = "+ targetFolder);
ftpClient.rename(srcFolderPath + fileName, targetFolder + fileName);
}
}
}
Any help to achieve this in route itself would be appreciated!
Thank you in advance!
Sounds like what you are looking for is the flatten option, ie.:
from(fromUri)
.to(destDir + "?flatten=true");
I have simple java console application, where I have used Jasypt1.9.2 for encryption purpose of the database connection string. The code runs perfectly fine in Eclipse,but application exported as Jar give this error
This is my simple Main class:
public static void main (String args[])
{
try {
GetConnection config = new GetConnection();
out = config.getLogFile();
System.out.println("Start");
Interface.writeLine(out, "Start");
while (true) {
new Interface().ProcessProcedure();
new Interface().ProcessKTP();
new Interface().TransferToPRF();
System.out.println("End");
Thread.sleep(90000);
}
}
catch (java.lang.Exception ex) {
System.out.println("Error:"+ex);
}
}
And this my GetConnection class
public Connection getSql(BufferedWriter out){
try{
StandardPBEStringEncryptor encryptor = new StandardPBEStringEncryptor();
prop = new EncryptableProperties(encryptor); //--fOR ENCRYPTED PASSWORD
String fileName = System.getProperty("user.dir") + "\\configuration.properties";
InputStream is = new FileInputStream(fileName);
//InputStream inputStream;
if (is != null) {
prop.load(is);
} else {
throw new FileNotFoundException("Configuration property not found");
}
String Driver = prop.getProperty("sqlserver.driver");
String url = prop.getProperty("sqlserver.durl");
String id = prop.getProperty("sqlserver.uid");
key = "123"
encryptor.setPassword(key);
String sqlpass = prop.getProperty("sqlserver.pass").trim();
Class.forName(Driver).newInstance();
Connection conMSSQL = DriverManager.getConnection (url,id,sqlpass);
if(conMSSQL != null && conMSSQL.isValid(0)) {
GetConnection.writeLine(out,"Connection established");
}
return conMSSQL;
}
catch( Exception ex ){
GetConnection.writeLine(out,"Exception at Kondor connection");
GetConnection.writeLine(out,ex.getMessage());
return null;
}
}
I have check and verified the classpath for the Jasypt1.9.2.jar entry.
Please help in this.
NoClassDefFoundError means that the JVM tried to access a class but didn't find it.
You've exported your project to a JAR file without exporting the dependencies with it.
Select the package or repack option while exporting it from eclipse.
This adds the dependencies to the exported JAR file.
I've a Java program that is copying a file from Unix to hdfs. It is running fine however I am looking for impersonating a different account when it runs and copies file.
Input: Apart form input file and target hdfs directory path, another input should be properties file containing account, keytab directory, domain
Please kindly let me know the best way to move forward.
I am currently exploring using a shell to first issue a kinit command and then run the jar
I am also reading about Jaas and how this can be done in Java itself - from - https://henning.kropponline.de/2016/02/14/a-secure-hdfs-client-example/
Need inputs and any reference of available options.
My Java program that copies file is as below:
public class FileCopy implements Runnable {
#Option(names = {"-i","--input"}, required=true, description="file name to copy to hadoop")
String input;
#Option(names = {"-o","--output"}, required=true, description="hdfs directory path to be copied into")
String output;
public void run() {
Properties hadoop_properties = new Properties();
HdfsFileDeploy hdfsFileDeploy = new HdfsFileDeploy();
try {
hadoop_properties.load(FileCopy.class.getClassLoader().getResourceAsStream("hadoop.properties"));
} catch (IOException e) {
e.printStackTrace();
}
FileSystem fs = hdfsFileDeploy.configureFilesystem(hadoop_properties.getProperty("coreSitePath"),hadoop_properties.getProperty("hdfsSitePath"));
String status = hdfsFileDeploy.writeToHDFS(fs,input,output);
if (status == "SUCCESS") {
System.out.println("completed copying");
} else {
System.out.println("copying error");
}
hdfsFileDeploy.closeFileSystem(fs);
}
public static void main(String[] args) throws IOException {
CommandLine.run(new FileCopy(), args);
}
}
public class HdfsFileDeploy {
public FileSystem configureFilesystem(String coreSitePath, String hdfsSitePath) {
FileSystem fileSystem = null;
try {
Configuration conf = new Configuration();
Path hdfsCoreSitePath = new Path(coreSitePath);
Path hdfsHDFSSitePath = new Path(hdfsSitePath);
conf.addResource(hdfsCoreSitePath);
conf.addResource(hdfsHDFSSitePath);
fileSystem = FileSystem.get(conf);
System.out.println(fileSystem);
return fileSystem;
} catch (Exception ex) {
ex.printStackTrace();
return fileSystem;
}
}
public void closeFileSystem(FileSystem fileSystem) {
try {
fileSystem.close();
} catch (Exception ex) {
System.out.println("Unable to close Hadoop filesystem : " + ex);
}
}
//
public String writeToHDFS(FileSystem fileSystem, String sourcePath, String destinationPath) {
String failure = "FAILURE";
String success = "SUCCESS";
Boolean doNotDelSrc = false;
Boolean overwrite = true;
try {
Path inputPath = new Path(sourcePath);
Path outputPath = new Path(destinationPath);
if(!fileSystem.exists(outputPath)) {
System.out.println("Output path " + outputPath + " does not exist. Creating outputPath directory now..");
if (fileSystem.mkdirs(outputPath)) {
System.out.println("Output path " + outputPath + " created...");
}
}
System.out.println("about to copy from " + inputPath + " to " + outputPath);
fileSystem.copyFromLocalFile(doNotDelSrc, overwrite, inputPath, outputPath);
return success;
} catch (IOException ex) {
System.out.println("Some exception occurred while writing file to hdfs");
ex.printStackTrace();
return failure;
}
}
}
Input1: input file
Input2: target hdfs directory
Reference Input: file (say yaml) containing account, domain, keytab path.
jar should impersonate and copy the input file to target hdfs directory.
I am using a Windows machine and Java. I'm just trying to backup a file, but I ran into an issue with an illegal character in the path ("#"). I really tried and I'm stuck. I rewrote it trying all the variations I could find or think of. Any help would be greatly appreciated.
public class SyncActionMachine {
/**
* #param args the command line arguments
*/
public static void main(String[] args) throws IOException, URISyntaxException {
String MSI_one, MSI_two, dropBox;
GetDate getDate = new GetDate();
MSI_one = "C:\\Users\\Brian\\AppData\\Roaming\\Macromedia\\Flash Player\\#SharedObjects\\Q2965ZS7\\localhost\\ActionMachine.sol";
MSI_two = "C:\\Users\\Brian\\Desktop\\test.txt";
dropBox = "C:\\Users\\Brian\\Dropbox\\Action Machine History\\ActionMachine.sol";
File source = new File(MSI_one);
File destination = new File(dropBox);
// Attempt #1 using string with special characters
try {
Files.copy(source.toPath(), destination.toPath());
} catch (IOException iOException) {
System.out.println("Didn't work: " + iOException);
}
// Attempt #2 using URI - not really sure how to use it.
URI uri;
uri = new URI("file:///C:/Users/Brian/AppDate/Roaming/Macromedia/Flash%20Player/%23SharedObjects/Q2965ZS7/localhost/ActionMachine.sol");
Path uriSelfMadePath = Paths.get(uri);
try {
Files.copy(uriSelfMadePath, destination.toPath());
} catch (IOException iOException) {
System.out.println("Didn't work: " + iOException);
}
// Attempt #3 Suggestion from Aurasphere. Thanks again for quick response.
// Not sure what I'm suppose to do with the URL
String thePath = MSI_one;
thePath = URLEncoder.encode(thePath, "UTF-8");
Path aurasphereThePath = Paths.get(thePath);
try {
Files.copy(aurasphereThePath, destination.toPath());
} catch (IOException iOException) {
System.out.println("Didn't work: " + iOException);
}
// Attempt #4 build path using Patha and passing in augruments separately
Path pathOneByOne = Paths.get("C:", "Users", "Brian", "AppDate", "Roaming", "Macromedia", "Flash Player",
"#SharedObjects", "Q2965ZS7", "localhost", "ActionMachine.sol");
try {
Files.copy(pathOneByOne, destination.toPath());
} catch (IOException iOException) {
System.out.println("Didn't work: " + iOException);
}
// Seeing what all these path's look like
URL fileUrl = source.toURI().toURL();
URI fileUri = source.toURI();
System.out.println("------------Path Print out------------------");
System.out.println("URLEncoder : " + thePath);
Path from = Paths.get(fileUri);
System.out.println("URL : " + fileUrl);
System.out.println("URI : " + fileUri);
System.out.println("source: " + source);
}
}
Thanks for any advice.
Just use URLEncode:
String thePath = "your_path";
thePath = URLEncoder.encode(thePath, "UTF-8");
Thank you everyone that looked and commented. Must have been some sleep derived moment. Anyway here is the source, it worked fine. Turned out # was a big deal, I'm not even sure what my hang up was.
public static void main(String[] args) throws IOException, URISyntaxException {
String MSI_one, MSI_two, dropBox;
GetDate getDate = new GetDate();
MSI_one = "C:\\Users\\Brian\\AppData\\Roaming\\Macromedia\\Flash Player\\#SharedObjects\\Q2965ZS7\\localhost\\ActionMachine.sol";
MSI_two = "C:\\Users\\brian\\AppData\\Roaming\\Macromedia\\Flash Player\\#SharedObjects\\HSTARDTM\\localhost\\ActionMachine.sol";
dropBox = "C:\\Users\\brian\\Dropbox\\Action Machine History\\";
// Create new file name for backup file
dropBox = dropBox + "ActionMachine-" + getDate.today() + ".sol";
File source = new File(MSI_two);
File destination = new File(dropBox);
copyNewFile cf = new copyNewFile(source, destination);
}
public class copyNewFile {
public copyNewFile(File source, File dest) throws IOException {
CopyOption[] options = new CopyOption[]{
StandardCopyOption.REPLACE_EXISTING,
StandardCopyOption.COPY_ATTRIBUTES
};
Files.copy(source.toPath(), dest.toPath(), options);
System.out.println("File sucessfully copied.");
}
}