I've been trying to configure a Java application (specifically my integration tests) with absolutely no success. I originally had a single XMLConfiguration which I could load and use without issue. My directory setup looks like:
src
test
java
mypackage
resources
Before, I had a file myconfig.xml in the resources directory. I was loading it it like so:
private static XMLConfiguration configuration = null;
public static void configure() {
try {
configuration = new XMLConfiguration("myconfig.xml");
}
catch (ConfigurationException e) {
System.err.println(e);
}
}
Now I'm trying to make it so that users can configure the application themselves by writing a configuration file in /etc or /home or whatever, so I made a new file in the same location called config-test.xml:
<?xml version="1.0" encoding="ISO-8859-1"?>
<configuration>
<!-- Attempt to load configuration provided on the system -->
<xml fileName="/etc/myconf/myconfig.xml" config-optional="true" />
<!-- Load default configuration from the classpath -->
<xml fileName="myconfig.xml" />
</configuration>
The file /etc/myconf/myconfig.xml does not exist, but that's the whole point of making it optional. I've tried a few different approaches to loading my configuration, including how the documentation does it: http://commons.apache.org/configuration/userguide/howto_configurationbuilder.html. What I have currently is the following:
private static CombinedConfiguration configuration = null;
public static void configure() {
try {
DefaultConfigurationBuilder builder =
new DefaultConfigurationBuilder("config-test.xml");
configuration = builder.getConfiguration(true);
System.err.println("Yay");
}
catch (ConfigurationException e) {
System.err.println("Herp");
}
catch (Exception e) {
System.err.println("Derp");
}
}
When I run my integration tests, I see no Yay, I see no Herp and I see no Derp. If I place a println before I try to load things, it does print so configure is being called. The output I get from commons configuration amounts to:
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running TestSuite
13:36:57.752 [main] DEBUG o.a.c.c.ConfigurationUtils - ConfigurationUtils.locate(): base is null, name is config-test.xml
13:36:57.753 [main] DEBUG o.a.c.c.DefaultFileSystem - Could not locate file config-test.xml at null: no protocol: config-test.xml
13:36:57.756 [main] DEBUG o.a.c.c.ConfigurationUtils - Loading configuration from the context classpath (config-test.xml)
13:36:57.766 [main] DEBUG o.a.c.c.DefaultConfigurationBuilder - Creating configuration null with name null
13:36:57.795 [main] DEBUG o.a.c.c.ConfigurationUtils - ConfigurationUtils.locate(): base is null, name is config-test.xml
13:36:57.795 [main] DEBUG o.a.c.c.DefaultFileSystem - Could not locate file config-test.xml at null: no protocol: config-test.xml
13:36:57.796 [main] DEBUG o.a.c.c.ConfigurationUtils - Loading configuration from the context classpath (config-test.xml)
13:36:57.800 [main] DEBUG o.a.c.c.DefaultConfigurationBuilder - Creating configuration null with name null
Tests run: 56, Failures: 2, Errors: 0, Skipped: 16, Time elapsed: 2.589 sec <<< FAILURE!
Results :
Failed tests: initialize(mypackage.ImportTestIT): org/apache/commons/beanutils/PropertyUtils
initialize(mypackage.TransferTestIT): org/apache/commons/beanutils/PropertyUtils
Tests run: 56, Failures: 2, Errors: 0, Skipped: 16
Those initialize methods are just calling configure and then trying to get data from the configuration that is loaded. I've been working with this for a couple hours now and could really use another pair of eyes. Any ideas? Is there anything I could have changed that I'm not mentioning that would affect this? Thanks.
Edit #1 (1:58pm): After making sure beanutils were on the classpath, I now get a brutal explosion of stack trace that starts with
Running TestSuite
14:00:16.088 [main] DEBUG o.a.c.c.ConfigurationUtils - ConfigurationUtils.locate(): base is null, name is config-test.xml
14:00:16.089 [main] DEBUG o.a.c.c.DefaultFileSystem - Could not locate file config-test.xml at null: no protocol: config-test.xml
14:00:16.093 [main] DEBUG o.a.c.c.ConfigurationUtils - Loading configuration from the context classpath (config-test.xml)
14:00:16.103 [main] DEBUG o.a.c.c.DefaultConfigurationBuilder - Creating configuration null with name null
14:00:16.178 [main] DEBUG o.a.c.c.ConfigurationUtils - ConfigurationUtils.locate(): base is file:///home/pgarrity/projects/myproject/target/test-classes/config-test.xml, name is /etc/myconf/myproject/myconfig.xml
14:00:16.179 [main] DEBUG o.a.c.c.DefaultFileSystem - Could not locate file /etc/myconf/myproject/myconfig.xml at file:///home/pgarrity/projects/myproject/target/test-classes/config-test.xml: /etc/myconf/myproject/myconfig.xml (No such file or directory)
14:00:16.181 [main] DEBUG o.a.c.c.DefaultConfigurationBuilder - Load failed for optional configuration xml: Cannot locate configuration source /etc/myconf/myproject/myconfig.xml
14:00:16.185 [main] WARN o.a.c.c.DefaultConfigurationBuilder - Internal error
org.apache.commons.configuration.ConfigurationException: Cannot locate configuration source /etc/myconf/myproject/myconfig.xml
at org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:259) ~[commons-configuration-1.8.jar:1.8]
at org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:238) ~[commons-configuration-1.8.jar:1.8]
And so on and so on... but I do see 'Yay' now. Now what I don't get is that it doesn't seem to be looking for my non-optional configuration source, or assuming it's in a location I never told it to look.
Edit #2:
I looked through my output a bit more and buried in the complaints from commons I found that it seemed to load the correct file eventually. I think I may have it working? Maybe my path to the properties change when I load it like this... I've got some stuff to try. Anyway, the problem that I asked about has been fixed. Thanks for the help.
You seem to be missing commons-beanutils on your classpath. Make sure that jar is there.
Pfft. A multi-page document talks about how to load a configuration file. But did they forget to implement detection of file URLs in absolute Windows paths such as c:\Users\USER\FOO\bar.xml? Commons 1.9 will show this scrupulous message,
DEBUG DefaultFileSystem - Could not locate file C:\Users\USER\FOO\bar.xml at null: unknown protocol: c
https://github.com/apache/commons-configuration/blob/CONFIGURATION_1_9/src/main/java/org/apache/commons/configuration/DefaultFileSystem.java#L281
Related
I have a dataflow running in NiFi 1.12.0, the relevant properties from this installation is here:
nifi.sensitive.props.key=
nifi.sensitive.props.key.protected=
nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSL
nifi.sensitive.props.provider=BC
nifi.sensitive.props.additional.keys=
I am facing migration issue in nifi when I upgrade base version to 1.16.3 from 1.12.0. which having following properties.
nifi.sensitive.props.key=testPassword
nifi.sensitive.props.key.protected=
nifi.sensitive.props.algorithm=NIFI_ARGON2_AES_GCM_256
nifi.sensitive.props.provider=BC
nifi.sensitive.props.additional.keys=
I am getting following exception while I execute command for migrating flow file.
nifi#nifi-service-0:/opt/nifi/nifi-toolkit-current/bin$ ./encrypt-config.sh -n $NIFI_HOME/conf/nifi.properties -f /opt/nifi/data/old_flow.xml.gz -s testPassword -x
[main] WARN org.apache.nifi.properties.ConfigEncryptionTool - The source nifi.properties and destination nifi.properties are identical [/opt/nifi/nifi-current/conf/nifi.properties] so the original will be overwritten
[main] WARN org.apache.nifi.properties.ConfigEncryptionTool - The source flow.xml.gz and destination flow.xml.gz are identical [/opt/nifi/data/old_flow.xml.gz] so the original will be overwritten
[main] WARN org.apache.nifi.properties.AbstractBootstrapPropertiesLoader - System Property [nifi.properties.file.path] not found: Using Relative Path [conf/nifi.properties]
[main] INFO org.apache.nifi.properties.NiFiPropertiesLoader - Loading Application Properties [/opt/nifi/nifi-current/conf/nifi.properties]
[main] INFO org.apache.nifi.properties.NiFiPropertiesLoader - Loading Application Properties [/opt/nifi/nifi-current/conf/nifi.properties]
[main] INFO org.apache.nifi.properties.ConfigEncryptionTool - Loaded NiFiProperties instance with 138 properties
[main] INFO org.apache.nifi.properties.NiFiPropertiesLoader - Loading Application Properties [/opt/nifi/nifi-current/conf/nifi.properties]
[main] INFO org.apache.nifi.properties.ConfigEncryptionTool - Migrating flow.xml file at /opt/nifi/data/old_flow.xml.gz. This could take a while if the flow XML is very large.
[main] ERROR org.apache.nifi.properties.ConfigEncryptionTool - Encountered an error: Decryption Failed with Algorithm [AES/GCM/NoPadding]
Encountered an error migrating flow content
usage: org.apache.nifi.properties.ConfigEncryptionTool [-h] [-v] [-n <file>] [-o <file>] [-l <file>] [-i <file>] [-a <file>] [-u <file>] [-f <file>] [-g <file>]
[-b <file>] [-S <protectionScheme>] [-k <keyhex>] [-e <keyhex>] [-H <protectionScheme>] [-p <password>] [-w <password>] [-r] [-m] [-x] [-s
<password|keyhex>] [-A <algorithm>] [-P <algorithm>] [-c]
This tool reads from a nifi.properties and/or login-identity-providers.xml file with plain sensitive configuration values, prompts the user for a root key, and
encrypts each value. It will replace the plain value with the protected value in the same file (or write to a new file if specified). It can also be used to
migrate already-encrypted values in those files or in flow.xml.gz to be encrypted with a new key.
Please help to solve this issue
Solved by myself.
Issue can be resolved by following steps
Before migration if you don't have nifi.sensitive.props.key set, set it using following command ${NIFI_TOOLKIT_PAT}/bin/encrypt-config.sh -f /opt/nifi/nifi-current/data/flow.xml.gz -p ${NIFI_HOME}/conf/nifi.properties -s <NEW_KEY_TO_SET> -x
Once key is set upgrade nifi. Since in newer version algorithm is changed set it using command ${NIFI_HOME}/bin/nifi.sh set-sensitive-properties-algorithm <NEW_ALGORITHM>
Once algorithm set, encrypt again using command ${NIFI_TOOLKIT_PAT}/bin/encrypt-config.sh -f /opt/nifi/nifi-current/data/flow.xml.gz -p ${NIFI_HOME}/conf/nifi.properties -s <NEW_KEY_TO_SET> -x
Now you will get all compatible files with respect your latest version
Posting this self-answered so when I break it again in a few months an answer will actually come up in google.
Simple Java project where Flyway API is used to migrate database schema. A resource directory of sql scripts following the V1__init.sql, V2__updateCustomerTable.sql convention is used to check against the schema_version metadata table and migrate if needed.
Everything works fine until the jar is proguarded. The sql scripts are definitely packed into the jar file, but they are not found:
2017-04-10 17:17:13,612 [main] DEBUG (?:?) - Validating migrations ...
2017-04-10 17:17:13,884 [main] DEBUG (?:?) - Scanning for classpath resources at 'classpath:db/migration/postgres' (Prefix: '', Suffix: '.sql')
2017-04-10 17:17:13,885 [main] DEBUG (?:?) - Determining location urls for classpath:db/migration/postgres using ClassLoader sun.misc.Launcher$AppClassLoader#8b819f ...
2017-04-10 17:17:13,885 [main] WARN (?:?) - Unable to resolve location classpath:db/migration/postgres
2017-04-10 17:17:13,899 [main] DEBUG (?:?) - Scanning for classpath resources at 'classpath:db/migration/postgres' (Prefix: 'V', Suffix: '.sql')
2017-04-10 17:17:13,900 [main] DEBUG (?:?) - Scanning for classpath resources at 'classpath:db/migration/postgres' (Prefix: 'R', Suffix: '.sql')
An un-proguarded jar finds them with the following log messages (note that it looks in the jar file now):
2017-04-10 17:07:16,612 [main] DEBUG (?:?) - Validating migrations ...
2017-04-10 17:07:16,613 [main] DEBUG (?:?) - Scanning for classpath resources at 'classpath:db/migration/postgres' (Prefix: 'V', Suffix: '.sql')
2017-04-10 17:07:16,614 [main] DEBUG (?:?) - Scanning URL: jar:file:/C:/Dev/Sanbox/myjar.jar!/db/migration/postgres
2017-04-10 17:07:16,614 [main] DEBUG (?:?) - JBoss VFS v2 available: false
2017-04-10 17:07:16,615 [main] DEBUG (?:?) - Filtering out resource: db/migration/postgres/ (filename: )
2017-04-10 17:07:16,615 [main] DEBUG (?:?) - Found resource: db/migration/postgres/V1__init.sql
2017-04-10 17:07:16,616 [main] DEBUG (?:?) - Found resource: db/migration/postgres/V2__updateCustomerTable.sql
The culprit was that you need to tell proguard to leave your db/migrations directory alone (or wherever you put your migration scripts) in your config file, else flyway will not be able to find your scripts even if they seem to be in the right place in the jar. This is the line I needed in my proguard config:
-keepdirectories db/migration/**
If you change where you put your sql scripts down the road, this config will also need to be updated.
I have a jar in server/lib/sample.jar.
sample.jar file consists .class files related to application
so how should access server/lib instated of application.war/web-inf/classes .
jboss-web.xml
<class-loading java2ClassLoadingCompliance="false">
<loader-repository>
com.example:loader=sample.jar
<loader-repository-config>java2ParentDelegation=false</loader-repository-config>
</loader-repository>
</class-loading >
keeping sample.jar in application.war/web-inf/lib got exception is
2013-05-16 10:28:58,388 INFO [STDOUT] [Mobee]- INFO 2013-05-16 10:28:58,388 [] LoadSystemDataCommand - ....................LoadSystemDataCommand.......................
2013-05-16 10:35:25,747 DEBUG [org.jboss.deployment.scanner.URLDeploymentScanner] Added url: file:/D:/Mobee Admin-ManamDB/.metadata/.plugins/org.jboss.ide.eclipse.as.core/JBoss_4.2_Runtime_1_Server1368615125350/deploy/
2013-05-16 10:36:02,472 DEBUG [org.jboss.resource.connectionmanager.IdleRemover] run: IdleRemover notifying pools, interval: 450000
2013-05-16 10:38:40,215 WARN [com.arjuna.ats.arjuna.logging.arjLoggerI18N] [com.arjuna.ats.arjuna.coordinator.BasicAction_58] - Abort of action id 7f000001:e258:51948a81:33 invoked while multiple threads active within it.
2013-05-16 10:38:40,215 WARN [com.arjuna.ats.arjuna.logging.arjLoggerI18N] [com.arjuna.ats.arjuna.coordinator.CheckedAction_2] - CheckedAction::check - atomic action 7f000001:e258:51948a81:33 aborting with 1 threads active!
2013-05-16 10:38:40,242 INFO [STDOUT] [Mobee]-ERROR 2013-05-16 10:38:40,242 [] SystemStartupObserver - Error While Loading System Propertiesorg.hibernate.PersistentObjectException: detached entity passed to persist: com.manam.mobee.persist.entity.MobeeVariable
Please refer below answer by skaffman
jboss-4-2-class-loading
Have a look at JBossClassLoadingUseCases. Your situation seems to be Use Case 4 (i.e. the class will be loaded from server/lib in preference to WEB-INF/classes.
What you seem to want is Use Case 3 (i.e. WEB-INF/classes preferred to server/lib), in which case you should omit the java2ParentDelegation=true config option from jboss-web.xml (or set it to false, which is the default).
ClassLoading Configuration
Other reference worth evaluating :
JBoss classloading problem
How do I tell JBoss 4.2.3.GA to load jars in my war file lib directory first?
JBoss classloading problem
I'm trying to use Spring. A dependency of Spring is
So I have added commons-logging-1.1.1.jar to my build path. This all works fine. And I get stuff like this:
Feb 28, 2013 2:40:39 PM org.springframework.context.support.AbstractApplicationContext prepareRefresh
INFO: Refreshing org.springframework.context.support.ClassPathXmlApplicationContext#c63a8af: startup date [Thu Feb 28 14:40:39 GMT 2013]; root of context hierarchy
This is all great but I don't want this printed to my standard out, I would prefer it to be sent to a file.
DirectoryStructure:
project
src
com.myproject.classes
myspring.xml
log4j.properties
test
com.myproject.classes
logs
myproject.log
So I would like to get all logs redirected from the standard out to the log file instead. Is this possible with Spring?
I've used the properties file from here: http://www.tutorialspoint.com/spring/logging_with_log4j.htm,
where I got my Spring example from. But this doesn't seem to make any difference. I've added some rubbish in there too to see if any exceptions get thrown, but nothing. I've tried moving the file around to different directories, but it still doesn't work.
Here it is:
# Define the root logger with appender file
log4j.rootLogger = DEBUG, FILE
# Define the file appender
log4j.appender.FILE=org.apache.log4j.FileAppender
# Set the name of the file
log4j.appender.FILE.File=logs/myproject.log
# Set the immediate flush to true (default)
log4j.appender.FILE.ImmediateFlush=true
# Set the threshold to debug mode
log4j.appender.FILE.Threshold=debug
# Set the append to false, overwrite
log4j.appender.FILE.Append=false
# Define the layout for file appender
log4j.appender.FILE.layout=org.apache.log4j.PatternLayout
log4j.appender.FILE.layout.conversionPattern=%m%n
Anyone got any ideas?
Thanks in advance.
Try this configuration : http://forum.springsource.org/showthread.php?94000-org-springframework-util-Log4jConfigurer-is-overriding-my-log4j-xml
Read the configuration class: http://static.springsource.org/spring/docs/1.1.x/api/org/springframework/util/Log4jConfigurer.html
For source code of Log Configuration - go through this :
http://git.springsource.org/sandbox/cbeams/blobs/bb96df1ed8e3720f33353824b108104027289e5c/org.springframework.core/src/main/java/org/springframework/util/Log4jConfigurer.java
I'm trying to debug some hibernate functionality in a spring app with a junit test and commons logging, but I can't seem to get anything other than the default INFO messages to appear. I'm also running these junit tests from Eclipse.
I've had no luck from the spring forums either.
I'm particularly interested in the debug logging output by Hibernate (to try and figure out why it takes 23 seconds to run this test).
Current output shows the default setting of INFO:
Mar 29, 2011 4:44:35 PM org.springframework.test.AbstractTransactionalSpringContextTests onSetUp
INFO: Began transaction: transaction manager [org.springframework.orm.hibernate3.HibernateTransactionManager#5f873eb2]; defaultRollback true
testGetSubjectsForSite time: [00:00:00:068]
Mar 29, 2011 4:44:58 PM org.springframework.test.AbstractTransactionalSpringContextTests endTransaction
INFO: Rolled back transaction after test execution
I've tried to add a commons-logging.properties file to the classpath (the same location as the hibernate.properties and test-components.xml) but still only the default INFO messages appear.
Here's the commons-logging.properties file:
org.apache.commons.logging.Log=org.apache.commons.logging.impl.Jdk14Logger
# handlers
handlers=java.util.logging.ConsoleHandler
# default log level
.level=FINE
org.springframework.level=FINE
org.hibernate.level=FINE
# level for the console logger
java.util.logging.ConsoleHandler.level=FINE
java.util.logging.ConsoleHandler.formatter=java.util.logging.SimpleFormatter
Is anyone able to shed any light on why I can't get the DEBUG messages to print out? Is there a logging setting I'm missing?
Edit: I've tried FINEST and DEBUG to no avail.
Unfortunately, it seems the logging configuration file used by Jdk14Logger should be specified at runtime.
See the following file in your JDK directory: JDK_HOME/jre/lib/logging.properties (it's the default one used if no config file is found)
Moreover, the file path should be absolute, otherwise it's relative to the folder where the JRE is executed - see the code of java.util.logging.LogManager.readConfiguration()
Also see:
http://www.javapractices.com/topic/TopicAction.do?Id=143
http://cyntech.wordpress.com/2009/01/09/how-to-use-commons-logging/
Your default and hibernate logging is at Level "FINE" which is more of a "INFO" in log4j terms.
You need set DEBUG level for org.hibernate which in JDK logging is equal to 'FINEST'
Set
org.hibernate.level=FINEST (in the above log which should enable debug logs)