Develop Reference

Isn't it possible to configure logging INSIDE the application when using jboss AS 7.1.1.Final?

I read a lot but I couldn't figure out how I could specify for example the log level for specific classes.
Only way I could figure out was in the standalone.xml but why should I configure some application specific setting very general in the server? This complicates the deployment process unnecessary.
Isn't it somehow possible to define the specific log level and the output files somewhere inside the war without touching the server?
Btw. it doesn't matter if log4j or commons-logging or slf4j or whatever is used.

Using a logging.properties file or a log4j configuration file in your deployment will work in JBoss EAP 6.x and WildFly (formerly JBoss AS). Note though that a log4j configuration would only work if you use log4j for your logging facade.
That said I agree with Marko that this should probably be done in the server configuration. I would also encourage you to use the CLI or web interface rather than editing the raw XML as well. You can get some more information on the documentation.

I am sorry for not providing a direct answer, but consider this: the application being in charge of logging levels is a bad idea most of the time as this is something an AS admin should be able to change at any time. For example, the point of the DEBUG or TRACE log levels is to be able to place a lot of such statements in the code without hurting the production server's performance. However, once a bug is detected, you want to be able to lower the logging level without rebuilding the application. This should be a purely administrative task.
On the other hand, I do recognize the need to at least have a decent starting point for the logging configuration and I don't know of any architecture which would allow the application to provide defaults which are overridable by the server configuration.

Purpose of storing variables in web.xml?

A lot of the advice on the web on storing variables which may change depending on the env/other conditions is to put them in web.xml, but isn't the web.xml within the war file? even if you find the exploded war and change it, wouldn't it get overriden if you update the war file? Or does the webcontainer provide any method to configure the web.xml without tinkering with the war file?

The web.xml variables are of very limited use, in my experience - the only advantage is that it's a standard location to look for hard-coded "configuration".
There are several common solutions to get a more sensible way to configure web apps, none of which is standard:
Use system properties (which usually involves fiddling around with startup scripts, and it can be hard to get a good overview of your entire config)
Use environment variables (same drawbacks as system properties)
Read a config file from a predefined location; often from the classpath by using getResourceAsStream (IIRC that usually means putting the config files in Tomcat's lib directory)
You can also use JNDI, which has the disadvantage of being rather heavy-weight both to set up and read (if you're using vanilla Java, anyways - Spring for example has rather good support for reading from JNDI). However, JNDI is rather good because it's per-application, and not a process-global setting. If you need to run several instances of the same app on the same server, JNDI is pretty much the only option (although you can use it to just point out a config file somewhere, which makes things easier to work with).
This may be relevant to your interests: How can I store Java EE configuration parameters outside of an EAR or WAR?

Advantages of specifying Parameter Values in web.xml
Using your own settings file requires additional coding and management.
Hard-coding parameter values directly into your application code makes them more difficult to change in the future, and more difficult to use different settings for different deployments (eg: JDBC settings, mail server address).
Other developers using your code will be able to find any relevant parameters more easily, as this is a standard location for such parameters to be set.
See also:
Advantages of specifying Parameter Values in web.xml
Web.xml.EnvEntry
Referencing Environment Variables in web.xml

As far as I know web.xml does not provide ability to store custom variables. Typical way to configure your web application is to store configuration in database, separate properties/xml/json/other file, get configuration from separate web service or provide it through environment variables.
Often a mixture of all these is used. For example you can add system variable using -D switch when running your container. This variable will contain path to file or URL where your configuration can be found.
You can supply parameters using OS environment.
You choice should depend on how many parameters do you have, what kind of application are you developing and how can you configure application server or computer OS. For example if you a hosting application on server you cannot configure these ways are not for you, so DB or web service are your only ways.

The folks that work on the Tomcat container recognize the irony that you have identified and have implemented a way to work-around the issue.
The solution that they implemented for the issues that you have alluded to is to create another xml file... the context.xml file, which is read by the server.
It appears that you can edit this file and have the new values read by the Tomcat without a restart... as long as you keep the elements out of the server.xml.
I do not use Tomcat so I might be mis-interpreting the docs
The GlassFish web container supports a similar feature, but does it via a couple admin cli command (asadmin):
set-web-env-entry
set-web-context-param
There is probably web admin console support and you can set them up by editing the domain.xml. It seems like it isn't as flexible as the Tomcat implementation... but it does make it really easy to use.
You need to disable and then enable your application for the changed values to 'take'. Do not redeploy you app, since that will delete the value that you just set.

Best practices for an application-wide logger

After a lot of attempts, I have am finally able to formulate a question. I hope it makes some sense and is easy to understand. Working on a web-application that uses Spring and Jersey. I am required to implement an application-wide logger that should log all the activities performed on the application into a database. For the time being I have used the HAS-A implementation and called the logging method everywhere a CRUD operation is performed. Something like this:
LogBean lBean=new LogBean("rickesh#email.com","update","address","127.0.0.1");
logToDatabase(lBean);
But this causes a lot of repeated lines of code and I have to repeatedly keep instantiating and calling the log method every section the CRUD operation in performed. Is there any way I can pull out the logging from the controller layer, the REST layer? Are there any specific functionality in Spring or Jersey with which I can perform logging on a separate layer and I don't have to keep repeating the same lines of code everywhere. Please advice.

If you are getting messy with writing your logs you should look into AOP. If you are already using spring you should read about Spring AOP, and for logging per se, read using Spring AOP for logging

AOP is a better way but if getting to learn it to resolve this issue is going to take longer then here is a simple alternative.
Add log4j JAR to your WebApp. Make sure, log4j version bundled in your WebApp doesn't conflict with the version already with your Application Server
Place a log4j.xml for FILE and CONSOLE appendars in your class path (src/main/resources)
There is a Log4JConfigureListener provided by Spring which you can declare under section in your web.xml
Additionally, if too much log is generated, you can limit the output of the loggers via either log4j.xml in your application to certain packages only or via using Root logger configuration for your project specific packages. For example, in JBoss you can add a "category" for a package at a specific LOG level
Your specific "separate layer" should have a separate and distinct package name to target the log appendars for that layer
Hope this helps!

LOG4J as a singleton, anyway to avoid log4j logging conflicts in web environment

We have a j2ee web environment. The server is configured to share session and possibly classloaders across multiple webapps. Basically, one classloader could server multiple web apps.
This seems to cause issues with log4j. Different webapps could have different log4j configurations but the logging will move to the same file.
Reading online, it looks like log4j uses singletons a lot, in terms of the appenders and other functionality.
Is there a way to completely separate the log4j configurations from one webapp from the other.
Server: websphere6+
Log4j: 1.4.2
Java: 1.5
Example log4j.properties (webapp1):
log4j.appender.R=org.apache.log4j.RollingFileAppender
log4j.appender.R.File=/usr/local/file1.log
log4j.additivity.com.app=false
Example log4j.properties (webapp2):
log4j.appender.Z=org.apache.log4j.RollingFileAppender
log4j.appender.Z.File=/usr/local/file2.log
log4j.additivity.com.app=false
Right now, logging from webapp2 may appear in the webapp1 logs and vice verse. We don't want that.
Possible Solution:
It might be possible to add a custom file appender? Would that fix the issue and what code would I add to the custom appender?
Is it possible to change the log4j initialization. E.g., could I use some startup servlet to load logj4 for each webapp.

You have two ways to solve your problem:
Configure your app. server so it doesn't share classloaders across multiple webapps. When it does do that, because of log4j's nature, only one log4j.properties file will be loaded.
If you leave the app. server so it shares classloaders, then use one "master" log4j.properties file. In it, define appenders for root of every one of your applications (example com.mycompany.webapp1, com.mycompany.webapp2)
No solution is perfect. Particularly, the second one will be problematic if your web apps share some classes that use log4j. In that case, logs from both apps will end up in the same file.
Your problem is a common one. To understand more about this topic, google for "log4j and j2ee".
Edit: since solutions 1 and 2 aren't feasible, you could try something else:
Use log4j.properties file per application. In every one of them, define an appender for their root (as explained in solution 2), and set additivity to false. This also won't be perfect if there is any class sharing between them.
Configure log4j programatically for every application. Since they're legacy applications, this could be tough. One way to do it is to use ServletContextListener for every application and configure log4j on application startup. I haven't personally tried this, so I'm not 100% sure if there will be clashes due to shared classloaders.
Log4j won't be really updated anymore. Ceki Gülcü, who created log4j, stated that he will focus his efforts on slf4j and logback, to correct some mistakes he did during development of log4j.

log4j.xml in client jars

I have some jar files that will be distributed to clients that are using log4j for logging. My question is should I include a log4j.xml configuration in the jar file or have the client provide one if they want logging?
My feeling is to leave the log4j.xml configuration file out of the client jars, since the apache jar files all come with log4j logging, but sans log4j.xml.

Yes, leave it out. It's an utter nuisance when your log4j configuration file is ignored because one of the 60 third-party libraries of your app contains its own.

The good thing about log4j in your case is that your jar really shouldn't have to worry about it. The basic use case of log4j is:
Obtain a logger object for the current class
Call one of the methods on that logger, such as debug("some message");
If the jars you are shipping are to be used by a larger application, then ideally your code will only do the two steps listed above. In this way, your code will simply obtain logger objects from the already-configured log4j instance in the client's application. Your production code is then decoupled from having to know how to configure log4j.
Any logging you need to see for your development of the jars can be accomplished by configuring a log4j instance in unit test setUp() methods or something similar that won't get bundled with the production code going to the client.

I would put a default log4j configuration that you expect will be useful to your clients in the documentation. This way interested people can see what logging options you have (usually certain classes have more interesting log messages, from the user's perspective). I find it annoying when I have a third-party lib using log4j and it has no documentation, and there are log messages filling my screen and I have to try to figure out how to enable or suppress certain log messages.

If you are using log4j in your application then you include it in your project. If you are not, then why would you put it in there? What if client A wants log4j version 1.2 and client B wants log4j version 1.3.
Let them decide what they need for their projects and worry about what you need for yours.

I would add the configuration xml and load it up with instruction for the user showing different configuration and options. This will make it easier for either them or support to enable addition logging.