I believe I am missing some configuration either on load balancing side or in tomcat, it would great if someone could help and provide the solution.
What am i trying to do?
Setup two tomcat instances with P2P Hazelcast session replication for tomcat6 by referring to this guide
I have placed the hazelcast-all-.jar, hazelcast-tomcat-sessionmanager-.jar and hazelcast.xml in the folder $CATALINA_HOME/lib/
Updated the listener and the context configs as shown below
Updated Tomcat Failover and the jvmRoute Parameter as shown below
So far, i was able to bring up the tomcat instances which connect to each others hazelcast instances, I see that in the tomcat logs.
The users connects to the load balancer and are routed to the balancer members - no issues here.
But when I take one of the tomcat instance down to test failover and to make sure users who were connected to tom instance were able to continue to cat instance with the same session and without having to sign in again. But I believe the load balancer seems to be creating a new session and forces user to sign in again to establish a session with cat instance. I suspect it this but not sure.
Or are the sessions not being replicated and synced in both hazelcast instances? - This is happening
Do we have a client where I could monitor hazelcast sessions? - Now I do
I did refer to this issue which refers setting up hazelcast.sessionId instead of JSESSIONID but does not provide any specifics about how to.
Versions -
Apache Haus - 2.2.32 (for load balancing)
Hazelcast - 3.8.6
Java 8
Tomcat - 6.0.48
Session objects that need to be clustered are Serializable.
stickysession=JSESSIONID
Load Balancer Configuration
<Proxy balancer://mycluster>
BalancerMember http://IPAddress1:8080/app/ route=tom
BalancerMember http://IPAddress2:8080/app/ route=cat
ProxySet lbmethod=byrequests stickysession=JSESSIONID|jsessionid
</Proxy>
ProxyPass /app/ balancer://mycluster/
ProxyPassReverse /app/ balancer://mycluster/
tom instance - server.xml
<Listener className="com.hazelcast.session.P2PLifecycleListener"/>
<Engine name="Catalina" defaultHost="localhost" jvmRoute="tom">
tom instance - context.xml
<Manager className="com.hazelcast.session.HazelcastSessionManager" sticky=true/>
cat instance - server.xml
<Listener className="com.hazelcast.session.P2PLifecycleListener"/>
<Engine name="Catalina" defaultHost="localhost" jvmRoute="cat">
cat instance - context.xml
<Manager className="com.hazelcast.session.HazelcastSessionManager" sticky=true/>
Update:
I believe the below is the same issue I have session sync failure
Although I don't have much experience of using Apache HTTP server as a load balancer, looking at this page, I think you need to update your load balancer configuration as follows:
<Proxy balancer://mycluster>
BalancerMember http://IPAddress1:8080/app/ route=tom
BalancerMember http://IPAddress2:8080/app/ route=cat
ProxySet lbmethod=byrequests stickysession=hazelcast.sessionId
</Proxy>
ProxyPass /app/ balancer://mycluster/
ProxyPassReverse /app/ balancer://mycluster/
Do we have a client where I could monitor hazelcast sessions?
You can use Hazelcast Management Center to monitor the cluster and browse the session entries that are stored in the Hazelcast IMap. All sessions are stored in a Hazelcast IMap keyed by their IDs. Please note that you need to enable Management Center on the members of your cluster before being able to see any data on Management Center.
You can also have a look into the Docker code samples for container based session replication to see a working configuration of session replication. Although there's none that uses Apache as a load balancer, there's one that uses Nginx which might be of help to you.
I got it working with the below configuration. Figured it needed both jsessionid and hazelcast.sessionid, the session replication works great now.
<Proxy balancer://mycluster>
BalancerMember http://IPAddress1:8080/app/ route=tom
BalancerMember http://IPAddress2:8080/app/ route=cat
ProxySet lbmethod=byrequests stickysession=JSESSIONID|jsessionid|hazelcast.sessionId
</Proxy>
ProxyPass /app/ balancer://mycluster/
ProxyPassReverse /app/ balancer://mycluster/
Related
I have a configured reverse proxy in apache server to authenticate user throughs oauth2.
When the proxy path is different than the one in the server, the session cookie is lost and i get authorization_request_not_found error.
The following configuration is working fine while /backend in both:
ProxyPass "/backend" "http://localhost:8085/backend"
ProxyPassReverse "/backend" "http://localhost:8085/backend"
When i configure a different path i can't see the session Cookie so the authentication fail:
ProxyPass "/service" "http://localhost:8085/backend"
ProxyPassReverse "/service" "http://localhost:8085/backend"
In this case /service is different than /backend
Can you please help me through the required config to pass the cookie to the backend to keep the functionality as expected.
Thank you
I configured Apache 2.4 server, which used as a proxy server to my tomcat. There are two(maybe more) applications deployed on my Tomcat & both are accessible only using apache web server IP and port(443-https).
But whenever I tried to access Login.htm page, it took almost 10-15 sec to load a single page. When I tried to log in using username & password, here welcome page took 1 min 20 sec to load.
However, in my local environment (local IP and port(8080-http)) it took 1 sec for login page & 3-4 sec for welcome page. Note - there is no apache web server configured on my local environment.
I tried a lot of performance tuning methods on my apache web server, but none of them working for me. I am not sure what is missing here. Any help appreciated.
Thanks in advance.
Below is my apache server configuration.
Listen 443 https
SSLPassPhraseDialog exec:/usr/libexec/httpd-ssl-pass-dialog
SSLSessionCache shmcb:/run/httpd/sslcache(512000)
SSLSessionCacheTimeout 300
SSLRandomSeed startup file:/dev/urandom 512
SSLRandomSeed connect builtin
SSLCryptoDevice builtin
<VirtualHost <APP_SERVER_IP_ADDR>:443>
ServerName <APP_SERVER_IP_ADDR>
ServerAlias <APP_SERVER_IP_ADDR>
ErrorLog /var/log/httpd/ssl_error_log
TransferLog /var/log/httpd/ssl_access_log
LogLevel info
SSLEngine on
SSLProtocol all -SSLv2
SSLCipherSuite HIGH:MEDIUM:!aNULL:!MD5:!SEED:!IDEA
SSLCertificateFile /etc/pki/tls/certs/localhost.crt
SSLCertificateKeyFile /etc/pki/tls/private/localhost.key
<Files ~ "\.(cgi|shtml|phtml|php3?)$">
SSLOptions +StdEnvVars
</Files>
<Directory "/var/www/cgi-bin">
SSLOptions +StdEnvVars
</Directory>
BrowserMatch "MSIE [2-5]" \
nokeepalive ssl-unclean-shutdown \
downgrade-1.0 force-response-1.0
CustomLog logs/ssl_request_log \
"%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"
ProxyRequests Off
SSLProxyEngine On
ProxyPreserveHost On
RewriteEngine On
HostnameLookups off
<Proxy <APP_SERVER_IP_ADDR>:443>
Order deny,allow
Allow from all
</Proxy>
ProxyPass /application1 http://<WEB_SERVER_IP_ADDR>:9660/application1
ProxyPassReverse /application1 http://<WEB_SERVER_IP_ADDR>:9660/application1
ProxyPass /application2 http://<WEB_SERVER_IP_ADDR>:9660/application2
ProxyPassReverse /application2 http://<WEB_SERVER_IP_ADDR>:9660/application2
<Location /proxy/>
ProxyPassReverse /
Order deny,allow
Allow from all
</Location>
Header edit Location ^http://<APP_SERVER_IP_ADDR>/ https://<APP_SERVER_IP_ADDR>/
Timeout 10
ProxyTimeout 10
ProxyBadHeader Ignore
</VirtualHost>
After digging a lot into apache API, I finally found that there is a big problem in my application.
We are trying to implement the 3-Tier architecture in our production system. As we are using apache web server (as a proxy server) from Web-Server to App Server, there are lot of Js, CSS and images files transferred from my application.
When user requesting Login.htm (or accessing any other page) from WebServer, this request will be responded by my app server. As there are lot of Js, CSS and images files are transferred from App to Web, and finally Web to browser, the signle request consumes around 15 seconds.
To overcome this situation, I moved all Js, CSS and images related files to web server. So all HTML related stuff now loading from Web server and not from app server. For this, I just added below lines in my ssl.conf
DocumentRoot "/var/www/html"
ProxyPass /application1/resources !
The path present in DocumentRoot is context path of my Web server, where I created 'application1' directory and added 'resources' directory where all js, css and image files are present.
Now my application take 4-7 seconds to load Login.htm page.
I downloaded mod_cluster native bundles with httpd from http://mod-cluster.jboss.org/mod_cluster/downloads/1-2-6-Final-bin like below. So I did not need to do any configuration for httpd since this tar file contain httpd and related mod_cluster modules.
mod_cluster-1.2.6.Final-linux2-x64.tar.gz
Then I downloaded tomcat7 and do the following two things.
1 . add <Listener className="org.jboss.modcluster.container.catalina.standalone.ModClusterListener" proxyList="192.168.91.128:80" advertise="true"/> to tomcat7/conf/server.xml
2.download mod_cluster-parent-1.2.6.Final-bin.tar.gz from jboss site and extract the jars to tomcat7/lib directory.
I referred all above steps from http://docs.jboss.org/mod_cluster/1.2.0/html/Quick_Start_Guide.html#d0e250.
But after I started up httpd, tomcat7, I found there are error messages in catalina.out generated again and again like below.
ERROR: MODCLUSTER000042: Error null sending INFO command to 192.168.91.128/192.168.91.128:80, configuration will be reset: nul
I pasted the critical configuration from httpd.conf.
<IfModule ssl_module>
SSLRandomSeed startup builtin
SSLRandomSeed connect builtin
</IfModule>
<IfModule manager_module>
Listen 192.168.91.128:6666
ManagerBalancerName mycluster
<VirtualHost 192.168.91.128:6666>
<Location />
Order deny,allow
Deny from all
Allow from 192.168.91
</Location>
KeepAliveTimeout 300
MaxKeepAliveRequests 0
AdvertiseFrequency 5
#AdvertiseSecurityKey secret
#AdvertiseGroup #ADVIP#:23364
EnableMCPMReceive
<Location /mod_cluster_manager>
SetHandler mod_cluster-manager
Order deny,allow
Deny from all
Allow from 192.168.91
</Location>
</VirtualHost>
</IfModule>
ServerName localhost
Error
Communication between Apache HTTP Server and Tomcat is bidirectional, i.e. Tomcat's mod_cluster library needs to talk to a VirtualHost configured in your Apache HTTP Server -- the exactly one VirtualHost where you have EnableMCPMReceive set. In this case, please, follow Federico's advice and change the proxyList to match your EnableMCPMReceiveVirtualHost IP and port.
Advertising
Furthermore, is it intentional that you use both advertising (Apache HTTP Server using UDP multicast to advertise its presence to Tomcats) and you have proxyList set at the same time? You can get about without one or the other, depending on your desired setup and environment.
Do not use mod_cluster 1.2.6, please
mod_clister 1.2.6 is desperately obsolete; it contains not only severe performance issue but also CVE and several noteworthy bugs. Please, upgrade to mod_cluster 1.3.1.
See http://modcluster.io for 1.3.1.Final release bits or alternatively http://mod-cluster.jboss.org.
Regarding Apache HTTP Server 2.2.x vs 2.4.x
The aforementioned mod_cluster 1.3.1.Final is offered with httpd 2.4.x. Do you have any special desire for older httpd 2.2.x? It is possible to compile mod_cluster 1.3.1 with httpd 2.2.x, it's just not what we do regularly as noted in my comment on the subject.
Happy mod_clustering!
K.
I have a user configured in AD with delegated kerberos ticket:
klist
Ticket cache: FILE:/tmp/krb5cc_527
Default principal: user1#EXAMPLE
Valid starting Expires Service principal 11/27/15 16:28:27 11/28/15 02:28:27 krbtgt/EXAMPLE.com#EXAMPLE.COM
How can I get this 'user1'? On this client domain account (client side jsp? Or server side?) I want to extract this value after button click and pass it back (with backurl) to another java app.
EDIT:
My Apache configuration:
<Location /kerb >
AuthType Kerberos
AuthName "auth-realm"
KrbMethodNegotiate off
KrbMethodK5Passwd off
KrbServiceName HTTP
Krb5Keytab /etc/krb5.keytab
require valid-user
</Location>
ProxyPreserveHost On
ProxyPass /kerb ajp://120.201.131.169:8019/myApp
ProxyPassReverse /kerb ajp://120.201.131.169:8019/myApp
But I received
[Sun Dec 13 18:17:32 2015] [debug] src/mod_auth_kerb.c(1944): [client
126.185.3.202] kerb_authenticate_user entered with user (NULL) and auth_type Kerberos
It depends. In my case I have an Apache server configured with mod_kerb and forwarding http requests to Tomcat by means of AJP.
In such scenario, Tomcat AJP conector is configured with tomcatAuthentication=false and I can get user authenticated from JSP and Servlet using request.getRemoteUser().
Obviously, the user string comes with domain info after # so you have to consider it.
If you are interested in my solution, I can elaborate my answer.
Edit
I edit my answer to give more info about configuring Tomcat to use Kerberos.
Configure NTP
First, it is quite common to have NTP clients configured in every system AD server, Apache server and Tomcat server. If there is no date and time synchronization, it is quite common to get clock skew too great or postdating problems.
Create an AD principal for the server
You need to create a principal into AD to use for server principal authentication. It is necessary to get a keytab file for this principal. I am sorry, I can't tell you how to do this.
Install and configure Kerberos on Apache server
Once you have your server principal and keytab file, it is time to configure Apache server. Install kerberos into that system and configure /etc/krb5.conf. A sample of this file is:
HERE.YOUR.KERB.DOMAIN = {
kdc = your.dns.kerb.domain
admin_server = your.dns.kerb.domain
}
Check with:
kinit -k -t keytab.file HTTP/principal.dns.name#HERE.YOUR.KERB.DOMAIN
klist
that your server is right configured.
Install and configure mod_auth_kerb
Install mod_auth_kerb apache module and configure its use in every location, directory, virtual host or whatever you need, see below. This configuration is very dependent on your kerberos server, you will have to play with some parameters as KDC verification, negotiation, be or not authoritative, ...
The most important parameter is Krb5Keytab, but you can check this page to understand all parameters. Here you are with a sample location:
<Location /sample/>
AuthType Kerberos
AuthName "auth-realm"
KrbMethodNegotiate on
KrbMethodK5Passwd off
Krb5Keytab /your/path/to/keytab.file
require valid-user
</Location>
When you try to access this location http://your.apache.server/sample Apache will try to check user credentials by means of kerberos.
AJP Configuration
For AJP configuration (I already answer this before, but I cut and paste here and adapt for this question) the procedure is the following:
Install Apache module for AJP, usually it is called something like libapache2-mod-jk. (In debian/ubuntu you can run sudo apt-get install libapache2-mod-jk).
Then you will have a new module calledjk or similar. You have to enable it (In debian/ubuntu you can run sudo a2enmod jk).
Default configuration will serve mostly, open it a see where does JkWorkersFile point. This file is needed to configure the workers that manage communication with tomcat apps.
Create workers file (if it does not exists). A workers file is more or less as following.
Sample workers file:
ps=/
worker.list=worker1,worker2,...
# worker1 definition
worker.worker1.port=8009
worker.worker1.host=host or ip
worker.worker1.type=ajp13
# worker2 definition
....
Every worker can point to different tomcat server. Port must be the same that configured into $CATALINA_HOME/conf/server.xml. In this file there is a connector for AJP protocol:
<Connector port="8009" protocol="AJP/1.3"
redirectPort="8443" tomcatAuthentication="false"/>
Every worker has to point to this port.
Finally, you can configure your location (or whatever) using JkMount workerName to indicate Apache that this url has to be forwarded to the proper worker:
<Location /sample/>
JkMount worker1
AuthType Kerberos
AuthName "auth-realm"
KrbMethodNegotiate on
KrbMethodK5Passwd off
Krb5Keytab /your/path/to/keytab.file
require valid-user
</Location>
There are plenty of samples an documentation. Here you are with Tomcat official docs: https://tomcat.apache.org/connectors-doc/webserver_howto/apache.html
Web app authentication
You don't need to configure anything about security constraint in web.xml, with this configuration, Apache will authenticate users instead Tomcat, and Tomcat will receive user's principal name into HTTP request.
Tomcat (and any other servlet container) will encapsulate the user's principal into request.getRemoteUser().
Hope it helps.
I got it! I don't know what exactly was wrong but now It works.
I can simple get principal from HTTP Header with the following configuration (I moved this configuration from /conf.d to main /conf/httpd.conf file). What is imported. On RHEL httpd server apache user should have rights to read /etc/krb5.keytab. In my case:
ps -ef | grep httpd
apache 27537 27535 0 16:18 ? 00:00:00 /usr/sbin/httpd
<VirtualHost myhost.domain.com:80>
ServerName myhost.domain.com
<Location /myApp >
# SSLRequireSSL
AuthType Kerberos
KrbMethodNegotiate On
KrbMethodK5Passwd Off
KrbServiceName HTTP/nmyhost.domain.com#EXAMPLE.COM
KrbAuthRealms EXAMPLE.COM
Krb5KeyTab /etc/krb5.keytab
require valid-user
RewriteEngine On
RewriteCond %{LA-U:REMOTE_USER} (.+)
RewriteRule . - [E=RU:%1]
Header add X-Remote-User "%{RU}e" env=RU
</Location>
ProxyRequests Off
ProxyPreserveHost On
ProxyPass /myApp ajp://126.101.100.169:8029/myApp
ProxyPassReverse /myApp ajp://126.101.100.169:8029/myApp
</VirtualHost>
Try checking your keytab file. I had similar issue and when i ran
cat httpd.keytab
it showed plain text, which isnt normal, keytab file should contain ASCII chars, its a binary file. Had to regenerate it with Domain Admin account "$user.name" and then it worked.
I've set of JSON APIs that are exposed and implemented on Tomcat.
I would like to implement Multi-tenancy for these APIs on Tomcat with the following URL approach:
companyname1.domain.com/api/getUsers...
companyname2.domain.com/api/getUsers...
companyname3.domain.com/api/getUsers...
Let me know if there is a best practice for implementing it using context or other mechanism. I don't want to create a separate Tomcat instance for each and every company.
In addition is there any way to create it dynamically once company is registered.
Thank you in advance,
Moshe
Create several virtual hosts in your Tomcats server.xml, that listen to your domains. These should point to different webapps directories, hosting your particular application in the ROOT directory:
<Host name="localhost" appBase="domain1-webapps" autoDeploy="true" unpackWARs="true"></Host>
<Host name="companyname1.domain.com" appBase="domain1-webapps" autoDeploy="true" unpackWARs="true"></Host>
<Host name="companyname2.domain.com" appBase="domain2-webapps" autoDeploy="true" unpackWARs="true"></Host>
...
This can be done using multiple reverse proxies that provide different parameters to the Tomcat webapp. The simplest setup (with Apache HTTP and mod_proxy_ajp) would probably be to preserve the original request's host and resolve that inside the web app.
<VirtualHost *:80>
ServerName companyname1.domain.com
ProxyPass / ajp://localhost:8009/
ProxyPassReverse / ajp://localhost:8009/
ProxyPreserveHost On
</VirtualHost>
<VirtualHost *:80>
ServerName companyname2.domain.com
ProxyPass /api ajp://localhost:8009/
ProxyPassReverse / ajp://localhost:8009/
ProxyPreserveHost On
</VirtualHost>
<VirtualHost *:80>
ServerName companyname3.domain.com
ProxyPass / ajp://localhost:8009/
ProxyPassReverse / ajp://localhost:8009/
ProxyPreserveHost On
</VirtualHost>
I can not comment because of my reputation is below than 50.
if URI is same like
companyname1.domain.com/api/getUsers
companyname2.domain.com/api/getUsers
companyname3.domain.com/api/getUsers
then Apache web server found domain companyname1.domain.com and from httpd file, it send call to tomcat application server through AJP connector. But problem is that how can it found which war file to run.
So application name is required in URL
companyname1.domain.com/abc/api/getUsers
companyname2.domain.com/xyz/api/getUsers
companyname3.domain.com/fgf/api/getUsers
We can use server context setting. As per tomcat server specification.
The Host element represents a virtual host, which is an association of a network name for a server (such as "www.mycompany.com")
with the particular server on which Tomcat is running. For clients to be able to connect to a Tomcat server using its network name,
this name must be registered in the Domain Name Service (DNS) server that manages the Internet domain you belong to - contact your Network Administrator
for more information.
In many cases, System Administrators wish to associate more than one network name (such as www.mycompany.com and company.com)
with the same virtual host and applications. This can be accomplished using the Host Name Aliases feature discussed below.
One or more Host elements are nested inside an Engine element.
Inside the Host element, you can nest Context elements for the web applications associated with this virtual host.
Exactly one of the Hosts associated with each Engine MUST have a name matching the defaultHost attribute of that Engine.
Clients normally use host names to identify the server they wish to connect to. This host name is also included in the HTTP request headers.
Tomcat extracts the host name from the HTTP headers and looks for a Host with a matching name.
If no match is found, the request is routed to the default host.
The name of the default host does not have to match a DNS name (although it can) since any request where the
DNS name does not match the name of a Host element will be routed to the default host.
For more info. go through this link.
:https://tomcat.apache.org/tomcat-8.0-doc/config/host.html