Is there a way to secure Hadoop (version 2.7.2) web user interface - JobTracker, etc. - by using the available Linux accounts of the system? My goal is somewhat similar to what the old mod_auth_pam and the newer mod_authnz_external and pwauth is to Apache. I found a description on how to password protect these interfaces, however it would be much easier if an additional user-password list would not be required to be maintained just for this.
Configuring Hadoop into secure mode and using Kerberos authentication would be one option, but since I have a small cluster with few users, mainly for testing purposes, it would be nice if a not so robust solution would be available for the Jetty web server as well, since my goal is just to make the web UI not public towards the world.
Related
Background Context:
Due to enterprise limitations, an uncooperative 3rd party vendor, and a lack of internal tools, this approach has been deemed most desirable. I am fully aware that there are easier ways to do this, but that decision is a couple of pay grades away from my hands, and I'm not about to fund new development efforts out of my own pocket.
Problem:
We need to send an internal file to an external vendor. The team responsible for these types of files only transfers with SFTP, while our vendor only accepts files via REST API calls. The idea we came up with (considering the above constraints) was to use our OpenShift environment to host a "middle-man" SFTP server (running from a jar file) that will hit the vendor's API after our team sends it the file.
I have learned that if we want to get SFTP to work with OpenShift we need to set up of our cluster and pods with an ingress/external IP. This looks promising, but due to enterprise bureaucracy, I'm waiting for the OpenShift admins to make the required changes before I can see if this works, and I'm running out of time.
Questions:
Is this approach even possible with the technologies involved? Am I on the right track?
Are there other configuration options I should be using instead of what I explained above?
Are there any clever ways in which an SFTP client can send a file via HTTP request? So instead of running an embedded SFTP server, we could just set up a web service instead (this is what our infrastructure supports and prefers).
References:
https://docs.openshift.com/container-platform/4.5/networking/configuring_ingress_cluster_traffic/configuring-externalip.html
https://docs.openshift.com/container-platform/4.5/networking/configuring_ingress_cluster_traffic/configuring-ingress-cluster-traffic-service-external-ip.html#configuring-ingress-cluster-traffic-service-external-ip
That's totally possible, I have done it in the past as well with OpenShift 3.10. The approach to use externalIPs is the right way.
I'm a Java EE developer and we typically use Weblogic to deploy our apps. Now I'm faced with a new desktop application which requires logging, database connectivity and mail.
After some investigation I'm realizing that desktop apps are a completely new world to me and I'm not sure if I'm choosing the right libraries to support my app.
These are my questions:
In our Weblogic projects we used Log4j and I want to use it again in my desktop app. Is it a bad idea? Should I use a better logging framework?
In Weblogic we retrieve database connections with JNDI but now it seems impossible to do the same. How do I perform the same action in a desktop application so I can connect with a remote database? Is the combination c3p0 + database driver a good approach for this?
Is there any framework/JAR which provides all this stuff (log + ddbb + mail) as an integrated solution? Workmates told me Spring could help. I also found Warework.
In our Weblogic projects we used Log4j and I want to use it again in
my desktop app. Is it a bad idea? Should I use a better logging
framework?
No, it is not a bad idea and perfectly works. Personally, I'd go with java.util.logging as it does the job fairly well and it reduces your applications' footprint (storage). Although, it's configuration is a bit tricky.
In Weblogic we retrieve database connections with JNDI but now it
seems impossible to do the same. How do I perform the same action in a
desktop application so I can connect with a remote database? Is the
combination c3p0 + database driver a good approach for this?
You can directly connect to your database using pure java.sql JDBC API (tons of examples available in the internet), but always have to distribute the proprietary database drivers as part of your application (mySQL, Oracle, DB2, etc.). Furthermore it's possible to directly use connection pools provided with those drivers by using their proprietary APIs (fairly easy to encapsulate). Nevertheless, there are a number of issues:
latency; database protocols are fairly sensitive when it comes to latency (distance between client and database server). Having a database in the UK and desktop clients in US is probably not a good idea.
security 1; you have to distribute database user credentials to each and every desktop client. Be aware of that.
security 2; your database security requirements may demand for transport security (packet encryption).
change management; applying non-backward compatible updates to your database requires you to update all desktop clients (believe me - it's not fun).
network; depending on your environment, certain ports and/or protocols may be blocked.
Is there any framework/JAR which provides all this stuff (log + ddbb +
mail) as an integrated solution? Workmates told me Spring could help.
I also found Warework.
Logging and database access are not an issue and work fairly well without any third-party framework. Of course, those frameworks might provide value regarding other aspects (abstraction, DI, JDBC abstraction, etc.), but this is a topic of detailed software design. Sending emails directly from a desktop application might become an issue, regardless of the framework in use. Just some things to keep in mind:
which SMTP relay server do you want to use?
in case of an enterprise environment, your IT operations teams might not allow you to use their SMTP server from each desktop (keep spam in mind).
Conclusion: In desktop scenarios an application server is not a bad idea either. You should have your desktop application to communicate with an application server only by using e.g. JSON, XML, SOAP over HTTP/HTTPS or RMI, etc. The application should be responsible for the complex tasks like database access, transaction management, fine grained security, email, etc.
I am using ActiveMQ to connect a number of application modules written in Java.
I eventually would have a web interface for the application, developed in either Grails, Struts2, or Rails.
My 2 main concerns are:
to have an external security module that is not bound to the Web Framework in use.
to have an independent security db
Any recommendations for this Architecture?
You should place all your components within a secured firewall. Then you wouldn't need to worry about any kind of security for ActiveMQ. If not a firewall, you should have a way to whitelist your components so only you can connect to them.
For the database, I recommend having one user that read data and one user that writes data. Separating this permissions will be a closer step to someone deleting you data.
You need to secure both parts of your application. For the first part go with Amir Raminfar's answer, and insure that your running on secure servers. Also make sure to use what ever security features are built into MQ to allow the components to communicate securely. For Web Security there is no good way I know of to have a framework agnostic security setup. An option for you may be Spring Security You should be able to integrate it with Struts and there is a Grails Plugin This should make it easier to do security in a relatively common way whether you use Struts or Grails but you will probably not be able to easily use Spring Security from Ruby.
I'am developing a java servlet application, and tesing it on Eclipse + Apache Tomcat (refer: http://www.vogella.de/articles/EclipseWTP/article.html#overview_wtp).
The application is now tested on the localhost and accesed by any clients on the same LAN.
Now, I need to deploy it into the web server, where everyone from anywhere can access this servlet.
Coud you guide me the way that I've to do to archive this task.
You need to have a computer accessible to everyone - i.e. placed on the internet and not behind a firewall - with the appropriate software installed (and hardened against hacker attacks).
If you do not have such a computer, you can have a look at the Google Application Engine which allows you to deploy Java web applications (with some additional restrictions) to the Google cloud. This is free for low-volume applications.
Yes, you can do it by deploying your application in the Cloud Instance. Since we cannot able to make our server instance or computer to be run always(We may come across internet connection problems, Power Fluctuation, etc.,), We have lot of problems while making our instance public(In Security perspective too). So it is better to use cloud instances.
We have many Cloud Service Providers such as AWS by Amazon, Google Cloud, Microsoft Cloud, etc.,
Take a look on this List of Cloud Services Providers.(You have links for all top 10 providers)
Why do we need Application Server in Java like JBoss or WebSphere. Is it possible that we develop large scale website only with Java (Apache Tomcat). Where thousand of user connect on site at a moment. An example is a website like b2b.
What is the cost of a Application Server? I will be thankful if you compare price among different application server and if there is any free version kindly highlight it.
Application Servers are mostly used if you want to use advanced features like transaction management, hot code swapping and advanced persistence.
There are application servers that are open source. E.g. GlassFish and JBoss.
I don't think you need an application server for building a popular web site, you'll also be fine with a servlet container like Tomcat or Jetty.
In short Application Servers provide you with few services like
Transaction Management
Load Balancing
Security
Threading
etc.
You have to take care of these things yourself in a Web Server.
There are few Open Source Application servers which are free of cost.
I have used Glassfish.
Apart from answers given above, App Servers are required for EJBs.
You need Application Server as follow:
It provides you useful services like automatic transaction,Authentication,Authorization,Lifecycle management.
To remember large user data across pages using ejb's pertaining to a client.
Load balance the user request and buisness logic.
To interact with different Client UI like Java Swing,Browsers.
It is possible to handle the httpheaders yourself. We have done socket servers in java for 20 years. You do not need a container for java swing.
Persistence can be done through databases or server side files unless you need real high speed stuff. I have yet to find a real requirement for an ejb
except that some systems simply require them
This may be because jboss can provide better after-sales service, and jboss, etc. can provide operation and maintenance support, etc. This may be the reason why many large companies choose commercial versions of servers.
But you must know that tomcat and netty are not bad. For example, many large B2B or C2C or B2C companies still use tomcat, such as Internet companies such as Alibaba.
Choose a server
Operation and maintenance costs
Scalable costs
Server cost