Network communication between dedicated web and database server - java

We're at the juncture (I can even say, right time) that we need to separate our database server apart from our application web server. This we arrived at after careful & thorough monitoring of the server (load average, CPU usage, etc.) itself.
When the database server is separated:
What would be the ideal setup/best practice for network communication path between 2 servers? If it were running on the same server, connectivity between database and application would make use of TCP/IP communication, but how it should be running/setup when it is separated, so that performance is not majorly affected? For example, currently our JDBC URL looks like jdbc:postgresql://localhost:9999/ourproductiondb, probably this localhost would be changed to private IP address of the database server.
In our application, we also allow application users to submit/attach attachments (.txt, .doc, .pdf, .xls, etc.), which are actually stored physically in server disk and only its file path is stored in the database. Which is the ideal place/approach for maintaining/keeping documents physically, either in application web server or in database server?
We're very new into this area of separating database as a separate server and our knowledge on this is very limited. Therefore, any resources, links, documentation, articles, guidelines, suggestion etc. in this regard are highly welcome and really appreciated.
NOTE: Our tools of trade are: RHEL5 64-bit, Apache Tomcat 6.0.10, JDK 1.6, PostgreSQL 8.2.22, pgpool 2.0.1

I received a response for this very same question after posting in serverfault. Here is the answer. Am posting this as an answer here with the hope that this may be useful for other users who might have similar kind of question.

Related

How can I allow other computers to access my programm's database?

I have a java application that I programmed, and I also created a Wampserver database to store the app's data. When I install the application in another computer, it basically can't do anything because wampserver isn't installed on that computer, and even if it was, the database wouldn't be imported on that same computer.
Can I install the database with the program itself, without making a database server in my computer? (For security reasons)
Sorry if this is a dumb question, im new to programming.
Assuming you are doing this in a Home/office environment. Where your IP Addresses do not change so often. Or you have the power to assign them statically.
You could try using a multi-threaded client server model. Where your wampserver computer(server) will accept a connection from another computer(client), and from there any queries to the database will be handled by that thread. This way, you could get away without installing wampserver on every client.
It would require some networking codes to be added to your java program. There are many examples on multi-threaded client server model -> Multithreading with client server program
You can open your MySQL server to the outside world or to certain IP (comment or edit bind value in your MySQL configuration file).
This can be a security issue. What is your objective? Usually, you'd do an API allowing access to your data from your application.
You are talking about "hosting" your database. You need to either:
Create, develop, and run a hosting service (not trivial), or
Pay to one the myriad of hosting services available, and install your database there (cheap these days). Just a couple of well known examples: Digital Ocean or Amazon Web Services.
In any case, it's a huge security risk to open your database to any client online.
Normally you write a program/application that accesses the database in the same [local] network, and that application serves web pages (or other web service) to the world.

How do I host (non-local host) MySQL for JDBC

sorry if this is a super easy question, but I have been googling a lot and I have failed to get a result.
I am looking for a way that I can either purchase or host so that I and others can connect remotely.
I have all my code working locally, but I would just like to know how to host for others to join.
There are many ways to provide a MySQL server instance that's accessible over the public internet.
For this to be secure, you'll need to use transport layer security (TLS) for the connections from JDBC clients to the server. Even when you do this you'll need to be vigilant about the security of your database.
If your users' data is sensitive, or needs protection against cybercriminals, this is probably a bad idea. (Sensitive data: personally identifiable information, payment/credit card data, health data, personal financial data). It's much safer to put MySQL instances behind solid firewalls, and use web services and/or web sites to grant access to the public internet.
You could run a virtual machine instance on AWS or some other cloud service provider, and put MySQL on it. AWS also offers a Relational Database managed service you could use.
You could use a GoDaddy style service, and enable remote access to your own MySQL instance on that service.
You could put a MySQL instance on a machine in the DMZ -- the publicly visible segment -- of your home or office network. Most home and small-office routers allow a particular port on a particular machine to be made visible to the public network.
Be careful, k?

Host MySQL on a shared folder

I am trying to host MySQL Database on the shared location of the organization.
Is there any way through which I can achieve this because I don't have a shared server which will run all the time.
MySQL is a "client-server" database management system (like Microsoft SQL Server, PostgreSQL, Oracle, etc.) so it requires a server on which the centralized database management application (the "database server") can run. The client software for such databases can only "talk to" the application running on the server, and the server application updates the database file(s); the clients cannot update the database file(s) directly.
By contrast, a "peer-to-peer" (sometimes called a "shared-file") database like Microsoft Access does not have a centralized database application. Instead, each client machine has a copy of the database engine that can update the database file(s) directly in a co-operative fashion.
So, you cannot use MySQL without having a MySQL database server running somewhere. As for a peer-to-peer solution, refer to your previous question here.
While I understand what you are doing it is simply a very bad idea and cannot be made to work. Much of the purpose of a DBMS system (like mySQL) is to handle concurrent access to the data responsibly.
The basic idea of the architecture is to talk (in small messages) to a server software stack that can rapidly access its files and grab the data you need and send it back to you. This server software stack does two things:
Negotiates that clients looking at the same data all see a consistent meaningful picture of the data
Does all the searching, sorting, indexing etc on the server without sending all this data back and forth across the network.
So if you are using MySQL shared between different client machines you need to have the mysql server running ONCE on the server (as opposed to running on each client accessing a shared drive as you suggest).

JDBC Remote MySQL Connectivity

I'm developing a Java Swing based app which uses JDBC to connect to a MySQL database. As such, the software directly remotely access the database from whichever computer it happens to be running on. Additionally, the app uses prepared statements to query the database and the database is hosted on a shared CPanel hosting account (if that matters).
The snippet of code I use to connect to the database is as follows (fairly standard connect code I think and all strings in all caps contain the correct contents):
String url = "jdbc:mysql://URL:PORT/DB_NAME?connectTimeout=3000";
Connection conn = DriverManager.getConnection(url, USERNAME, PASSWORD);
I have only ever successfully used the app from one IP. Before I use the app from an IP, I have to manually whitelist the IP by adding it as an allowed remote MySQL access host. If I don't add the IP as an allowed access host, the server refuses my connection and I get the resultant error:
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
Then if I whitelist an IP and try to connect from it, I don't get that error and the app connects to the database properly.
This system would be okay if the app were only going to be used from one IP, but it needs to work from any IP since I cannot predict who will download and use it. The only solution I see would be to do a global whitelist of all IPs in the allowed MySQL access hosts area. However, that seems like it has many drawbacks, such as being insecure as anyone who has the correct password could log in (and would thus be susceptible to brute force attacks). This seems to corroborate the hypothesis that that method is insecure. Thus, I would like to have a system of communicating between the app and database that is IP-independent (doesn't require whitelisting all the IPs).
Additionally (I don't know if this makes sense or matters), but I believe some of the areas I expect the app to be used in block certain protocols. Thus, (I think) I would like it if the selected method of communication only used HTTP or some other widely-used protocol.
I did some research into this problem and my efforts led me to 2-tier and n-tier models of database communication. Perhaps I could do something like make a PHP page which accepts a statement and a series of parameters (plus a password to gain entry), executes the statement, and then returns the result back as JSON. However, this seems like another less-than-ideal method as it seems like it would also have security problems.
I'm sure someone more experienced and knowledgeable than I has already come across this problem and developed a solution.
Therefore, my question: What is the preferred method of connecting to a MySQL database from a Java app in an IP-independent way?
I greatly appreciate and thank you for your time.
You're on the right track:
1) If you want any arbitrary client to connect directly to your database, and the clients can have any arbitrary IP address ... then you're probably going to have to effectly disable IP security be whitelisting all possible client IP addresses.
2) On the other hand, if you only allow local access to mySql (by far the most common scenario), then you can create a web app to interface between your clients and mySql.
SUGGESTION:
Consider creating a "REST" web service that your clients can talk to.
Here's a good tutorial that might help you get started:
REST with Java (JAX-RS) using Jersey - Tutorial
Q: Does your Swing app really need to emit "raw SQL"? Or can it make "high level" queries? REST is ideally suited for the latter.
PS:
Here's another, short example that might help suggest some design alternatives with REST, mySQL and Java for you:
http://www.9lessons.info/2012/09/restful-web-services-api-using-java-and.html
You are up against the policies -- primarily the security policies -- of your hosting provider. It's generally considered insecure to allow port 3306 (MySQL) connections from the whole internet. It certainly lays your MySQL server open to trivial denial-of-service attacks. (It just takes some knucklehead controlling a botnet to send in lots of port 3306 connection attempts. They don't even have to be successful connection attempts.) If you're sharing your MySQL server with other customers of your hosting provider, they have every incentive to restrict your remote access to their server.
Most folks who build database applications for deployment on the public internet do it by providing web services to hit the database with the specific operations required by the application. The application deployed at the end-user's machine then uses HTTP (or HTTPS for security) to access those web services. In turn the web services access the database. That's what multitier operations do. You're right that there are security problems, but you can mitigate them with careful development of your web service code.
You could use SSH tunneling to handle your database access. The SSH suite of remote-access applications allows port forwarding. To use this, you would establish (authenticated and encrypted) ssh connections between your end-users' machines and your database machine, that forward port 3306. Then your users could connect to localhost:3306, and that net traffic would be forwarded to your database server. It's pretty flexible and quite secure, if not completely simple to configure.
You might also investigate using SQL Relay. It's generally used for connection pooling and management within a data center network, but it might work for this purpose.
Be careful opening up your MySQL server to the world! If you do that you may want to require the use of TLS encrypted conections.

Connecting Access Databases remotely?

Currently I have a Java application which performs all I wanted and the limitation
in it is "It only resides on the Local machine for which it needs to work". I have the Access Databases located on a Remote Server and I install the Java app on the remote server to connect the databases using a connection string jdbc:odbc:DRIVER={Microsoft Access Driver (*.mdb)}; DBQ=D:\\DATA\\DPPI\\DATA\\DPPI\\DPPIPR01.mdb;
It connects to Access Databases and do all the business logic with them.
Enhancements (to be done):
Making the application online - not a big issue.
Connecting the databases remotely - its the only concern.
If I make the Application Online the Access Databases need to be connecting through network (somewhat like IP or some web address).
Problem:
But after going through this Connect to Access database remotely It seems my requirement can not be completed.
Is there any way if I'm able to upload the database to the online web hosting server and access the database from there only? If yes, then how should I do the connectivity since Access Database connects by giving the fully qualified name containing drive letter and all.
But how should I proceed further so that I may be able to connect the mdb Databases remotely?
ADO Solution:
But after going through the link: http://webcheatsheet.com/ASP/access_connection_strings.php, it seems we can perform remote Access Database connectivity.
The solution for the ADO is something like:
connectionString="Provider=MS Remote; Remote Server=http://your_remote_server_ip;" &_
"Remote Provider=Microsoft.Jet.OLEDB.4.0; Data Source=c:\your_database_name.mdb"
I'm a tad confused to how Access Database we can connect.
There is one more question in my head.
I'm making use of Type-1 Driver. Should the Type-4 Driver support the remote connectivity?
If your application is a server side application (like a web application) and your server's OS is Windows then theoretically it is possible to use your Access database. In this scenario, the Access database is local to your application. In your connection string you can simply use the path to the Access database inside your server's file system.
But keep in mind, that Access is a desktop database not made to use it as a server database. This can lead to poor performance and maybe system crashes. And it is possible that there are some licencing issues for this use case.
So even it may be possible to use Access with a server application it is strongly recommended to use a real server database like MySQL, MS SQL Server, etc. Even an embedded database like Apache Derby or H2 would be a better fit.
If your application runs at client side like a fat client, then your only chance to use Access is to put it on a network filesystem, so it is visible from every client. But this works only inside a LAN. If you cannot put it on a network filesystem then you have to use a real server database.
Do not use Provider=MS Remote (RDS) for new development. Here is a quote from the MSDN article
Microsoft OLE DB Remoting Provider (ADO Service Provider)
Important
Beginning with Windows 8 and Windows Server 2012, RDS server components are no longer included in the Windows operating system (see Windows 8 and Windows Server 2012 Compatibility Cookbook for more detail). RDS client components will be removed in a future version of Windows. Avoid using this feature in new development work, and plan to modify applications that currently use this feature. Applications that use RDS should migrate to WCF Data Service.
As others have suggested, you should seriously consider using a different back-end database for this particular application.
What if I'm able to call another java application residing on the Server housing the Microsoft Access Databases? Can you please let me know if it can be the solution such that I will have the division of my application into two:
1. The Online java Application performing 2 things: Accepting the requests coming in and waiting for the response to come from another application running on the Server housing the Microsoft Access Databases.
2. The Java application installed on the Server housing the Microsoft Access Databases only responsible for connecting and returning the DTO (Data Transfer Objects) -or some kind of objects in Collections.
So, I would expect a serverside program to handle the communication with the DB and the client application would communicate with the server app.
Please let me know if I should proceed in this manner.
it can be done by marking the traffic and queueing the traffic from the server---to----remote site ---(QOS..quality of service ) .....the two sites will be connected through VPN methods to reduce the unnecessary hops that will improve the latency plus add the security , thus server and remote site will logically act as if they are on the same network (LAN)
regards ,
Ali Rehan

Categories

Resources