I have a GAE project and I would like to make requests to an external server. The server needs to whitelist the IP address but GAE works with dynamic IP addresses and the list is too long. I have been looking at 2 options:
1. Move the GAE project to Compute Engine
My biggest concern about this is the time it would take to move the project to Compute Engine. So far we've been using App Engine for free and we would like to avoid extra costs.
2. Use our domain to make the requests
We could white list the IPs ourselves and receive requests from the GAE project to our domain and then make requests from the domain's static host IP itself to their server.
For this option I'd also like to know if transferring the domain to GAE has any advantage. Will the outgoing requests from GAE through the transferred domain be made through the domain's IP?
Note:
This is largely based on assumption, I am highly inexperienced with networking so if there's anything that could be better expressed please tell me. I tried to search online for a way to do this but couldn't get to a satisfactory answer.
You don't need to think about domains in this context, because you're whitelisting IPs, not domains. And it's not necessary to move the whole project, you need only move that part that makes this requests.
Just create a proxy on any server with a static IP, like micro instance on Google Compute, and do all requests through this server.
As I understand you do all request to the same external server, right? In this case it could be even simpler. You don't need to install a full featured proxy, just install an Nginx on a micro instance (with SSL and some authentication, of course) that will proxy all requests to the target server.
I ended up using RabbitMQ to send messages from Google App Engine to Compute Engine, the compute engine then forward those messages as Http requests.
Related
Background Context:
Due to enterprise limitations, an uncooperative 3rd party vendor, and a lack of internal tools, this approach has been deemed most desirable. I am fully aware that there are easier ways to do this, but that decision is a couple of pay grades away from my hands, and I'm not about to fund new development efforts out of my own pocket.
Problem:
We need to send an internal file to an external vendor. The team responsible for these types of files only transfers with SFTP, while our vendor only accepts files via REST API calls. The idea we came up with (considering the above constraints) was to use our OpenShift environment to host a "middle-man" SFTP server (running from a jar file) that will hit the vendor's API after our team sends it the file.
I have learned that if we want to get SFTP to work with OpenShift we need to set up of our cluster and pods with an ingress/external IP. This looks promising, but due to enterprise bureaucracy, I'm waiting for the OpenShift admins to make the required changes before I can see if this works, and I'm running out of time.
Questions:
Is this approach even possible with the technologies involved? Am I on the right track?
Are there other configuration options I should be using instead of what I explained above?
Are there any clever ways in which an SFTP client can send a file via HTTP request? So instead of running an embedded SFTP server, we could just set up a web service instead (this is what our infrastructure supports and prefers).
References:
https://docs.openshift.com/container-platform/4.5/networking/configuring_ingress_cluster_traffic/configuring-externalip.html
https://docs.openshift.com/container-platform/4.5/networking/configuring_ingress_cluster_traffic/configuring-ingress-cluster-traffic-service-external-ip.html#configuring-ingress-cluster-traffic-service-external-ip
That's totally possible, I have done it in the past as well with OpenShift 3.10. The approach to use externalIPs is the right way.
I have separate application for client side which is in ReactJs and NodeJS (Express server) and Web Services in Java application running in tomcat.
My query is which is better approach in terms of making web service call.
One is making direct web service call from ReactJS and get the data.
Other one is calling web service in Express server. Request from client browser will go to Express and Express will make all web services call.
I know one issue in making direct call to web service will be cross domain policy which can be handle by setting configuration in Java server.
Apart from it what should be better approach.
From my experience it ended up better using direct calls from UI application and avoiding intermediate server.
Reason for doing this directly is that our servers ended up with a lot of restrictions based on IP addresses, and all requests have been coming from intermediate server (nodeJS server), so DDOS protection of end server had to have some exceptions (our node server which could be on ACS with dynamic IP addresses so it might be hard to manage).
Also If you want to pass and track end users IP addresses, you need to manage headers on Node server (to be sure you are passing it as it was in original request).
It is way simpler to manage this kind of situation if calls are comming from React app and simply set up CORS on java server.
Also its way easier to debug it on UI app directly, so you will be watching response logs on one place only. Other way around you could end up debugging your node server and UI app.
Hope this helps a bit.
Best way IMO is to create a router in Node js specifically for all your Java webservices and act as a proxy.
Suppose if your url pattern is like http://domain/java-ws/api then all these requests will be routed to your Java service.
If you have an Apache server directing requests to your node JS then configure url pattern proxy using proxy module.
Browsers are blocking CORS requests for a reason. You may get away by setting things on your server.
I am creating 3 applications that are written for different platforms (.NET (C#), Android (Java) and PHP). I'm using C# for the WPF application that is going to run on Windows PCs, PHP on the server side and Java for the mobile app. I am using a MySQL database where I'm storing all the information that 3 apps are going to be using.
I am using web requests to my Apache server (JSON and POST basically) when I need some specific stuff to do with PHP.
But, how safe is:
When I'm connecting to the MySQL database via C# and Java?
When I'm sending GET and POST web requests with C# and Java?
Can you somehow spy on the traffic that is going on between the device (PC / Android device) and the server and find out the user and the password of the database, or even get the post request parameters that the app is sending?
Because I know there are a lot of network-monitoring software and I wouldn't be surprised if this is possible.
If it is, then how to avoid it?
"How secure are Java and C#?" isn't quite the right question, because the answer depends on what you do rather than the features in the languages. They both have plenty of good options for implementing various types of security in various ways. What really matters in your case is how the machines communicate.
Can you somehow spy on the traffic that is going on between the device (PC / Android device) and the server and find out the user and the password of the database
Your clients (the PCs and Android devices) should not be connecting directly to your database. They should submit requests to your server, where you have much more control, and can authenticate clients and validate their data. The server then connects to the DB.
If the clients call the DB directly, not only are the credentials transmitted over the internet, but they must also be present locally on the client in some form. This means that someone could potentially crack your app and get access to them.
or even get the post request parameters that the app is sending?
Yes, these can be intercepted and read. Again, preventing this is a matter of how you implement the communication. Use the HTTPS protocol, which you can do in both C# and Java, and the content of your requests will be protected from being intercepted by third parties along the way.
When your traffic is noticed or intercepted it will be freely interpretatable to the reader. You can see an example of such traffic in the console window of your browser, or if you want to view the actual application traffic use a proxy (such as Fiddler2).
If you want to prevent your traffic from being read, you have to take measures to ensure authorization and access control. You can do this by encrypting the traffic with TLS/SSL. If you have web-endpoints you can often enable https trough the libraries configuration. You may need to pass it as a parameter to the code that builds your connection.
Furthermore, it is best practice not to divulge sensitive information in your application output. You will want to use strong passwords and refrain from storing or sending these in plaintext.
I would also advice you to break down the need for securing in smaller bits.
Example:
You are using a lot of different technologies. These all have best practices and guidelines related to security. Separate your applications from your networking/operational assets. Encrypting your communication is a measure in your application. Whereas your MySQL configuration works in a different way entirely, mostly trough configuration.
Why are you connecting directly to your DB from the Android/WPF apps?
If the MySQL DB is sitting on a secure server, perhaps wrap the database calls/services in RESTful APIs implemented in your PHP solution, then call the APIs from your client apps, this also saves you from writing SQL statements and DB specific tasks in multiple languages (Java/C#)
not knowing your situation makes it hard though...
I am developing an android application. It does http communication with a web server. As of now, the web server IP address is hard coded. In the future, if I change to a different server with some other IP address, then I have to change the code. I know that probably having domain name/static address will solve this issue. But since my native app is specific for small devices, I need to find alternative easy solution.
As a solution, I am thinking of storing the key/value pairs in an online store service. So the app can query the online service for server ip and use the configured value. If IP changes, then I need to do modification in online service and need not change the app code. Is there any service which suits to my purpose?
Also are there any other better solutions to my requirement?
You can use keyvalue.xyz. It is cloud key/value store service with REST API.
Disclaimer: I have developed the system.
You might be interested in kvdb.io. It's a key-value store like DynamoDB or Redis, but with a much simpler REST API that you can use from Curl, jQuery or anywhere you can make an HTTP request.
Disclaimer: I built the service.
Your requirements really sound like dynamic dns, but to be specific on the key/value store, what about having some object storage on Amazon S3?
Overkill? Or maybe you don't want to spend money?
The good thing is that you would get a REST interface to your objects, so it works quite easily from an application point of view (Amazon SDK).
Sounds like you might want to look at a dynamic DNS hosting service. The service would handle the DNS updates if your IP address should ever change. Read up on
DNS on wikipedia or this great explanation on godaddy.com
Hello
I have a cache server (written with Java+Lucene Framework) which keeps large amount of data and provides them according to request query.
It basically works like this:
On the startup, it connects DB and stores all tables to the RAM.
It listens for requests and provides the proper data as array lists (about 1000 - 20000 rows)
When a user visits to the web page, it connects to the cache server, requests, and show the server response.
I planned to run web and cache applications in different instances because of memory issues. Cache Server is as service and web is on Tomcat.
What is your suggestion about how the communication should be built between web side and cache server ?
I need to pass large amount of data with array lists from one instance to another. Should I think web services (xml communication), nio socket communication (maybe Apache MINA) or the solutions like CORBA ?
Thanks.
It really depends very much on considerations you have not specified.
What are the clients? for example, if your clients are javascript running AJAX, obviously something over HTTP is more useful than a proprietary UDP solution.
What network is it working on? Local networks behave differently than internet, and mobile internet is quite different than both.
How elaborate use can you make of caching? If you use HTTP you can have a rather good control (through HTTP headers) of both client cache and network caches, and a plethora of existing software that can make use of both.
There are many other considerations to be taken into account, and there are many existing implementations of systems matching the more-common needs. From your (not very detailed) description you gave, I would recommend having a look at Redis.