Develop Reference

Web app security architecture

I have a Java Spring driven REST API server connecting with PostgreSQL database and a Spring Web Server in Java that is serving content from the REST API to the client using JavaScript (now browsers, but in the future also mobile apps).
I've read a number of articles and topics how one can secure a REST API, but I haven't been able to make a final decision yet. I don't want to have Basic Authorization, because it doesn't make sense as I would need to store credentials in JavaScript that can be easily accessed and read by anyone entering the webpage and developer console. I'd like not to show any credentials to the end user so I can't keep them on the client's side.
I've read a lot about JWT and almost decided to implement it, but I've heard it has some drawbacks and haven't been so sure since then if it's the option I would like to choose. I know there is also oAuth 1.0 or oAuth 2.0 but I don't know if I want to have something this complicated. I would also like to store hashed user credentials in my own database in order not to be depended to any other credential providers like social media or Google.
Now I'm making another layer on my web server as a proxy hoping that it will allow me to authenticate user on this proxy level using Spring Security and having some kind or cookies or something to authenticate, but I'm not so sure if its doable this way and it increases the respond time, adds complexity and needs me to write controller methods for these endpoints. My architecture now is of the following:
Client (browser) -> Webserver -> REST API server -> db
I've also denied all external connections and allowed only localhost access to REST API on tomcat level so that I'd have to implement the security level only on the web server allowing free information transit between the webserver and REST API as it is not reachable anyway.
Web server and REST API are on the same server running as Tomcat instances.
I'm also not so sure if this kind of architecture will allow me to authenticate mobile app clients through the web server.
I would be very grateful for any piece of advice you would have for me in this matter. I'm not so experienced in security, so I'm a bit lost what I should do. Does this kind of architecture any sense or should I simply ask REST API directly from any type of clients, be it a webpage or a mobile app from different IPs and secure Rest API only? And if I want to secure some subpages of my webpage or parts of mobile app should that be an completely other layer?
Thank you for your help.

You have already gone through OAuth, JWT tokens etc. If you don't want to use them,then you can create your own token based authentication system.(say 'TokenHandler').
How this TokenHandler will work ?
TokenHandler will be like a gateway server i.e your every REST API request will route through this server application. So you will address your confusion of mobile and web application call with a authToken in header. Main responsibility of this server applciation is to accept the token and validate against the database where all token details are maintain. This DB will have information regarding timestamp when token was last used to validate, to decide your validation rule .
How Token will get generated ?
Token can be any random 64 digit alphanumeric string and will be generated and updated in DB during every single login activity. Login webservice returns this token in response body.
What can be rules for validation ?
It can be depending on your business logic. I preferred to keep active session window of 15 mins. Means if you access the webservice you will get active window of 15 more minutes. If you didn't access any service for 15 consecutive minutes then from 16th Minute you will need to login again to access further calls. This part can change according to requirements.
How client side will handle this ?
Client side will store this token and pass this token with every request call. Token Handler will validate and redirects it's request to the application server . So your one token handler can be used to server authentication of multiple applcation servers. This you can achieve by application end point identifer.
I will like to discuss further if you have any questions or suggestions .

Use API Gateway Architecture Pattern for your use case -
http://microservices.io/patterns/apigateway.html .
The API Gateway (The webserver in your question) can act as single entry point for all desktop/mobile clients. You can use either session cookies or jwts to authenticate clients at gateway.
With respect to authentication between gateway and micro services, and between micro services, i would suggest mutual ssl - https://www.codeproject.com/Articles/326574/An-Introduction-to-Mutual-SSL-Authentication. If you are using spring boot, this might help - http://www.opencodez.com/java/implement-2-way-authentication-using-ssl.htm
The problem with IP white-listing approach is that - it's not well suitable for cloud architecture as the IPs might change with every server reboot. Even if you are using dedicated IPs, you should be careful to secure the communication with SSL/TLS, else attackers can easily sniff your traffic.

Exposing a web site through web services

I know what I am asking is somehow weird. There is a web application (which we don't have access to its source code), and we want to expose a few of its features as web services.
I was thinking to use something like Selenium WebDriver, so I simulate web clicks on the application according to the web service request.
I want to know whether this is a better solution or pattern to do this.
I shall mention that the application is written using Java, Spring MVC (it is not SPA) and Spring Security. And there is a CAS server providing SSO.

There are multiple ways to implement it. In my opinion Selenium/PhantomJS is not the best option as if the web is properly designed, you can interact with it only using the provided HTML or even some API rather than needing all the CSS, and execute the javascript async requests. As your page is not SPA it's quite likely that an "API" already exists in form of GET/POST requests and you might be lucky enough that there's no CSRF protection.
First of all, you need to solve the authentication against the CAS. There are multiple types of authentication in oAuth, but you should get an API token that enables you access to the application. This token should be added in form of HTTP Header or Cookie in every single request. Ideally this token shouldn't expire, otherwise you'll need to implement a re-authentication logic in your app.
Once the authentication part is resolved, you'll need quite a lot of patience, open the target website with the web inspector of your preferred web browser and go to the Network panel and execute the actions that you want to run programmatically. There you'll find your request with all the headers and content and the response.
That's what you need to code. There are plenty of libraries to achieve that in Java. You can have a look at Jsop if you need to parse HTML, but to run plain GET/POST requests, go for RestTemplate (in Spring) or JAX-RS/Jersey 2 Client.
You might consider implementing a cache layer to increase performance if the result of the query is maintained over the time, or you can assume that in, let's say 5 minutes, the response will be the same to the same query.
You can create your app in your favourite language/framework. I'd recommend to start with SpringBoot + MVC + DevTools. That'd contain all you need + Jsoup if you need to parse some HTML. Later on you can add the cache provider if needed.

We do something similar to access web banking on behalf of a user, scrape his account data and obtain a credit score. In most cases, we have managed to reverse-engineer mobile apps and sniff traffic to use undocumented APIs. In others, we have to fall back to web scraping.
You can have two other types of applications to scrape:
Data is essentially the same for any user, like product listings in Amazon
Data is specific to each user, like in a banking app.
In the firs case, you could have your scraper running and populating a local database and use your local data to provide the web service. In the later case, you cannot do that and you need to scrape the site on user's request.
I understand from your explanation that you are in this later case.
When web scraping you can find really difficult web apps:
Some may require you to send data from previous requests to the next
Others render most data on the client with JavaScript
If any of these two is your case, Selenium will make your implementation easier though not performant.
Implementing the first without selenium will require you to do lots of trial an error to get the thing working because you will be simulating the requests and you will need to know what data is expected from the client. Whereas if you use selenium you will be executing the same interactions that you do with the browser and hence sending the expected data.
Implementing the second case requires your scraper to support JavaScript. AFAIK best support is provided by selenium. HtmlUnit claims to provide fair support, and I think JSoup provides no support to JavaScript.
Finally, if your solution takes too much time you can mitigate the problem providing your web service with a notification mechanism, similar to Webhooks or Resthooks:
A client of your web service would make a request for data providing a URI they would like to get notified when the results are ready.
Your service would respond immediatly with an id of the request and start scraping the necessary info in the background.
If you use skinny payload model, when the scraping is done, you store the response in your data store with an id identifying the original request. This response will be exposed as a resource.
You would execute an HTTPPOST on the URI provided by the client. In the body of the request you would add the URI of the response resource.
The client can now GET the response resource and because the request and response have the same id, the client can correlate both.

Selenium isn't a best way to consume webservices. Selenium is preferably an automation tool largely used for testing the applications.
Assuming the services are already developed, the first thing we need to do is authenticate user request.
This can be done by adding a HttpHeader with key as "Authorization" and value as "Basic "+ Base64Encode(username+":"+password)
If the user is valid (Users login credentials match with credentials in server) then generate a unique token, store the token in server by mapping with the user Id and
set the same token in the response header or create a cookie containing token.
By doing this we can avoid validating credentials for the following requests form the same user by just looking for the token in the response header or cookie.
If the services are designed to chcek login every time the "Authorization" header needs to be set in request every time when the request is made.

I think it is a lot of overhead using a webdriver but it depends on what you really want to achieve. With the info you provided I would rather go with a restTemplate implementation sending the appropriate http messages to the existing webapp, wrap it with a nice #service layer and build your web service (rest or soap) on top of it.
The authentication is a matter of configuration, you can pack this in a microservice with #EnableOAuth2Sso and your restTemplate bean, thanks to spring boot, will handle the underlining auth part for you.

May be overkill..... But RPA? http://windowsitpro.com/scripting/review-automation-anywhere-enterprise

Secure Rest-Service before user authentification

I have a web application that provides several rest services (Jersey). Most of the endpoints are secured by BASIC authentification. Further more I use SSL for transport and demand POSTs for every call.
The clients/consumers are android apps.
So far so good. The only service that seems to be vulnerable is the registration. It's the 'first' service to call and a user does not exist yet. So I cannot use OAuth, etc. I also have to keep the endpoint easy accessible to enable the user to regster.
How do I secure this service, so it's not spammed by a bot flooding my database?

How about these?
Use a registration link with a token in the request parameter. Ensure that the tokens expire after sometime. You could create a token endpoint url as well for a client to get a valid token.
Use a custom header or a dynamic custom header in your request. Additionally, you could check for a dynamic custom header to validate the request's authenticity.
Use registration confirmation workflows, such as an email / text verification as soon the registration is done. Run a process every day to delete any user accounts, which are not validated in say x days.
I do not think you can really secure the registration URL in a HTTP way. IMHO, anyone who has the registration url can be a right guy trying to register. So if you ask me, option 3 is better than others.

OAuth + spring security for internal REST communication

Setup
We're developing a distributed application with Java and Spring where our existing client front end (complete with its own authentication, database, accounts, etc.) uses REST calls to access our new server for additional services. We want to protect these resources with Oauth.
Access should be restricted by role or account. However we don't want the user on the client side to have to worry about any additional authentication apart from the already existing account. At the same time we need to provide a means for third party applications to access some resources from the outside after going through some kind of registration against the server (which is why we're distributing in the first place).
So we have set up spring security on the server side to provide accounts that should be used to restrict access to the resources. The user should log in on the client side and then be able to access only those server resources assigned to him. We have some kind of registration process that sets up the user on the client side to be able to access the server services so any account setup I think should be done there.
So the questions are
How can I enable the client side to obtain an access token for the protected resources without the user having to log in to his server-side account?
And how do I setup the server side account without needing any user input?
My thoughts
This won't do
I'm thinking I'll have to either tell the client about a new account created on the server side for that user (but then, how would I choose and communicate a password?) or synchronize the client side account to the server, and use those credentials to authenticate the client against the server and generate access tokens. But how save can that be? Also the server has a much higher security (one way encrypted, salted passwords) on its accounts and I don't really want to compromise this by using the less save client accounts.
Maybe this will?
Maybe the way to go will be to tell the server about the client account during the first authentication, create an account on the server side, store the generated token on the client side and then authenticate the client against the server with that token for each subsequent request..? Will the server be able to log-in the client using its server-side account via that token for each request?
I'd need a special resource for that initial (2-legged?) handshake that can only be accessed from the client server, right?
Also:
Which would be better suited for the task, OAuth 1 or 2?
I'm hoping someone understands my problem and can help me sort through possible missunderstandings and knowledge gaps (I'm reading through Oauth and spring security documentations right now, so I'll update if I come up with a clearer picture and thus clearer questions of what to do)
Thanks for any help!

So our current status is to use OAuth2 mostly for reasons of simplicity. We're also sure that the flaws it might have concerning security we can cover ourselves as needed and they will most likely be addressed in the future by the implementation vendors or the IETF.
To handle the communication between REST server and REST client (both in our control) we use the formerly known as 2-legged authentication, now client credentials grant. I've asked a few questions on SO about that including
our current spring-security context setup
the client credentials flow in particular
the use of long lived tokens versus reauthentication
and how to limit REST access by HTTP method
Concerning the use of client based user accounts for authentication against the server we didn't get any further.
For now we authenticate the user against our old client web application as before and then authenticate the client against the server 2-legged. In theory this will allow any user to access any resource using the client accesstoken but for now that's okay for us so we will not investigate further down that road.
Still, should anyone have a good idea on how this might be solved we'll pick it up, just to tighten security further. So, I'll leave this question open.
My thoughts currently are along the line of registering a new client ID for each user on the authentication server with a generated secret and then synchronize those back to the client server and use those client_id / secret combinations to access resources for a user represented by the generated client_id in a client credentials flow.
For our latest application we'll store accounts on the REST server (authentication provider) and have the user login against that server and then use the token to access the REST resources as intended by the spec.

JAX-WS, Authentication and Authorization - How to?

What is the best way to do authentication and authorization in web services?
I am developing a set of web services, requiring role based access control.
Using metro - SOAP, simple java without EJBs.
I want to Authenticate the user just one time, using username and
password, to be matched against a data base. In the subsequent calls.
I would like to use some kind of session management. Could be some
session id, retrieved to the client at login, to be presented in all
calls.
So Far:
Read authentication using a database - but I want application level validation;
Read application authentication with jax-ws - but i don't want to do the authentication mechanism every time;
I think I can use a SOAP Handler, to intercept all the messages, and do the authorization control in the hander, using some session identifier token, that comes with the message, that can be matched against an identifier saved in the data base, in the login web method.
EDIT:
I still have some questions:
How to know the name of the web method being called?
What kind of token should I use?
How to pass this token between calls?
EDIT 2
Because of #ag112 answer:
I'm using Glassfish.
I use WS-Policy and WS-Security to encrypt and sign the messages. Using Mutual Certificate Authentication. I would like to complement this message level security between applications, with the authentication and authorization for the users also in message level.
I am just developing the services, and I don't know almost nothing the clients, just that they could be created in different languages.
At this point I think the most important thing is to do what ever I need to do to authenticate and authentication the users, I the most easy way to be implemented for the client applications.

#Luis: Here are my inputs.
Well exact solution for your problem depends upon kind of web service clients you expect, do you have control over web service client system, your app server etc.....but assuming you don't have any control over web service client, for you it is just a SOAP message over HTTP transport, here is probable solution.
You can of course performs session management & authentication at message level or transport level. It means either you can have session token and auth token information in SOAP message or you can use standard HTTP Session and HTTP authentication mechanism.
Of course transport level solution is much simpler and industry wide standard in case if transport layer is HTTP. For message level, ws specifications like ws-security can be used.
Your each web service request is simple HTTP GET/POST identified by a unique HTTP URI.
Typically in jax-ws metro environment, WSServlet is one which entry servlet for any web service call and which eventually delegates the call to right service provider implementation class. Since you application is going to be deployed in web server, you can exploit all session and authentication facilities provided by J2ee web container.
Since you are looking for role-based access control, I would use standard <web-resource-collection> in web.xml to specify which role you would like to have in case of particular HTTP URI. You can use standard JAAS login module which can do authentication and populates the JAAS subject with role. If user name/password are provided in SOAP XML, JAAS login module can also search/parse SOAP XML to retrieve those information. JAAS/app server will automatically create auth token and store it as cookie so that each subsequent request need not to go through authentication process again. This is all J2ee standard. You can find plenty of help on internet on this. Please let me know your app server so that I can provide you additional details.
If you still want to use SOAP message level session management, authentication & authorization process, then to provide you more details, may I know more details about your client side.
EDIT1:
Well based on your further inputs, here is my more thoughts:
Message security namely encryption and signature needs to happen each message travels between server and client. where as message authentication- you intend to do once and give a session token/auth token to client for subsequent calls.
Question still remains: if you put a unique session Identifier in SOAP response of first time authentication, do you expect client to parse SOAP response XML and ensure that client should send you session identifier each time in subsequent SOAP requests.
OR
You want to keep session management transparent to client and for client it needs to send username/password token first time and subsequent calls need not require any username/password token. In this case you would need to rely on transport based session management for e.g. HTTP cookies
Now what is the best for you depends upon your use case. Can you tell me what is expected use case flow? how another system (web service client) makes more than one service call to your system? Is another system user driven/some background process? What is exact need that you want only first service call to go through authentication process not subsequent calls?
PS: Glassfish server provides a way of configuring message authentication provider which automatically enables/disables message level authentication.
EDIT2:
I understand you do not want to store user credentials in client app and web service server need those user credentials.
OAuth is open standard protocol which allows site A to access user's private data on site B. Ultimate idea is site A gets auth token which has specific expiry time. So Token containing encrypted from of user credentials or jsession id help you avoid need of re-authentication. You only need to decide where you want to keep token at client app side
You can keep token as cookie if transport is HTTP protocol.
Having said that ofcourse passing user credentials each time seems bit easier and straight forward.

You can also go for OpenEJB.
It used JAAS with WS-Security.
I hope the link is useful.

After all the help, I create this answer to simplify, and summarize all the ideas that was discussed.
The questions has 2 requisites:
Message level security;
One time authentication.
With ag112 help, this is hard to do, or to elegant in any way. So here are to conclusions:
For message level security send the user
credentials every time (place it in SOAP header);
For one time authentication use transport level security, and do a
session management.
I prefer the first one, because the message level was the biggest requisite.

As had no answers, following #unhillbilly advise, I answer my own question, with the progress so far:
How to know the name of the web method
being called;
Using a SOAP handler, finding the name of the first element in the body.
What kind of token should I use;
I decide to use a 128 bits token, representing each session. The Webservices, continue to be session-less, the key is just for authorizations purposes.
How to pass this token between calls.
For the login web method the result has the token, in each subsequent calls, the token is a parameter.
is there a better answer?