I am currently in the early stages of creating a web application (HTML5, JS, etc.) that uses a REST API on the backend (Java, specifically Jersey v1.18). The nature of the data that will be stored is highly sensitive, so security is something that I’ve started looking at, even though the application is only in the early stages. The eventual goal will be to have native mobile applications as well, and to potentially provide access to the data for external clients via the same API.
In my research thus far, I have identified a variety of authentication methods, including HTTP Basic, token-based, session cookie, OAuth, HMAC, etc. The key component here is that the REST API will be primarily accessed by users, rather than other applications or backends. Thus, having a “login/logout” equivalent is important, and this boils down to user level authentication.
So far, HMAC authentication looks to be the most promising, as we have no plans to integrate with any OAuth provider at this moment.
I have already read through dozens of SO posts, as well as articles such as:
http://www.thebuzzmedia.com/designing-a-secure-rest-api-without-oauth-authentication/
http://www.errant.me.uk/blog/2013/04/authenticating-restful-web-applications/ (note: this is clearly bad, as salting with the username is not recommended)
Ideally, HMAC seems like the way to go, but I have yet to see a recommended approach to handling the shared secret. The idea of using a resource to validate credentials, which then provides a token/nonce to be used with the HMAC scheme, seems to be an option, but I’m questioning the advantages over just using this token/nonce strictly as a token.
I know that HMAC authentication for a REST API has been discussed at length, but when used in conjunction with the authentication details that users have come to expect (username, email, password, etc.), is there any recommended approach that doesn’t require a pre-shared secret key?
This is primarily an opinion-based question, but I'll offer my two cents: just go for a session cookie.
If your primary audience is humans, and you don't need to integrate with third parties, don't bother with OAuth. Just make sure your API is only available over HTTPS, and issue a session token that the server can revoke after login. Strictly speaking it doesn't need to be a cookie; I've seen APIs that stash the token in HTML5 session storage and provide it in the Authorization header or as a query param.
If you have SSL set up properly, your users will get the expected padlock in the browser, and you'll be safe from anyone in between you and the client. If the client is compromised, you're screwed anyways. And since the client can't keep a secret, there aren't a lot of advantages to more complex HMAC schemes.
Related
In my app, I need to pass client_id and client_secret to make my API calls.
What is the most secure way of storing client_id and client_secret's on the android app?
I have read about SharedPreferences and I am sure they are not secure. Also read about Keystore but not sure if that is the right approach. Can you please suggest what would be the most secure way of storing such information?
Thanks
R
The best approach is to use Android's Keystore. It will handle the key generation, storage for you. Your app will be communicating through KeyChain API which offers the following advantages:
It performs all the cryptography for you
Hard to extract from device
Each app can only access their own keys (enforced by Keystore)
Securing the user's secrets in a keychain
Mobile OAuth Authorization flow
In my app, I need to pass client_id and client_secret to make my API calls.
It seems to me that you are not using the correct OAuth authorization flow in your mobile app because the one you are using now requires the client_secret. I think you may be trying to use the flow meant for m2m(machine to machine) authorization.
The correct flow to use for a mobile app is the Authorization Code Flow with Proof Key for Code Exchange (PKCE).
From auth0.com/docs:
The PKCE-enhanced Authorization Code Flow introduces a secret created by the calling application that can be verified by the authorization server; this secret is called the Code Verifier. Additionally, the calling app creates a transform value of the Code Verifier called the Code Challenge and sends this value over HTTPS to retrieve an Authorization Code. This way, a malicious attacker can only intercept the Authorization Code, and they cannot exchange it for a token without the Code Verifier.
You can read that this is the recommended approach in the OAuth 2.0 RFC8252 for Native Apps, that is about the best practices for mobile apps:
Abstract
OAuth 2.0 authorization requests from native apps should only be made
through external user-agents, primarily the user's browser. This
specification details the security and usability reasons why this is
the case and how native apps and authorization servers can implement
this best practice.
Status of This Memo
This memo documents an Internet Best Current Practice.
This document is a product of the Internet Engineering Task Force
(IETF). It represents the consensus of the IETF community. It has
received public review and has been approved for publication by the
Internet Engineering Steering Group (IESG). Further information on
BCPs is available in Section 2 of RFC 7841.
Information about the current status of this document, any errata,
and how to provide feedback on it may be obtained at
https://www.rfc-editor.org/info/rfc8252.
Securely storing secrets in the mobile app
What is the most secure way of storing client_id and client_secret's on the android app?
Also read about Keystore but not sure if that is the right approach. Can you please suggest what would be the most secure way of storing such information?
Yes, the Android Keystore is the correct way to go. You can use it from the Android Security Library to store your secrets but bear in mind that an attacker can use an instrumentation framework to hook at runtime into the code that uses the client_id and client_secret already decrypted and extract them for use outside of your mobile app. A popular instrumentation framework used for this propose is Frida:
Inject your own scripts into black-box processes. Hook any function, spy on crypto APIs or trace private application code, no source code needed. Edit, hit save, and instantly see the results. All without compilation steps or program restarts.
I invite you to read my answer to the question Store Client Certificate and key (.pem) in Android securely to see more details about using the Android Keystore with the security library, and the answer includes some code examples.
Do You Want To Go The Extra Mile?
In any response to a security question I always like to reference the excellent work from the OWASP foundation:
OWASP Mobile Security Project - Top 10 risks
The OWASP Mobile Security Project is a centralized resource intended to give developers and security teams the resources they need to build and maintain secure mobile applications. Through the project, our goal is to classify mobile security risks and provide developmental controls to reduce their impact or likelihood of exploitation.
OWASP - Mobile Security Testing Guide:
The Mobile Security Testing Guide (MSTG) is a comprehensive manual for mobile app security development, testing and reverse engineering.
I'm developing an budgeting application that uses a set of tokens and secret keys to access an external financial service, where a token-secret pair map to a single account. In this system, a user can have multiple accounts and therefore multiple sets of token-secret pairs. The token and secret can be used to access the transactions for an account, which means that the token-secret pairs should be securely stored (and be guarded against nefarious access or tampering). Although these pairs should be securely stored, the external financial service API requires the token and secret in plaintext.
What is a secure technique for storing these credentials at rest but providing the external service API with the original, plaintext credentials?
My application is a REST-based Spring Boot application written in Java 9+. Although there have been other answers I've seen on this topic, many of them are specific to Android and use Android security techniques (as are thus not applicable to my application). This application also uses Spring Data MonogDB to store other non-sensitive information, but if another technology is required for satisfying the above security requirements, I am open to suggestions.
This is not a problem that can be solved inside Java.
The token and secret can be used to access the transactions for an account, which means that the token-secret pairs should be securely stored (and be guarded against nefarious access or tampering).
The fundamental issue is who you are securing this against:
If you are trying to secure it against the people who manage the platform, it is pretty much unsolvable1.
If you are trying to secure it against "ordinary" (i.e. non-privileged) users, then you can either rely on ordinary file system security (plus standard service "hardening"), or you can use something like a Spring Vault or a Hardware Security Module if local file system security is insufficient2.
If you are trying to secure against a hacker who might be able to acquire the privilege of a full administrator, this is probably unsolvable too. (Though a hacker may need to be sophisticated ... )
Note that you can do things like saving the secrets in a Java keystore, but in order to do that the JVM needs the secret key to the keystore in order to get the secret. And where do you store that secret?
Aside:
... many of them are specific to Android and use Android security techniques (as are thus not applicable to my application).
Those techniques typically assume that the platform itself is secure, and / or that it hasn't been "rooted" by the user.
1 - So, if your goal is to embed some secrets in an app that you give to a client to use ... give up. If that problem was solvable, priracy of software, music, videos, etcetera would have a simple and reliable technological solution. Clearly ... it hasn't.
2 - For a lot of purposes, it isn't sufficient. However, the decision should be based on an assessment the risks, and balancing risks versus the cost / severity of the consequences of security failure.
I am searching for the best approach of authenticating users of mobile clients when accessing my RESTful API. For example, how approximately AirBnb uses it's auth module.
Should the authentication be different for RESTful and basic session-based resource, working with the same data?
I am not a mobile developer therefore, I am interested in what is the best way to provide authentication from server-side, so the mobile-platform developers could use it simply.
I googled for few approaches using OAuth, OAuth2, HTTPBasic authentication and still wonder how the mobile developers can use such API, how they will store this token (cookie is stored by browser in browser-oriented apps).
Could you please suggest me some links/code samples/techiques that you used in production or pet-projects or something?
An easy and manageable alternative to OAuth(2) for authentication is JWT.
You don't need additional infrastructure, the workflow and use is straightforward and there are ready to use libraries for all major languages already available.
Compared to HTTP Basic Authentication JWT is more flexible by transmitting additional information not just credentials, you can store the JWT token as JSON or you can use cookies, you don't need to store the credentials on client side and you don't transmit the credentials on every request.
Also based on JWT you can realize very easy a single sign on function. So if you need more than just a simple system user then you should definitely try JWT.
I have read a lot of about the way to provide acces to a REST API and I still cannot come with a decision what to use.
In my case I am writing a REST API that will be used by the users of the mobile application(android&iOS), thus I do not provide or require access from third parties and this makes me think that I don't have to use OAuth.
However I have considerations about how to provide access of one user's account from multiple devices and how to provide offline access.
Another consideration I have is how should I restrict the API access, for example if using API Tokens what are the best practices for expiration and renewal of the tokens?
You have several topics in your question:
What are the benefits of OAuth2 for an internal API exposed on the Internet?
How should I manage tokens?
How can a user gain access via multiple devices?
How can a user have offline access?
I discuss these questions below.
Oauth2
OAuth2 offers a standardized protocol for several authentication schemes of varying complexity. One of the most complex use cases is the 'Authorization Code Grant' flow which allows a resource owner (user) to grant specific access to a client application via an intermediary, the Authorization server. This is what happens when you 'login using google'. The advantage of using OAuth2 over a homebrew solution is that the protocol is clear to all parties and less likely to contain fundamental flaws. A drawback can be that the protocol is not that flexible so some custom scenario's might be hard to support within the boundaries of OAuth2. If you don't have the immediate need for any of the typical OAuth2 scenario's (or a stakeholder demanding use of OAuth2) then I suggest not starting off with it, but to implement a simple token scheme yourself.
Managing tokens
The most common way to manage API access is by using tokens. A token is generated when the user logs in, typically with username and password over HTTPS. The token is persisted on the server and must be supplied by the app in each request. This is similar to the session ID used in web applications which is automatically generated and handled in-memory by the application container on the server and passed via a cookie or request parameter. An API token is typically handled by the security layer of the application itself, persisted in the database and passed via the 'Authorization' header.
A token should have an expiration date. One should decide on the best interval for this and whether token renewal is automatic (each time the user accesses the API) or explicit (force the user to re-enter credentials after expiration). This depends on the type of application and the level of security required. Tokens can also be revoked manually on the server.
Multiple devices
Each token can be associated with a specific user and device to allow access on multiple devices. This means each device must be uniquely identified, typically with the IMEI code. This makes it easy to revoke all tokens for a specific device or user at once.
Offline access
The typical way to offer offline access is to cache relevant data on the device. For example the Google Maps app allows you to make specific regions of the map available offline. To avoid (too) stale data you could keep track of the token's expiration date and invalidate the cached data after this date. An issue to be aware of is the handling of offline edits by the user. These edits have to be processed when the device comes online again. When simultaneous edits on the same data are encountered a strategy is needed to resolve the conflict, e.g.:
one edit overrides the other depending on the type of edit or the role of the user
the last edit is ignored or offered for resolution to the last editor
some types of edits might be 'merged' automatically
etc.
Another nice and simple strategy is to disallow all edits whilst offline.
There are 2 things you want to protect / authenticate
That the client app is authorized to use the service
That the user is authorized to access personal data
App authentication
A mobile application is an untrusted client. Even if you gave nobody access to the app source you must expect that any kind of authorization secret or mechanism is unsafe and can come from a hacked app or other malicious tool that emulates the behaviour of your apps.
For authenticating the app, all you can do is to have a client id, but not a client secret. E.g.
http://service.com/rest?client_id=android
Reply method(String client_id) {
if (!client_id in ["andoid", "ios"])
return Unauthorized();
}
You can change that schema to something a little harder to guess but anything you do boils down to the same security level.
User authentication
Protecting user data is crucial and luckily possible. The key difference is that the secret is not statically hardcoded into the app, it is only known to the user.
One "easy" way to authenticate users is to use other accounts they have. Schemas like http://openid.net/connect/faq/ allow you to do exactly that.
You basically delegate the authentication to some other service. and get a (per service) unique user id which you can use in your code as key to all user data. An attacker can not forge this since your server can authenticate that the token is valid by asking another service. Looks roughly like
http://service.com/rest?client_id=android&user_token=aasjkbn9nah9z23&user_auth_service=facebook
Reply method(String client_id, user_token, user_auth_service) {
if (!client_id in ["andoid", "ios"])
return Unauthorized();
authenticated_user_id = user_auth_service.getUserIdOrFail(user_token);
accessDatabase(authenticated_user_id);
}
An attacker can still use your service from some evil app but there is no way to access accounts he has no access to anyways.
And if you hardcode access tokens into the app, you better don't expire them or make sure to handle that case specifically in the app somehow. There are always users with outdated app versions.
I would like to implement OAuth 2.0 authorization on my JAX-RS RESTful services.
After some researches, I've found Apache CXF to do that. However, I haven't found any examples about it and it's unclear for me. Where can I find some examples of JAX-RS with OAuth 2.0?
Disclaimer: This answer doesn't really provide a solution for securing a JAX-RS with OAuth 2.0. But it aims to give some insights to Mohasin Ali, who started a bounty on my question. Maybe, the solution I used can be useful for him.
Regarding the bounty:
The question is widely applicable to a large audience. A detailed canonical answer is required to address all the concerns.
After asking this question a while ago, I realized that OAuth 2.0 would be too complex for my requirements. Even Basic Authentication would be enough for my requirements. But I ended up using an authentication scheme based on JWT tokens signed on server side. I described my solution in this answer.
Apache CXF provides an implementation of OAuth 2.0. It may worth looking at it if you want to use OAuth for securing you API. Apache CXF also supports OAuth 1.0.
It doesn't matter the authentication method you decide to use, do it on the top of a HTTPS connection. You'll need a certificate for that. As a suggestion, have a look at Let's Encrypt. They claim to be a free, automated, and open Certificate Authority, currently sponsored by Mozilla, Akamai, Cisco, Chrome, Facebook and others.
Regarding the following situation, mentioned in the comments:
[...] a malicious user visits someone's computer, open the browser, see the access token and copies the access token to his own browser [...]
If a malicious user have physical access to a computer, HTTPS won't prevent this malicious user from stealing an authentication token from someone's computer. Actually, if it happens, I think you should have bigger concerns...
For an additional layer of security, you could consider storing the token along with the IP address of the user you issued the token for. For each request that hits your API, compare the IP of the incoming request with the IP of the user you issued the token for. If the IPs don't match, refuse the request.
If you go for JWT tokens, instead of storing the whole token, store only the JWT ID claim (jti). Just ensure this value is unique (java.util.UUID should be enough for generating the jti value).
For a completely stateless authentication (not storing the whole token neither storing token ID), you could store the IP address in a JWT token claim, but mind the token will be a few bytes longer.
Please see https://github.com/Talend/tesb-rt-se/tree/master/examples/cxf/jaxrs-oauth2 for one example, it has a collocated example (all endpoints in the same container) and more complex one with the endpoints distributed, with SAML SSO Web profile supporting SSO.