I'm trying to get my head around the OAuth2 Java library that Google provides.
I have everything I need to make the request to Google's token endpoint manually using Springs built-in WebClient. However, this is very verbose and feels like re-inventing the wheel. It got me thinking that there must be a way to get this data using the classes provided by the library. Right?
Currently I am using the com.google.auth.oauth2.UserAuthorizer class to build up a request for the exchange of information.
val userCredentials: UserCredentials = UserAuthorizer.newBuilder()
.setClientId(googleOauthConfig.clientId)
.setTokenStore(tokenStore)
.setScopes(googleOauthConfig.scopes)
.setTokenServerUri(URI.create("https://oauth2.googleapis.com/token"))
.setCallbackUri(redirectUri)
.build()
.getCredentialsFromCode(authorizationCode, redirectUri)
The internals of getCredentialsFromCode() parses the response and it contains all the tokens. Including the id_token but, it gets discarded when constructing the UserCredentials object further down.
return UserCredentials.newBuilder()
.setClientId(clientId.getClientId())
.setClientSecret(clientId.getClientSecret())
.setRefreshToken(refreshToken)
.setAccessToken(accessToken)
.setHttpTransportFactory(transportFactory)
.setTokenServerUri(tokenServerUri)
.build(); // no mention of id_token
Regardless, I want to get this value so I can know basic information about the user such as their name, birthday and email address from a single request.
There does exist a method called idTokenWithAudience() which returns a Google ID Token from the refresh token response. If I call this, I get a token back that doesn't contain all the data that was available in the identically named id_token mentioned earlier making it a no-go either.
You can use IdTokenCredentials to access the ID token like so:
var credentials = UserCredentials.newBuilder()
.setClientId("...")
.setClientSecret("...")
.setRefreshToken("...")
.build()
.createScoped("openid email");
var idToken = IdTokenCredentials
.newBuilder()
.setIdTokenProvider((IdTokenProvider)credentials)
.build();
idToken.refresh();
System.out.println(idToken.getIdToken().getTokenValue());
I have a server using Kotlin 1.5, JDK 11, http4k v4.12, and I've got the Twilio Java SDK v8.19, hosted using Google Cloud Run.
I've created a predicate using Twilio's Java SDK RequestValidator.
import com.twilio.security.RequestValidator
import mu.KotlinLogging
import org.http4k.core.Filter
import org.http4k.core.HttpHandler
import org.http4k.core.Method
import org.http4k.core.Response
import org.http4k.core.Status
import org.http4k.core.body.form
import org.http4k.core.queries
import org.http4k.core.then
import org.http4k.core.toParametersMap
import org.http4k.filter.RequestPredicate
import org.http4k.filter.ServerFilters
import org.http4k.lens.Header
private val twilioAuthHeaderLens = Header.optional("X-Twilio-Signature")
/** Twilio's helper [RequestValidator]. */
private val twilioValidator = RequestValidator("my-auth-token")
/**
* Use the Twilio helper validator, [RequestValidator]
*/
val twilioAuthPredicate: RequestPredicate = { request ->
when (val requestSignature: String? = twilioAuthHeaderLens(request)) {
null -> {
logger.debug { "Request has no Twilio request header valid" }
false
}
else -> {
val uri: String = request.uri.toString()
val paramMap: Map<String, String?> = request.form().toMap()
logger.info { "Validating request with uri: $uri, paramMap: $paramMap, signature: $requestSignature" }
val isTwilioSignatureValid = twilioValidator.validate(uri, paramMap, requestSignature)
logger.info { "Request Twilio valid: $isTwilioSignatureValid" }
isTwilioSignatureValid
}
}
}
This works using the example Twilio provide, as demonstrated with this Kotest unit test.
(the test and the example code are mismatched - but OperatorAuth is a class that applies the twilioAuthPredicate, and ApplicationProperties fetches the Twilio auth key from a .env file.)
test("demo https://www.twilio.com/docs/usage/security") {
val twilioApiKey = "12345"
val appProps = ApplicationProperties(
TWILIO_API_AUTH_TOKEN(twilioApiKey, TEST_ENV)
)
// system-under-test
val handler: HttpHandler = OperatorAuth(appProps).then { Response(OK) }
// construct a GET request: https://mycompany.com/myapp.php?foo=1&bar=2
val urlProto = "https"
val urlBase = "mycompany.com"
val requestSignature = "0/KCTR6DLpKmkAf8muzZqo1nDgQ="
val request = Request(Method.GET, "$urlProto://$urlBase/myapp.php")
.query("foo", "1")
.query("bar", "2")
.form("CallSid", "CA1234567890ABCDE")
.form("Caller", "+12349013030")
.form("Digits", "1234")
.form("From", "+12349013030")
.form("To", "+18005551212")
.header("X-Twilio-Signature", requestSignature)
.header("X-Forwarded-Proto", urlProto)
.header("Host", urlBase)
val response = handler(request)
response shouldHaveStatus OK
}
However, aside from this simple example, no other requests work, either when creating a unit test, or when live. All Twilio requests fail validation and my server returns 401. The information in the Twilio website is completely opaque. It's incredibly frustrating. It doesn't tell me how it's calculated the hash so I can't tell what is going wrong.
Warning 15003
Message Got HTTP 401 response to https://my-gcr-server.run.app/twilio
Here is an example test using real values gathered from a log (although I have edited the identifiers).
test("real request") {
val appProps = ApplicationProperties() // this loads the Twilio Auth Key from my environment variables
val handler: HttpHandler = OperatorAuth(appProps).then { Response(OK) }
// construct a GET request
val urlProto = "https"
val urlBase = "my-gcr-server.run.app"
val requestSignature = "GATG2313LSuCYRbPASD4axJ26XyTk="
val request = Request(Method.GET, "$urlProto://$urlBase/voicemail/transcript")
.query("ApplicationSid", "AP1234567890abcdefg")
.query("ApiVersion", "2010-04-01")
.query("Called", "")
.query("Caller", "client:Anonymous")
.query("CallStatus", "ringing")
.query("CallSid", "CA1234567890abcdefg")
.query("From", "client:Anonymous")
.query("To", "")
.query("Direction", "inbound")
.query("AccountSid", "AC1234567890abcdefg")
// note, changing these variables to be form parameters doesn't affect the result, Twilio's validator still says the request is invalid.
.header("X-Twilio-Signature", requestSignature)
.header("I-Twilio-Idempotency-Token", "337aaaa-1111-2222-3333-ffffb5333")
.header("Content-Type", "text/html")
.header("User-Agent: ", "TwilioProxy/1.1")
.header("X-Forwarded-Proto", urlProto)
.header("Host", urlBase)
val response = handler(request)
response shouldHaveStatus OK // this fails, Status: expected:<200 OK> but was:<401 Unauthorized>
}
Sometimes validation fails because of Google Cloud. I had previously hosted my server on Google Cloud Functions until I discovered there's an issue where GCF silently omits part of the URI https://github.com/GoogleCloudPlatform/functions-framework-java/issues/90
There is also an issue where if a request is 'modified', for example if I set a Twilio callback URL to include a query param, e.g. https://my-gcr-server.app.run/twilio/callback?type=recording, then the Twilio signature ignores this parameter, but when validating the auth it's impossible to know which parameters Twilio is ignoring. The same is true if the headers are altered.
Is there a working method of validating that a request originates from Twilio? Or an alternative validation solution?
Update
I've just found that Twilio's RequestValidator is really under-tested, there's only one example RequestValidatorTest
Twilio developer evangelist here.
The documentation describes how the signature is created and that may show up some differences in the way you are testing. On your server, the algorithm to check the signature is:
Take the full URL of the request URL you specify for your phone number or app, from the protocol (https...) through the end of the query string (everything after the ?).
If the request is a POST, sort all of the POST parameters alphabetically (using Unix-style case-sensitive sorting order).
Iterate through the sorted list of POST parameters, and append the variable name and value (with no delimiters) to the end of the URL string.
Sign the resulting string with HMAC-SHA1 using your AuthToken as the key (remember, your AuthToken's case matters!).
Base64 encode the resulting hash value.
Compare your hash to ours, submitted in the X-Twilio-Signature header. If they match, then you're good to go.
You are using GET requests, so you can discard steps 2 and 3.
There are some things I can see from this algorithm that might cause differences in the way you are testing the validator.
Your error from testing in real life used the URL https://my-gcr-server.run.app/twilio, but your test script from a real request uses https://my-gcr-server.run.app/voicemail/transcript. The URL matters in the generation of the signature.
Your test also adds query parameters to the request, but it's hard to know what the order of those params will be. The order of the query params in the URL should be the exact same as the URL Twilio made the request to.
On the other hand, if the original Twilio request was a POST request, then those parameters should be added as form parameters, as the algorithm takes the form parameters, sorts them and appends them to the URL, with no delimiters.
You said:
There is also an issue where if a request is 'modified', for example if I set a Twilio callback URL to include a query param, e.g. https://my-gcr-server.app.run/twilio/callback?type=recording, then the Twilio signature ignores this parameter, but when validating the auth it's impossible to know which parameters Twilio is ignoring. The same is true if the headers are altered.
This is not true, a query param is part of the URL, as I've said above. Twilio does not ignore parameters, it deals with them according to the algorithm described above. As for headers, aside from the X-Twilio-Signature which is used to test the signature against, they do not come into play.
Having said all that, I am not sure why a real life request would fail the validator as it should be handling all of the things I have discussed above. You can inspect the code used to validate a request and get a signature.
In your code:
val uri: String = request.uri.toString()
val paramMap: Map<String, String?> = request.form().toMap()
logger.info { "Validating request with uri: $uri, paramMap: $paramMap, signature: $requestSignature" }
val isTwilioSignatureValid = twilioValidator.validate(uri, paramMap, requestSignature)
logger.info { "Request Twilio valid: $isTwilioSignatureValid" }
isTwilioSignatureValid
Can you guarantee that the uri is indeed the original URL that Twilio made the request to, not a URL that has been parsed into parts and put back together with the query parameters in a different order? In a GET request, does request.form().toMap() return an empty Map?
Sorry this isn't a full answer, I'm not much of a Java/Kotlin developer. I'm hoping this gives you a good idea of what to look into though.
I was having trouble because I was testing with ngrok to route the request to my local server while developing. I had was I was running the algorithm (see above in philnash 's answer as per the Twilio docs )
However while I was setting the callback to the https ngrok enpoint which Twilio used to calculate the signature, the actual request that cam to me was the http endpoint, ngrok forwards the https to http on the free account.
so I was testing a http endpoint but Twilio was calculating against the https endpoint.
when I told Twilio to callback on the http endpoint there was no ngrok misdirection and the signature matched!
Also I notice from the "validate a request: and "get a signature" links above in philnash's answer that the code tried both with a port (eg 443 or 80) in the URL and without and accepts either signature as a match.
I am trying to complete the Instagram Oauth flow,
I currently have the authorization code which I'm to exchange for the access token. I am to make an x-www-form-urlencoded POST request to this endpoint
"https://api.instagram.com/oauth/access_token?"
This is what I've done so far.
String query = "https://api.instagram.com/oauth/access_token/?client_id=" + clientId +"&client_secret="+ clientSecret+ "&grant_type=authorization_code&redirect_uri="+ redirectUri + "&code=" + code
String response = new URL(query).getText()
A JSON string is expected as response.
Please Keep in mind that I'm a beginner.
I haven't read the Instagram documentation but based on your example code there's a couple of things to keep in mind:
you mentioned that you have to make a POST request, your example makes a GET request
never build a URL with untrusted parameter values. This basically means: always encode parameters, never trust them.
There are dozens of 3rd party HTTP Request libraries that give you flexibility and easier insight into aspects like timeouts and redirects. Java 11 has a built-in HTTP client that might ease this as well. But building on your code provided in your question using basic Java connection primitives this might work:
URL url = new URL("https://api.instagram.com/oauth/access_token/?client_id=${URLEncoder.encode(clientId, 'UTF-8')}&client_secret=${URLEncoder.encode(clientSecret, 'UTF-8')}&grant_type=authorization_code&redirect_uri=${URLEncoder.encode(redirectUri, 'UTF-8')}&code=${URLEncoder.encode(code, 'UTF-8')}")
def jsonString = ((HttpURLConnection) url.openConnection()).with {
setRequestMethod('POST')
setRequestProperty('Accept', 'application/json')
setDoInput(true)
connect()
if (getResponseCode() >= 400)
throw new Exception("Error code = ${getResponseCode()}")
inputStream.text
}
Every URL parameter is encoded so that any non-URL safe characters they contain are made safe, then we tell the connection that it will be a 'POST' and that we expect to get back json as input. inputStream.text is groovy code that takes an inputstream from the connection and reads all of the contents and then closes the stream. Since it is the last line of the with closure it is automatically returned as the value of the closure and assigned to the variable jsonString.
I am working on a project in which I need, not only to authenticate but also to have the real value of the token.
We have a Spring Boot application with oAuth2.0 Spring Security and the problem is that I am not able to find a method that gives me a valid token every time I call it.
At this moment, I have a post method raw coded in Java, but there must be a Spring Security implementation that does something like the following:
The first time that it is called, it asks for the token and stores it.
The following times checks if the token has expired and, just if it has expired, it asks for a new one.
Where could I find it?
EDIT
There are 2 different Spring Instances in my project: The Authorization server - which is a Cloud Foundry UAA server - and the resource server - which is the one that asks for the token and is coded by me.
The Authorization server uses AuthorizationServerTokenServices in JWT version and when the Resource server gets a token from there, I want it to be kept, not only decoded and used because I need to send it to another server.
Moreover, my application is not a web app, so there is no login page to log in on Facebook and I have to get the token using the Client Credentials Grant Type.
In my case, Single Sign-On is not possible because I have to use it not decoded.
This is my current implementation:
public String obtainAccessToken() throws ClientProtocolException, IOException {
HttpClient httpclient = HttpClients.createDefault();
String userPass64 = new String("User and password");
HttpPost httppost = new HttpPost("localhost:8080/uaa/oauth/token?grant_type=client_credentials");
httppost.setHeader("Content-Type", "application/x-www-form-urlencoded");
httppost.setHeader("Authorization", "Basic " + userPass64);
//Execute and get the response.
HttpResponse response = httpclient.execute(httppost);
String responseBody = EntityUtils.toString(response.getEntity());
ObjectMapper mapper = new ObjectMapper();
TokenMessage tokenMessage = mapper.readValue(responseBody, TokenMessage.class);
return tokenMessage.getAccess_token();
}
From what I have seen is that there are few different ways that Spring security can handle this.
The default way is to have AuthorizationServerTokenServices interface handle it. And with it you can have different ways of storing the token. For example JDBCTokenStore, InMemoryTokenStore and JwtTokenStore. More about this here : http://projects.spring.io/spring-security-oauth/docs/oauth2.html#managing-tokens
But since I do not know what kind of application you are creating, you could maybe develop a single sign on functionality and let Facebook, for example, handle the authentication Token. Quite good tutorial about that with Spring boot can be found here: https://spring.io/guides/tutorials/spring-boot-oauth2/
I have succeed using openID and OAuth separately, but I can't make them work together.
Am I doing something incorrect:
String userSuppliedString = "https://www.google.com/accounts/o8/id";
ConsumerManager manager = new ConsumerManager();
String returnToUrl = "http://example.com:8080/app-test-1.0-SNAPSHOT/GAuthorize";
List<DiscoveryInformation> discoveries = manager.discover(userSuppliedString);
DiscoveryInformation discovered = manager.associate(discoveries);
AuthRequest authReq = manager.authenticate(discovered, returnToUrl);
session.put("openID-discoveries", discovered);
FetchRequest fetch = FetchRequest.createFetchRequest();
fetch.addAttribute("email","http://schema.openid.net/contact/email",true);
fetch.addAttribute("oauth", "http://specs.openid.net/extensions/oauth/1.0",true);
fetch.addAttribute("consumer","example.com" ,true);
fetch.addAttribute("scope","http://www.google.com/calendar/feeds/" ,true);
authReq.addExtension(fetch);
destinationUrl = authReq.getDestinationUrl(true);
then destinationUrl is
https://www.google.com/accounts/o8/ud?openid.ns=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0&openid.claimed_id=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&openid.identity=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&openid.return_to=http%3A%2F%2Fexample.com%3A8080%2FgoogleTest%2Fauthorize&openid.realm=http%3A%2F%2Fexample.com%3A8080%2FgoogleTest%2Fauthorize&openid.assoc_handle=AMlYA9WVkS_oVNWtczp3zr3sS8lxR4DlnDS0fe-zMIhmepQsByLqvGnc8qeJwypiRQAuQvdw&openid.mode=checkid_setup&openid.ns.ext1=http%3A%2F%2Fopenid.net%2Fsrv%2Fax%2F1.0&openid.ext1.mode=fetch_request&openid.ext1.type.email=http%3A%2F%2Fschema.openid.net%2Fcontact%2Femail&openid.ext1.type.oauth=http%3A%2F%2Fspecs.openid.net%2Fextensions%2Foauth%2F1.0&openid.ext1.type.consumer=example.com&openid.ext1.type.scope=http%3A%2F%2Fwww.google.com%2Fcalendar%2Ffeeds%2F&openid.ext1.required=email%2Coauth%2Cconsumer%2Cscope"
but in the response from google request_token is missing
http://example.com:8080/googleTest/authorize?openid.ns=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0&openid.mode=id_res&openid.op_endpoint=https%3A%2F%2Fwww.google.com%2Faccounts%2Fo8%2Fud&openid.response_nonce=2011-11-29T17%3A38%3A39ZEU2iBVXr_zQG5Q&openid.return_to=http%3A%2F%2Fexample.com%3A8080%2FgoogleTest%2Fauthorize&openid.assoc_handle=AMlYA9WVkS_oVNWtczp3zr3sS8lxR4DlnDS0fe-zMIhmepQsByLqvGnc8qeJwypiRQAuQvdw&openid.signed=op_endpoint%2Cclaimed_id%2Cidentity%2Creturn_to%2Cresponse_nonce%2Cassoc_handle%2Cns.ext1%2Cext1.mode%2Cext1.type.email%2Cext1.value.email&openid.sig=5jUnS1jT16hIDCAjv%2BwAL1jopo6YHgfZ3nUUgFpeXlw%3D&openid.identity=https%3A%2F%2Fwww.google.com%2Faccounts%2Fo8%2Fid%3Fid%3DAItOawk8YPjBcnQrqXW8tzK3aFVop63E7q-JrCE&openid.claimed_id=https%3A%2F%2Fwww.google.com%2Faccounts%2Fo8%2Fid%3Fid%3DAItOawk8YPjBcnQrqXW8tzK3aFVop63E7q-JrCE&openid.ns.ext1=http%3A%2F%2Fopenid.net%2Fsrv%2Fax%2F1.0&openid.ext1.mode=fetch_response&openid.ext1.type.email=http%3A%2F%2Fschema.openid.net%2Fcontact%2Femail&openid.ext1.value.email=example%40gmail.com
why?
In the above code, you have added OAuth extension parameters with the Attribute Exchange extension parameters. But since OAuth and Attribute Exchange are different extensions, therefore you have to create a different extension message for OAuth parameters and then add it to Authentication request message.
But since there is no mechanism to add OAuth parameters to the Authentication message, therefore you'll have to create such a mechanism. You can get information about it in the following link
http://code.google.com/p/openid4java/wiki/ExtensionHowTo
You can then use the code provided in the following link to hard code this mechanism
http://code.google.com/p/openid4java/issues/detail?id=110&q=oauth