Develop Reference

Plaid Quickstart Issues (Java)

I am trying to implement Plaid using the sample code provided on the Java Quickstart [sandbox] and am getting issues when I show the Plaid Dialog (javascript). I am able to successfully get a link_token, but I'm never able to show the dialog. It spins for a brief second, then shows me:
oauth uri does not contain a valid oauth_state_id query parameter. Request ID: DBoT92FCo8AORay
I have tried this with an empty redirectUri, as well as "http://localhost:8080/plaid_test.html", which is registered in my developer account.
I am a bit stuck and hoping someone can direct me in the right direction. I've tested with both versions 9.10.0 and the latest (11.9.0).
Curiously, I am able to get the Java Quickstart working directly, but ONLY if I leave the .env PLAID_REDIRECT_URI blank. If I put localhost in there, it fails when trying to get the link token.
Does anyone have any suggestions on how to overcome this setup issue?
Thank you!

I got this error (oauth uri does not contain a valid oauth_state_id query parameter) while creating a new test application in Plaid's Sandbox environment.
Important note: My application does not use OAuth.
The problem turned out to be, in my configuration parameters being passed to usePlaidLink, I was including a receivedRedirectUri key-value pair. Removing that key-value pair entirely resolved the issue for me.
In other words, my React component looked something like:
import { usePlaidLink } from 'react-plaid-link';
function PlaidLink(props) {
const onSuccess = React.useCallback((public_token, metadata) => {
// ...
});
const config = {
token: props.linkToken,
receivedRedirectUri: window.location.href,
onSuccess,
};
const { open, ready } = usePlaidLink(config);
// ...
}
Removing the line with the receivedRedirectUri was the solution for me, getting me past the oauth uri does not contain a valid oauth_state_id query parameter error, and getting the Plaid Link UI to appear in my app successfully.
This Plaid article, which has a number of mentions of "OAuth state ID" (as mentioned in the error message), helped point me toward this solution.

The issue may be the location you are trying to use -- unless you have manually modified the ports or other code used by the Quickstart, you should use http://localhost:3000/ as the PLAID_REDIRECT_URI (make sure to add this to your Dashboard as an allowed redirect URI). When I tried this just now on the Java quickstart (non-Docker version) it worked fine.

How to write a Java code to read fields from a website that requires login and uses POST request?

Need some help with fetching some data from a website.
Previously , we had following code in our application and it used to fetch the required data. We just used to read the required fields by forming a URL by passing username , password and search parameter (DEA number). The same URL (with parameters ) could also be hit from browser directly to see the results. It was a simple GET request:
{URL url = new URL(
"http://www.deanumber.com/Websvc/deaWebsvc.asmx/GetQuery?UserName="+getUsername()+"&Password="+getPassword()+"&DEA="
+ deaNumber
+ "&BAC=&BASC=&ExpirationDate=&Company=&Zip=&State=&PI=&MaxRows=");
Document document = parser.parse(url.toExternalForm());
// Ask the document for a list of all <sect1> tags it contains
NodeList sections = document.getElementsByTagName("DEA");
//Followed by a loop code to get each element by using sections.item(index).getFirstChild() etc.
}
Now, the website URL has got changed to following:
https://www.deanumber.com/RelId/33637/ISvars/default/Home.htm
I am able to login to the URL with credentials , go to the search page , enter the DEA number and search. The login page comes as a pop-up once I click 'Login' link on home page. Also, the final result comes as a pop-up. This is a POST request so I am unable to form the complete URL which I could use in my code.
I am not an expert in Web Services , but I think I need a web service URL like the one mentioned in the code above. Not sure how to get that !! Even if I get the URL , I am not sure how to perform the login through Java code and search the DEA number.
Also, it would be great if I could validate the URL manually before using in Java. Let me know if there is any way.
Or, in case there is any alternate approach in Java; kindly suggest.
Thanks in advance.

First of all, the previous approach provided by the website was completely wrong and insecure, because it passes the username and password as querystring parameters in plain text. I think, they would have realized this thing and changed their way of authentication.
Also, it looks like that they have restricted the direct URL based requests from the client applications like yours. For such requests from clients, they have published the web services. Check this link. They also have mentioned the rates for web service request counts.
So, you may need to open a formal communication channel to get authentication and other details to access their web services for this purpose. Depends on what they use for web service client authentication, you may code your client to access the web services.
I hope this helps.

Android, AccountManager and OAuth

I'm sure this is basic and I'm missing something. I've read through other answers on SO, I've googled, I've read resources and I just can't wrap my head around what I need to do.
I'm trying to figure out how to write an app that connects to Twitch's API, specifically how to authenticate with Twitch's api. Their documentation is here: https://github.com/justintv/Twitch-API/blob/master/authentication.md
I've created an app and stored my keys.
Now comes the part where I want my user to click a button which launches the authentication on their website. From what I can tell I do this by using an AccountManager. Except... I can't figure out what I'm supposed to do.
Here's the excerpt I've found online:
AccountManager am = AccountManager.get(this);
Bundle options = new Bundle();
am.getAuthToken(
myAccount_, // Account retrieved using getAccountsByType()
"Manage your tasks", // Auth scope
options, // Authenticator-specific options
this, // Your activity
new OnTokenAcquired(), // Callback called when a token is successfully acquired
new Handler(new OnError())); // Callback called if an error occurs
According to twitch's documentation I want to send the user to:
https://api.twitch.tv/kraken/oauth2/authorize
?response_type=code
&client_id=[your client ID]
&redirect_uri=[your registered redirect URI]
&scope=[space separated list of scopes]
&state=[your provided unique token]
And I simply have no idea how these two things need to be combined.

Firstly, I recommend to read the OAuth2 RFC. This should cover everything you need to know.
The AccountManager code snippet won't help you much unless there already is an app that provides authentication for Twitch. If that's not the case you either need to use an existing OAuth2 library or implement your own.
You could write your own AccountAuthenticator but that's a different challenge (and you still need some kind of OAuth2 client).
Doing it yourself is not that hard, see below.
Steps to implement it yourself
Twitch recommends to use the "Implicit Grant Flow" for mobile apps. That's what I'm going to describe below.
1. Get a client ID
Register your app as outlined in Developer Setup to get a client ID
As redirect URI you can use something like https://localhost:12398/, the actual port doesn't really matter.
2. Build the authentication URL
In your client app you need to construct the authentication URL like so:
https://api.twitch.tv/kraken/oauth2/authorize?
response_type=token&
client_id=[your client ID]&
redirect_uri=[your registered redirect URI]&
scope=[space separated list of scopes]
Apparently [your client ID] should be replaced by the client ID you've received from Twitch, same goes for [your registered redirect URI] (that's the URL above, i.e. https://localhost:12398/). [space separated list of scopes] is the list of scopes (i.e. features your want to access), see Scopes. Make sure you URL-encode the parameter values properly.
Assuming your client ID is 123456 and the scopes you need are user_read and channel_read your URL would look like this:
https://api.twitch.tv/kraken/oauth2/authorize?
response_type=token&
client_id=123456&
redirect_uri=https%3A%2F%2Flocalhost%3A12398%2F&
scope=user_read%20channel_read
Note that you should also pass a state parameter, just use a randomly generated value. You can also append the (non-standard) force_verify parameter to make sure the user actually needs to log in each time (instead of continuing a previous session), but I think you can achieve the same by clearing the cookie store (given that you open the URL in a webview in the context of your app) before you open the login page.
With a random state the URL would look like this:
https://api.twitch.tv/kraken/oauth2/authorize?
response_type=token&
client_id=123456&
redirect_uri=https%3A%2F%2Flocalhost%3A12398%2F&
scope=user_read%20channel_read&
state=82hdknaizuVBfd9847guHUIhndzhuehnb
Again, make sure the state value is properly URL encoded.
3. Open the authentication URL
Ideally you just open the URL in a WebView inside of your app. In that case you need to intercept all request to load a new URL using WebViewClient.shouldOverrideUrlLoading
Once the client is redirected to your redirect URL you can close the webview and continue with step 4.
Theoretically it's possible to utilize the default browser to do the authentication, but I would have security concerns since an external app could learn about your client ID and the access token.
4. Extract the access token
The actual URL you get redirected to in step #3 will have the form:
https://[your registered redirect URI]/#access_token=[an access token]&scope=[authorized scopes]
or to pick up the example
https://localhost:12398/#access_token=xxx&scope=user_read%20channel_read
Where xxx is the actual access token.
If you passed a state it will be present like so:
https://localhost:12398/#access_token=xxx&scope=user_read%20channel_read&state=82hdknaizuVBfd9847guHUIhndzhuehnb
All you have to do now is to parse the (URL encoded) access token, scope and state. Compare the scopes and state to the ones that you actually sent. If they match you can start using the access_token to authenticate.
Note According to the OAuth2 RFC, the response URL MUST also contain a token_type and it SHOULD contain an expires_in duration in seconds.
Once you received the access token you can use it to authenticate as described here.
Access tokens issued by the Implicit Grant Flow usually expire after a certain time and the user needs to authenticate again. The Twitch documentation doesn't mention any expiration time, so it's possible that the token is valid forever. So make sure your app doesn't store it or store it in a secure way (like using Android's key store provider to generate and store a key to encrypt the access token).
If the implicitly issued access token expires you could consider using the "Authorization Code Flow". That's quite similar but it contains an additional step to receive the access token and a "refresh token" that can be used to renew the access token. I leave it up to you to figure out how that works.

PUT file to S3 with presigned URL

I've been playing with Amazon S3 presigned URLs all night attempting to PUT a file. I generate the presigned URL in java code.
AWSCredentials credentials = new BasicAWSCredentials( accessKey, secretKey );
client = new AmazonS3Client( credentials );
GeneratePresignedUrlRequest request = new GeneratePresignedUrlRequest( bucketName, "myfilename", HttpMethod.PUT);
request.setExpiration( new Date( System.currentTimeMillis() + (120 * 60 * 1000) ));
return client.generatePresignedUrl( request ).toString();
I then want to use the generated, presigned URL to PUT a file using curl.
curl -v -H "content-type:image/jpg" -T mypicture.jpg https://mybucket.s3.amazonaws.com/myfilename?Expires=1334126943&AWSAccessKeyId=<accessKey>&Signature=<generatedSignature>
I assumed that, like a GET, this would work on a bucket which is not public (that's the point of presigned, right?) Well, I got access denied on every attempt. Finally out of frustration I changed the permission of the bucket to allow EVERYONE to write. Of course, then the presigned URL worked. I quickly removed the EVERYONE permission from the bucket. Now, I don't have permission to delete the item that was uploaded into my bucket by my own self-pre-signed URL. I see now that I probably should have put a x-amz-acl header on what I uploaded. I suspect I'll create several more undelete-able objects before I get that right.
This leads to a few questions:
How can I upload with curl using PUT and a generated presigned URL?
How can I delete the uploaded file and the bucket I created to test it with?
The end goal is that a mobile phone will use this presigned URL to PUT images. I'm trying to get it going in curl as a proof of concept.
Update: I asked a question on the amazon forums. If an answer is provided there I'll put it as an answer here.

This is indeed a bit puzzling, I consider it to be a bug in the AWS SDK for Java (see below) - but first and foremost, the following curl command will upload your file as such (assuming an updated pre-signed URL of course):
curl -v -T mypicture.jpg https://mybucket.s3.amazonaws.com/myfilename?Expires=1334126943&AWSAccessKeyId=<accessKey>&Signature=<generatedSignature>
That is, I've excluded the Content type header, which yields application/octet-stream (or binary/octet-stream) as a result, which is obviously not desired; thus, further digging had been order.
Background / Analysis
Pre-signed URLs for PUT (and DELETE as well as HEAD) requests to Amazon S3 are known to work in principle, not the least evidenced in related questions on this site (see e.g. my answer to Upload to s3 with curl using pre-signed URL (getting 403)).
The facilitated Query String Request Authentication Alternative is documented to use the following pseudo-grammar that illustrates the query string request authentication method:
StringToSign = HTTP-VERB + "\n" +
Content-MD5 + "\n" +
Content-Type + "\n" +
Expires + "\n" +
CanonicalizedAmzHeaders +
CanonicalizedResource;
It does include the Content-Type header, and (as you already discovered) this has been the missing piece in some documented cases, see e.g. the AWS team response to GetPreSignedURL with PUT request, yielding a working pre-signed URL once added.
This is easy to achieve with the AWS SDK for .NET indeed, which provides the convenience method GetPreSignedUrlRequest.WithContentType to do just that:
Sets the ContentType property for this request. This property defaults
to "binary/octet-stream", but if you require something else you can
set this property.
Accordingly, extending the respective sample Upload an Object Using Pre-Signed URL - AWS SDK for .NET as follows yields a working pre-signed URL with content type, that can be uploaded via curl as expected (i.e. exactly as you attempted to):
// ...
GetPreSignedUrlRequest request = new GetPreSignedUrlRequest();
// ...
request.WithContentType("image/jpg");
// ...
Now, one would like to extend the semantically identical sample Upload an Object Using Pre-Signed URL - AWS SDK for Java in a similar fashion, but (as you've discovered already as well), there is no dedicated method to achieve this. This might just be a lacking convenience method though and could be achievable via addRequestParameter() or setResponseHeaders() eventually, e.g.:
// ...
request.setExpiration( new Date( System.currentTimeMillis() + (120 * 60 * 1000) ));
request.addRequestParameter("content-type", "image/jpg");
return client.generatePresignedUrl( request ).toString();
// ...
However, both method's documentation suggests other purposes, and it doesn't work indeed, i.e. they always yield the identical signature, no matter which content type is set like so (if any).
Debugging further into the SDKs reveals, that both provide a semantically similar core method to calculate the query string authentication according to the pseudo-grammar referenced above, see buildSigningString() for .NET and makeS3CanonicalString() for Java.
But the respective code in the Java version to Add all interesting headers to a list, then sort them, where "Interesting" is defined as Content-MD5, Content-Type, Date, and x-amz- is never executed in fact, because there is indeed no method to provide these headers somehow, which are only available for class DefaultRequest and not class GeneratePresignedUrlRequest used to initialize the former, which is used as input for calculating the signature in turn, see protected method createRequest().
Interestingly/Notably, the two methods to calculate the query string authentication in .NET vs. Java compose their input from an almost inverse combination of header vs. parameter sources on the call stack, which could hint on the cause of the Java bug, but obviously that might as well be just difficult to decipher, i.e. the internal architecture could differ significantly of course.
Preliminary Conclusion
There are two angles to this:
The AWS SDK for Java is definitely lacking the convenience method for setting the content type, which might be a comparatively rare, but nonetheless obvious use case accounted for in other AWS SDKs accordingly - this is surprising, given its widespread use in AWS related backend services.
Regardless, there seems to be something fishy with the way the Query String Request Authentication is implemented in comparison to the .NET version for example - again this is surprising, given it is a core functionality, however, this is still within the S3 model/namespace and thus might only be required by the respective uses cases above.
In conclusion, the only reasonable way to resolve this would be an updated SDK, so a bug report is in order - obviously one could as well duplicate/extend the SDK functionality to account for this special case separately (ideally in a way allowing to submit a pull request for the aws-sdk-for-java project), but getting this right in a compatible and maintainable way seems to be a bit tricky, thus is likely best done by the SDK maintainers themselves.

Ran into this problem as well. We're already tracking when the file is uploaded on the backend, so our work around was to set the content type after the client uploads the file using the Rails app with a call to copy_from.

clear hash on redirect in java [duplicate]

I am going through the facebook authentication process to log my users into my site. Once a user is logged in I redirect to the profile page using:
resp.sendRedirect("/l/profile");
But when I get to the profile page, the URL ends /profile#_=_
This seems to be at the end of the URL redirected by facebook when it returns a code. Why is it sticking around, how do I get rid of it?

I'm guessing it's a byproduct of new feature called "Authenticated Referrals" whereby they add a valid accessToken to the end of the url as either an #anchor or ?param (depending on whether you're reading it client- or server-side). You can read more about it here: http://developers.facebook.com/docs/opengraph/authentication/
At any rate, because it's appended Facebook-side, you'll have to file a bug with them to fix it, although at the moment the tracker seems to be down for me.