Develop Reference

How to use Apache Olingo behind a proxy

I can successfully reach following OData-service using different browsers and also using Postman even so I am behind a proxy:
String SERVICE_ROOT = http://services.odata.org/V4/TripPinService/
However, using Apache Olingo in Java I am not able to access this service.
JVM parameters like -Dhttp.proxySet=true -Dhttp.proxyHost=http-proxy.example.com -Dhttp.proxyPort=8080 allow me to perform basic URL functions, like retrieving HTTP status codes (google returns 200). Nevertheless, access of the OData-Service using an ODataClient is not possible (code below). No errors are thrown.
ODataClient client = ODataClientFactory.getClient();
ODataServiceDocumentRequest request = client.getRetrieveRequestFactory().getServiceDocumentRequest(SERVICE_ROOT);
ODataRetrieveResponse<ClientServiceDocument> response = request.execute();
I tried using the proxy capabilities within Olingo, however without any success:
client.getConfiguration().setHttpClientFactory(new ProxyWrappingHttpClientFactory(URI.create("http://http-proxy.example.com:8080")));
What am I doing wrong, what options do I have left?
Thank you very much.

If you are behind an NTLM proxy you can try with NTLMAuthHttpClientFactory.
NTLMAuthHttpClientFactory ntlm = new NTLMAuthHttpClientFactory(username, password, workstation, domain);
client.getConfiguration().setHttpClientFactory(ntlm);
In case that doesn't work, you can try with cntlm. Install it, change username, password, domain and proxy in C:\Program Files (x86)\Cntlm\cntlm.ini and then invoke net start cntlm.
Use this for Olingo:
client.getConfiguration().setHttpClientFactory(new ProxyWrappingHttpClientFactory(URI.create("http://localhost:3128")));

URI uri;
String scheme = "http";
try {
uri = new URI (scheme,null,host,port,null,null,null);
} catch (URISyntaxException e) {
throw(e);
}
HttpClientFactory clientProxy = new ProxyWrappingHttpClientFactory(uri,userName,password );
client.getConfiguration().setHttpClientFactory(clientProxy);

I somehow solved the problem by myself. Within VM arguments I now only have
-Djava.net.preferIPv4Stack=true
Further I defined a proxy config only within the application:
client = ODataClientFactory.getClient();
client.getConfiguration().setHttpClientFactory(
new ProxyWrappingHttpClientFactory(URI.create("http-prox.example.com:8080")));
This worked for me. :)

Google Drive API - OAuth2.0: How to Automate Authentication Process? Doubts and Questions

I'm trying to integrate Google APIs inside a project (Thesis project) and I have some doubts and questions. So, here it is the scenario:
I wrote a back-end application in Java that runs solely from a command-line and has absolutely no interaction with a user. Its goal is to allow communication and interaction between sensors and actuators. Everything works great. Now I'd like to integrate something in order to let the sensors backup data both with a certain periodicity and due to some detected threshold value. So I thought, why not trying with Google Drive. The first very useful links have been:
https://developers.google.com/drive/web/quickstart/quickstart-java
https://developers.google.com/accounts/docs/OAuth2InstalledApp
Quick start examples work like a charm. However it requires quite a bit of settings: create a project inside the Developer Console (therefore an account), enable Drive API, then create a Client ID and a Client Secret. Once you've done these steps, you can hard-coded client ID and secret to form the request URL for google drive. Then you're kindly asked to enter the url in a browser, log in if you're not, accept and finally copy and paste into your console the authorization code for obtaining an access token. Wow, quite a security proccess. But hey, I completely agree with it, above all in a scenario where we have either a web app, a smartphone app or a web service that needs users' authentication and authorization in order to let the app doing its job by accessing someone else account. But in my case, I just would like that sensors will backup data on my google drive.
These facts lead to my first question: in order to use Google APIs (Drive in this case), do I have to create a project anyway? Or is there another approach? If I'm not wrong, there aren't other ways to create a client Id and secret without creating a project inside the Developer Console. This puzzles me a lot. Why should I create a project to use basically some libraries?
So, let's assume the previous as justifiable constraints and move on the real question: how to automate the authentication process? Given my scenario where a sensor (simply a Java module) want to backup data, it would be impossible to complete all that steps. The google page about OAuth 2.0 has a great explanations about different scenarios where we can embed the authentication procedure, included one for "devices with limited input capabilities". Unluckily, this is more complicated then the others and requires that "The user switches to a device or computer with richer input capabilities, launches a browser, navigates to the URL specified on the limited-input device, logs in, and enters the code." (LOL)
So, I didn't give up and I ended up on this post that talks about OAuth Playground: How do I authorise an app (web or installed) without user intervention? (canonical ?). It really looks like as a solution for me, in particular when it says:
NB2. This technique works well if you want a web app which access
your own (and only your own) Drive account, without bothering to write
the authorization code which would only ever be run once. Just skip
step 1, and replace "my.drive.app" with your own email address in step
5.
However if I'm not wrong, I think that OAuth Playground it's just for helping test and debug projects that use Google APIs, isn't it? Moreover, Google drive classes such as GoogleAuthorizationCodeFlow and GoogleCredential (used inside the Java quick start example) always need Client ID, Client Secret and so on, which brings me to point zero (create a project and do the whole graphical procedure).
In conclusion: is there a way to avoid the "graphical" authentication interaction and convert it into an automated process using only Drive's APIs without the user intervention? Thanks a lot, I would be grateful for any tip, hint, answer, pointer :-)

This is just a snippet of code that I wrote thanks to pinoyyid suggestions. Just to recap what we should do in this case (when in your program there isn't a user interaction for completing all the Google GUI authentication process). As reported in https://developers.google.com/drive/web/quickstart/quickstart-java
Go to the Google Developers Console.
Select a project, or create a new one.
In the sidebar on the left, expand APIs & auth. Next, click APIs. In the list of APIs, make sure the status is ON for the Drive API.
In the sidebar on the left, select Credentials.
In either case, you end up on the Credentials page and can create your project's credentials from here.
From the Credentials page, click Create new Client ID under the OAuth heading to create your OAuth 2.0 credentials. Your application's client ID, email address, client secret, redirect URIs, and JavaScript origins are in the Client ID for web application section.
The pinoyyd post is neater and get straight to the point: How do I authorise a background web app without user intervention? (canonical ?)
Pay attention to step number 7
Finally the snippet of code is very simple, it's just about sending a POST request and it's possible to do that in many ways in Java. Therefore this is just an example and I'm sure there is room for improvements ;-)
// Both to set access token the first time that we run the module and in general to refresh the token
public void sendPOST(){
try {
URL url = new URL("https://www.googleapis.com/oauth2/v3/token");
Map<String,Object> params = new LinkedHashMap<>();
params.put("client_id", CLIENT_ID);
params.put("client_secret", CLIENT_SECRET);
params.put("refresh_token", REFRESH_TOKEN);
params.put("grant_type", "refresh_token");
StringBuilder postData = new StringBuilder();
for (Map.Entry<String,Object> param : params.entrySet()) {
if (postData.length() != 0) postData.append('&');
postData.append(URLEncoder.encode(param.getKey(), "UTF-8"));
postData.append('=');
postData.append(URLEncoder.encode(String.valueOf(param.getValue()), "UTF-8"));
}
byte[] postDataBytes = postData.toString().getBytes("UTF-8");
HttpsURLConnection conn = (HttpsURLConnection)url.openConnection();
conn.setRequestMethod("POST");
conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
conn.setRequestProperty("Content-Length", String.valueOf(postDataBytes.length));
conn.setDoOutput(true);
conn.getOutputStream().write(postDataBytes);
BufferedReader in_rd = new BufferedReader(new InputStreamReader(conn.getInputStream(), "UTF-8"));
// Read response body which should be a json structure
String inputLine;
StringBuilder responseBody = new StringBuilder();
while ((inputLine = in_rd.readLine()) != null) {
responseBody.append(inputLine);
}
in_rd.close();
//Parsing Response --> create a json object
JSONObject jsonResp = new JSONObject(responseBody);
//Modify previous access token String
ACCESS_TOKEN = jsonResp.getString("access_token");
}
catch(MalformedURLException ex_URL){
System.out.println("An error occured: " + ex_URL.getMessage());
}
catch(JSONException ex_json) {
System.out.println("An error occured: " + ex_json.getMessage());
}
catch(IOException ex_IO){
System.out.println("An error occured: " + ex_IO.getMessage());
}
} //end of sendRefreshPOST method
Hope this snippet of code will help others that will face the same situation !

I wrote the SO post at How do I authorise an app (web or installed) without user intervention? (canonical ?)
What it describes is indeed the solution to your use-case. The key bit you'd missed is step 7 where you enter the details of your own application into the OAuth Playground. From that point, the playground is impersonating your app and so you can do the one-time authorization and obtaining a refresh token.

Getting an SSL error for the java client library that I'm writting in Java. Requests still fail after following documentation

I'm starting an e-commerce site and one of the drop shippers that I plan on using offers an api that can be used to automate the placing and tracking of orders. Even thought they have clients in several languages (PHP, Python, Ruby,Node), they don't have one in Java yet so I decided to write my own. I'm trying to test it, but I keep getting this error
Hostname in certificate didn't match: <api.theprintful.com> != <behappy.me> OR <behappy.me> OR <www.behappy.me>
I've followed the directions that they have for their documentation:
Some API requests (like the Product catalog and Country codes) are available without an
API key, but for majority of requests you will need to authenticate your store.
...
To perform authorization, you need to add the Authorization header with a Base64 encoded
API key when performing a request. So if the API key is vv0gtldg-1d7v-qclq:e2vv-
lmhg676ak0z1, then you need to add this header to the API request:
Authorization: Basic dnYwZ3RsZGctMWQ3di1xY2xxOmUydnYtbG1oZzY3NmFrMHox
I've followed the directions
//constructor that is used to make the object
public ProductsRequest(String apiKey)
{
super();
this.apiKey = apiKey;
this.encodedApiKey = codec.encodeBase64String(apiKey.getBytes());
}
//function that is used to call the api
public List<Product> getAllProductList(String path)
{
//the list of products that will be returned
List<Product> returnedProducts = null;
//set the endpoint that will be used to get the list of products
httpGet = new HttpGet(path);
String basicString = "Basic "+this.encodedApiKey;
httpGet.addHeader("Authorization",basicString.substring(0,basicString.length()-2));
try
{
response = httpClient.execute(httpGet);
if(response.getStatusLine().getStatusCode() == 200)
{
entity = response.getEntity();
jsonResponse = EntityUtils.toString(entity);
Type listType = new TypeToken<List<Product>>(){}.getType();
JsonNode root = mapper.readTree(jsonResponse);
ArrayNode products = (ArrayNode) root.path("result");
returnedProducts = (List<Product>) gson.fromJson(products.toString(),listType);
}
}
catch (IOException e)
{
e.printStackTrace();
}
return returnedProducts;
}
I can't figure out why I'm getting this error. I tried emailing the dev team at the company and they suggested that I try creating a fake store that isn't connected to any ecommerce software that I can use just for testing but they said that everything looked good on their end. I tried following that suggestion but I still get the same error.
I found this thread on Stackoverflow where it looks like somebody had a similar problem. I'm just wondering why would I have to do this in Java when none of the other client libraries in other languages have to go through this process?
As far as I can tell I'm encoding everything the way the documentation says I should and I'm setting the headers correctly. Do I need to use different libraries?
I'm currently using the apache commons-codec
<dependency>
<groupId>commons-codec</groupId>
<artifactId>commons-codec</artifactId>
<version>1.4</version>
</dependency>
and apache http-components
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.3.2</version>
</dependency>
I've tried running the examples from one of the client libraries that the company has in another language and everything is working correctly. I know that it is something wrong with my code and not their SSL certificates, but I can't figure out what it is.

Your server is using Server Name Indication to present a different certificate depending on the name you ask for (which is necessary to host multiple certificate with SSL/TLS on the same IP address and port).
You can check this with:
openssl s_client -connect api.theprintful.com:443 -servername api.theprintful.com
and
openssl s_client -connect api.theprintful.com:443
SNI is currently only available since Java 7 (and only on the client side, server-side support being planned for Java 8).
An application doing Java 7 should do this automatically for you when using HttpsURLConnection.
The fix for this in Apache HTTP Client is recent. You should make sure you are indeed using version 4.3.2 and Java 7. The following would fail with version 4.3.1 (even with Java 7), or with 4.3.2 on Java 6:
CloseableHttpClient httpClient = HttpClients.createDefault();
HttpGet httpGet = new HttpGet("https://sni.velox.ch/");
HttpResponse response = httpClient.execute(httpGet);

Are you connecting via SSL? i.e. does your path begin with 'https://www.behappy.me'? If so, then i think what's happening is that their SSL certificate is issued for 'api.theprintful.com'. To prevent man-in-the-middle attacks, SSL clients are supposed to check whether the certificate they got from the server was really issued for the server in question, see RFC 6125. In your case, this check fails and results in the exception you see. This has nothing to do with your side of the authentication, you are probably doing everything right, they see the correct request coming in, but your side croaks on the wrong certificate.
What can you do to get around this?
Ask them to provide the correct certificate. That's the easiest and best solution. Unfortunately, it requires cooperation from their side, which you may or may not get.
Fake you own DNS, that is, put an entry in your hosts file pointing api.theprintful.com to the IP address of www.behappy.me. This is the suggestion in this thread.
Suppress the hostname check by supplying an empty HostnameVeriier. That's indicated in the same thread above, and can be done without messing with any external settings. But make sure, i mean really absolutely sure that you remove that workaround before moving the code to production, or else goto fail.

How to make all network traffic go via a proxy?

I have an app that makes http requests to a remote server. I do this with the following code:
HttpClient httpClient = new DefaultHttpClient();
HttpPost httpPost = new HttpPost("myURL");
try {
ArrayList<BasicNameValuePair> postVariables = new ArrayList<BasicNameValuePair>(2);
postVariables.add(new BasicNameValuePair("key","value"));
httpPost.setEntity(new UrlEncodedFormEntity(postVariables));
HttpResponse response = httpClient.execute(httpPost);
String responseString = EntityUtils.toString(response.getEntity());
if (responseString.contains("\"success\":true")){
//this means the request succeeded
} else {
//failed
}
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
This goes really well, but one of our customers has set up an APN that requires requests to go via a certain proxy server. If I add the following to the request this works, the request gets rerouted via the proxy to the server:
HttpHost httpHost = new HttpHost("proxyURL",8080);
httpClient.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, httpHost);
So far so good, however, I use a library that makes some http requests as well. The library's code is not accesible for me, so I can't add those two lines to the code. I contacted the creators of that library, and they told me it should be possible to set up the android environment so that all requests will automatically go through the proxy. Is there something like that? I didn't find anything on google.
I'm basically looking for a way to set the above two lines as a standard for all http requests. Please note that the APN does not set the proxy as a default for the entire phone, so apps will have to do this manually (and yes that means the majority of the apps don't work on that customer's phone).

It's been a year or two since I've needed to use it, but if I remember correctly, you can use the System.setProperty(String, String) in order to set an environment-wide setting for your application to route all HTTP traffic through a proxy. The properties that you should need to set are "http.proxyHost" and "http.proxyPort" and then use your HttpClient normally without specifying a proxy because the VM will handle routing requests.
Docs for more information about what I'm talking about can be found here: ProxySelector (just so you know what keys to use) and here for documentation about the actual System.setProperty(String, String) function
If that doesn't work for you, let me know and I'll try to dig out my old code that set a system-level proxy. BTW, it's really only "system-level" since each app runs in it's own Dalvik so you won't impact other app's network communications.

Authenticate programmatically to Google with OAuth2

How can I authenticate programmatically to Google?
Now that ClientLogin (https://developers.google.com/accounts/docs/AuthForInstalledApps)
is deprecated, how can we perform a programmatic authentication to Google with OAuth2?
With ClientLogin we could perform a post to
https://www.google.com/accounts/ClientLogin
with email and password parameters and obtain the authentication token.
With OAuth2 i can't find a solution!
#
My app is a java background process.
I saw, following this link: developers.google.com/accounts/docs/OAuth2InstalledApp#refresh, how to obtain a new access token using a refreshed token.
The problem is that I can't find a java example about how to instantiate an Analytics object (for example) to perform a query when I have a new valid access token
This is my code that returns a 401 Invalid credentials when invoke the "execute()":
public class Test {
static final String client_id = "MY_CLIENT_ID";
static final String client_secret = "MY_SECRET";
static final String appName = "MY_APP";
private static final HttpTransport HTTP_TRANSPORT = new NetHttpTransport();
private static final JsonFactory JSON_FACTORY = new JacksonFactory();
static String access_token = "xxxx";
static String refreshToken = "yyyyy";
public static void main (String args[]){
try {
GoogleCredential credential =
new GoogleCredential.Builder()
.setTransport(HTTP_TRANSPORT)
.setJsonFactory(JSON_FACTORY)
.setClientSecrets(client_id, client_secret).build();
credential.setAccessToken(access_token);
credential.setRefreshToken(refreshToken);
//GoogleCredential
Analytics analytics = Analytics.builder(HTTP_TRANSPORT, JSON_FACTORY)
.setApplicationName(appName)
.setHttpRequestInitializer(credential)
.build();
Accounts accounts = analytics.management().accounts().list().execute();
} catch (Exception e) {
e.printStackTrace();
}
}
What is the problem?

Check the OAuth 2 flow for Installed Application:
https://developers.google.com/accounts/docs/OAuth2InstalledApp
It still requires the user to authenticate with a browser the first time, but then you can store the refresh token and use it for subsequent requests.
For alternative solutions, check the Device flow or Service Accounts, they are explained in the same documentation set.

I found the Google Java client to be overly complex and poorly documented. Here's plain and simple Servlet example with Google Oauth2. For a background process you'll need to request access_type=offline. As others have mentioned you need the user to do a one time authorization. After that you can request refresh tokens as google tokens expire in an hour.

Although I appreciate that the OP was originally targeting the OAuth2InstalledApp approach, I would like to point out a working solution using the OAuth2WebServer approach. They don't differ significantly and this worked for me. I have found the google OAuth library to be pretty good as it will handle most of the OAuth dance for you and it makes it easy to refresh the access token. The solution below depends on using a pre-obtained refresh token.
As the accepted answer states, to get OAuth authentication working (even for a Java background process) where the request relies upon access to user data
requires the user to authenticate with a browser the first time, but then you can store the refresh token and use it for subsequent requests.
From previous comments by the OP I see the following
So I followed OAuth2 for Web Server Applications (here offline access is documented) but I have still problems.
1) I perform the first request via browser and I obtain autenticaton code for offline access
2) I perform a java post of the authentication code and obtain acces token and refresh token
The approach I used is more like
1) I perform the first request via a browser and obtain the refresh token for offline access
2) In java I provide the refresh token to the library and the library will obtain the access token etc
specifically, using the google-api-java-client library the code is quite straightforward and note that I haven't set an access token as the OP did, as I am calling credential.refreshToken(); elsewhere. (I check if I have a valid access token already and if not call refresh prior to the API call)
private Credential generateCredentialWithUserApprovedToken() throws IOException,
GeneralSecurityException {
JsonFactory jsonFactory = JacksonFactory.getDefaultInstance();
HttpTransport httpTransport = GoogleNetHttpTransport.newTrustedTransport();
InputStreamReader inputStreamReader =
new InputStreamReader(jsonFileResourceForClient.getInputStream());
GoogleClientSecrets clientSecrets = GoogleClientSecrets.load(jsonFactory, inputStreamReader);
return new GoogleCredential.Builder().setTransport(httpTransport).setJsonFactory(jsonFactory)
.setClientSecrets(clientSecrets).build().setRefreshToken(REFRESH_TOKEN);
}
Note this covers step 2 of my approach, and the REFRESH_TOKEN mentioned in step 1 can be obtained as explained below.
First there is a prior set up of a web app creating an OAuth 2.0 client ID on the Google console for Credentials where you end up with a downloaded json file which will be read into the GoogleClientSecrets object.
i.e.
Make sure you add the Google playground callback uri into Authorized redirect URIs
Then you have your client id and the client secret ready for the playground and you can also download the json which you can pull into your Java code.
The REFRESH_TOKEN is obtained by sending a request to the google oauth playground with the following configuration. Note that prior to Step 1 and selecting your scope you should go to settings to check that you are providing you own credentials and add your client id and secret just below that
Note that the Access type is Offline, which corresponds to this.
There is also a nice explanation on grabbing the refresh token here https://www.youtube.com/watch?v=hfWe1gPCnzc
That is enough to get going and is a one time set up!
Regarding refresh tokens you should be aware of their lifecycle as discussed in the docs here
In the oauthplayground you will see this
but on point 4 of the docs here it says this
Hmmm.
Also for reference see How do I authorise an app (web or installed) without user intervention? (canonical ?)

For applications that authenticate on behalf of themselves (i.e., to another application, traditionally by signing into a role account using a shared password), the OAuth2 alternative to ClientLogin offered by Google is Service Accounts:
https://developers.google.com/accounts/docs/OAuth2ServiceAccount