The situation is as follows:
On the one side I have created an OData-Service which should create an entry when it receives a POST-Request. The Service is created in an S/4HANA System and is reachable via the SAP-Gateway.
On the other hand I have a Java Application (OpenJDK 11) which does essentially a loop and must issue every loop a POST-Request to the OData-Service.
I'm using IntelliJ IDEA Community Edition and OpenJDK 11.
Also this is my first time using OData both with Java and SAP.
At first I tried the following:
private static void postRequest() throws IOException {
//Setting authenticator needed for login
Authenticator authenticator = new Authenticator() {
#Override
protected PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication(user, password.toCharArray());
}
};
Authenticator.setDefault(authenticator);
//Creating the connection
URL url = new URL("<my_service_link>");
HttpsURLConnection con = (HttpsURLConnection) url.openConnection();
con.setRequestMethod("POST");
con.setRequestProperty("Content-Type", "application/json; utf-8");
con.setRequestProperty("Accept", "application/json");
con.setDoOutput(true);
try(OutputStream os = con.getOutputStream()) {
byte[] input = this.getJsonRequest().getBytes("utf-8");
os.write(input, 0, input.length);
}
//Reading response
int status = con.getResponseCode();
Reader streamReader = null;
if (status > 299) {
streamReader = new InputStreamReader(con.getErrorStream());
} else {
streamReader = new InputStreamReader(con.getInputStream());
}
BufferedReader in = new BufferedReader(streamReader);
String inputLine;
StringBuffer content = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
content.append(inputLine);
}
in.close();
con.disconnect();
System.out.println(content.toString());
}
But I got the error, that my CSRF-Token is invalid.
So after googling to find out what an CSRF-Token is I tried to create a GET-Request first with its own HttpsURLConnection:
private static String getRequest() {
//Setting authenticator needed for login
Authenticator authenticator = new Authenticator() {
#Override
protected PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication(user, password.toCharArray());
}
};
Authenticator.setDefault(authenticator);
//Creating the connection
URL url = new URL("<my_service_link>");
HttpsURLConnection con = (HttpsURLConnection) url.openConnection();
con.setRequestMethod("GET");
con.setRequestProperty("Content-Type", "application/json; utf-8");
con.setRequestProperty("X-CSRF-Token","fetch");
con.connect();
return con.getHeaderField("x-csrf-token").toString();
}
Then I would issue the actual POST-Request to the same URL and set the previous X-CSRF-Token into the HTTPS-Header with
con.setRequestProperty("X-CSRF-Token",theGETToken); in postRequest()
But I still got the same error.
What am I doing wrong?
After some more googling I eventually understood what I was missing.
The CSRF-Token is only valid for a specific session of a user. The session is identified by the cookies passed in the HTTPS-Header.
What needs to be done is the following (also see: https://blogs.sap.com/2021/06/04/how-does-csrf-token-work-sap-gateway/):
Open a session by issuing a non-modification request and specify the header to fetch a CSRF-Token and session-cookies
HTTP-Request:
Type: GET
Header-Fields: x-csrf-token = fetch
set-cookie = fetch
Save the CSRF-Token and session-cookies as you need them for the POST-Request
Issue a POST-Request and set the session-cookies and CSRF-Token from the saved values
HTTP-Request:
Type: POST
Header-Fields: x-csrf-token = <tokenFromGet>
cookie = <allSessionCookies>
Beware that the header field of a request is named cookie instead of set-cookie and to pass all values of the HeaderField of set-cookie to the POST-Request-Header.
It is also important to mention, that the CSRF-Token as well as the session-cookies expire after a provided or adjusted timeframe or any changes to the session are made and both must be fetched anew (see https://blogs.sap.com/2021/06/04/how-does-csrf-token-work-sap-gateway/#comment-575524).
Example of my Working Code:
import javax.net.ssl.HttpsURLConnection;
import java.io.*;
import java.net.Authenticator;
import java.net.PasswordAuthentication;
import java.net.URL;
public class ODataLogger {
private String sessionCookies;
private String csrfToken;
public ODataLogger() {}
public void logOdata (String user, String pass, String jsonBody) throws IOException {
this.setDefaultAuthenticator(user, pass);
fetchSessionHeaderFields();
postRequest(jsonBody);
}
private void setDefaultAuthenticator (String user, String pass) {
Authenticator authenticator = new Authenticator() {
#Override
protected PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication(user, pass.toCharArray());
}
};
Authenticator.setDefault(authenticator);
}
private void fetchSessionHeaderFields() throws IOException {
URL url = new URL("<my-service-link>");
HttpsURLConnection con = (HttpsURLConnection) url.openConnection();
con.setRequestMethod("GET");
con.setRequestProperty("Content-Type", "application/json");
con.setRequestProperty("x-csrf-token", "fetch");
con.setRequestProperty("set-cookie","fetch");
//Reading Response
int status = con.getResponseCode();
Reader streamReader = null;
if (status < 299) {
StringBuffer sb = new StringBuffer(con.getHeaderFields().get("set-cookie").toString());
//Delete leading [ and trailing ] character
sb.deleteCharAt(this.sessionCookies.length()-1);
sb.deleteCharAt(0);
this.sessionCookies = sb.toString();
this.csrfToken = con.getHeaderField("x-csrf-token");
return;
}
}
private void postRequest(String jsonBody) throws IOException {
//Creating the connection
URL url = new URL("<my-service-link>");
HttpsURLConnection con = (HttpsURLConnection) url.openConnection();
con.setRequestMethod("POST");
con.setRequestProperty("Content-Type", "application/json");
con.setRequestProperty("x-csrf-token", this.csrfToken);
con.setRequestProperty("Cookie", this.sessionCookies);
con.setRequestProperty("Accept", "application/json");
//Setting JSON Body
con.setDoOutput(true);
try(OutputStream os = con.getOutputStream()) {
byte[] input = jsonBody.getBytes("utf-8");
os.write(input, 0, input.length);
}
//Reading response
int status = con.getResponseCode();
Reader streamReader = null;
if (status > 299) {
streamReader = new InputStreamReader(con.getErrorStream());
} else {
streamReader = new InputStreamReader(con.getInputStream());
}
BufferedReader in = new BufferedReader(streamReader);
String inputLine;
StringBuffer content = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
content.append(inputLine);
}
in.close();
con.disconnect();
I would like to open an URL and submit the following parameters to it, but it only seems to work if I add the BufferedReader to my code. Why is that?
Send.php is a script what will add an username with a time to my database.
This following code does not work (it does not submit any data to my database):
final String base = "http://awebsite.com//send.php?";
final String params = String.format("username=%s&time=%s", username, time);
final URL url = new URL(base + params);
final HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestProperty("User-Agent", "Agent");
connection.setRequestMethod("POST");
connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
connection.connect();
But this code does work:
final String base = "http://awebsite.com//send.php?";
final String params = String.format("username=%s&time=%s", username, time);
final URL url = new URL(base + params);
final HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestProperty("User-Agent", "Agent");
connection.setRequestMethod("POST");
connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
connection.connect();
final BufferedReader in = new BufferedReader(new InputStreamReader(connection.getInputStream()));
String line;
while ((line = in.readLine()) != null) {
System.out.println(line);
}
connection.disconnect();
As far as I know. When you called the connect() function, it will only create the connection.
You need to at least call the getInputStream() or getResponseCode() for the connection to be committed so that the server that the url is pointing to able to process the request.
Getting unauthorize execption while connecting to share point rest web service:
URL myURL = new URL("http://test:2014/PWA/_api/ProjectData/Projects");
URLConnection uc = myURL.openConnection();
HttpURLConnection myURLConnection = (HttpURLConnection)myURL.openConnection();
String userCredentials = "admin:pasword";
String basicAuth = "Basic " + javax.xml.bind.DatatypeConverter.printBase64Binary("password".getBytes());
uc.setRequestProperty ("Authorization", basicAuth);
InputStream in = uc.getInputStream();
Getting following errors while reading from url
java.io.IOException: Server returned HTTP response code: 401 for URL: http://test:2014/PWA/_api/ProjectData/Projects
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at com.jw.sharepoint.examples.XMLParser.getDocumentFromUrl(XMLParser.java:127)
at com.jw.sharepoint.examples.XMLParser.main(XMLParser.java:27)
java.lang.NullPointerException
at com.jw.sharepoint.examples.XMLParser.main(XMLParser.java:29)
Add request property and request method solved the problem.
InputStream getAuthenticatedResponse(final String urlStr, final String domain,final String userName, final String password) throws IOException {
Authenticator.setDefault(new Authenticator() {
#Override
public PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication(
domain + "\\" + userName, password.toCharArray());
}
});
URL urlRequest = new URL(urlStr);
HttpURLConnection conn = (HttpURLConnection) urlRequest.openConnection();
conn.setDoOutput(true);
conn.setDoInput(true);
conn.setRequestMethod("GET");
conn.setRequestProperty("Accept", "*/*");
return conn.getInputStream();
}
In my project I must strictly use HttpURLConnection class
I have this following code which I got from the internet
MultipartEntity multiPart = new MultiPartEntity(HttpMultipartMode.BROWSER_COMPATIBLE, null Chartset.forName("UTF-8");
File f = new File("/home/abhishek/foo.docx");
FileBody fb = new FileBody(f);
multiPart.addPart("file", fb);
HttpPost post = new HttpPost();
post.setHeader("ENCTYPE", "multipart/form-data");
post.setEntity(multiPart);
Problem is that I cannot use HttpPost ... In my project only HttpURLConnection class works!
So I need to translate the code above into HttpURLConnection.
I cannot find anything similar to setEntity on the HttpUrlConnection.
Edit::
Based on the suggestions below. I have this code
public class RESTFileUpload {
public static void main(String[] args) throws Exception {
Authenticator.setDefault(new Authenticator() {
#Override
public PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication("domain\\user", "Password".toCharArray());
}
});
String filePath = "/home/abhishek/Documents/HelloWorld.docx";
String fileName = "HelloWorld.docx";
String fileNameShort = "HelloWorld";
String urlStr = "https://sp.company.com/sites/abhi_test/_vti_bin/listdata.svc/SharedDocuments/RootFolder/Files/add(url=#TargetFileName,overwrite='true')&#TargetFileName=" + fileName;
String crlf = "\r\n";
String twoHypens = "--";
String boundary = "*****";
URL url = new URL(urlStr);
HttpURLConnection con = (HttpURLConnection) url.openConnection();
con.setDoOutput(true);
con.setDoInput(true);
con.setUseCaches(false);
con.setRequestMethod("POST");
con.setRequestProperty("Connection", "Keep-Alive");
con.setRequestProperty("Cache-Control", "no-cache");
con.setRequestProperty("Content-Type", "multipart/form-data;boundary=" + boundary);
DataOutputStream request = new DataOutputStream(con.getOutputStream());
request.writeBytes(twoHypens + boundary + crlf);
request.writeBytes("Content-Disposition: form-data;name=\"" + fileNameShort + "\";fileName=\"" + fileName + "\"" + crlf);
request.writeBytes(crlf);
request.write(convertToByteArray(filePath));
request.writeBytes(crlf);
request.writeBytes(twoHypens + boundary + twoHypens + crlf);
request.flush();
request.close();
InputStream responseStream = new BufferedInputStream(con.getInputStream());
BufferedReader responseStreamReader = new BufferedReader(new InputStreamReader(responseStream));
String line = "";
StringBuilder strBuilder = new StringBuilder();
while((line = responseStreamReader.readLine()) != null) {
strBuilder.append(line).append("\n");
}
responseStreamReader.close();
String response = strBuilder.toString();
responseStream.close();
con.disconnect();
System.out.println(response);
}
private static byte[] convertToByteArray(String filePath) {
File f = new File(filePath);
byte[] retVal = new byte[(int)f.length()];
try {
FileInputStream fis = new FileInputStream(f);
fis.read(retVal);
}
catch (FileNotFoundException ex) {
ex.printStackTrace();
}
catch(IOException ex2) {
ex2.printStackTrace();
}
return retVal;
}
}
But I get the error
Exception in thread "main" java.io.IOException: Server returned HTTP response code: 400 for URL: https://sp.web.gs.com/sites/abhi_test/_vti_bin/listdata.svc/SharedDocuments/
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1626)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254)
at RESTFileUpload.main(RESTFileUpload.java:62)
HttpURLConnection has .getInputStream() and .getOutputStream() methods. If you wish to send body content with an Http request, you call .setDoOutput(true) on your HttpURLConnection object, call .getOutputStream() to get an Output stream and then write the content of your entity to the output stream (either as raw bytes, or using a Writer implementation of some sort), closing it when you are finished writing.
For more details, see the API docs for HttpURLConnection here.
To post files using the HttpURLConnection you have to compose the file wrapper manually. Take a look at this answer, it should be helpful for you.
How do I connect to a remote URL in Java which requires authentication. I'm trying to find a way to modify the following code to be able to programatically provide a username/password so it doesn't throw a 401.
URL url = new URL(String.format("http://%s/manager/list", _host + ":8080"));
HttpURLConnection connection = (HttpURLConnection)url.openConnection();
There's a native and less intrusive alternative, which works only for your call.
URL url = new URL(“location address”);
URLConnection uc = url.openConnection();
String userpass = username + ":" + password;
String basicAuth = "Basic " + new String(Base64.getEncoder().encode(userpass.getBytes()));
uc.setRequestProperty ("Authorization", basicAuth);
InputStream in = uc.getInputStream();
You can set the default authenticator for http requests like this:
Authenticator.setDefault (new Authenticator() {
protected PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication ("username", "password".toCharArray());
}
});
Also, if you require more flexibility, you can check out the Apache HttpClient, which will give you more authentication options (as well as session support, etc.)
You can also use the following, which does not require using external packages:
URL url = new URL(“location address”);
URLConnection uc = url.openConnection();
String userpass = username + ":" + password;
String basicAuth = "Basic " + javax.xml.bind.DatatypeConverter.printBase64Binary(userpass.getBytes());
uc.setRequestProperty ("Authorization", basicAuth);
InputStream in = uc.getInputStream();
If you are using the normal login whilst entering the username and password between the protocol and the domain this is simpler. It also works with and without login.
Sample URL: http://user:pass#example.com/url
URL url = new URL("http://user:pass#example.com/url");
URLConnection urlConnection = url.openConnection();
if (url.getUserInfo() != null) {
String basicAuth = "Basic " + new String(new Base64().encode(url.getUserInfo().getBytes()));
urlConnection.setRequestProperty("Authorization", basicAuth);
}
InputStream inputStream = urlConnection.getInputStream();
Please note in the comment, from valerybodak, below how it is done in an Android development environment.
As I have came here looking for an Android-Java-Answer I am going to do a short summary:
Use java.net.Authenticator as shown by James van Huis
Use Apache Commons HTTP Client, as in this Answer
Use basic java.net.URLConnection and set the Authentication-Header manually like shown here
If you want to use java.net.URLConnection with Basic Authentication in Android try this code:
URL url = new URL("http://www.example.com/resource");
URLConnection urlConnection = url.openConnection();
String header = "Basic " + new String(android.util.Base64.encode("user:pass".getBytes(), android.util.Base64.NO_WRAP));
urlConnection.addRequestProperty("Authorization", header);
// go on setting more request headers, reading the response, etc
Was able to set the auth using the HttpsURLConnection
URL myUrl = new URL(httpsURL);
HttpsURLConnection conn = (HttpsURLConnection)myUrl.openConnection();
String userpass = username + ":" + password;
String basicAuth = "Basic " + new String(Base64.getEncoder().encode(userpass.getBytes()));
//httpsurlconnection
conn.setRequestProperty("Authorization", basicAuth);
few of the changes fetched from this post. and Base64 is from java.util package.
Be really careful with the "Base64().encode()"approach, my team and I got 400 Apache bad request issues because it adds a \r\n at the end of the string generated.
We found it sniffing packets thanks to Wireshark.
Here is our solution :
import org.apache.commons.codec.binary.Base64;
HttpGet getRequest = new HttpGet(endpoint);
getRequest.addHeader("Authorization", "Basic " + getBasicAuthenticationEncoding());
private String getBasicAuthenticationEncoding() {
String userPassword = username + ":" + password;
return new String(Base64.encodeBase64(userPassword.getBytes()));
}
Hope it helps!
Use this code for basic authentication.
URL url = new URL(path);
String userPass = "username:password";
String basicAuth = "Basic " + Base64.encodeToString(userPass.getBytes(), Base64.DEFAULT);//or
//String basicAuth = "Basic " + new String(Base64.encode(userPass.getBytes(), Base64.No_WRAP));
HttpURLConnection urlConnection = (HttpURLConnection)url.openConnection();
urlConnection.setRequestProperty("Authorization", basicAuth);
urlConnection.connect();
Since Java 9, you can do this
URL url = new URL("http://www.example.com");
HttpURLConnection connection = (HttpURLConnection)url.openConnection();
connection.setAuthenticator(new Authenticator() {
protected PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication ("USER", "PASS".toCharArray());
}
});
I'd like to provide an answer for the case that you do not have control over the code that opens the connection. Like I did when using the URLClassLoader to load a jar file from a password protected server.
The Authenticator solution would work but has the drawback that it first tries to reach the server without a password and only after the server asks for a password provides one. That's an unnecessary roundtrip if you already know the server would need a password.
public class MyStreamHandlerFactory implements URLStreamHandlerFactory {
private final ServerInfo serverInfo;
public MyStreamHandlerFactory(ServerInfo serverInfo) {
this.serverInfo = serverInfo;
}
#Override
public URLStreamHandler createURLStreamHandler(String protocol) {
switch (protocol) {
case "my":
return new MyStreamHandler(serverInfo);
default:
return null;
}
}
}
public class MyStreamHandler extends URLStreamHandler {
private final String encodedCredentials;
public MyStreamHandler(ServerInfo serverInfo) {
String strCredentials = serverInfo.getUsername() + ":" + serverInfo.getPassword();
this.encodedCredentials = Base64.getEncoder().encodeToString(strCredentials.getBytes());
}
#Override
protected URLConnection openConnection(URL url) throws IOException {
String authority = url.getAuthority();
String protocol = "http";
URL directUrl = new URL(protocol, url.getHost(), url.getPort(), url.getFile());
HttpURLConnection connection = (HttpURLConnection) directUrl.openConnection();
connection.setRequestProperty("Authorization", "Basic " + encodedCredentials);
return connection;
}
}
This registers a new protocol my that is replaced by http when credentials are added. So when creating the new URLClassLoader just replace http with my and everything is fine. I know URLClassLoader provides a constructor that takes an URLStreamHandlerFactory but this factory is not used if the URL points to a jar file.
i did that this way you need to do this just copy paste it be happy
HttpURLConnection urlConnection;
String url;
// String data = json;
String result = null;
try {
String username ="user#gmail.com";
String password = "12345678";
String auth =new String(username + ":" + password);
byte[] data1 = auth.getBytes(UTF_8);
String base64 = Base64.encodeToString(data1, Base64.NO_WRAP);
//Connect
urlConnection = (HttpURLConnection) ((new URL(urlBasePath).openConnection()));
urlConnection.setDoOutput(true);
urlConnection.setRequestProperty("Content-Type", "application/json");
urlConnection.setRequestProperty("Authorization", "Basic "+base64);
urlConnection.setRequestProperty("Accept", "application/json");
urlConnection.setRequestMethod("POST");
urlConnection.setConnectTimeout(10000);
urlConnection.connect();
JSONObject obj = new JSONObject();
obj.put("MobileNumber", "+97333746934");
obj.put("EmailAddress", "danish.hussain#example.com");
obj.put("FirstName", "Danish");
obj.put("LastName", "Hussain");
obj.put("Country", "BH");
obj.put("Language", "EN");
String data = obj.toString();
//Write
OutputStream outputStream = urlConnection.getOutputStream();
BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(outputStream, "UTF-8"));
writer.write(data);
writer.close();
outputStream.close();
int responseCode=urlConnection.getResponseCode();
if (responseCode == HttpsURLConnection.HTTP_OK) {
//Read
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(urlConnection.getInputStream(), "UTF-8"));
String line = null;
StringBuilder sb = new StringBuilder();
while ((line = bufferedReader.readLine()) != null) {
sb.append(line);
}
bufferedReader.close();
result = sb.toString();
}else {
// return new String("false : "+responseCode);
new String("false : "+responseCode);
}
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} catch (JSONException e) {
e.printStackTrace();
}
ANDROD IMPLEMENTATION
A complete method to request data/string response from web service requesting authorization with username and password
public static String getData(String uri, String userName, String userPassword) {
BufferedReader reader = null;
byte[] loginBytes = (userName + ":" + userPassword).getBytes();
StringBuilder loginBuilder = new StringBuilder()
.append("Basic ")
.append(Base64.encodeToString(loginBytes, Base64.DEFAULT));
try {
URL url = new URL(uri);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.addRequestProperty("Authorization", loginBuilder.toString());
StringBuilder sb = new StringBuilder();
reader = new BufferedReader(new InputStreamReader(connection.getInputStream()));
String line;
while ((line = reader.readLine())!= null){
sb.append(line);
sb.append("\n");
}
return sb.toString();
} catch (Exception e) {
e.printStackTrace();
return null;
} finally {
if (null != reader){
try {
reader.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}