Develop Reference

Google Form fill from Java

I need to fill basic Google Form from my Java code but it throws org.apache.http.client.ClientProtocolException: Unexpected response status: 405
Here is my code :
private boolean sendMessage(UserInfo userInfo) {
final HttpPost req = new HttpPost("my-form-url");
try (CloseableHttpClient httpclient = HttpClients.createDefault()) {
List<NameValuePair> form = new ArrayList<>();
form.add(new BasicNameValuePair("entry.1301726507", userInfo.getName()));
form.add(new BasicNameValuePair("entry.1466759457", "hello"));
UrlEncodedFormEntity entity = new UrlEncodedFormEntity(form, Consts.UTF_8);
req.setEntity(entity);
System.out.println("Executing request " + req.getRequestLine());
ResponseHandler<String> responseHandler = response -> {
int status = response.getStatusLine().getStatusCode();
if (status >= 200 && status < 300) {
HttpEntity responseEntity = response.getEntity();
return responseEntity != null ? EntityUtils.toString(responseEntity) : null;
} else {
throw new ClientProtocolException("Unexpected response status: " + status);
}
};
String responseBody = httpclient.execute(req, responseHandler);
System.out.println("----------------------------------------");
System.out.println(responseBody);
return true;
} catch (IOException e) {
e.printStackTrace();
return false;
}
And here is the form :
what am I doing wrong?

You may use my pet project to do the job for you:
Add dependency:
<dependency>
<groupId>io.github.stepio.jgforms</groupId>
<artifactId>jgforms</artifactId>
<version>1.0.1</version>
</dependency>
Define your form:
public enum MyForm implements MetaData {
USER_NAME(1301726507L),
MESSAGE(1466759457L);
private long id;
JGForm(long id) {
this.id = id;
}
#Override
public long getId() {
return this.id;
}
}
Fill the form and submit:
private boolean sendMessage(UserInfo userInfo) {
URL url = Builder.formKey("my-form-key-from-url")
.put(MyForm.USER_NAME, userInfo.getName())
.put(MyForm.MESSAGE, "hello")
.toUrl();
Submitter submitter = new Submitter(
new Configuration()
);
try {
submitter.submitForm(url);
} catch (NotSubmittedException ex) {
// TODO: log & handle the exception properly
return false;
}
return true;
}
Check README and unit tests for more details and examples:
https://github.com/stepio/jgforms

Status code 404 when calling spring-boot API, but not for postman or browser

I want to make an API call to a local REST server built with Spring-Boot which is interacting with mongodb. I already checked some posts which I found to this topic, but my problem seems to be a little bit different.
Here are some code snippets which are relevant:
protected static CoreEntity[] sendGET(CoreEntity[] resultList, String path) throws IOException {
path = String.join("%20", path.split(" "));
return handleResponse(resultList, getConnection(path, "Get"));
}
private static HttpURLConnection getConnection(String path, String requestMethod) throws IOException {
URL url = new URL(REQUEST_URL + path);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setDoOutput(true);
connection.setDoInput(true);
connection.setRequestProperty("Content-Type", "application/json");
connection.setRequestProperty("accept", "application/json");
connection.setConnectTimeout(50000);
connection.setReadTimeout(50000);
connection.setRequestMethod(requestMethod);
initializeGSON();
return connection;
}
private static CoreEntity[] handleResponse(CoreEntity[] resultList, HttpURLConnection connection) {
try {
final int status = connection.getResponseCode();
if (status == HttpURLConnection.HTTP_OK) { // Success
try (InputStreamReader reader = new InputStreamReader(connection.getInputStream()); BufferedReader in = new BufferedReader(reader)) {
String inputLine;
StringBuilder response = new StringBuilder();
while ((inputLine = in.readLine()) != null) { response.append(inputLine); }
reader.close();
in.close();
JSONArray jsonArray = getJSONAsArray(response.toString());
resultList = (CoreEntity[]) Array.newInstance(resultList.getClass().getComponentType(), jsonArray.length());
for (int i = 0; i < jsonArray.length(); i++)
resultList[i] = (CoreEntity) GSON.fromJson(jsonArray.get(i).toString(), resultList.getClass().getComponentType());
} catch (IOException | JSONException e) { e.printStackTrace(); }
} else {
System.out.println("\nRequest failed with error code: " + status);
}
connection.disconnect();
return resultList;
} catch (ConnectException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
The response for http://www.google.com or any other homepage is successful with status code 200. But as soon as I call my API I get an error with status code 404. Strange is that everything works when I am using Postman or the browser. So when I do a get request via postman to the following method (http://localhost:8080/pets/3), I can see the print out and get the data from mongodb, but not for the code above. For the code above nothing happens on server side, no print out, no exception, nothing.
#RequestMapping(value = "/{id}", method = RequestMethod.GET)
public ResponseEntity<T> getById(#PathVariable final long id) {
System.out.println("TEST ===> " + id);
T entity = getService().getById(id);
return entity == null ? ResponseEntity.notFound().build() : ResponseEntity.ok(entity);
}
It seems like my application is not able to find the API, but I already verified that the URL is correct which is why I don’t understand the error code 404.
I also read about package visibility, but my structure looks like the following which is why I don't think that this is the reason.
Package Structure (Don't be confused from name Aerospike)
I've spend way too much time for this now, and I am really desperate for help and hope you can help me or at least point me in the right direction.
Edit
Here is the entire RestController:
#Controller
public abstract class CoreController<S extends CoreService<T>, T extends CoreEntity> {
public static final String SERVER = "http://localhost", PORT = ":8080",
CORE_API = SERVER + PORT + "/"; // controller/v2/
public static final String ID = "id";
private String api;
public CoreController() { }
public CoreController(final String api) { this.api = CORE_API + api; }
#RequestMapping(value = "/{" + ID + "}", method = RequestMethod.GET)
public ResponseEntity<T> getById(#PathVariable final long id) {
System.out.println("TEST ===> " + id);
T entity = getService().getById(id);
return entity == null ? ResponseEntity.notFound().build() : ResponseEntity.ok(entity);
}
public abstract S getService();
}
#Controller
#RequestMapping(value = "pets/")
public class PetController extends CoreController<PetService, Pet> {
#Autowired
protected PetService service;
public static final String API = "pets";
public PetController() { super(API); }
public PetService getService() { return service; }
}
Here the evidence that the spring-boot is listening on 8080 and also postman works with port 8080.
Server print out on start up

I think you missing the slash ("/") in the start and you have duplicate in the end of the exposed value so it's looking for pets//{id} in the controller change to value = { "/pets"}
Anyway, when starting the service syou should see in the logs the exposed uri's

How to check if website link has data (picture) or website is invalid?

I want to test if website has data in it or it's invalid link in android studio. For example:
Has data:
https://media-cdn.tripadvisor.com/media/photo-o/03/d8/a8/70/zinfandel-s.jpg
Has no data cause link is invalid:
https://media-cdn.tripadvisor.com/media/po/03/d8/a8/70/zdel-s.jpg

We can also use java.net.url class to validate a URL. A MalformedURLExceptio will be thrown if no protocol is specified, or an unknown protocol is found, or spec is null. Then we will call method toURI()that will throw a URISyntaxException if the URL cannot be converted to URI
class URLValidator {
public static boolean urlValidator(String url) {
try {
new URL(url).toURI();
return true;
}
catch (URISyntaxException exception) {
return false;
}
catch (MalformedURLException exception) {
return false;
}
}
public static void main(String[] args)
{
String url = "https://media-cdn.tripadvisor.com/media/po/03/d8/a8/70/zdel-s.jpg";
if (urlValidator(url))
System.out.print("The given URL: " + url + " , contain image.");
else
System.out.print("The given URL: " + url + " , is not contain image.");
}
}

well, you can use okhttp library for maing http-requests.
here's the snippet for you:
private final OkHttpClient client = new OkHttpClient();
public void run() throws Exception {
Request request = new Request.Builder()
.url("https://media-cdn.tripadvisor.com/media/po/03/d8/a8/70/zdel-s.jpg")
.build();
try (Response response = client.newCall(request).execute()) {
if (!response.isSuccessful()) {
//this will run if image is not available
}
}
}

GSSException createCredential

Major edit: 2015-05-27: After some degree of success updated on where I'm currently stuck rather than leaving a rambling post....could really do with some pointers on this one - a little bogged down....
I'm running some code on a Linux app server (WebSphere) that needs to authenticate to an IIS web service which is configured for "Integrated Authentication", but I'm having some problems forming the Authorization: Negotiate token.
I should also say that I need to put this token into the HTTP header for a JAX-WS SOAP request that I will subsequently build. I know my SOAP request itself works because we were using WS-Security Username token profile previously and it worked fine - trying to swap to kerberos is proving difficult...
My problem is with initSecContext I think. It appears that on the first call the context is configured in "some" way and there is some returned token data, but .isEstablished is false. The problem I'm having is putting the initSecContext call into a loop - it seems IIS just closes the connection when I do this. Can anyone give me some pointers - I seem to be taking the approach used by other posters and the Oracle samples (although the IBM/WebSphere sample only makes a single initSecContext call and doesn't check .isEstablished which seems odd to me based on the Oracle documentation).
Anyway, the error I get is below (note the Ready: property seems to clearly say initSecContext needs to loop - to me at least);
[5/27/15 6:51:11:605 UTC] 0000004f SystemOut O INFO: com.mycorp.kerberosKerberosTokenGenerator/getKerberosToken/run: After initSecContext:
--- GSSContext ---
Owner: domainuser#MYDOMAIN.COM
Peer: HTTP/iishost.mycorp.com
State: initialized
Lifetime: indefinite
Ready: no
Flags:
Confidentiality off
Delegation on
Integrity off
MutualAuthn on
ReplayDetection off
SequenceDetection off
DelegatedCred: unknown
--- End of GSSContext ---
[5/27/15 6:51:11:605 UTC] 0000004f SystemOut O INFO: com.mycorp.kerberosKerberosTokenGenerator/getKerberosToken/run: Context is not established, trying again
[5/27/15 6:51:11:606 UTC] 0000004f SystemOut O ERROR: com.mycorp.kerberosKerberosTokenGenerator/getKerberosToken/run: IOException during context establishment: Connection reset
My code is below;
LoginContext lc = getLoginContext(contextName);
final Subject subject = lc.getSubject();
String b64Token = (String) Subject.doAs(subject, new PrivilegedExceptionAction() {
#Override
public Object run() throws PrivilegedActionException, GSSException {
// Create socket to server
Socket socket;
DataInputStream inStream = null;
DataOutputStream outStream = null;
try {
socket = new Socket("iishost.mycorp.com", 443);
inStream = new DataInputStream(socket.getInputStream());
outStream = new DataOutputStream(socket.getOutputStream());
} catch (IOException ex) {
System.out.println("Exception setting up server sockets: " + ex.getMessage());
}
GSSName gssName = manager.createName(userName, GSSName.NT_USER_NAME, KRB5_MECH_OID);
GSSCredential gssCred = manager.createCredential(gssName.canonicalize(KRB5_MECH_OID),
GSSCredential.DEFAULT_LIFETIME,
KRB5_MECH_OID,
GSSCredential.INITIATE_ONLY);
gssCred.add(gssName, GSSCredential.INDEFINITE_LIFETIME,
GSSCredential.INDEFINITE_LIFETIME,
SPNEGO_MECH_OID,
GSSCredential.INITIATE_ONLY);
GSSName gssServerName = manager.createName(servicePrincipal, KERBEROS_V5_PRINCIPAL_NAME);
GSSContext clientContext = manager.createContext(gssServerName.canonicalize(SPNEGO_MECH_OID),
SPNEGO_MECH_OID,
gssCred,
GSSContext.DEFAULT_LIFETIME);
clientContext.requestCredDeleg(true);
clientContext.requestMutualAuth(true);
byte[] token = new byte[0];
while (!clientContext.isEstablished()) {
try {
token = clientContext.initSecContext(token, 0, token.length);
// IF I LOOK AT token HERE THERE IS CERTAINLY TOKEN DATA THERE - .isEstablished IS STILL FALSE
outStream.writeInt(token.length);
outStream.write(token);
outStream.flush();
// Check if we're done
if (!clientContext.isEstablished()) {
token = new byte[inStream.readInt()];
inStream.readFully(token);
}
} catch (IOException ex) {
// THIS EXCEPTION IS THROWN ON SECOND ITERATION - LOOKS LIKE IIS CLOSES THE CONNECTION
System.out.println("IOException during context establishment: " + ex.getMessage());
}
}
String b64Token = Base64.encode(token);
clientContext.dispose(); // I'm assuming this won't invalidate the token in some way as I need to use it later
return b64Token;
}
});
This doc tells me I don't need to loop on initSecContext, but .isEstablished returns false for me: http://www-01.ibm.com/support/knowledgecenter/SS7K4U_8.5.5/com.ibm.websphere.zseries.doc/ae/tsec_SPNEGO_token.html?cp=SS7K4U_8.5.5%2F1-3-0-20-4-0&lang=en
The Oracle docs tell me I should: https://docs.oracle.com/javase/7/docs/api/org/ietf/jgss/GSSContext.html
My only hesitation is that from the Oracle docs it seems like I'm starting the application conversation, but what I'm trying to do it obtain the token only & it's later on in my code when I will use JAX-WS to post my actual web service call (including the spnego/kerberos token in the http header) - is this the cause of my issue?

Just an update. I have this working now - my previous code was largely ok - it was just my understanding of how the Kerberos token would be added to the JAX-WS request. Turns out it's just a matter of attaching a Handler to the bindingProvider. The handler then obtains the Kerberos token and adds it to the header of the request - nice and easy.
Below is my working Handler which is added to the Handler chain obtained from a call to bindingProvider.getBinding().getHandlerChain()
public class HTTPKerberosHandler implements SOAPHandler<SOAPMessageContext> {
private final String contextName;
private final String servicePrincipal;
private static Oid KRB5_MECH_OID = null;
private static Oid SPNEGO_MECH_OID = null;
private static Oid KERBEROS_V5_PRINCIPAL_NAME = null;
final String className = this.getClass().getName();
static {
try {
KERBEROS_V5_PRINCIPAL_NAME = new Oid("1.2.840.113554.1.2.2.1");
KRB5_MECH_OID = new Oid("1.2.840.113554.1.2.2");
SPNEGO_MECH_OID = new Oid("1.3.6.1.5.5.2");
} catch (final GSSException ex) {
System.out.println("Exception creating mechOid's: " + ex.getMessage());
ex.printStackTrace();
}
}
public HTTPKerberosHandler(final String contextName, final String servicePrincipal) {
this.contextName = contextName;
this.servicePrincipal = servicePrincipal;
}
#Override
public Set<QName> getHeaders() {
return null;
}
#Override
public boolean handleFault(SOAPMessageContext context) {
return false;
}
#Override
public void close(MessageContext context) {
// No action
}
#Override
public boolean handleMessage(SOAPMessageContext context) {
if (((Boolean) context.get(SOAPMessageContext.MESSAGE_OUTBOUND_PROPERTY))) {
return handleRequest(context);
} else {
return handleResponse(context);
}
}
private boolean handleRequest(SOAPMessageContext context) {
byte[] token = getKerberosToken(contextName, servicePrincipal);
HashMap<String, String> sendTransportHeaders = new HashMap<String, String>();
sendTransportHeaders.put("Authorization", "Negotiate " + Base64.encode(token));
context.put(com.ibm.websphere.webservices.Constants.REQUEST_TRANSPORT_PROPERTIES, sendTransportHeaders);
return true;
}
private boolean handleResponse(SOAPMessageContext context) {
logger.logInformation(className, "handleResponse", "Inbound response detected");
return true;
}
public byte[] getKerberosToken(final String contextName, final String servicePrincipal) {
try {
LoginContext lc = getLoginContext(contextName);
final Subject subject = lc.getSubject();
byte[] token = (byte[]) Subject.doAs(subject, new PrivilegedExceptionAction() {
#Override
public Object run() throws PrivilegedActionException, GSSException {
final String methodName = "getKerberosToken/run";
final GSSManager manager = GSSManager.getInstance();
Set<Principal> principals = subject.getPrincipals();
Iterator it = principals.iterator();
String principalName = ((Principal) it.next()).getName();
logger.logInformation(className, methodName, "Using principal: [" + principalName + "]");
GSSName gssName = manager.createName(principalName, GSSName.NT_USER_NAME, KRB5_MECH_OID);
GSSCredential gssCred = manager.createCredential(gssName.canonicalize(KRB5_MECH_OID),
GSSCredential.DEFAULT_LIFETIME,
KRB5_MECH_OID,
GSSCredential.INITIATE_ONLY);
gssCred.add(gssName, GSSCredential.INDEFINITE_LIFETIME,
GSSCredential.INDEFINITE_LIFETIME,
SPNEGO_MECH_OID,
GSSCredential.INITIATE_ONLY);
logger.logInformation(className, methodName, "Client TGT obtained: " + gssCred.toString());
GSSName gssServerName = manager.createName(servicePrincipal, GSSName.NT_USER_NAME);
GSSContext clientContext = manager.createContext(gssServerName.canonicalize(SPNEGO_MECH_OID),
SPNEGO_MECH_OID,
gssCred,
GSSContext.DEFAULT_LIFETIME);
logger.logInformation(className, methodName, "Service ticket obtained: " + clientContext.toString());
byte[] token = new byte[0];
token = clientContext.initSecContext(token, 0, token.length);
clientContext.dispose();
return token;
}
});
return token;
} catch (PrivilegedActionException ex) {
logger.logError(HTTPKerberosHandler.class.getName(), methodName, "PrivilegedActionException: " + ex.getMessage());
} catch (Exception ex) {
logger.logError(HTTPKerberosHandler.class.getName(), methodName, "Exception: " + ex.getMessage());
}
return null;
}
private LoginContext getLoginContext(String contextName) {
LoginContext lc = null;
try {
lc = new LoginContext(contextName);
lc.login();
} catch (LoginException le) {
logger.logError(HTTPKerberosHandler.class.getName(), methodName, "Login exception: [" + le.getMessage() + "]");
le.printStackTrace();
}
return lc;
}
}

Handling Http status code 302 Moved Temporarily in Netty

I am getting 302 status code for the http request I am making to my URL.. I want it to be handled by my netty code..
My client code :
public static void main(String[] args) throws Exception {
URI uri = new URI("http://myurl.mydomain.com/v1/v2?param1=value1&param2=value2");
String host = uri.getHost();
int port = 80;
// Configure the client.
EventLoopGroup group = new NioEventLoopGroup();
try {
Bootstrap b = new Bootstrap();
b.group(group)
.channel(NioSocketChannel.class)
.handler(new NettyClientInitializer());
// Make the connection attempt.
Channel ch = b.connect(host, port).sync().channel();
// Prepare the HTTP request.
HttpRequest request = new DefaultHttpRequest(
HttpVersion.HTTP_1_1, HttpMethod.GET, uri.toString());
request.headers().set(HttpHeaders.Names.HOST, host);
request.headers().set(HttpHeaders.Names.CONNECTION, HttpHeaders.Values.KEEP_ALIVE);
request.headers().set(HttpHeaders.Names.CACHE_CONTROL, HttpHeaders.Values.NO_CACHE);
/*// Set some example cookies.
request.headers().set(
HttpHeaders.Names.COOKIE,
ClientCookieEncoder.encode(
new DefaultCookie("my-cookie", "foo"),
new DefaultCookie("another-cookie", "bar")));
*/
// Send the HTTP request.
ch.writeAndFlush(request);
// Wait for the server to close the connection.
ch.closeFuture().sync();
} finally {
// Shut down executor threads to exit.
group.shutdownGracefully();
}
}
My handler code :
public class NettyClientHandler extends SimpleChannelInboundHandler<HttpObject> {
#Override
public void channelRead0(ChannelHandlerContext ctx, HttpObject msg) throws Exception {
if (msg instanceof HttpResponse) {
HttpResponse response = (HttpResponse) msg;
System.out.println("STATUS: " + response.getStatus());
System.out.println("VERSION: " + response.getProtocolVersion());
System.out.println();
if (!response.headers().isEmpty()) {
for (String name: response.headers().names()) {
for (String value: response.headers().getAll(name)) {
System.out.println("HEADER: " + name + " = " + value);
}
}
System.out.println();
}
if (HttpHeaders.isTransferEncodingChunked(response)) {
System.out.println("CHUNKED CONTENT {");
} else {
System.out.println("CONTENT {");
}
}
if (msg instanceof HttpContent) {
HttpContent content = (HttpContent) msg;
System.out.print(content.content().toString(CharsetUtil.UTF_8));
System.out.flush();
if (content instanceof LastHttpContent) {
System.out.println("} END OF CONTENT");
}
}
}
#Override
public void exceptionCaught(
ChannelHandlerContext ctx, Throwable cause) throws Exception {
cause.printStackTrace();
ctx.close();
}
}
My initializer code :
public class NettyClientInitializer extends ChannelInitializer<SocketChannel> {
#Override
public void initChannel(SocketChannel ch) throws Exception {
// Create a default pipeline implementation.
ChannelPipeline p = ch.pipeline();
p.addLast("log", new LoggingHandler(LogLevel.INFO));
// Enable HTTPS if necessary.
/*
if (ssl) {
SSLEngine engine =
SecureChatSslContextFactory.getClientContext().createSSLEngine();
engine.setUseClientMode(true);
p.addLast("ssl", new SslHandler(engine));
}
*/
p.addLast("codec", new HttpClientCodec());
// Remove the following line if you don't want automatic content decompression.
// p.addLast("inflater", new HttpContentDecompressor());
// Uncomment the following line if you don't want to handle HttpChunks.
p.addLast("aggregator", new HttpObjectAggregator(1048576));
p.addLast("handler", new NettyClientHandler());
}
}
I referred to this link with similar problem :
redirect - handling http 302 moved temporarily using netty
but the code in this using 3.x version of the library and also there is no answer to this question as of now..
I am using Netty 4.0.12 library..
Please tell me how to handle this using Netty

You can modify your NettyClientHandler to check for 302 Redirects, and open a new connection to handle the HTML content of the redirect.
Changes made to NettyClientHandler:
//We know this is a redirect...
if(response.getStatus().code() == HttpResponseStatus.FOUND.code()){//When its a 302...
if(response.headers().names().contains("Location"))
{
System.out.println("We have a redirect...");
//Now we will do the process over to get the actual content...
Main.main(new String[]{response.headers().get("Location")});
}
}
Changes made to main() as an example to handle the content of the redirect:
String urlPlace = "http://initial.com";
if(args != null && args.length > 0)
{
urlPlace = args[0];
}
URI uri = new URI(urlPlace);
String host = uri.getHost();
int port = uri.getPort();
if(port == -1)
{
port = 80;
}
When we get a HTTP Status code 302, it is the "servers" responsibility to set the Location header for the new URL location in order for the client to handle appropriately.
See Wikipedia 302