I have a java program, that connects to a website to retrieve some XML from it. This works fine on my computer, as well as others outside our company. One of our customers is now not able to connect to the website. I figured out, that they are behind a proxy. I have now found which settings I need to use, and in my test program it works (partially).
In the code below, the downloadFile() call works as expected, and the file can be downloaded without problems. The contactHost() fails on our client machines with an UnknownHostException:
java.net.UnknownHostException: No such host is known (api.myserver.de)
at java.base/java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
at java.base/java.net.InetAddress$PlatformNameService.lookupAllHostAddr(InetAddress.java:925)
at java.base/java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1505)
at java.base/java.net.InetAddress$NameServiceAddresses.get(InetAddress.java:844)
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1495)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1354)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1288)
at org.apache.http.impl.conn.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:45)
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:111)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
Background: Windows 10 machines, our program is shipped with an internal OpenJDK, version "10.0.2" 2018-07-17. The program is started with the following defines -Djdk.http.auth.tunneling.disabledSchemes="" -Djava.net.preferIPv4Stack=true in order to use IP4 only, and to enable BasicAuthentification for the Proxy. With these settings, the file can be downloaded, however the UnknownHostException is still there.
We have also tried to open the used URL in an browser, and this works as excepted, i.e. in the browser the website is opened.
Here is my code for testing:
import java.io.BufferedInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.Authenticator;
import java.net.URL;
import java.net.URLConnection;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClientBuilder;
public class LFTProxyTest {
private static String uname = null;
private static String pass = null;
public static void main(String[] args) {
System.setProperty("java.net.useSystemProxies", "true");
// uname = "test"; // whatever that user provides
// pass = "sectret"; // whatever that user provides
Authenticator.setDefault(new ProxyAuth(uname, pass));
contactHost();
downloadFile();
}
private static boolean downloadFile() {
System.out.println("CHECK connection");
int cp = contactHost();
if (cp == 200)
return true;
if (cp == 407)
return false;
else {
try {
System.out.println("Try loading file: ");
URL url = new URL("https://www.google.de");
URLConnection urlConnection = url.openConnection();
InputStream in = new BufferedInputStream(urlConnection.getInputStream());
DocumentBuilderFactory dbFactory = DocumentBuilderFactory.newInstance();
DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
dBuilder.parse(in);
System.out.println(" FILE DOWNLOAD successfull!");
} catch (Exception e) {
System.out.println(" FILE DOWNLOAD failed:");
System.out.println("***EXCEPTION: " + e.getMessage());
return false;
}
}
System.out.println("CHECK done");
return true;
}
private static int contactHost() {
HttpClient client = HttpClientBuilder.create().build();// new DefaultHttpClient();
String catalogURI = "https://api.myserver.de/query";
HttpGet request = new HttpGet(catalogURI);
try {
int ret = 0;
HttpResponse response = client.execute(request);
ret = response.getStatusLine().getStatusCode();
System.out.println("PROXY test: " + ret);
((CloseableHttpClient) client).close();
return ret;
} catch (IOException e) {
e.printStackTrace();
return -1;
}
}
}
I don't know what do do know, I'm not even sure where the error could be. Any ideas are highly appreciated!
Ok, so after some further digging, I found out that org.apache.http.client.HttpClient is not respecting java.net.useSystemProxies at all, be it set via System or via -D. And it is also ignoring http.proxyHost etc. Solution is to use a ProxySelector like this:
ProxySelector.setDefault(new ProxySelector() {
#Override
public List<Proxy> select(URI uri) {
ArrayList<Proxy> list = new ArrayList<Proxy>();
list.add(new Proxy(Proxy.Type.HTTP, new InetSocketAddress("proxy1.de", 8000)));
list.add(new Proxy(Proxy.Type.HTTP, new InetSocketAddress("proxy2.de", 8080)));
return list;
}
#Override
public void connectFailed(URI uri, SocketAddress sa, IOException ioe) {
logger.error("Error in ProxySelector, connection Failed: ", ioe);
}
});
I'm getting another exception now, but I might open another thread for this.
UnknownHostException designates a pretty straight forward problem. That the IP address of the remote host you are trying to reach cannot be resolved. So the solution to this is very simple. You should check the input of Socket (or any other method that throws an UnknownHostException), and validate that it is the intended one. If you are not whether you have the correct host name, you can launch a UNIX terminal and use the nslookup command (among others) to see if your DNS server can resolve the host name to an IP address successfully.
If you are on Windows you can use the host command. If that doesn’t work as expected then, you should check if the host name you have is correct and then try to refresh your DNS cache. If that doesn’t work either, try to use a different DNS server, eg Google Public DNS is a very good alternative.
History for context:
I am trying to run a web job from an HTTP Client. The file is a ZIP file . and contains a java class and bat file to run that java class. This runs okay when i do from POSTMAN. But when i use HTTP client, i get the following error always " '---i-NPsGbTVUpaP0CeJxMQVrHoDHvaxo3' is not recognized as an internal or external command" - Please help – Jagaran yesterday
#Jagaran if it only happen from some clients, it is likely unrelated. Please ask a new question – David Ebbo 21 hours ago
No any HTTP Client i am using in java, it is the same. it works in CURL or loading from web console. My sample code below – Jagaran 2 hours ago
No any HTTP Client i am using in java, it is the same. it works in CURL or loading from web console.
Do you have any sample Java based HTTP Client where I can publish Azure Web Job? I have tried all Java REST clients.
May be i am doing something wrong. The error I get in Azure console is '---i-NPsGbTVUpaP0CeJxMQVrHoDHvaxo3' is not recognized as an internal or external command, [08/25/2017 09:30:22 > e7f683: ERR ] operable program or batch file.o
I feel Content type = applciation /zip is not happening correctly when using java. Please help us.
Sample Code:
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.MalformedURLException;
import java.net.URI;
import java.net.URL;
import java.util.zip.ZipEntry;
import java.util.zip.ZipFile;
import org.apache.http.entity.ContentType;
import com.mashape.unirest.http.HttpResponse;
import com.mashape.unirest.http.Unirest;
/**
* #author jagaran.das
*
*/
public class AIPHTTPClient {
/**
* #param args
* #throws IOException
*/
#SuppressWarnings({ "unused", "rawtypes" })
public static void main(String[] args) throws IOException {
try {
URI uri = new AIPHTTPClient().getURI();
HttpResponse<InputStream> jsonResponse = Unirest.put("https://<URL>/api/triggeredwebjobs/TestJOb")
.basicAuth("$AzureWebJobTestBRMS", "XXXXX")
.header("content-disposition","attachement; filename=acvbgth.bat")
.field("file", new FileInputStream(new File(uri))
,ContentType.create("content-type: application/zip"),"AzureWebJob.zip").asBinary();
System.out.println(jsonResponse.getStatusText());
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
public InputStream readZip() {
ZipFile zipFile = null;
ZipEntry zipEntry = zipFile.getEntry("run.bat");
InputStream stream = null;
/* try {
zipFile = new ZipFile("/Users/jagaran.das/Documents/work/AIP/AzureWebJob.zip");
java.util.Enumeration<? extends ZipEntry> entries = zipFile.entries();
while(entries.hasMoreElements()){
ZipEntry entry = entries.nextElement();
stream = zipFile.getInputStream(entry);
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} */
try {
stream = zipFile.getInputStream(zipEntry);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return stream;
}
public URI getURI() throws MalformedURLException {
File file = new File("/Users/jagaran.das/Documents/work/AIP/azure-poc/AzureWebJob.zip");
URI fileUri = file.toURI();
System.out.println("URI:" + fileUri);
URL fileUrl = file.toURI().toURL();
System.out.println("URL:" + fileUrl);
URL fileUrlWithoutSpecialCharacterHandling = file.toURL();
System.out.println("URL (no special character handling):" + fileUrlWithoutSpecialCharacterHandling);
return fileUri;
}
}
I've been a little too harsh in my answer before really trying stuff out. Apologies. I've now tried out your snippet and looks like you're hitting an issue with Unirest - probably this one.
My advice would be to just move to Apache's HTTP library.
Here's a working sample:
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.client.HttpClient;
import org.apache.http.client.entity.EntityBuilder;
import org.apache.http.client.methods.HttpPut;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.util.EntityUtils;
import java.io.File;
public class App
{
public static void main( String[] args )
{
File sourceZipFile = new File("webjob.zip");
String kuduApiUrl = "https://yoursitename.scm.azurewebsites.net/api/zip/site/wwwroot/app_data/jobs/triggered/job988/";
HttpEntity httpEntity = EntityBuilder.create()
.setFile(sourceZipFile)
.build();
CredentialsProvider provider = new BasicCredentialsProvider();
UsernamePasswordCredentials credentials = new UsernamePasswordCredentials(
"$yoursitename", "SiteLevelPasSw0rD"
);
provider.setCredentials(AuthScope.ANY, credentials);
HttpClient client = HttpClientBuilder.create()
.setDefaultCredentialsProvider(provider)
.build();
HttpPut putRequest = new HttpPut(kuduApiUrl);
putRequest.setEntity(httpEntity);
// Kudu's Zip API expects application/zip
putRequest.setHeader("Content-type", "application/zip");
try {
HttpResponse response = client.execute(putRequest);
int statusCode = response.getStatusLine().getStatusCode();
HttpEntity entity = response.getEntity();
String resBody = EntityUtils.toString(entity, "UTF-8");
System.out.println(statusCode);
System.out.println(resBody);
}
catch (Exception e) {
e.printStackTrace();
}
}
}
That's sending Content-Type: application/zip and the raw zip contents in the body (no multipart horse manure). I've probably over-engineered the sample.. but it is what it is.
The upload is successful and the WebJob published:
Glad for you that you have solved the issue and I try to provide a workaround for your reference.
Deploy WebJob to azure , in addition to using REST API, you can also use the FTP way. Of course, the premise is that you need to know the directory uploaded by webjob via KUDU.
I offer you the snippet of code below via FTP4J libiary:
import java.io.File;
import it.sauronsoftware.ftp4j.FTPClient;
public class UploadFileByFTP {
private static String hostName = <your host name>;
private static String userName = <user name>;
private static String password = <password>;
public static void main(String[] args) {
try {
// create client
FTPClient client = new FTPClient();
// connect host
client.connect(hostName);
// log in
client.login(userName, password);
// print address
System.out.println(client);
// change directory
client.changeDirectory("/site/wwwroot/App_Data/jobs/continuous");
// current directory
String dir = client.currentDirectory();
System.out.println(dir);
File file = new File("D:/test.zip");
client.upload(file);
} catch (Exception e) {
e.printStackTrace();
}
}
}
You can follow this tutorial to configure your parameters.
I'm trying to use my phone as a realtime MJPEG video source. So far, capturing frames and converting them into JPEGs is no big deal. My real issue is sending the multipart response properly. There's tons of documentation about sending multipart responses out there, but the issue with them is that they all expect that all of the images are available at the time the HTTP request comes in (such as would be used for a multi-image upload). In order to stream in realtime, of course, I need to be able to begin to send the multipart response while continually adding jpegs in the body. I'm by no means a HTTP buff, so it's not desirable for me be required to roll my own HTTP response and write directly to a socket. Is there a library out there that supports this kind of behavior? I've scoured the internet for solutions, but I really don't see anything useful out there.
Any ideas? Worst case scenario, I'd be willing to look at human-readable documentation of how to write a multipart response by hand, but I'd really just rather use a library if that's possible.
Thanks in advance.
edit: got it working using the orielly servlet library as per sigmavirus' suggestion. Note that the MJPEG stream is more or less implicitly inferred from the fact that I'm sending a multipart/x-mixed-replace that only has image/jpeg's in it. Check out the comment in my code for a tutorial that shows what jetty libraries you'll need to get this running. Of course, you'll additionally need cos.jar, the Orielly servlet library. The code follows:
package edu.stevens.arpac.webclient;
import java.io.IOException;
import java.net.InetAddress;
import java.net.NetworkInterface;
import java.util.Collections;
import java.util.List;
import javax.servlet.ServletException;
import javax.servlet.ServletOutputStream;
import javax.servlet.http.HttpServletResponse;
import javax.servlet.http.HttpServletRequest;
import org.apache.http.conn.util.InetAddressUtils;
import org.eclipse.jetty.server.Handler;
import org.eclipse.jetty.server.Server;
import org.eclipse.jetty.server.handler.AbstractHandler;
import org.eclipse.jetty.server.Request;
import com.oreilly.servlet.MultipartResponse;
import com.oreilly.servlet.ServletUtils;
import android.os.Environment;
import android.util.Log;
// holla at http://puregeekjoy.blogspot.com/2011/06/running-embedded-jetty-in-android-app.html
public class JettyServer extends Thread
{
private static final String TAG = "JettyServer";
private Server webServer;
private Boolean isStarted = false;
public JettyServer()
{
super();
Log.i(TAG, "Initializing server to port 8080");
webServer = new Server(8080);
Handler handler = new AbstractHandler() {
public void handle(String target, Request request, HttpServletRequest servletRequest,
HttpServletResponse servletResponse) throws IOException, ServletException {
ServletOutputStream out = servletResponse.getOutputStream();
MultipartResponse multi = new MultipartResponse(servletResponse);
Boolean go = true;
while( go )
{
try
{
multi.startResponse("image/jpeg");
ServletUtils.returnFile(Environment.getExternalStorageDirectory().getPath() + "/ARPac/twi.jpg", out);
multi.endResponse();
}
catch(IOException ex)
{
go = false;
Log.i(TAG, "IO Failed with exception " + ex.getMessage());
}
}
request.setHandled(true);
}
};
webServer.setHandler(handler);
try {
webServer.start();
Log.d(TAG, "started Web server # " + getIPAddress());
isStarted = true;
}
catch (Exception e) {
Log.d(TAG, "unexpected exception starting Web server: " + e);
}
}
/**
* Get IP address from first non-localhost interface
* #return address or empty string
*/
private String getIPAddress()
{
try
{
List<NetworkInterface> interfaces = Collections.list(NetworkInterface.getNetworkInterfaces());
for (NetworkInterface intf : interfaces)
{
List<InetAddress> addrs = Collections.list(intf.getInetAddresses());
for (InetAddress addr : addrs)
{
if (!addr.isLoopbackAddress())
{
String sAddr = addr.getHostAddress().toUpperCase();
if (InetAddressUtils.isIPv4Address(sAddr))
{
//Log.d(TAG, "IP address is: " + sAddr);
return sAddr;
}
}
}
}
}
catch (Exception ex)
{
Log.e(TAG, "could not get IP address: " + ex.getMessage());
} // for now eat exceptions
Log.e(TAG, "Could not find a non-loopback IPv4 address!");
return "";
}
public void teardown()
{
if( isStarted )
{
try {
webServer.stop();
isStarted = false;
} catch (Exception e) {
Log.e(TAG, "Couldn't stop server. Probably was called when server already stopped.");
}
}
}
public void run()
{
}
}
Have you seen this? http://www.servlets.com/cos/javadoc/com/oreilly/servlet/MultipartResponse.html It looks like the example sends each part individually and waits a specified time limit before sending the next or receiving an interrupt.
Is there a way to create a very basic HTTP server (supporting only GET/POST) in Java using just the Java SE API, without writing code to manually parse HTTP requests and manually format HTTP responses? The Java SE API nicely encapsulates the HTTP client functionality in HttpURLConnection, but is there an analog for HTTP server functionality?
Just to be clear, the problem I have with a lot of ServerSocket examples I've seen online is that they do their own request parsing/response formatting and error handling, which is tedious, error-prone, and not likely to be comprehensive, and I'm trying to avoid it for those reasons.
Since Java SE 6, there's a builtin HTTP server in Sun Oracle JRE. The Java 9 module name is jdk.httpserver. The com.sun.net.httpserver package summary outlines the involved classes and contains examples.
Here's a kickoff example copypasted from their docs. You can just copy'n'paste'n'run it on Java 6+.
(to all people trying to edit it nonetheless, because it's an ugly piece of code, please don't, this is a copy paste, not mine, moreover you should never edit quotations unless they have changed in the original source)
package com.stackoverflow.q3732109;
import java.io.IOException;
import java.io.OutputStream;
import java.net.InetSocketAddress;
import com.sun.net.httpserver.HttpExchange;
import com.sun.net.httpserver.HttpHandler;
import com.sun.net.httpserver.HttpServer;
public class Test {
public static void main(String[] args) throws Exception {
HttpServer server = HttpServer.create(new InetSocketAddress(8000), 0);
server.createContext("/test", new MyHandler());
server.setExecutor(null); // creates a default executor
server.start();
}
static class MyHandler implements HttpHandler {
#Override
public void handle(HttpExchange t) throws IOException {
String response = "This is the response";
t.sendResponseHeaders(200, response.length());
OutputStream os = t.getResponseBody();
os.write(response.getBytes());
os.close();
}
}
}
Noted should be that the response.length() part in their example is bad, it should have been response.getBytes().length. Even then, the getBytes() method must explicitly specify the charset which you then specify in the response header. Alas, albeit misguiding to starters, it's after all just a basic kickoff example.
Execute it and go to http://localhost:8000/test and you'll see the following response:
This is the response
As to using com.sun.* classes, do note that this is, in contrary to what some developers think, absolutely not forbidden by the well known FAQ Why Developers Should Not Write Programs That Call 'sun' Packages. That FAQ concerns the sun.* package (such as sun.misc.BASE64Encoder) for internal usage by the Oracle JRE (which would thus kill your application when you run it on a different JRE), not the com.sun.* package. Sun/Oracle also just develop software on top of the Java SE API themselves like as every other company such as Apache and so on. Moreover, this specific HttpServer must be present in every JDK so there is absolutely no means of "portability" issue like as would happen with sun.* package. Using com.sun.* classes is only discouraged (but not forbidden) when it concerns an implementation of a certain Java API, such as GlassFish (Java EE impl), Mojarra (JSF impl), Jersey (JAX-RS impl), etc.
Check out NanoHttpd
NanoHTTPD is a light-weight HTTP server designed for embedding in other applications, released under a Modified BSD licence.
It is being developed at Github and uses Apache Maven for builds & unit testing"
The com.sun.net.httpserver solution is not portable across JREs. Its better to use the official webservices API in javax.xml.ws to bootstrap a minimal HTTP server...
import java.io._
import javax.xml.ws._
import javax.xml.ws.http._
import javax.xml.transform._
import javax.xml.transform.stream._
#WebServiceProvider
#ServiceMode(value=Service.Mode.PAYLOAD)
class P extends Provider[Source] {
def invoke(source: Source) = new StreamSource( new StringReader("<p>Hello There!</p>"));
}
val address = "http://127.0.0.1:8080/"
Endpoint.create(HTTPBinding.HTTP_BINDING, new P()).publish(address)
println("Service running at "+address)
println("Type [CTRL]+[C] to quit!")
Thread.sleep(Long.MaxValue)
EDIT: this actually works! The above code looks like Groovy or something. Here is a translation to Java which I tested:
import java.io.*;
import javax.xml.ws.*;
import javax.xml.ws.http.*;
import javax.xml.transform.*;
import javax.xml.transform.stream.*;
#WebServiceProvider
#ServiceMode(value = Service.Mode.PAYLOAD)
public class Server implements Provider<Source> {
public Source invoke(Source request) {
return new StreamSource(new StringReader("<p>Hello There!</p>"));
}
public static void main(String[] args) throws InterruptedException {
String address = "http://127.0.0.1:8080/";
Endpoint.create(HTTPBinding.HTTP_BINDING, new Server()).publish(address);
System.out.println("Service running at " + address);
System.out.println("Type [CTRL]+[C] to quit!");
Thread.sleep(Long.MAX_VALUE);
}
}
I like this question because this is an area where there's continuous innovation and there's always a need to have a light server especially when talking about embedded servers in small(er) devices. I think answers fall into two broad groups.
Thin-server: server-up static content with minimal processing, context or session processing.
Small-server: ostensibly a has many httpD-like server qualities with as small a footprint as you can get away with.
While I might consider HTTP libraries like: Jetty, Apache Http Components, Netty and others to be more like a raw HTTP processing facilities. The labelling is very subjective, and depends on the kinds of thing you've been call-on to deliver for small-sites. I make this distinction in the spirit of the question, particularly the remark about...
"...without writing code to manually parse HTTP requests and manually format HTTP responses..."
These raw tools let you do that (as described in other answers). They don't really lend themselves to a ready-set-go style of making a light, embedded or mini-server. A mini-server is something that can give you similar functionality to a full-function web server (like say, Tomcat) without bells and whistles, low volume, good performance 99% of the time. A thin-server seems closer to the original phrasing just a bit more than raw perhaps with a limited subset functionality, enough to make you look good 90% of the time. My idea of raw would be makes me look good 75% - 89% of the time without extra design and coding. I think if/when you reach the level of WAR files, we've left the "small" for bonsi servers that looks like everything a big server does smaller.
Thin-server options
Grizzly
UniRest (multiple-languages)
NanoHTTPD (just one file)
Mini-server options:
Spark Java ... Good things are possible with lots of helper constructs like Filters, Templates, etc.
MadVoc ... aims to be bonsai and could well be such ;-)
Among the other things to consider, I'd include authentication, validation, internationalisation, using something like FreeMaker or other template tool to render page output. Otherwise managing HTML editing and parameterisation is likely to make working with HTTP look like noughts-n-crosses. Naturally it all depends on how flexible you need to be. If it's a menu-driven FAX machine it can be very simple. The more interactions, the 'thicker' your framework needs to be. Good question, good luck!
Have a look at the "Jetty" web server Jetty. Superb piece of Open Source software that would seem to meet all your requirments.
If you insist on rolling your own then have a look at the "httpMessage" class.
Once upon a time I was looking for something similar - a lightweight yet fully functional HTTP server that I could easily embed and customize. I found two types of potential solutions:
Full servers that are not all that lightweight or simple (for an extreme definition of lightweight.)
Truly lightweight servers that aren't quite HTTP servers, but glorified ServerSocket examples that are not even remotely RFC-compliant and don't support commonly needed basic functionality.
So... I set out to write JLHTTP - The Java Lightweight HTTP Server.
You can embed it in any project as a single (if rather long) source file, or as a ~50K jar (~35K stripped) with no dependencies. It strives to be RFC-compliant and includes extensive documentation and many useful features while keeping bloat to a minimum.
Features include: virtual hosts, file serving from disk, mime type mappings via standard mime.types file, directory index generation, welcome files, support for all HTTP methods, conditional ETags and If-* header support, chunked transfer encoding, gzip/deflate compression, basic HTTPS (as provided by the JVM), partial content (download continuation), multipart/form-data handling for file uploads, multiple context handlers via API or annotations, parameter parsing (query string or x-www-form-urlencoded body), etc.
I hope others find it useful :-)
Spark is the simplest, here is a quick start guide: http://sparkjava.com/
All the above answers details about Single main threaded Request Handler.
setting:
server.setExecutor(java.util.concurrent.Executors.newCachedThreadPool());
Allows multiple request serving via multiple threads using executor service.
So the end code will be something like below:
import java.io.IOException;
import java.io.OutputStream;
import java.net.InetSocketAddress;
import com.sun.net.httpserver.HttpExchange;
import com.sun.net.httpserver.HttpHandler;
import com.sun.net.httpserver.HttpServer;
public class App {
public static void main(String[] args) throws Exception {
HttpServer server = HttpServer.create(new InetSocketAddress(8000), 0);
server.createContext("/test", new MyHandler());
//Thread control is given to executor service.
server.setExecutor(java.util.concurrent.Executors.newCachedThreadPool());
server.start();
}
static class MyHandler implements HttpHandler {
#Override
public void handle(HttpExchange t) throws IOException {
String response = "This is the response";
long threadId = Thread.currentThread().getId();
System.out.println("I am thread " + threadId );
response = response + "Thread Id = "+threadId;
t.sendResponseHeaders(200, response.length());
OutputStream os = t.getResponseBody();
os.write(response.getBytes());
os.close();
}
}
}
It's possible to create an httpserver that provides basic support for J2EE servlets with just the JDK and the servlet api in a just a few lines of code.
I've found this very useful for unit testing servlets, as it starts much faster than other lightweight containers (we use jetty for production).
Most very lightweight httpservers do not provide support for servlets, but we need them, so I thought I'd share.
The below example provides basic servlet support, or throws and UnsupportedOperationException for stuff not yet implemented. It uses the com.sun.net.httpserver.HttpServer for basic http support.
import java.io.*;
import java.lang.reflect.*;
import java.net.InetSocketAddress;
import java.util.*;
import javax.servlet.*;
import javax.servlet.http.*;
import com.sun.net.httpserver.HttpExchange;
import com.sun.net.httpserver.HttpHandler;
import com.sun.net.httpserver.HttpServer;
#SuppressWarnings("deprecation")
public class VerySimpleServletHttpServer {
HttpServer server;
private String contextPath;
private HttpHandler httpHandler;
public VerySimpleServletHttpServer(String contextPath, HttpServlet servlet) {
this.contextPath = contextPath;
httpHandler = new HttpHandlerWithServletSupport(servlet);
}
public void start(int port) throws IOException {
InetSocketAddress inetSocketAddress = new InetSocketAddress(port);
server = HttpServer.create(inetSocketAddress, 0);
server.createContext(contextPath, httpHandler);
server.setExecutor(null);
server.start();
}
public void stop(int secondsDelay) {
server.stop(secondsDelay);
}
public int getServerPort() {
return server.getAddress().getPort();
}
}
final class HttpHandlerWithServletSupport implements HttpHandler {
private HttpServlet servlet;
private final class RequestWrapper extends HttpServletRequestWrapper {
private final HttpExchange ex;
private final Map<String, String[]> postData;
private final ServletInputStream is;
private final Map<String, Object> attributes = new HashMap<>();
private RequestWrapper(HttpServletRequest request, HttpExchange ex, Map<String, String[]> postData, ServletInputStream is) {
super(request);
this.ex = ex;
this.postData = postData;
this.is = is;
}
#Override
public String getHeader(String name) {
return ex.getRequestHeaders().getFirst(name);
}
#Override
public Enumeration<String> getHeaders(String name) {
return new Vector<String>(ex.getRequestHeaders().get(name)).elements();
}
#Override
public Enumeration<String> getHeaderNames() {
return new Vector<String>(ex.getRequestHeaders().keySet()).elements();
}
#Override
public Object getAttribute(String name) {
return attributes.get(name);
}
#Override
public void setAttribute(String name, Object o) {
this.attributes.put(name, o);
}
#Override
public Enumeration<String> getAttributeNames() {
return new Vector<String>(attributes.keySet()).elements();
}
#Override
public String getMethod() {
return ex.getRequestMethod();
}
#Override
public ServletInputStream getInputStream() throws IOException {
return is;
}
#Override
public BufferedReader getReader() throws IOException {
return new BufferedReader(new InputStreamReader(getInputStream()));
}
#Override
public String getPathInfo() {
return ex.getRequestURI().getPath();
}
#Override
public String getParameter(String name) {
String[] arr = postData.get(name);
return arr != null ? (arr.length > 1 ? Arrays.toString(arr) : arr[0]) : null;
}
#Override
public Map<String, String[]> getParameterMap() {
return postData;
}
#Override
public Enumeration<String> getParameterNames() {
return new Vector<String>(postData.keySet()).elements();
}
}
private final class ResponseWrapper extends HttpServletResponseWrapper {
final ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
final ServletOutputStream servletOutputStream = new ServletOutputStream() {
#Override
public void write(int b) throws IOException {
outputStream.write(b);
}
};
private final HttpExchange ex;
private final PrintWriter printWriter;
private int status = HttpServletResponse.SC_OK;
private ResponseWrapper(HttpServletResponse response, HttpExchange ex) {
super(response);
this.ex = ex;
printWriter = new PrintWriter(servletOutputStream);
}
#Override
public void setContentType(String type) {
ex.getResponseHeaders().add("Content-Type", type);
}
#Override
public void setHeader(String name, String value) {
ex.getResponseHeaders().add(name, value);
}
#Override
public javax.servlet.ServletOutputStream getOutputStream() throws IOException {
return servletOutputStream;
}
#Override
public void setContentLength(int len) {
ex.getResponseHeaders().add("Content-Length", len + "");
}
#Override
public void setStatus(int status) {
this.status = status;
}
#Override
public void sendError(int sc, String msg) throws IOException {
this.status = sc;
if (msg != null) {
printWriter.write(msg);
}
}
#Override
public void sendError(int sc) throws IOException {
sendError(sc, null);
}
#Override
public PrintWriter getWriter() throws IOException {
return printWriter;
}
public void complete() throws IOException {
try {
printWriter.flush();
ex.sendResponseHeaders(status, outputStream.size());
if (outputStream.size() > 0) {
ex.getResponseBody().write(outputStream.toByteArray());
}
ex.getResponseBody().flush();
} catch (Exception e) {
e.printStackTrace();
} finally {
ex.close();
}
}
}
public HttpHandlerWithServletSupport(HttpServlet servlet) {
this.servlet = servlet;
}
#SuppressWarnings("deprecation")
#Override
public void handle(final HttpExchange ex) throws IOException {
byte[] inBytes = getBytes(ex.getRequestBody());
ex.getRequestBody().close();
final ByteArrayInputStream newInput = new ByteArrayInputStream(inBytes);
final ServletInputStream is = new ServletInputStream() {
#Override
public int read() throws IOException {
return newInput.read();
}
};
Map<String, String[]> parsePostData = new HashMap<>();
try {
parsePostData.putAll(HttpUtils.parseQueryString(ex.getRequestURI().getQuery()));
// check if any postdata to parse
parsePostData.putAll(HttpUtils.parsePostData(inBytes.length, is));
} catch (IllegalArgumentException e) {
// no postData - just reset inputstream
newInput.reset();
}
final Map<String, String[]> postData = parsePostData;
RequestWrapper req = new RequestWrapper(createUnimplementAdapter(HttpServletRequest.class), ex, postData, is);
ResponseWrapper resp = new ResponseWrapper(createUnimplementAdapter(HttpServletResponse.class), ex);
try {
servlet.service(req, resp);
resp.complete();
} catch (ServletException e) {
throw new IOException(e);
}
}
private static byte[] getBytes(InputStream in) throws IOException {
ByteArrayOutputStream out = new ByteArrayOutputStream();
byte[] buffer = new byte[1024];
while (true) {
int r = in.read(buffer);
if (r == -1)
break;
out.write(buffer, 0, r);
}
return out.toByteArray();
}
#SuppressWarnings("unchecked")
private static <T> T createUnimplementAdapter(Class<T> httpServletApi) {
class UnimplementedHandler implements InvocationHandler {
#Override
public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
throw new UnsupportedOperationException("Not implemented: " + method + ", args=" + Arrays.toString(args));
}
}
return (T) Proxy.newProxyInstance(UnimplementedHandler.class.getClassLoader(),
new Class<?>[] { httpServletApi },
new UnimplementedHandler());
}
}
You may also have a look at some NIO application framework such as:
Netty: http://jboss.org/netty
Apache Mina: http://mina.apache.org/ or its subproject AsyncWeb: http://mina.apache.org/asyncweb/
This code is better than ours, you only need to add 2 libs: javax.servelet.jar and org.mortbay.jetty.jar.
Class Jetty:
package jetty;
import java.util.logging.Level;
import java.util.logging.Logger;
import org.mortbay.http.SocketListener;
import org.mortbay.jetty.Server;
import org.mortbay.jetty.servlet.ServletHttpContext;
public class Jetty {
public static void main(String[] args) {
try {
Server server = new Server();
SocketListener listener = new SocketListener();
System.out.println("Max Thread :" + listener.getMaxThreads() + " Min Thread :" + listener.getMinThreads());
listener.setHost("localhost");
listener.setPort(8070);
listener.setMinThreads(5);
listener.setMaxThreads(250);
server.addListener(listener);
ServletHttpContext context = (ServletHttpContext) server.getContext("/");
context.addServlet("/MO", "jetty.HelloWorldServlet");
server.start();
server.join();
/*//We will create our server running at http://localhost:8070
Server server = new Server();
server.addListener(":8070");
//We will deploy our servlet to the server at the path '/'
//it will be available at http://localhost:8070
ServletHttpContext context = (ServletHttpContext) server.getContext("/");
context.addServlet("/MO", "jetty.HelloWorldServlet");
server.start();
*/
} catch (Exception ex) {
Logger.getLogger(Jetty.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
Servlet class:
package jetty;
import java.io.IOException;
import java.io.PrintWriter;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
public class HelloWorldServlet extends HttpServlet
{
#Override
protected void doGet(HttpServletRequest httpServletRequest, HttpServletResponse httpServletResponse) throws ServletException, IOException
{
String appid = httpServletRequest.getParameter("appid");
String conta = httpServletRequest.getParameter("conta");
System.out.println("Appid : "+appid);
System.out.println("Conta : "+conta);
httpServletResponse.setContentType("text/plain");
PrintWriter out = httpServletResponse.getWriter();
out.println("Hello World!");
out.close();
}
}
I can strongly recommend looking into Simple, especially if you don't need Servlet capabilities but simply access to the request/reponse objects. If you need REST you can put Jersey on top of it, if you need to output HTML or similar there's Freemarker. I really love what you can do with this combination, and there is relatively little API to learn.
Starting in Java 18, you can create simple web servers with Java standard library:
class Main {
public static void main(String[] args) {
var port = 8000;
var rootDirectory = Path.of("C:/Users/Mahozad/Desktop/");
var outputLevel = OutputLevel.VERBOSE;
var server = SimpleFileServer.createFileServer(
new InetSocketAddress(port),
rootDirectory,
outputLevel
);
server.start();
}
}
This will, by default, show a directory listing of the root directory you specified. You can place an index.html file (and other assets like CSS and JS files) in that directory to show them instead.
Example (I put these in Desktop which was specified as my directory root above):
index.html:
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<title>Java 18 Simple Web Server</title>
<link rel="stylesheet" href="styles.css">
<style>h1 { color: blue; }</style>
<script src="scripts.js" defer>
let element = document.getElementsByTagName("h1")[0];
element.style.fontSize = "48px";
</script>
</head>
<body>
<h1>I'm <i>index.html</i> in the root directory.</h1>
</body>
</html>
Sidenote
For Java standard library HTTP client, see the post Java 11 new HTTP Client API.
An example of a very basic HTTP server on TCP sockets level:
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.PrintWriter;
import java.net.InetAddress;
import java.net.ServerSocket;
import java.net.Socket;
public class NaiveHttpServer {
public static void main(String[] args) throws IOException {
String hostname = InetAddress.getLocalHost().getHostName();
ServerSocket serverSocket = new ServerSocket(8089);
while (true) {
Socket clientSocket = serverSocket.accept();
PrintWriter out = new PrintWriter(clientSocket.getOutputStream(), true);
BufferedReader in = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
String s = in.readLine();
System.out.println(s);
while ("\r\n".equals(in.readLine()));
if ("GET /hostname HTTP/1.1".equals(s)) {
out.println("HTTP/1.1 200 OK");
out.println("Connection: close");
out.println("Content-Type: text/plain");
out.println("Content-Length:" + hostname.length());
out.println();
out.println(hostname);
} else {
out.println("HTTP/1.1 404 Not Found");
out.println("Connection: close");
out.println();
}
out.flush();
}
}
}
The example serves the hostname of the computer.
checkout Simple. its a pretty simple embeddable server with built in support for quite a variety of operations. I particularly love its threading model..
Amazing!
Check out takes. Look at https://github.com/yegor256/takes for quick info
Try this https://github.com/devashish234073/Java-Socket-Http-Server/blob/master/README.md
This API has creates an HTTP server using sockets.
It gets a request from the browser as text
Parses it to retrieve URL info, method, attributes, etc.
Creates dynamic response using the URL mapping defined
Sends the response to the browser.
For example the here's how the constructor in the Response.java class converts a raw response into an http response:
public Response(String resp){
Date date = new Date();
String start = "HTTP/1.1 200 OK\r\n";
String header = "Date: "+date.toString()+"\r\n";
header+= "Content-Type: text/html\r\n";
header+= "Content-length: "+resp.length()+"\r\n";
header+="\r\n";
this.resp=start+header+resp;
}
How about Apache Commons HttpCore project?
From the web site:...
HttpCore Goals
Implementation of the most fundamental HTTP transport aspects
Balance between good performance and the clarity & expressiveness of
API
Small (predictable) memory footprint
Self contained library (no external dependencies beyond JRE)
You can write a pretty simple embedded Jetty Java server.
Embedded Jetty means that the server (Jetty) shipped together with the application as opposed of deploying the application on external Jetty server.
So if in non-embedded approach your webapp built into WAR file which deployed to some external server (Tomcat / Jetty / etc), in embedded Jetty, you write the webapp and instantiate the jetty server in the same code base.
An example for embedded Jetty Java server you can git clone and use: https://github.com/stas-slu/embedded-jetty-java-server-example
The old com.sun.net.httpserver is again a public and accepted API, since Java 11. You can get it as HttpServer class, available as part of jdk.httpserver module. See https://docs.oracle.com/en/java/javase/11/docs/api/jdk.httpserver/com/sun/net/httpserver/HttpServer.html
This class implements a simple HTTP server. A HttpServer is bound to an IP address and port number and listens for incoming TCP connections from clients on this address. The sub-class HttpsServer implements a server which handles HTTPS requests.
So, apart from its limitations, there is no reason to avoid its use anymore.
I use it to publish a control interface in server applications. Reading the User-agent header from a client request I even respond in text/plain to CLI tools like curl or in more elegant HTML way to any other browser.
Cool and easy.
Here is my simple webserver, used in JMeter for testing webhooks (that's why it will close and end itself after request is received).
import java.io.IOException;
import java.io.PrintWriter;
import java.net.ServerSocket;
import java.net.Socket;
public class HttpServer {
private static int extractContentLength(StringBuilder sb) {
int length = 0;
String[] lines = sb.toString().split("\\n");
for (int i = 0; i < lines.length; i++) {
String s = lines[i];
if (s.toLowerCase().startsWith("Content-Length:".toLowerCase()) && i <= lines.length - 2) {
String slength = s.substring(s.indexOf(":") + 1, s.length()).trim();
length = Integer.parseInt(slength);
System.out.println("Length = " + length);
return length;
}
}
return 0;
}
public static void main(String[] args) throws IOException {
int port = Integer.parseInt(args[0]);
System.out.println("starting HTTP Server on port " + port);
StringBuilder outputString = new StringBuilder(1000);
ServerSocket serverSocket = new ServerSocket(port);
serverSocket.setSoTimeout(3 * 60 * 1000); // 3 minutes timeout
while (true) {
outputString.setLength(0); // reset buff
Socket clientSocket = serverSocket.accept(); // blocking
PrintWriter out = new PrintWriter(clientSocket.getOutputStream(), true);
try {
boolean isBodyRead = false;
int dataBuffer;
while ((dataBuffer = clientSocket.getInputStream().read()) != -1) {
if (dataBuffer == 13) { // CR
if (clientSocket.getInputStream().read() == 10) { // LF
outputString.append("\n");
}
} else {
outputString.append((char) dataBuffer);
}
// do we have Content length
int len = extractContentLength(outputString);
if (len > 0) {
int actualLength = len - 1; // we need to substract \r\n
for (int i = 0; i < actualLength; i++) {
int body = clientSocket.getInputStream().read();
outputString.append((char) body);
}
isBodyRead = true;
break;
}
} // end of reading while
if (isBodyRead) {
// response headers
out.println("HTTP/1.1 200 OK");
out.println("Connection: close");
out.println(); // must have empty line for HTTP
out.flush();
out.close(); // close clients connection
}
} catch (IOException ioEx) {
System.out.println(ioEx.getMessage());
}
System.out.println(outputString.toString());
break; // stop server - break while true
} // end of outer while true
serverSocket.close();
} // end of method
}
You can test it like this:
curl -X POST -H "Content-Type: application/json" -H "Connection: close" -d '{"name": "gustinmi", "email": "gustinmi at google dot com "}' -v http://localhost:8081/
I had some fun, I toyed around and pieced together this. I hope it helps you.
You are going to need Gradle installed or use Maven with a plugin.
build.gradle
plugins {
id 'application'
}
group 'foo.bar'
version '1.0'
repositories {
mavenCentral()
}
application{
mainClass.set("foo.FooServer")
}
dependencies {}
FooServer
The main entry point, your main class.
package foo;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.ServerSocket;
import java.net.Socket;
import java.nio.ByteBuffer;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentMap;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
public class FooServer {
public static void main(String[] args) throws IOException {
ServerSocket serverSocket = new ServerSocket(7654);
serverSocket.setPerformancePreferences(0, 1, 2);
/* the higher the numbers, the better the concurrent performance, ha!
we found that a 3:7 ratio to be optimal
3 partitioned executors to 7 network executors */
ExecutorService executors = Executors.newFixedThreadPool(3);
executors.execute(new PartitionedExecutor(serverSocket));
}
public static class PartitionedExecutor implements Runnable {
ServerSocket serverSocket;
public PartitionedExecutor(ServerSocket serverSocket) {
this.serverSocket = serverSocket;
}
#Override
public void run() {
ExecutorService executors = Executors.newFixedThreadPool(30);
executors.execute(new NetworkRequestExecutor(serverSocket, executors));
}
}
public static class NetworkRequestExecutor implements Runnable{
String IGNORE_CHROME = "/favicon.ico";
String BREAK = "\r\n";
String DOUBLEBREAK = "\r\n\r\n";
Integer REQUEST_METHOD = 0;
Integer REQUEST_PATH = 1;
Integer REQUEST_VERSION = 2;
String RENDERER;
Socket socketClient;
ExecutorService executors;
ServerSocket serverSocket;
public NetworkRequestExecutor(ServerSocket serverSocket, ExecutorService executors){
this.serverSocket = serverSocket;
this.executors = executors;
}
#Override
public void run() {
try {
socketClient = serverSocket.accept();
Thread.sleep(19);//do this for safari, its a hack but safari requires something like this.
InputStream requestInputStream = socketClient.getInputStream();
OutputStream clientOutput = socketClient.getOutputStream();
if (requestInputStream.available() == 0) {
requestInputStream.close();
clientOutput.flush();
clientOutput.close();
executors.execute(new NetworkRequestExecutor(serverSocket, executors));
return;
}
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
ByteBuffer byteBuffer = ByteBuffer.allocate(1024);
int bytesRead;
while ((bytesRead = requestInputStream.read(byteBuffer.array())) != -1) {
byteArrayOutputStream.write(byteBuffer.array(), 0, bytesRead);
if (requestInputStream.available() == 0) break;
}
String completeRequestContent = byteArrayOutputStream.toString();
String[] requestBlocks = completeRequestContent.split(DOUBLEBREAK, 2);
String headerComponent = requestBlocks[0];
String[] methodPathComponentsLookup = headerComponent.split(BREAK);
String methodPathComponent = methodPathComponentsLookup[0];
String[] methodPathVersionComponents = methodPathComponent.split("\\s");
String requestVerb = methodPathVersionComponents[REQUEST_METHOD];
String requestPath = methodPathVersionComponents[REQUEST_PATH];
String requestVersion = methodPathVersionComponents[REQUEST_VERSION];
if (requestPath.equals(IGNORE_CHROME)) {
requestInputStream.close();
clientOutput.flush();
clientOutput.close();
executors.execute(new NetworkRequestExecutor(serverSocket, executors));
return;
}
ConcurrentMap<String, String> headers = new ConcurrentHashMap<>();
String[] headerComponents = headerComponent.split(BREAK);
for (String headerLine : headerComponents) {
String[] headerLineComponents = headerLine.split(":");
if (headerLineComponents.length == 2) {
String fieldKey = headerLineComponents[0].trim();
String content = headerLineComponents[1].trim();
headers.put(fieldKey.toLowerCase(), content);
}
}
clientOutput.write("HTTP/1.1 200 OK".getBytes());
clientOutput.write(BREAK.getBytes());
Integer bytesLength = "hi".length();
String contentLengthBytes = "Content-Length:" + bytesLength;
clientOutput.write(contentLengthBytes.getBytes());
clientOutput.write(BREAK.getBytes());
clientOutput.write("Server: foo server".getBytes());
clientOutput.write(BREAK.getBytes());
clientOutput.write("Content-Type: text/html".getBytes());
clientOutput.write(DOUBLEBREAK.getBytes());
clientOutput.write("hi".getBytes());
clientOutput.close();
socketClient.close();
executors.execute(new NetworkRequestExecutor(serverSocket, executors));
} catch (IOException ex) {
ex.printStackTrace();
} catch (InterruptedException ioException) {
ioException.printStackTrace();
}
}
}
}
Run it:
gradle run
Browse to:
http://localhost:7654/