I'm making an URL shortener with the Javalin framework and have this endpoint set up:
app.routes(()->{
path("",()->{
get("/:id", ctx->{
//do stuff
ctx.redirect("somewhere.com");
});
});
});
Problem is when I need to serve a javascript file to load into my html files. It tries to load from http://localhost:7000/qrcode.min.js but ends up going to the endpoint mentioned above. From what I read in the documentation this is normal behaviour, Javalin first runs the endpoint handler and then (if it doesn't find an endpoint) runs the file handler.
So how can I fix this? should I define a GET request at "/qrcode.min.js"?, I dont think the javalin context handler has a function that lets me return a .js file.
As Matt already suggested in a comment, it would be way cleaner if you'd prefix either path. That way, you could have /r/:id (or /u/:id with "u" for "URL") and the static files would not get in your way, or you could prefix your static files with e.g. /static/, or even just /s/ for brevity, and your shortened URLs would not get in your way.
If you, however, prefer to stick with your current scheme, you can simply filter out JavaScript files (or any other non-id request) in the handler and instead provide the file (however, if you previously had auto-generated ETags, you'd lose caching if you don't want to handle that yourself).
The latter solution would look like so:
app.routes (() -> {
path ("", () -> {
get ("/:id", ctx -> {
String id = ctx.pathParam ("id");
if (id.endsWith (".js")) {
String resourcePath = "your/classpath/resources/folder/" + id;
try {
InputStream resultStream = Thread.currentThread ()
.getContextClassLoader ()
.getResourceAsStream (resourcePath);
if (resultStream == null)
throw new NullPointerException ("Script not found");
ctx.contentType ("application/javascript");
ctx.result (resultStream);
} catch (NullPointerException e) { // script does not exist
e.printStackTrace (); // for development only!
ctx.status (404);
}
return;
}
// do stuff
ctx.redirect ("somewhere.com");
});
});
});
Depending on your preference, you can also handle the resultStream == null case where my code is currently throwing an NPE to be caught by the outer try/catch and omit the try/catch completely.
Setting the Content-Type is essential so that the browser knows that you're actually responding with JavaScript code. Also, I'm typically using Thread.currentThread ().getContextClassLoader () because we'd want the resource to be resolved based upon the current HTTP handler thread, which could, in theory, have a different class path/class loader than the class we're currently in.
Please note that, as stated above, this will not support client-side caching as the handler simply ignores all ETag headers sent with the request* and instead respond with the complete file which, with many requests in a short amount of time and large scripts, will certainly put way more stress on your disks and CPUs.
Thus, I'd actually recommend to prefix the static files route and let Javalin/Jetty handle all the caching and files magic.
* Actually, the header sent by the client is If-None-Match most of the time. The server would respond with an ETag to allow for caching in the browser.
Related
According to this link, if the source has a problem when opening and throws an exception and it is also in try parentheses, JVM closes it. My question is how to inform the user now that this source is closed and we encountered a problem when opening this resource? In other words, how can this exception be handled?
Seems trivial. Usually, java code is running in some sort of 'no user interaction' environment (servers and the like). The right move is to let the exception bubble up - you want the daily job that is halfway through reading through the database to open the related file to then send the logs to long term storage or whatever it is to completely abort and write a note in the log file. Usually for jobs like that, there's some atomary functionality (in this case, perhaps each such file is independent of the others, and its okay to leave the 'broken' one in place for now until a server admin can look at it whilst continuing to process the remainder - in that case, the 'do the backup rotation thing on THIS file' is the atomary functionality): Catch all exceptions and write code that does what you want when the job fails. For example, my servers can send notifications straight to admin phones (via telegram or pushover, or using slack API, and there are many services that automate this for you too), if it's important, you'd write that in your catch block.
For code that is directly 'triggered' by a user, let's say a 'save file' function, then it's not so much 'the resource is now closed' - resources are not long lived (they cannot be - not if you use try-with-resources). They were either never open in the first place (you attempt to save a file to a dir that doesn't exist - the act of trying to make the new OutputStream already failed, it was never open to begin with), or, perhaps it did open, but it was to a USB stick and the user pulled it out halfway through saving. The resource is just closed, effectively, whether in java you .close() it or not - the entire stick is gone!!
The only thing the 'safe close' aspect of try-with-resources did for you is ensure that your Java Process isn't wasting a file handle.
You handle it the same way you handle pretty much any 'unrecoverable' (you can't write software that hypnotises the user into sticking that USB stick back into the machine, obviously - it is not recoverable as a consequence, like most exceptions) problem: You toss up a dialog box that explains the situation.
try (OutputStream out = Files.newOutputStream(saveGameFile)) {
boardState.save(out);
} catch (IOException e) {
// show dialog here
}
Even when using a try-with-resources, the catch clause still works.
private static void printFile() throws MyCustomException {
try(FileInputStream input = new FileInputStream("file.txt")) {
int data = input.read();
while(data != -1){
System.out.print((char) data);
data = input.read();
}
} catch (IOException e) {
throw new MyCustomException("There was an error while opening the resource", e);
}
}
I am working on fixing Veracode issues in my application. Veracode has highlighted the flaw "External Control of File Name or Path (CWE ID 73) " in below code.
Thread.currentThread().getContextClassLoader().getResourceAsStream(lookupName)
How do I validate the parameter? If I need to use below ESAPI validation, then what is the exact parameter I should be passing in getValidFileName() method. Currently I am passing the parameters as below.
ESAPI.validator().getValidFileName(lookupName, lookupName,
ESAPI.securityConfiguration().getAllowedFileExtensions(), false);
Correct me whether I am following the right approach for fixing this issue.
There are several suggestions at: https://community.veracode.com/s/article/how-do-i-fix-cwe-73-external-control-of-file-name-or-path-in-java
You can use hardcoded values, if these files are stored in the server side.
(i.e.: in a HashMap).
Another solution is to use a custom validator (from veracode page) :
// GOOD Code
String extension = request.getParameter("extension");
File f = new File(buildValidAvatarPath(extension))
#FilePathCleanser
public String buildValidAvatarPath(extension) {
String[] allowedExtensions = new String[]{"jpg","gif","png"};
String extension = "png"; // Default extension
for (String allowedExtension: allowedExtensions) {
if (allowedExtension.equals(request.getParameter("extension"))) {
extension = request.getParameter("extension");
}
}
// See "Note on authorization"
User user = getCurrentUser();
if (!userMayAccessFile(user, path)) {
throw new AuthorizationException("User may not access this file", user);
}
File(configPath + "avatar." + extension)
return path;
}
Okay, so the problem is that you are allowing user-control of that file path. Imagine its on a UNIX box and they enter:
../../../../../../../etc/shadow
Whatever user privileges are granted to the user running that java Thread is possible to expose to the user in question. I don't know what processing is going on in your application, but the danger is that you need to prevent user control of that lookup variable.
The call you're making is consistent with the single test in ValidatorTest.java, which is definitely a deficiency in code coverage on our behalf.
Now, there's an excellent chance that even if you use this call that Veracode might still flag it: the default file list in ESAPI.properties will need to be either truncated for your use case, or you'll have to create your own Validator rule for legal file extensions for your specific use case.
Which brings up the next bit: There's a lot of mischief that can happen in regards to file uploads.
In short, to be actually secure about file uploads will require more than what ESAPI currently offers, which is unfortunately, only an extension check. In your particular case, make sure you try some directory traversal attacks. And use that OWASP link to help analyze your application.
Given that the OP wants to clear the issue in Veracode, you would want to chain a couple calls:
ESAPI.validator().getValidDirectoryPath() and ESAPI.Validator.getValidFileName()
But be sure you've properly truncated the extension list in HttpUtilities.ApprovedUploadExtensions in validator.properties as the default list is too permissive, at least until we release 2.1.0.2.
I have to stress however that even with this particular combination there is absolutely nothing ESAPI does to prevent a user from renaming "netcat.exe" to "puppies.xlsx" and bypassing your validation check, that's why the rant on the first part of this answer.
ESAPI's file validation is NOT secure, it's quite simply better than nothing at all.
Doing this correctly requires more work than just using 1-2 calls to ESAPI.
DISCLAIMER: as of this writing I am the project co-lead for ESAPI.
You can change file name by sanitizing it as below code snippet:
private static String sanitizeFileName(String name) {
return name
.chars()
.mapToObj(i -> (char) i)
.map(c -> Character.isWhitespace(c) ? '_' : c)
.filter(c -> Character.isLetterOrDigit(c) || c == '-' || c == '_' || c == ':')
.map(String::valueOf)
.collect(Collectors.joining());
}
How do i delete a file after serving it over http,
Files.TemporaryFile file = null;
try {
file = new Files.TemporaryFile(f);
return ok().sendFile(file.file());
} catch (IllegalArgumentException e) {
return badRequest(Json.newObject().put("message", e.getMessage()));
} finally {
file.clean();
}
with this code, the file gets deleted before it is served. i receive an empty file on the client.
Play framework in version 2.8 should support onClose argument in sendFile method also in Java (so far it seems to be supported only in Scala version).
In older versions (I have tried only on 2.7.x) you may apply the same approach like in the fix for 2.8, so:
public Result doSomething() {
final File fileToReturn = ....;
final Source<ByteString, CompletionStage<IOResult>> source = FileIO.fromFile(fileToReturn);
return Results.ok().streamed(wrap(source, () -> fileToReturn.delete()), Optional.of(fileToReturn.length()), Optional.of("content type, e.g. application/zip"));
}
private Source<ByteString, CompletionStage<IOResult>> wrap(final Source<ByteString, CompletionStage<IOResult>> source, final Runnable handler) {
return source.mapMaterializedValue(
action -> action.whenCompleteAsync((ioResult, exception) -> handler.run())
);
}
From reading the JavaFileUpload documentation for 2.6.x, it sounds like you don't need that finally block to clean up the file afterwards. Since you are using a TemporaryFile, garbage collection should take care of deleting the resource:
...the idea behind TemporaryFile is that it’s only in scope at completion and should be moved out of the temporary file system as soon as possible. Any temporary files that are not moved are deleted [by the garbage collector].
The same section goes on to describe that there is the potential that the file will not get garbage collection causing Denial Of Service issues. If you find that the files are not getting removed, then you can use the TemporaryFilesReaper:
However, under certain conditions, garbage collection does not occur in a timely fashion. As such, there’s also a play.api.libs.Files.TemporaryFileReaper that can be enabled to delete temporary files on a scheduled basis using the Akka scheduler, distinct from the garbage collection method.
I am not forcing all the project, but you can use a Scala for only this controller, then you can use onClose parameter of the sendFile method. The only attention - that parameter is not workable in all versions, it looks like in 2.5 there was an issue so it was not triggered (was not work: https://github.com/playframework/playframework/issues/6351).
Another way - you can use Akka streams, like here: https://www.playframework.com/documentation/2.6.x/JavaStream#Chunked-responses.
I want to create a URL structure for my Vert.x Web powered API that makes it clear how some entities are "contained" inside other entities and how you "traverse the entity path" to find child entities, so I'm thinking of using something like this to address a "grand child" (I don't expect anything deeper than a grand child):
GET /entity/:parent_id/sub-entity/:child_id/subsub-entity/:grandchild_id
So normally my Router configuration will look something like this:
router.get("/entity/:parent_id/sub-entity/:child_id/subsub-entity/:grandchild_id")
.handler(r -> {
HttpServerRequest req = r.request();
Minipart p = Entities.get(req.getParameter("parent_id"))
.getPart(req.getParameter("child_id"))
.getMinipart(req.getParameter("grandchild_id"));
// do something with p
});
When I add a lot of operations (each entity class at each level has catalog and create operations, and each level entity instance has get, update and delete operations, as well as a few other tidbits), my router class gets really large.
I was thinking of using sub-routers to offload the sub-entity management down the line, so the Entities Router configuration might do:
router.mountSubRouter("/entity/:parent_id/sub-entity", PartsRouter.init(vertx));
and then PartsRouter can do:
router.get("/:child_id").handler(r -> {
String parentEntityId = r.request().getParameter("parent_id");
Entity parent = Entities.get(parentEntityId);
String myid = r.request().getParameter("child_id");
Part myself = parent.getPart(myid);
// do something with myself
});
But when I do try that and try to access the sub-router operations, I get a 404 error from Vert.x...
Update:
Apparently Vert.x explicitly does not support this - it threw an exception that my wrapper code just logged and ignored, saying:
java.lang.IllegalArgumentException: Can't use patterns in subrouter mounts
So, is there another way to achieve what I'm trying to do (split a large router configuration into a proper class hierarchy)?
I can image 2 ways of solving your problem:
The first one would be to have a initial handler that processes the common part of the request and calls next so the following one will continue where the first stopped e.g.:
router.route("/entity/:parent_id/sub-entity", ctx -> {
// common part here...
ctx.next();
});
And then:
router.route("/entity/:parent_id/sub-entity/:child_id", ctx -> {
String parentEntityId = r.request().getParameter("parent_id");
Entity parent = Entities.get(parentEntityId);
String myid = r.request().getParameter("child_id");
Part myself = parent.getPart(myid);
// do something with myself
});
Alternatively you can use internal redirects, so you handle the initial code as before but instead of calling next() you redirect to another URL. In that case you should store in the context what you want to reuse since the request will be restarted in the new location.
I'm writing an app that loads javascript dynamically using rhino(or a browser); I got 2 files:
// in a file called fooDefinition.js
var definition = {
foo: function(data){return bar(data)},
loadFile: "barLib.js"
}
now, bar() is defined like this:
// in a file called barLib.js
function bar(data){
return data + " -> bar!";
}
This is what I want to do:
load fooDefinition.js into the environment
read the value of loadFile (in this case: "barLib.js") and load the file (NOTE: load the file through external mechanism, not through javascript itself!)
call foo
external mechanism & example usage (Java pseudo code):
// assume engine is a statefull engine (rhino for example)
String str = /*content of fooDefinition.js*/;
engine.eval(str);
String fileToLoad = engine.eval("definition.loadFile");
engine.load(IOUtils.readFileToString(new File(fileToLoad)));
String result = engine.eval("definition.foo('myData')");
I've tried this in Google Chrome's JS console and no error was thrown
I wonder is this the correct way of accomplish such task?
TL;DR:
Are the attributes of an object loaded and checked when the object is defined?
If your engine is statefull that is it keeps track of defined variables, yes your approach is corrent and will work as expected
But if it is not, your way will fail, because when you call the following
String fileToLoad = engine.eval("definition.loadFile");
your engine haven't any info about definition object and as a result it return an exception (in JavaScript).
It seems your engine is statefull and all things will work correctly