I am working with ESAPI to try and validate windows directory paths. For some reason, the part of my directory path named \14\ is getting converted into a CRLF. The error I am receiving is below, what am I not understanding correctly? I feel like my regex should be working.
WARN IntrusionDetector [SECURITY FAILURE Anonymous:null#unknown -> /project-test/IntrusionDetector] Invalid input: context=directoryPath, type(DirectoryName)=^[a-zA-Z0-9:/\\!##$%^&{}\[\]()_+\-=,.~'` ]{1,255}$, input=C:\UsersTESTUS~1AppDataLocalTempTestCase8002TempWorkSpace, orig=C:\Users\TESTUS~1\AppData\Local\Temp\14\TestCase8002TempWorkSpace
As you can see, I am using the regex:
^[a-zA-Z0-9:/\!##$%^&{}[]()_+-=,.~'` ]$
My input is:
C:\Users\TESTUS~1\AppData\Local\Temp\14\TestCase8002TempWorkSpace
Ouput, after ESAPI does canonicalization and validation:
C:\UsersTESTUS~1AppDataLocalTempTestCase8002TempWorkSpace
Here is the line of code that causes me to receive the error;
String validatedSourcePath = ESAPI.validator().getValidInput("directoryUnzip", directory, "DirectoryName", 255, false);
File validFile = new File(validatedSourcePath);
#C.Williams: I was about 30 minutes into writing up a detailed reply in an editor and accidentally excited my editor window. I'm too ticked off at my stupidity of not saving it to write it again, especially since I was only about 75% done.
However, if you want to email me I can arrange to talk to you via Google Hangouts or Signal to tell you want your problem is and how you can fix it. But it's long and complicated and partially related to a bug the ESAPI team just fixed but is not in any official release yet. But I am not going to take another 45 minutes or more trying to reply with any written text. My email address should be easy enough to find. Just google for my name and ESAPI. I am one of the project co-leaders on ESAPI.
-kevin wall
Related
I am using TextLocal tool to send SMS in my java code. I am using the same code snippet given here.
This is a transactional account. I am getting the following error
{"errors":[{"code":80,"message":"Invalid template"}],"status":"failure"}
This error code is nowhere found in the TextLocal documentation. Please let me know the solution if anyone has faced this issue before.
This was happening because the TextLocal transactional messages should be given a message template through the TextLocal dashboard and all the SMS sent should follow all the rules of these messages.
After the template has been created and followed, the issue is resolved.
go to Your login dashboard of textlocal
Then in send menu go to template and draft.
then open your template to see the message format.
And provide same massage format to your java method to send message.
Hope may it help you.
Few quick points to check
1.Is the message that you are passing in your API as per your approved
templates? If not, this error will come in response to your API. You
can check your approved templates under Send -> Templates & Drafts
2.Are all special characters in your template (&, #, #...) URL
encoded?
3.Are you using the same placeholder name
for multiple dynamic/replaceable parameters in your API? They have
to be unique.
4.Are you exceeding the max character length set for
the placeholder? Are you using a newline character? Replace all the
newline characters with %n
The above points are from textlocal SMS API documentation. You can also try copying the content of approved textlocal template into the Java code.
This because of the sending message content doesn't match the template which you created on the textlocal account.
check template is created or not. If not create a template and try.
If created check current message content match with your existing template.
This is happening because,
From a transactional account you can send only those messages that
match one of your pre-approved templates. Please ensure that your
messages match one of the approved template formats.
To create a Template,
Go to Your login dashboard of textlocal-> Send -> Templates and Drafts.
Then create a Template. Templates can be 2 types Static & Dynamic. You can create both by using this. Template message format should match the format in your code.
I am providing a link.. This will help to understand this more clearly.
How to create and format template
I am using JMeter to test HLS playback from a Streaming Server. So, the first HTTP request is for a master manifest file(m3u8). Say,
http://myserver/application1/subpath1/file1.m3u8
The reply to this will result in a playlist something like,
subsubFolder/360p/file1.m3u8
subsubFolder/480p/file1.m3u8
subsubFolder/720p/file1.m3u8
So, next set of URLs become
http://myserver/application1/subpath1/subsubFolder/360p/file1.m3u8
http://myserver/application1/subpath1/subsubFolder/480p/file1.m3u8
http://myserver/application1/subpath1/subsubFolder/720p/file1.m3u8
Now, individual reply to these further will be an index of chunks, like
0/file1.ts
1/file1.ts
2/file2.ts
3/file3.ts
Again, we have next set of URLs as
http://myserver/application1/subpath1/subsubFolder/360p/0/file1.ts
http://myserver/application1/subpath1/subsubFolder/360p/1/file1.ts
http://myserver/application1/subpath1/subsubFolder/360p/2/file1.ts
http://myserver/application1/subpath1/subsubFolder/360p/3/file1.ts
This is just the case of one set(360p). There will be 2 more sets like these(for 480p, 720p).
I hope the requirement statement is clear uptill this.
Now, the problem statement.
Using http://myserver/application1 as static part, regex(.+?).m3u8 is applied at 1st reply which gives subpath1/subsubFolder/360p/file1. This, is then added to the static part again, to get http://myserver/application1/subpath1/subsubFolder/360p/file1 + .m3u8
The problem comes at the next stage. As, you can see, with parts extracted previously, all I'm getting is
http://myserver/application1/subpath1/subsubFolder/360p/file1/0/file1.ts
The problem is obvious, an extra file1, 360p/file1 in place of 360p/0.
Any suggestions, inputs or alternate approaches appreciated.
If I understood the problem correctly, all you need is the file name as the other URLs can be constructed with it. Rather than using http://myserver/application1 as static part of your regex, I would try to get the filename directly:
([^\/.]+)\.m3u8$
# match one or more characters that are not a forward slash or a period
# followed by a period
# followed by the file extension (m3u8)
# anchor the whole match to the end
Now consider your urls, e.g. http://myserver/application1/subpath1/subsubFolder/360p/file1.m3u8, the above regex will capture file1, see a working demo here. Now you can construct the other URLs, e.g. (pseudo code):
http://myserver/application1/subpath1/subsubFolder/360p/ + filename + .m3u8
http://myserver/application1/subpath1/subsubFolder/360p/ + filename + /0/ + filename + .ts
Is this what you were after?
Make sure you use:
(.*?) - as Regular Expression (change plus to asterisk in your regex)
-1 - as Match No.
$1$- as template
See How to Load Test HTTP Live Media Streaming (HLS) with JMeter article for detailed instructions.
If you are ready to pay for a commercial plugin, then there is an easy and much more realistic solution which is a plugin for Apache JMeter provided by UbikLoadPack:
Besides doing this job for you, it will simulate the way a player would read the file. It will also scale much better than any custom script or player solution.
It supports VOD and Live which are quite difficult to script.
See:
http://www.ubik-ingenierie.com/blog/easy-and-realistic-load-testing-of-http-live-streaming-hls-with-apache-jmeter/
http://www.ubik-ingenierie.com/blog/ubikloadpack-http-live-streaming-plugin-jmeter-videostreaming-mpegdash/
Disclaimer, we are the providers of this solution
I have written a Servlet where I am reading an image from blobstore, another image from GCS and then after applying a composite on both these images I am storing the composite image back in GCS.
My code works well till here.
After that, when I am trying to get the serving url for the composite image, I am getting an OBJECT_NOT_FOUND.
Just to experiment I manually uploaded a image in GCS and gave all the necessary permissions. Added the serviceaccount as OWNER and gave READ access to All users. And then again I am just trying to get the serving url. Following is my code:-
BlobKey newImageKey = blobstoreService.createGsBlobKey(gcsPath);
//log.severe("GCS PATH: " + gcsPath + " BlobKey: " + newImageKey);
ServingUrlOptions options = ServingUrlOptions.Builder.withBlobKey(newImageKey);
String profilePicLink = imgService.getServingUrl(options);
I also tried the below code:-
ServingUrlOptions options = ServingUrlOptions.Builder.withGoogleStorageFileName(gcsPath);
String profilePicLink = imgService.getServingUrl(options);
And in both the cases this is the error that I am getting:
/controller javax.servlet.ServletException:
java.lang.IllegalArgumentException: OBJECT_NOT_FOUND:
Btw, I have not enable billing as I am using the default bucket with the free quota. This is still in development so the free quota works for me.
OK, so I found out where exactly the exception is happening...
byte[] responseBytes = ApiProxy.makeSyncCall(PACKAGE, "GetUrlBase",
request.build().toByteArray());
and the exception it is throwing is :
ApiProxy.ApplicationException Application Error 8
Enabled billing and tried, still of no use :(
Have been trying to solve this the whole day and tried to search a solution everywhere.
Though this actually does not answer my original question but I have found a workaround. I installed python and gsutil and set the default acl of my bucket to read. Now when I am saving an image file in GCS I am just showing the public url link.
The above can also be achieved if in the GCSFileOptions we add .acl("public-read").
Once the acl is applied by either of the above two methods, in the GCS cloud console you can see the images shared publicly link check box comes as a dash and it says you do not have permission to edit permissions. I was getting confused seeing it, as I was expecting the checkbox to be checked.
But even in the above scenario the publicly shared link will work which is:-
http://storage.googleapis.com/[bucket_name]/[gcs_object_name]
I would still appreciate if someone can explain why the getServingUrl is not not working. Yes, it is still not working after set default acl to read.
Thanks,
Sukalpo.
I could not reproduce this issue by either uploading to Google Cloud storage via the console or via the App Engine GCS Java client. In both cases I could create a public URL for the image
even without specifying any specific permissions.
Do you want to create a production issue request,
https://code.google.com/p/googleappengine/issues/entry?template=Production%20issue
, so we can get more details about your specific case?
What is your gcsPath? I have to use:
"/gs/" + gcsFileName.getBucketName() + "/" + gcsFileName.getObjectName();
Honestly, the only way I've run into this error (and run into this unsolved question) was when I was accidentally using the wrong filename in a difficult to notice way while fetching the BlobKey using Google's APIs.
So, check the obvious things first.
I am using Fortify SCA to find the security issues in my application (as a university homework). I have encountered some 'Log Forging' issues which I am not able to get rid off.
Basically, I log some values that come as user input from a web interface:
logger.warn("current id not valid - " + bean.getRecordId()));
and Fortify reports this as a log forging issue, because the getRecordId() returns an user input.
I have followed this article, and I am replacing the 'new line' with space, but the issue is still reported
logger.warn("current id not valid - " + Util.replaceNewLine(bean.getRecordId()));
Can anyone suggest a way to fix this issue?
I know this was already answered, but I thought an example would be nice :)
<?xml version="1.0" encoding="UTF-8"?>
<RulePack xmlns="xmlns://www.fortifysoftware.com/schema/rules">
<RulePackID>D82118B1-BBAE-4047-9066-5FC821E16456</RulePackID>
<SKU>SKU-Validated-Log-Forging</SKU>
<Name><![CDATA[Validated-Log-Forging]]></Name>
<Version>1.0</Version>
<Description><![CDATA[Validated-Log-Forging]]></Description>
<Rules version="3.14">
<RuleDefinitions>
<DataflowCleanseRule formatVersion="3.14" language="java">
<RuleID>DDAB5D73-8CF6-45E0-888C-EEEFBEFF2CD5</RuleID>
<TaintFlags>+VALIDATED_LOG_FORGING</TaintFlags>
<FunctionIdentifier>
<NamespaceName>
<Pattern/>
</NamespaceName>
<ClassName>
<Pattern>Util</Pattern>
</ClassName>
<FunctionName>
<Pattern>replaceNewLine</Pattern>
</FunctionName>
<ApplyTo implements="true" overrides="true" extends="true"/>
</FunctionIdentifier>
<OutArguments>return</OutArguments>
</DataflowCleanseRule>
</RuleDefinitions>
</Rules>
</RulePack>
Alina, I'm actually the author of the article you used to solve your log injection issue. Hope it was helpful.
Vitaly is correct with regards to Fortify. You'll need to build what Fortify calls a "custom rule".
It will likely be a dataflow cleanse rule. A basic example can be found here: http://www.cigital.com/newsletter/2009-11-tips.php. If you own Fortify, there should be a custom rule writing guide in your product documentation.
I don't know what the taint flag you'll use is, but it would look something like "-LOG_FORGING". You would essentially write a rule to remove the log forging "taint" whenever data is passed through your utility method. Fortify will them assume that any data passed through there is now safe to be written to a log, and will not cause log forging.
You need to mark your replaceNewLine as sanitiser in Fortify (if I remember correctly) and it will stop reporting the issue.
You can actually create a new rule from a particular method.
Navigate to the function on the right side of audit workbench after you've done a scan.
Find your sanitizing method and right click on it.
You can generate a rule from it. What you want is a general DataflowCleanseRule.
I just did this based on the xml someone posted above. You can save the rule as a .xml file.
When updating your scan you can pass the -rule argument and point at the .xml file.
My requirement is simple. At the beginning of each file there should be a block comment like this:
/*
* This file was last modified by {username} at {date} and has revision number {revisionnumber}
*/
I want to populate the {username}, {date} and {revisionnumber} with the appropriate content from SVN.
How can I achieve this with NetBeans and Subversion? I have searched a lot but I can't find exactly what I need.
I looked at this question and got some useful information. It is not exactly duplicate because I am working with NetBeans but the idea is the same. This is my header:
/*
* $LastChangedDate$
* $LastChangedRevision$
*/
Then I go to Team > Subversion > Svn properties and add svn:keywords as property name and LastChangedDate LastChangedRevision as property value.
And when I commit from NetBeans it looks like this:
/*
* $LastChangedDate: 2012-02-13 17:38:57 +0200 (Пн, 13 II 2012) $
* $LastChangedRevision: 27 $
*/
Thanks all for the support! I will accept my answer because other answers do not include the NetBeans information. Nevertheless I give +1 to the other answers.
As this data only exists after the file was committed it should be set by SVN itself, not a client program. (And client-side processing tends to get disabled or not configured at all.) This means there is no simple template/substitute like you want, because then after the first replacement the template variables would be lost.
You can find information abut SVN's keyword substitution here. Then things like $Rev$ can be replaced by $Rev: 12 $.
You can do this with The SubWCRev Program.
SubWCRev is Windows console program which can be used to read the
status of a Subversion working copy and optionally perform keyword
substitution in a template file. This is often used as part of the
build process as a means of incorporating working copy information
into the object you are building. Typically it might be used to
include the revision number in an “About” box.
This is typically done during the build process.
If you use Linux, you can find a Linux binary here. If you wish, you could also write your own using the output of svn log.
I followed Petar Minchev's suggestions, only I put the $LastChangedRevision$ tag not in a comment block but embedded it in a string. Now it is available to programmatically display the revision number in a Help -> About dialog.
String build = "$LastChangedRevision$";
I can later display the revision value in the about dialog using a String that has all of the fluff trimmed off.
String version = build.replace("$LastChangedRevision:", "").replace("$", "").trim();
I recommend a slightly different approach.
Put the following header at the top of your source files.
/*
* This file was last modified by {username} at {date} and has revision number {revisionnumber}
*/
Then add a shell script like this
post update, checkout script
USERNAME=# // use svnversion to get username
DATE=# // use svnversion to get revisio nnumber
sed -e "s#{username}#${USERNAME}#" -e "s#{date}#${DATE}#" ${SOURCE_CONTROL_FILE} > ${SOURCE_FILE}
pre commit script
cat standard_header.txt > ${SOURCE_CONTROL_FILE}
tail --lines $((${LENGTH}-4)) ${SOURCE_FILE} >> ${SOURCE_CONTROL_FILE}