I have been stuck looking around for all sorts of solutions to this problem and I have tried multiple suggestions like this one [1] and have been unsuccessful. The task is relatively simply and I have been successful in grabbing a single match so I know my regex is correct. I am working with Groovy inside of my Jenkins Pipeline script
Here is the problem, I have a file I read from like the following
<log><logentry revision="4813"><msg>nvcs-0909 haha nvpl-6700</msg></logentry></log>
All I want to do is grab the nvcs-0909 and nvpl-6700 and place them as the currentBuild.description. I can do it for one, but can't get multiple entries
Here is what I have so far
textOfFile = readFile("..\\builds\\$BUILD_NUMBER\\changelog0.xml")
regex = /[Nn][Vv]..[-\s]{1,3}\d+/
allIssues = (textOfFile =~ regex)
And when I print allIssues I get
java.util.regex.Matcher[pattern=[Nn][Vv]..[-\s]{1,3}\d+ region=0,377 lastmatch=]
How can I get this to work?
I have also tried
allIssues = textOfFile.findAll(regex)
And that fails as well but
allIssues = textOfFile.find(regex)
works and I get the first match I expected
works fine:
def textOfFile = '''<log><logentry revision="4813"><msg>nvcs-0909 haha nvpl-6700</msg></logentry></log>'''
def regex = /[Nn][Vv]..[-\s]{1,3}\d+/
println (textOfFile =~ regex).findAll()
result:
[nvcs-0909, nvpl-6700]
So the issue was one I faced before but was hidden because I added a try/catch this time around and not earlier with the find method. The current code was
def textOfFile
try {
textOfFile = readFile("..\\builds\\$BUILD_NUMBER\\changelog0.xml")
List<String> allIssues = textOfFile.findAll("[Nn][Vv]..[-\\s]{1,3}\\d+")
issueDescription = ""
for(int i = 0; i < allIssues.size(); i++)
issueDescription += allIssues[i].toUpperCase() + (i < allIssues.size() - 1 ? ", ":"")
currentBuild.description = issueDescription
}
catch(e) {
echo "ERROR"
}
Since I was trying to wrap my head around what in the world was going on I printed the error message with
println e.getMessage()
Inside the catch block and it gave the following message
Scripts not permitted to use staticMethod org.codehaus.groovy.runtime.DefaultGroovyMethods findAll java.lang.String
I ran into this message before and solved it with the help of this. But Jenkins wouldn't give me the option to add permissions and you can't manually add them yourself in (Manage Jenkins -> In-Process Script Approval).
So I commented out the try/catch, ran the script, let Jenkins catch the exception and I was given the option to allow my script to use the findAll method. Then I ran it one more time and it worked perfectly and re-wrapped the code in the try/catch.
I am guessing that the notification mechanism for allowing you to allow method permissions in Jenkins is triggered when your script throws the exception and never catches it..? Anyways it's working perfectly fine now.
Related
I am working with the following line of code on a project:
Document d = Jsoup.connect(url).timeout(3000).get();
I'm using an IntelliJ debugger and when I get to this line of code, the debugger disables all of it's buttons and just says "the application is running." It does not timeout and it never throws an exception or continue. Does anyone know why this might be happening? If I take the following code
String url = "https://www.cs.purdue.edu/homes/bxd/";
Document d = Jsoup.connect(url).timeout(3000).get();
System.out.println(d);
and run it in the main method of a seperate class it works just fine.
I'm using this code:
http://groovy.codehaus.org/Expect+for+Groovy
to attempt to automate a python based CLI.
My test main function is below.
Running this however, it seems that it never actually reads data from the process.
If I change the process to /bin/ls and expect some filename, it will work correctly, which leads me to believe it cant handle the fact that python is waiting for input, while /bin/ls closes the stream and flushes it.
Any ideas? Thanks.
public static void test2(String[] args){
println "Main"
def builder = new ProcessBuilder("/usr/bin/python");
builder.redirectErrorStream()
builder.redirectOutput(ProcessBuilder.Redirect.PIPE);
builder.redirectInput(ProcessBuilder.Redirect.PIPE);
def expectSession = new IOSession(builder.start());
expectSession.expect(">>>");
expectSession.send("print(%d) % (1+1)")
expectSession.expect("2");
expectSession.send("quit()");
expectSession.close();
println "Done...";
}
Looking through the source for IOSession it looks like this might be a bug in the constructor. Try:
def expectSession = new IOSession();
expectSession.addProcess(builder.start());
Also, you have to add \r to the end of the strings you are sending.
I have a code that that reads parameters from XML file. In debugger everything works fine, but after I built JAR file and run it - I get the following window
Java Virtual Machine Launcher
A Java Exception has occured.
But if I comment this piece of code:
else if (settingName.equals("log_level")) {
String value = element.getAttribute("value");
if (value.equals("full")) {
modelLogLevel = EnLogDetails.LOG_FULL;
} else if (value.equals("apdu")) {
modelLogLevel = EnLogDetails.LOG_APDU;
} else if (value.equals("none")) {
modelLogLevel = EnLogDetails.LOG_NONE;
} else {
throw new InvalidArgumentException(new String[]{"log_level"});
}
}
and rebuild JAR again - it works fine. How to fix this issue?
You've provided exactly none of the relevant information, such as the actual exception or the command line you are using, but clearly you are supplying a log_level argument that doesn't match any of those tests, so you are throwing an IllegalArgumentException which is terminating main() and therefore the launcher. If so it is a complete mystery to me why you need to come to StackOverflow to have your own code explained to you.
Or else the references to EnLogDetails are failing in some way which is shown in the exception which you haven't supplied.
I need to get a list of chunks after sharding inside my Java code. My code is simple and looks like this:
Mongo m = new Mongo( "localhost" , 27017 );
DB db = m.getDB( "admin" );
Object cr = db.eval("db.printShardingStatus()", 1);
A call of eval() returns an error:
Exception in thread "main" com.mongodb.CommandResult$CommandFailure: command failed [$eval]: { "serverUsed" : "localhost/127.0.0.1:27017" , "errno" : -3.0 , "errmsg" : "invoke failed: JS Error: ReferenceError: printShardingStatus is not defined src/mongo/shell/db.js:891" , "ok" : 0.0}
at com.mongodb.CommandResult.getException(CommandResult.java:88)
at com.mongodb.CommandResult.throwOnError(CommandResult.java:134)
at com.mongodb.DB.eval(DB.java:340)
at org.sm.mongodb.MongoTest.main(MongoTest.java:35)
And, really, if we look into the code of db.js, in line 891 there is a call to a method printShardingStatus() that is not defined inside a file. Inside of sh.status() method in utils_sh.js file, there is even a comment:
// TODO: move the actual commadn here
Important to mention, when I run these commands in mongo command line, everything works properly!
My questions are:
Is there any other possibility of getting a full sharding status within Java code? (eg. with DB.command() method)
If not, any other suggestions how to avoid my problem?
Many of the shell's helper functions are not available for server-side code execution. In the case of printShardingStatus(), it makes sense because there isn't a console to use for printing output and you'd rather have a string returned. Thankfully, you should be able to pull up the source of the shell function and reimplement it in your application (e.g. concatenating a returned string instead of printing directly).
$ mongo
MongoDB shell version: 2.2.0
connecting to: test
> db.printShardingStatus
function (verbose) {
printShardingStatus(this.getSiblingDB("config"), verbose);
}
So, let's look at the printShardingStatus() function...
> printShardingStatus
function (configDB, verbose) {
if (configDB === undefined) {
configDB = db.getSisterDB("config");
}
var version = configDB.getCollection("version").findOne();
// ...
}
Before turning all of the output statements into string concatenation, you'd want to make sure the other DB methods are all available to you. Performance-wise, I think the best option is to port the innards of this function to Java and avoid server-side JS evaluation altogether. If you dive deeper into the printShardingStatus() function, you'll see it's just issuing find() on the config database along with some group() queries.
If you do want to stick with evaluating JS and would rather not keep this code within your Java application, you can also look into storing JS functions server-side.
Have you deployed a shard cluster properly?
If so, you could connect to a mongo database that has sharding enabled.
Try calling the method db.printShardingStatus() with a that database within the mongo shell and see what happens.
Apparently the Javascript function 'printShardingStatus' is only available for the mongo shell and not for execution with server commands, to see the code start mongo.exe and type only 'printShardingStatus' and press enter.
In this case writing an extension method would be the best for solving this...
Javascript way of printing output of MongoDB query to a file
1] create a javascript file
test.js
cursor = db.printShardingStatus();
while(cursor.hasNext()){
printjson(cursor.next());
}
2] run
mongo admin --quiet test.js > output.txt
I run a bash script from my Java program which takes a chunk of data, manipulates it, and splits it up.
It's not a question of whether the bash script works -- I can see the split files in the directory.
Say the original file was "bigFile" in data/
Then
try
{
Process proc = Runtime.getRuntime().exec("bash " + SCRIPT_DIR + "/" + SPLIT_SCRIPT_NAME + " " + args[_MESSAGES_PER_UPLOAD_] + " " + args[_MAXIMUM_MESSAGES_PER_FEED_] + " " + (60000*Integer.parseInt(args[_DURATION_BEFORE_EACH_UPLOAD_IN_MINUTES_])/Integer.parseInt(args[_DURATION_OF_EACH_FEED_IN_MILLISECONDS_])));
proc.waitFor();
}
catch(IOException e) { error(e); }
String fileNames;
File folder = new File(DATA_DIR);
File[] filesToUpload = folder.listFiles();
for (int i = 0; i < filesToUpload.length; ++i)
if (filesToUpload[i].isFile())
{
fileNames = filesToUpload[i].getName();
System.out.println(fileNames);
}
Will print bigFile, not...
$ ls data/
dataChunk_00000
dataChunk_00001
dataChunk_00002
dataChunk_00003
dataChunk_00004
dataChunk_00005
dataChunk_00006
dataChunk_00007
dataChunk_00008
dataChunk_00009
dataChunk_00010
dataChunk_00011
dataChunk_00012
dataChunk_00013
dataChunk_00014
dataChunk_00015
dataChunk_00016
dataChunk_00017
dataChunk_00018
dataChunk_00019
dataChunk_00020
dataChunk_00021
dataChunk_00022
dataChunk_00023
dataChunk_00024
dataChunk_00025
dataChunk_00026
dataChunk_00027
as it should. I'm guessing this is a compiler optimization or something.
Edit: If somebody could explain to me why proc.waitFor() isn't working and/or a better way to solve this, I'd much appreciate it.
The problem with this is not compiler optimization or anything like that.
Its because you are invoking your script with a "bash" in front of it . This causes the process to fork -- so your bash command returns successfully immediately , but your script continues to run in the background and terminate.
The proc.waitFor() has nothing to wait for, the rest of the java program executes before your file has been "split".
You cannot change the directory with java.
If you want to "simulate" it, all you need to do is set the property "user.dir".
I am guessing that your bash script is performing actions asynchronously from its own process/thread. This means that the script finishes executing before the work is complete. This would still pass the waitFor() check and continue executing the code.
EDIT:
Kal's answer explains this more clearly, and it was posted first. The problem is the fact that you use the bash command to execute the script.
I suspect your arguments aren't all passed to your script.
Put all your arguments in an ArrayList instance, pass the instance to the ProcessBuilder, then call the start method on the builder instance, which returns the proc on which you call waitFor.
Here's sample Scala code to show what I mean (I can port it to Java if you're really interested ;-):
import java.lang.{ Process => JProcess, ProcessBuilder => JProcessBuilder }
import java.util.{ArrayList => JArrayList, List => JList, Map => JMap}
import java.io.{InputStreamReader, BufferedReader}
def call(args: String*) = {
val command: JList[String] = new JArrayList[String]()
args.foreach {arg =>
command.add(arg)
}
//log.debug("argument list: %s", command.toString)
val builder = new JProcessBuilder(command)
val proc: JProcess = builder.start()
proc.waitFor()
val read = new BufferedReader(new InputStreamReader(proc.getInputStream()));
val sb: StringBuffer = new StringBuffer()
while(read.ready()) {
sb.append(read.readLine)
}
// sb now has the output of the called process...
val exitValue: Int = proc.exitValue
// http://stuffthathappens.com/blog/2007/11/28/crash-boom-too-many-open-files/
read.close
proc.destroy
(exitValue, sb.toString) // return it
}
Example call in REPL:
scala> call("date")
res156: (Int, java.lang.String) = (0,Mon 18 Jul 2011 22:29:58 BST)
There are a number of wrong assumptions with this program:
Every time you do 'exec' you fork a new process, with its own environment, current directory, etc. Any change of the current directory would have been local to that process and will not affect the parent (your Java process). In other words, there is no way to change the current path of an application using a command in a sub-process, there is no Java API for that either - if you really need this, you have to use native call.
The 'cd' command on Unix is a real command, you do not need the shell in order to run it (unlike Windows).
When you fork a process, you need to make sure that you drain the stdout and stderr, or it is going to block when the OS buffer gets full (see next)
Process.waitFor() works. Always.
A better way to approach the problem is to read carefully the File API and as much as possible work with absolute paths. The 'current directory' is something very usefull when you are in shell, but for applications it ends up being more confusing, so the sooner you resolve it to absolute path - the better.