I have an #Test method with invocationCount=3.
Each time this method is run, there is a call for preparing some doc into another method.
All it's working GREAT when the #Test is run for the 1st time.
The doc is succesfully found and loaded:
final DocumentRegistryResource documentRegistryResource =
RestClientFactory.getInstance().createDocumentRegistryResource(
getUserRestAuth());
final File importFile = new File(
this.getClass().getResource("/documents-template-test.xml").getFile());
BUT, at the 2nd and 3rd invocation, i receive a null exception:
this.getClass().getResource("/documents-template-test.xml")
is no longer found.
Can anyone explain me WHY? The original file is on the same place, nothing was moved
during the 1st time invocation...
Do you close your file in the cleanup phase? Otherwise, your file might be locked on the second invocation of your test.
Also, be careful with the use of the final keyword combined with a test case. Using static or final can break a correct initialization or cleanup.
Related
First, some background why I want this crazy thing. I'm building a Plugin in Jenkins that provides an API for scripts that are started from a pipeline-script to independently communicate with jenkins.
For example a shell-script can then tell jenkins to start a new stage from the running script.
I've got the communication between the script and Jenkins working, but the problem is that I now want to try and start a stage from a callback in my code but I can't seem to figure out how to do it.
Stuff I've tried and failed at:
Start a new StageStep.java
I can't seem to find a way to correctly instantiate and inject the step into the lifecycle. I've looked into DSL.java, but cant seem to get to an instance to call invokeStep(), nor was I able to find out how to instantiate DSL.java with the right environment.
Look at StageStepExecution.java and do what it does.
It seems to either invoke the body with an Environment Variable and nothing else, or set some actions and save the state in a config file when it has no body. I could not find out how the Pipeline: Stage View Plugin hooks into this, but it doesn't seem to read the config file. I've tried setting the Actions (even the inner class through reflection) but that did not seem to do anything.
Inject a custom string as Groovy body and call it with csc.newBodyInvoker()
A hacky solution I came up with was just generating the groovy script and running it like the ParallelStep does. But the sandbox does not allow me to call new GroovyShell().evaluate(""), and If I approve that call, the 'stage' step throws a MissingMethodException. So I also do not instatiate the script with the right environment. Providing the EnvironmentExpander does not make any difference.
Referencing and modifying workflow/{n}.xml
Changing the name of a stage in the relevant workflow/{n}.xml and rebooting the server updates the name of the stage, but modifying my custom stage to look like a regular one does not seem to add the step as a stage.
Stuff I've researched:
If some other plugin does something like this, but I couldn't find any example of plugins starting other steps.
How Jenkins handles the scripts and starts the steps, but It seems as though every step is directly called through the method name after the script is parsed, and I found no way to hook into this.
Other plugins using the StageView through other methods, but I could not find any.
add an AtomNode as a head onto the running thread, but I couldn't find how to replace/add the head and am hesitant to mess with jenkins' threading.
I've spent multiple days on this seemingly trivial call, but I can't seem to figure it out.
So the latest thing I tried actually worked, and is displayed correctly, but it ain't pretty.
I basically reimplemented the implementation of DSL.invokeStep(), which required me to use reflection A LOT. This is not safe, and will break with any changes of course so I'll open an issue in the Jenkins' ticket system in the hopes they will add a public interface for doing this. I'm just hoping this won't give me any weird side-effects.
// First, get some environment stuff
CpsThread cpsThread = CpsThread.current();
CpsFlowExecution currentFlowExecution = (CpsFlowExecution) getContext().get(FlowExecution.class);
// instantiate the stage's descriptor
StageStep.DescriptorImpl stageStepDescriptor = new StageStep.DescriptorImpl();
// now we need to put a new FlowNode as the head of the step-stack. This is of course not possible directly,
// but everything is also outside of the sandbox, so putting the class in the same package doesn't work
// get the 'head' field
Field cpsHeadField = CpsThread.class.getDeclaredField("head");
cpsHeadField.setAccessible(true);
Object headValue = cpsHeadField.get(cpsThread);
// get it's value
Method head_get = headValue.getClass().getDeclaredMethod("get");
head_get.setAccessible(true);
FlowNode currentHead = (FlowNode) head_get.invoke(headValue);
// crate a new StepAtomNode starting at the current value of 'head'.
FlowNode an = new StepAtomNode(currentFlowExecution, stageStepDescriptor, currentHead);
// now set this as the new head.
Method head_setNewHead = headValue.getClass().getDeclaredMethod("setNewHead", FlowNode.class);
head_setNewHead.setAccessible(true);
head_setNewHead.invoke(headValue, an);
// Create a new CpsStepContext, and as the constructor is protected, use reflection again
Constructor<?> declaredConstructor = CpsStepContext.class.getDeclaredConstructors()[0];
declaredConstructor.setAccessible(true);
CpsStepContext context = (CpsStepContext) declaredConstructor.newInstance(stageStepDescriptor,cpsThread,currentFlowExecution.getOwner(),an,null);
stageStepDescriptor.checkContextAvailability(context); // Good to check stuff I guess
// Create a new instance of the step, passing in arguments as a Map
Map<String, Object> stageArguments = new HashMap<>();
stageArguments.put("name", "mynutest");
Step stageStep = stageStepDescriptor.newInstance(stageArguments);
// so start the damd thing
StepExecution execution = stageStep.start(context);
// now that we have a callable instance, we set the step on the Cps Thread. Reflection to the rescue
Method mSetStep = cpsThread.getClass().getDeclaredMethod("setStep", StepExecution.class);
mSetStep.setAccessible(true);
mSetStep.invoke(cpsThread, execution);
// Finally. Start running the step
execution.start();
I have test and class under test, which uses time in file name.
Test code:
SimpleDateFormat simpleDateFormatTimestamp = new SimpleDateFormat("yyMMddHHmmss");
String outputpath= inboundDir+inboundFilePrefix+simpleDateFormatTimestamp.format(new Date())+".txt";
PowerMockito.whenNew(File.class).withArguments(outputpath).thenReturn(outputFileToInboundDir);
Class under test Code:
File outputFile=new File(inboundDir+inboundFilePrefix+simpleDateFormatTimestamp.format(new Date())+".txt");
Also in test and class under test, i have other new file calls so i am not able to use withAnyArguments mocking. when i use withAnyArguments only one mock gets returned for all the new file calls.
My test case pass sometime and fails other times depending on test and class under test runs in same second ("yyMMddHHmmss") or not.
How do i remove this test failure, when class and test execute in different second.
Thanks
This is one possible solution.
String outputpath= inboundDir+inboundFilePrefix+simpleDateFormatTimestamp.format(new Date())+".txt";
PowerMockito.whenNew(File.class).withAnyArguments().thenAnswer(invocation -> {
String firstArgument = (String) invocation.getArguments()[0];
// do a pattern matching for firstArgument with a regex containing date in it.
// if its true then return outputpath
// else return something else
});
We could have used ArgumentCaptor but, PowerMockito.whenNew doesn't support that.
The Workaround which worked for me is as below.
Since i had only one such call in test, i removed variable part simpleDateFormatTimestamp.format(new Date())+".txt"
Now if i do below it worked fine.
String outputpath= inboundDir+inboundFilePrefix;
PowerMockito.whenNew(File.class).withArguments(startsWith(outputpath)).thenReturn(outputFileToInboundDir);
startsWith Matcher is available in org.mockito.Mockito
I've written a small method that is meant to tell me if another instance of the application is already running. I am aware that there are many ways to find out if another instance is running, but I chose this one. I am creating an empty file and keeping it locked for the duration of the application instance. If another instance is running, the tryLock() method is supposed to return null:
private static boolean alreadyRunning() throws IOException {
FileChannel fc = FileChannel.open(MYLOCKFILE,
StandardOpenOption.CREATE,
StandardOpenOption.WRITE,
StandardOpenOption.DELETE_ON_CLOSE);
return fc.tryLock() == null;
}
(MYLOCKFILE is a Path for a file in my temp directory.)
When testing this on Windows 7 Professional 64-bit, I found that it works as expected for the first instance and the second attempted instance. However, after the second instance exits (leaving just the first instance running), when a third instance is run, the tryLock() call throws java.nio.file.AccessDeniedException instead of returning null. Can you explain this behaviour? If this is considered normal behaviour, how can I differentiate between an existing instance having the file locked, and a real 'access denied' situation such as an idiot setting the TEMP directory to read-only?
I made a test project and tested the code the only problem because of which java.nio.file.AccessDeniedException is thrown is StandardOpenOption.DELETE_ON_CLOSE option used in the code.
I removed the option and it works fine now
FileChannel fc = FileChannel.open(MYLOCKFILE, StandardOpenOption.CREATE,
StandardOpenOption.WRITE);
Explanation that I can think because of which java.nio.file.AccessDeniedException is thrown is that as soon as your second instance terminates the option StandardOpenOption.DELETE_ON_CLOSE [More explaination] will attempt to delete the file on JVM exit and failing might have registered an event in kernel or OS to delete the file as and when possible. So if any other process tries to access, create or write the same file before deletion it throws java.nio.file.AccessDeniedException as a delete operation is already pending for that file.
EDIT
As per your new comment, you can add the following code in try finally block placed after checking alreadyRunning() code.
Snippet Example:
if(!alreadyRunning())
{
try
{
// YOUR CODE THAT RUNS
while(true)
{
//YOUR
Thread.sleep(35000);
}
}
finally
{
new File("f:\\test.lock").deleteOnExit();
}
}
Blockquote
I am facing a problem with jobParameters in spring batch.I have a jobParameter which is optional.For the first time when i am passing job parameter through commandLineJobRunner it is working.For the second time i am not passing any jobParameter but still it is taking the previous jobParameter.When i clear my Meta-Data then jobParameter is coming as null i am not passing.How can i fix this without clearing the Meta-Data.Is this happens normally in spring batch
edited code
I am using MapJobRegistry and next is used while launching the job.When i debugged i have observed that to increment the run.id it is loading all the previous parameters
public JobParameters More ...getNext(JobParameters parameters) {
if (parameters == null) {
parameters = new JobParameters();
}
long id = parameters.getLong(key, 0L) + 1;
return new JobParametersBuilder(parameters).addLong(key, id).toJobParameters();
}
First thing to mention is that you should not use all the Map... classes. They are not intendend for production and, therefore, you better of using the different Jdbc implementations. If you don't wanna use a real DB, you can use always an inmemory DB.
But about your initial question:
You are using the CommandLineJobRunner togehter with the option "next".
Having a look at method CommandLineJobRunner.start() you find the following lines:
if (opts.contains("-next")) {
JobParameters nextParameters = getNextJobParameters(job);
Map<String, JobParameter> map = new HashMap<String, JobParameter>(nextParameters.getParameters());
map.putAll(jobParameters.getParameters());
jobParameters = new JobParameters(map);
}
You can see that getNextJobParameters is called. Inside this method you can see that the data of the previous run is loaded 'jobExplorer.getJobInstances(jobIdentifier, 0, 1);' (if there was a previous run). If there is a previous run, then the job-parameters of this old run are returned after applying the incrementer.next method -> hence, this is the reason you get your old parameters.
Now, this is the technical explanation but the question that follows is how you should use the "next" and "restart" option in order to get what you want.
using of next:
- next works only as expected if you launch a job with the same name and the same jobparameters. Otherwise the results can be confusing. Actually, I use next only inside units and integration tests
using of restart:
- you can use "restart" if the previous jobexecution with the same jobname failed. Also here, the jobparameters will be taken from your previous launch.
For a normal start of a job, you shouldn't be using next nor restart. A normal start of a job should always have a unique jobparameter. For instance, a "runid" whose value is changed with every start of the job. (otherwise, you would get JobInstanceAlready... Exception).
In case of unit tests, I use a unique "runId" for every testcase. And here, I'm using the "next" option.
http://pastebin.com/m5fa7685e
It seems to fail when getting f3.. Output is:
not ready
File is null
Exception in thread "main" java.lang.NullPointerException
at BuabFile.parseBUAB(BuabFile.java:93)
at AddressBook.createBrowseForm(AddressBook.java:232)
at AddressBook.(AddressBook.java:51)
at Main.main(Main.java:4)"
But not before then - no file not found errors or anything...
My guess would be that the parseBUAB() method receives a "null" argument. Which means that it could be that it is the AddressBook class is responsible for the error.
It looks like you forgot to assign a value to BuabFile.file static field. You may want to add this to the end of your readFile() method:
BuabFile.file = f3;
I am guessing your AddressBook.createBrowseForm method looks something like this:
String filename = ...;
BuabFile buab = new BuabFile(filename);
buab.readFile();
ArrayList<String> buabLines = buab.returnFile(); // Returns null because readFile() never assigned a value to BuabFile.file
ArrayList<Buab> buabList = buab.parseBUAB(buabLines);
From all I can see, you just call parseBUAB(..) with a null value. I can't see the call to that method so you have to check the rest of your code.
For your 'not ready' output, which is created because your BufferedReader f3 is 'not ready', the API says
True if the next read() is guaranteed not to block for input, false otherwise.
Maybe you just call it too fast and the file is not loaded yet. Play with Thread.sleep() before calling ready() on the stream. Maybe a some-milliseconds blocking is just normal for File I/O.
And third - if f3 is the BufferedReader you want to keep, you have to assign it to the member file in the readFile() method. But now that's all I found ;)
I'm confused further but have found an answer sort of - I'm using windows 7 and have tried it on a windows xp computer and the code compiles fine and reads in the file (other errors you lot have noted are to be changed anyway through development - this was just one stick in the way...).
I'm wondering if there is some Windows 7 error with eclipse and opening/reading files...