I have a Jenkins pipeline;
#Library('sharedLib#master')
import org.foo.point
pipeline {
agent { label 'slaveone' }
// agent { label 'master' }
stages {
stage('Data Build'){
steps{
script{
def Point = new point()
Point.hello("mememe")
}
}
}
}
}
which runs a small bit of code in a library called 'jenkins-shared-library/src/sharedLib';
package org.foo
import java.io.File
class point{
def hello(name){
File saveFile = new File("c:/temp/jenkins_log.txt")
saveFile.write "hello"
}
}
It runs fine on both 'master' and 'slaveone', but in both cases the 'jenkins_log.txt' file appears on the master. The log file contains this;
Running on slaveone in d:\Jenkins_WorkDir\workspace\mypipeline
How is this code running on slaveone and writing files to master?
Edit: I should also mention that this is my third attempt at doing this. The first one was with Groovy code direct in the pipeline, and the second was using a 'def' type call in the vars directory. Both produced the same behaviour, seemingly oblivious to the agent it was being run on.
I think everything inside the script runs on master, but here I found a workaround: Jenkins Declarative Pipeline, run groovy script on slave agent
Jenkins stores all logs on master only, that's why you cannot find any log on nodes.
Related
I'm trying to understand a comment that a colleague made. We're using testcontainers to create a fixture:
import org.testcontainers.containers.GenericContainer;
import org.testcontainers.utility.DockerImageName;
public class SalesforceFixture extends GenericContainer<SalesforceFixture> {
private static final String APPLICATION_NAME = "salesforce-emulator";
public SalesforceFixture() {
// super(ImageResolver.resolve(APPLICATION_NAME));
super(DockerImageName.parse("gcr.io/ad-selfserve/salesforce-emulator:latest"));
...
}
...
The commented code is what it used to be. The next line is my colleague's suggestion. And on that line he commented:
This is the part I don't know. The [ImageResolver] gets the specific version of the emulator, rather than the latest. You need a docker-info file for that though, which jib doesn't automatically generate (but I think it can).
This is what I know or have figured so far:
SalesforceFixture is a class that will be used by other projects to write tests. It spins up a container in Docker, running a service that emulates the real service's API. It's like a local version of the service that behaves enough like the real thing that if one writes code and tests using the fixture, it should work the same in production. (This is where my knowledge ends.)
I looked into ImageResolver—it seems to be a class we wrote that searches a filesystem for something:
public static String resolve(String applicationName, File... roots) {
Stream<File> searchPaths = Arrays.stream(roots).flatMap((value) -> {
return Stream.of(new File(value, "../" + applicationName), new File(value, applicationName));
});
Optional<File> buildFile = searchPaths.flatMap((searchFile) -> {
if (searchFile.exists()) {
File imageFile = new File(searchFile + File.separator + "/target/docker/image-name");
if (imageFile.exists()) {
return Stream.of(imageFile);
}
}
return Stream.empty();
}).findAny();
InputStream build = (InputStream)buildFile.map(ImageResolver::fileStream).orElseGet(() -> {
return searchClasspath(applicationName);
});
if (build != null) {
try {
return IOUtils.toString(build, Charset.defaultCharset()).trim();
} catch (IOException var6) {
throw new RuntimeException("An exception has occurred while reading build file", var6);
}
} else {
throw new RuntimeException("Could not resolve target image for application: " + applicationName);
}
}
But I'm confused. What filesystem? Like, what is the present working directory? My local computer, wherever I ran the Java program from? Or is this from within some container? (I don't think so.) Or maybe the directory structure inside a .jar file? Or somewhere in gcr.io?
What does he mean about a "specific version number" vs. "latest"? I mean, when I build this project, whatever it built is all I have. Isn't that equivalent to "latest"? In what case would an older version of an image be present? (That's what made me think of gcr.io.)
Or, does he mean, that in the project using this project's image, one will not be able to specify a version via Maven/pom.xml—it will always spin up the latest.
Sorry this is long, just trying to "show my work." Any hints welcome. I'll keep looking.
I can't comment on specifics of your own internal implementations, but ImageResolver seems to work on your local filesystem, e.g. it looks into your target/ directory and also touches the classpath. I can imagine this code was just written for resolving an actual image name (not an image), since it also returns a String.
Regarding latest, using a latest tag for a Docker image is generally considered an anti-pattern, so likely your colleague is commenting about this. Here is a random article from the web explaining some of the issues with latest tag:
https://vsupalov.com/docker-latest-tag/
Besides, I don't understand why you ask these questions which are very specific to your project here on SO rather than asking your colleague.
Suppose I've got a minimal Scala WORKSPACE file like this:
workspace(name = "scala_example")
git_repository(
name = "io_bazel_rules_scala",
commit = "e9e65ada59823c263352d10c30411f4739d5df25",
remote = "https://github.com/bazelbuild/rules_scala",
)
load("#io_bazel_rules_scala//scala:scala.bzl", "scala_repositories")
scala_repositories()
load("#io_bazel_rules_scala//scala:toolchains.bzl", "scala_register_toolchains")
scala_register_toolchains()
And then a BUILD:
load("#io_bazel_rules_scala//scala:scala.bzl", "scala_binary")
scala_binary(
name = "example-bin",
srcs = glob(["*.scala"]),
main_class = "Example",
)
And an Example.scala:
object Example { def main(args: Array[String]): Unit = println("running") }
I can run bazel run example-bin and everything works just fine. My problem is that this recent rules_scala PR changed the way the Java binary path is set to use the following:
ctx.attr._java_runtime[java_common.JavaRuntimeInfo].java_executable_exec_path
…instead of the previous ctx.executable._java.short_path.
After this change the Java binary path includes an external directory in the path, which seems to be a legacy thing (?). This means that after this change, if I run the following:
bazel run --nolegacy_external_runfiles example-bin
It no longer works:
INFO: Running command line: bazel-bin/example-bin
.../.cache/bazel/_bazel_travis/03e97e9dbbfe483081a6eca2764532e8/execroot/scala_example/bazel-out/k8-fastbuild/bin/example-bin.runfiles/scala_example/example-bin_wrapper.sh: line 4: .../.cache/bazel/_bazel_travis/03e97e9dbbfe483081a6eca2764532e8/execroot/scala_example/bazel-out/k8-fastbuild/bin/example-bin.runfiles/scala_example/external/local_jdk/bin/java: No such file or directory
ERROR: Non-zero return code '127' from command: Process exited with status 127
It also breaks some scripts I have that expect non-external paths.
Why is java_executable_exec_path giving me this external path? Is there some option I can give bazel to convince it not to do this?
Sorry for the slow reply -- it appears that this is because the Scala rules erroneously used java_executable_exec_path whereas they should have used java_executable_runfiles_path.
I sent a pull request to fix it, then I realized that you already did in https://github.com/bazelbuild/rules_scala/commit/4235ef58782ce2ec82981ea70b808397b64fe7df
Since the latter is now available at HEAD with Bazel, I'll remove the ugly if at least.
I'm using jenkins as CI tool. I used restful api to build a job remotely but I don't know how to get test result remotely as well.
I can't be more thankful if anybody know a solution
Use the XML or Json API. At most pages on Jenkins you can add /api/ to the url and get data in xml, json and similar formats. So for a job you can go to <Jenkins URL>/job/<Job Name>/api/xml and get informaiton about the job, builds, etc. For a build you can go to <Jenkins URL>/job/<Job Name>/<build number>/api/xml and you will get a summary for the build. Note that you can use the latestXXXBuild in order to get the latest successful, stable, failing, complete build, like this; <Jenkins URL>/job/<Job Name>/lastCompletedBuild/api/xml.
Additionally if youre using any plugin which publishes test results to the build, then for a given job you can go to <Jenkins URL>/job/<Job Name>/lastCompletedBuild/testReport/api/xml and you will get an xml report with results.
There is a lot more to it, you can control what is exported with the tree parameter and depth parameter. For a summary go to <Jenkins URL>/api/
Well, if you are using a jenkins shared library or decided to permit the security exceptions (a less good approach) then you can access them via a job and send them out to whatever you like - push vs pull
def getCurrentBuildFailedTests() {
def failedTests = []
def build = currentBuild.build()
def action = build.getActions(hudson.tasks.junit.TestResultAction.class)
if (action) {
def failures = build.getAction(hudson.tasks.junit.TestResultAction.class).getFailedTests()
println "${failures.size()} Test Results Found"
for (def failure in failures) {
failedTests.add(['name': failure.name, 'url': failure.url, 'details': failure.errorDetails])
}
}
return failedTests
}
I have a pipeline job using Groovy script set up to run multiple tests in "parallel", but I am curious as to how to get the report(s) unified.
I am coding my Selenium tests in Java and using TestNG and Maven.
When I look at the report in target/surefire-reports, the only thing there is the "last" test ran of "suite".
How can I get a report that combines all of the tests within the Pipeline parallel job?
Example Groovy code:
node() {
try {
parallel 'exampleScripts':{
node(){
stage('ExampleScripts') {
def mvnHome
mvnHome = tool 'MAVEN_HOME'
env.JAVA_HOME = tool 'JDK-1.8'
bat(/"${mvnHome}\bin\mvn" -f "C:\workspace\Company\pom.xml" test -DsuiteXmlFile=ExampleScripts.xml -DenvironmentParam="$ENVIRONMENTPARAM" -DbrowserParam="$BROWSERPARAM" -DdebugParam="false"/)
} // end stage
} // end node
}, // end parallel
'exampleScripts2':{
node(){
stage('ExampleScripts2') {
def mvnHome
mvnHome = tool 'MAVEN_HOME'
env.JAVA_HOME = tool 'JDK-1.8'
bat(/"${mvnHome}\bin\mvn" -f "C:\workspace\Company\pom.xml" test -DsuiteXmlFile=ExampleScripts2.xml -DenvironmentParam="$ENVIRONMENTPARAM" -DbrowserParam="$BROWSERPARAM" -DdebugParam="false"/)
} // end stage
} // end node
step([$class: 'Publisher', reportFilenamePattern: 'C:/workspace/Company/target/surefire-reports/testng-results.xml'])
} // end parallel
There is a little more to this code after this in terms of emailing the test runner the result of the test and such.
This works great, other than the reporting aspect.
I prefer to use ExtentReports because it has a ExtentX server that allows to you report on multiple different test reports.
I used to use ReportNG but development on that stalled and so I don't recommend it any more. It doesn't allow you combine reports anyway.
Other than that, you could use CouchBase or similar JSON database to store test results and then generate your own report from that information.
I am writing a "Total Commander" like application in Java. There is quite obvious file system dependency here.
I want to unit test it. I created directory structure for test purposes, I keep it in known location in SVN repository. It works great so far.
Now, I have a method that should ignore hidden files. How can I go about this? Can I mark file hidden in the SVN somehow?
Other solution would be to make one of the files hidden in the build script before running tests, but I'm afraid this would mark file as modified and always show in a commit dialog.
Any suggestions?
I would put all the initialization of a test directory into the tests themselves. And such a case would be simple:
create a directory
put some hidden and visible files into it
test
tear down by removing the directory
Essentially, accessing the file system is a big no-no when unit testing. For starters, these tests are slow(er) than your in-system tests, thus reducing the likelihood of you running your tests at a high frequency (such as with every compilation).
Much better if you use an adapter-pattern to abstract away the access to the file system. I'm a .NET developer so my example will be in C#, but I expect you to be able to translate it simply enough:
public class MyManager
{
private readonly IFileAdapter _fileAdapter;
public MyManager(IFileAdapter fileAdapter)
{
_fileAdapter = fileAdapter;
}
}
public interface IFileAdapter
{
public FileStream Open(string fileName);
public string ReadLine(FileStream fileStream);
// More methods...
}
public class FileAdapter : IFileAdapter
{
public FileStream Open(string fileName)
{
return System.Io.File.Open(fileName);
}
public string ReadLine(FileStream fileStream)
{
return File.Open(fileStream);
}
}
Now, as usual, you can mock the file system, as you would any other class, supplying the returned results. Remember - you are not testing Java's IO classes it is assumed that they work. You are only testing your classes (i.e. MyManager, in the example above).
Leave the tests that actually use the file system to your integration / acceptance tests.
Hope this helps,
Assaf.
I would prefer to abstract file system, so that my unit-test wouldn't require access to real file system. Of course, this abstraction layer must be tested with real file system, but this allow you to reduce dependency on it.
As for storing hidden files in SVN, I second artemb. You should create all files necessary to test in JUnit set up. Presumably, you should prefer setup per test method (#Before and #After). But if you encounter test slowness problems, have a look at #BeforeClass and #AfterClass. I consider they can be used with test suites too.
artemb's answer is correct, you can use #Before and #After to create and remove your structure for each test.
Here is some code I use to create a new directory with some files in it, it will create the directory in the systems temp dir, this is important because depending on the machine your tests will run on, you may will not be allowed to create files or dirs somewhere else. (I had to write this code to allow my tests to be executed on our linux integration machine...)
final String tempdir = System.getProperty("java.io.tmpdir");
final String dirname = Long.toHexString(System.currentTimeMillis());
final File dir = new File(tempdir, dirname);
dir.deleteOnExit();
dir.mkdir();
final String path = dir.getAbsolutePath();
assertTrue(dir.exists());
// pre condition, the directory is empty
assertTrue(dir.list().length == 0);
// create temp files in the directory
final int nbFiles = 3;
for (int i = 0; i < nbFiles; i++) {
(File.createTempFile("test", ".txt", dir)).deleteOnExit();
}
BTW you will have to know on what platform you run to be able to create hiden files...