I have set up an end-to-end component test in Junit. It tests that objects generated at application startup are received at the other end of my application before they are sent over the network. I assert that the the number of objects received are the same as those generated. To be clear, this is only testing within the single application component. The network is mocked out.
When I run this test in Eclipse IDE, I send 4 and receive 4. When I run the test from Apache Ant, I send 4 but only 3 are received.
Does anybody know what this could be caused by? The test is probably run quicker with Ant, but my application is single threaded, so I don't see how this would make a difference.
Thanks!
Related
I created a project using cucumber to perform e2e tests of various apis I consume. I would like to know if I can run these tests through endpoints to further automate the application that was created.
That way I would be able to upload this app and would not need to keep calling locally.
You can do that if you create a Rest API with a get method which executes the test runner when called.
How to run cucumber feature file from java code not from JUnit Runner
But I don't recommend you to do that since what you are trying to achieve seems to me similar to a pipeline definition.
If you're in touch with the developers of these APIs, you can speak with them about including your test cases in their pipeline, since they probably have one in place.
If, for some reason, you still want to trigger your tests remotely and set it up by your own, I would recommend you to start reading about Jenkins. You can host it on any machine and run your tests from there, accessing from any machine to your jenkins instance:
https://www.softwaretestinghelp.com/cucumber-jenkins-tutorial/
If your code is hosted in any platform like github or gitlab, they already have its own way of creating pipelines and you can use it to run your tests. Read about Gitlab pipelines or Github actions.
So this is my situation:
I am fairly new to gitlab-ci. I don't host my own gitlab instance but rather push everything to gitab itself. I am not using and am not familiar with any build tools like Maven. I usually work and run my programms from an IDE rather than the terminal.
This is my problem:
When I push my Java project I want my pipeline to start the Junit tests I wrote. Whereas I've found various simple commands for other languages than Java to run unit tests I didn't come across anything for Junit. I've just found people using Maven, running the test locally and then pushing the test reports to gitlab. Is it even possible to easily run Junit tests on the gitlab server with the pipeline without build tools like Maven? Do I have to run them locally? Do I have to learn to start them with a Java terminal command? I've beeen searching for days now.
The documentation is clear:
To enable the Unit test reports in merge requests, you need to add artifacts:reports:junit in .gitlab-ci.yml, and specify the path(s) of the generated test reports.
The reports must be .xml files, otherwise GitLab returns an Error 500.
You then have various example in Ruby, Gio, Java (Gradle or Maven), and other languages.
But with GitLab 13.12 (May 2021), this gets better:
Failed test screenshots in test report
GitLab makes it easy for teams to set up end-to-end testing with automation tools like Selenium that capture screenshots of failed tests as artifacts.
This is great until you have to sort through a huge archive of screenshots looking for the specific one you need to debug a failing test.
Eventually, you may give up due to frustration and just re-run the test locally to try and figure out the source of the issue instead of wasting more time.
Now, you can link directly to the captured screenshot from the details screen in the Unit Test report on the pipeline page.
This lets you quickly review the captured screenshot alongside the stack trace to identify what failed as fast as possible.
See Documentation and Issue.
I know there is remote debugging but I want to go one step further: I would like to run tests in my eclipse that are run within another JVM - i.e. have access to static fields, resources, instances, etc from that JVM.
More specifically: I have an Apache server running locally on my machine and I would like to execute tests as if they were running natively in that very server.
Currently, I implemented my own JUnit test runner that runs within that server/JVM and creates test report XMLs that are written to a folder that I can inspect. But that's a bit bothersome so I would like to be able to run them directly with a mouseclick from eclipse and have them presented there nicely in the JUnit view.
So my question is:
Is there a way to run (not debug) code from eclipse within another JVM?
If so - is this also possible with tests, i.e. run and check the test reports with the JUnit view?
A presume that you are talking about an Apache Tomcat server. (Running unit tests within an arbitrary Apache server doesn't make a lot of sense.)
A Google search didn't yield a lot of leads, but I did come across this:
JUnit-Tomcat: No Mocking Just Testing
Be aware that the source code in the code in the downloadable hasn't been updated since 2006. Apparently doesn't support JUnit 4.0, and it was developed for Tomcat 5.5.
Another option would be to use an embedded Tomcat server within your unit tests; e.g.
https://github.com/mjeanroy/junit-servers
Note: this is not a recommendation.
I have around 200 testNG test cases can be executed through maven by and suite.xml file. But I want to convert these test cases into web service or any other similar webservice so that anybody can call any test case from there machine and will be able to know whether that particular functionality is working fine at that moment.
But what if no one calls the test webservices for longer time? You won't know the state of your application, if you have any failures/regressions.
Instead, you can use
continuous integration to run the tests automatically on every code push; see Jenkins for a more complete solution; or, more hacky, you can create your own cron job/demon/git hook on a server to run your tests automatically
a maven plugin that displays the results of the last execution of the automated tests; see Surefire for a html report on the state of the last execution of each test
I'm currently writing a Java Client Server Application. So i want to implement two Libraries, one for the Client and one for the Server. The Client Server Communication has a very strict protocol, that I wan't to test with JUnit.
As build tool im using Maven and a Husdon Server for continues Integration.
Actually I do not have any good Idea how to test these Client / Server Libraries.
I got following Approaches:
Just write a Dummy Client for testing the server and write a Dummy Server to test the Client.
Disadvantages: Unfortunately this will result in many extra work. I could not be 100% sure that client and Server could work together, because I'm not sure that the Tests are completely identical.
Write an separate Test Project that Tests the Client and the Server together.
Disadvantages: The Unit Tests does not belong to the Project it self, so Hudson will not run them automatically. Everyone who changes anything at one of these Libraries, will have to run the Tests manually to ensure, everything is correct. Also i will not receive any Code Coverage Report.
Are there any better approaches to test codes like that?
Maybe test a Maven Multi Module Project, or something like that.
I hope any one got a good solution for that Issue.
Thanks.
Think of all your code as "transforms input to output": X -> [A] -> Y
X is the data that goes in, [A] is the transformer, Y is the output. In your case, you have this setup:
[Client] -> X -> [Server] -> Y -> [Client]
So the unit tests work like this:
You need a test that runs the client code to generate X. Verify that the code actually produces X with an assert. X should be a final static String in the code.
Use the constant X in a second test to call the server code which transforms it into Y (another constant).
A third test makes sure that the client code can parse the input Y
This way, you can keep the tests independent and still make sure that the important parts work: The interface between the components.
My suggestion would be to use two levels of testing:
For your client/server project, include some mocking in your unit tests to ensure the object interfaces are working as expected.
Following the build, have a more extensive integration test run, with automation to install the compiled client and server on one or more test systems. Then you can ensure that all the particulars of the protocol are tested thoroughly. Have this integration test project triggered on each successful build of the client/server project. You can use JUnit for this and still receive the conventional report from Hudson.
The latest approach to solve this problem is by using Docker containers. Create a docker file containing a base image and all the necessary dependencies required for your client server application. Create a separate container for each node type of your distributed client-server system and test all the entry point server API/client interactions using TestNG or JUnit. The best part of this approach is that you are not mocking any service calls. In most cases you can orchestrate all the end-to-end client-server interactions.
There is a little bit of learning curve involved in this approach but Docker is becoming highly popular in the Dev community especially for solving this problem.
Here is an example of how you could use docker client api to pull docker images in your JUnit test:
https://github.com/influxdb/influxdb-java/blob/master/src/test/java/org/influxdb/InfluxDBTest.java
The approach described above is now opensource product : testcontainers
So finally the resolution was to build a Multi Module Project, with a separate Test Module that includes the Server and the Client Module
Works great in Husdon. And even better in the Eclipse IDE.
Thanks # Aaron for the hint