I´m running the GO-CD Server and Agents as Alpine Docker containers on a linux machine
I´m adding several pipelines through the json config plugin. The Server seems to start the first Pipeline and upload all the materials. Then he triggers 2 downstream pipes who´ll fetch the materials.
The 2 Downstream pipelines are beeing scheduled and asingned to an agent but they are not starting.
I´ve stopped the stages to see if the error was still comming and they stopped too.
2018-08-29 15:34:28,029 ERROR [85#MessageListener for WorkFinder] JMSMessageListenerAdapter:77 - Exception thrown in message handling by listener com.thoughtworks.go.server.messaging.scheduling.WorkFinder#2c731a16
java.lang.NullPointerException: null
2018-08-29 15:34:36,919 ERROR [85#MessageListener for WorkFinder] JMSMessageListenerAdapter:77 - Exception thrown in message handling by listener com.thoughtworks.go.server.messaging.scheduling.WorkFinder#2c731a16
java.lang.NullPointerException: null
2018-08-29 15:34:38,040 ERROR [85#MessageListener for WorkFinder] JMSMessageListenerAdapter:77 - Exception thrown in message handling by listener com.thoughtworks.go.server.messaging.scheduling.WorkFinder#2c731a16
java.lang.NullPointerException: null
Thats how the some of the pipelines are configured in the repository
{
"format_version" : 3,
"group": "group1",
"name": "Test-Pipeline-Stage",
"label_template": "${COUNT}",
"enable_pipeline_locking" : false,
"parameters": [],
"materials": [
{
"type": "git",
"url": ""
}
],
"stages": [
{
"name": "Test",
"fetch_materials": true,
"never_cleanup_artifacts": false,
"clean_working_directory": false,
"approval" : null,
"jobs": [{
"name": "RunUnittestAndCoverage",
"run_instance_count" : "all",
"environment_variables": [],
"timeout": 1,
"tabs": [
{
"name": "coverage",
"path": "coverage/coverage.html/index.html"
}
],
"artifacts": [
{
"source":"project.tar.gz",
"destination":"project_source",
"type":"build"
},
{
"source": "test_result.xml",
"destination": "testoutput",
"type": "test"
},
{
"source": "coverage.html",
"destination": "coverage",
"type": "build"
},
{
"source": "coverage.xml",
"destination": "coverage",
"type": "build"
}
],
"tasks":[
{
"type": "exec",
"run_if": "passed",
"on_cancel": null,
"command":"tox"
},
{
"type":"exec",
"run_if": "passed",
"on_cancel": null,
"command":"/usr/bin/xmllint",
"arguments": [
"--format",
"test_resulto.xml",
"--output",
"test_result.xml"
]
},
{
"type":"exec",
"run_if": "passed",
"on_cancel": null,
"command": "/bin/ls",
"arguments":[
"-al"
]
},
{
"type":"exec",
"run_if": "passed",
"on_cancel": null,
"command": "/bin/tar",
"arguments":[
"-cvzf",
"project.tar.gz",
"/godata/pipelines/Test-Pipeline-Stage"
]
}
]
}
]
}
]
}
{
"format_version" : 3,
"group": "group1",
"name": "Docker-Deployment-Pipeline-Stage",
"label_template": "${COUNT}",
"enable_pipeline_locking" : false,
"parameters": [],
"materials": [
{
"type": "dependency",
"pipeline": "Build-Pipeline-Stage",
"stage": "BuildPythonProduct",
"name": "project_source"
},
{
"type": "dependency",
"pipeline": "Analyse-Pipeline-Stage",
"stage":"Analyse",
"name":"analyse_result"
}
],
"stages":[
{
"name": "BuildDockerImage",
"fetch_materials": true,
"never_cleanup_artifacts": false,
"clean_working_directory": false,
"approval" : null,
"jobs":[
{
"name": "BuildImage",
"run_instance_count" : "all",
"environment_variables": [],
"timeout": 1,
"tasks": [
{
"type": "fetch",
"artifact_origin": "gocd",
"run_if": "any",
"pipeline": "Build-Pipeline-Stage",
"stage": "BuildPythonProduct",
"job": "BuildAndUpLoadWheel",
"is_source_a_file": true,
"source": "docker/Dockerfile",
"destination": ""
},
{
"type": "fetch",
"artifact_origin": "gocd",
"run_if": "any",
"pipeline": "Build-Pipeline-Stage",
"stage": "BuildPythonProduct",
"job": "BuildAndUpLoadWheel",
"is_source_a_file": true,
"source": "docker/production.ini",
"destination": ""
},
{
"type":"exec",
"run_if": "passed",
"on_cancel": null,
"command":"/usr/bin/docker",
"arguments": [
"build",
"-t",
"project_creator",
"."
]
}
]
}
]
}
]
}
Related
we are getting issue with multiple policies/permission.
Policy A - user A with scope A,B,C works fine.
Policy B - user B with scope B,C,D gives 403.
Policy C - user C with scope C,D,E gives 403.
If User B is added to Policy A it works fine and retunes combined result of scope A,B,C,D.
For User C we need to this add this user to both policy A,B and it works fine and retunes combined result of scope A,B,C,D,E.
We want this to work independently. For user B and C to work by adding them to policy B for user B and policy C for user C.
Both Policies are Positive and Both Permissions are Affirmative.
In Evaluate page it shows as expected but from Postman it gives error.
JSON from Authorization Export
{
"allowRemoteResourceManagement": false,
"policyEnforcementMode": "ENFORCING",
"resources": [
{
"name": "ResourceA",
"type": "urn:generic-rest-api:resources:employees",
"ownerManagedAccess": false,
"attributes": {},
"_id": "1b41c828-b78f-44da-84ab-e62a0b4843b5",
"uris": [
"/v1/employees/*"
],
"scopes": [
{
"name": "urn:generic-rest-api:scopes:employees:attribute:admindescription:view"
},
{
"name": "urn:generic-rest-api:scopes:employees:attribute:accessRole:view"
},
{
"name": "urn:generic-rest-api:scopes:employees:attribute:address:view"
}
]
},
{
"name": "ResourceB",
"type": "urn:generic-rest-api:resources:employees",
"ownerManagedAccess": false,
"displayName": "ResourceB",
"attributes": {},
"_id": "6237e427-55ef-4c1e-9e3d-714e5bfe31c9",
"uris": [
"/v1/employees/*"
],
"scopes": [
{
"name": "urn:generic-rest-api:scopes:employees:attribute:admindescription:view"
},
{
"name": "urn:generic-rest-api:scopes:employees:attribute:companyid:view"
},
{
"name": "urn:generic-rest-api:scopes:employees:attribute:authorizedAmount:view"
}
]
},
{
"name": "Default Resource",
"type": "urn:employee-rest-api:resources:default",
"ownerManagedAccess": false,
"attributes": {},
"_id": "fb476666-f779-4a86-8234-d49e897f6405",
"uris": [
"/*"
]
}
],
"policies": [
{
"id": "1e08f5f3-a766-416b-9448-50716dd0580b",
"name": "PolicyB",
"type": "user",
"logic": "POSITIVE",
"decisionStrategy": "UNANIMOUS",
"config": {
"users": "[\"UserB\"]"
}
},
{
"id": "8f90f109-0c7b-4caa-9620-50b7d36bd463",
"name": "Default Policy",
"description": "A policy that grants access only for users within this realm",
"type": "js",
"logic": "POSITIVE",
"decisionStrategy": "AFFIRMATIVE",
"config": {
"code": "// by default, grants any permission associated with this policy\n$evaluation.grant();\n"
}
},
{
"id": "c4ce02d8-cb2d-479f-ad22-82f092535b1b",
"name": "PolicyA",
"type": "user",
"logic": "POSITIVE",
"decisionStrategy": "UNANIMOUS",
"config": {
"users": "[\"UserA\"]"
}
},
{
"id": "29f9ae37-afa9-4a98-864f-bc9f8dd7a783",
"name": "PermissionA",
"type": "resource",
"logic": "POSITIVE",
"decisionStrategy": "AFFIRMATIVE",
"config": {
"resources": "[\"ResourceA\"]",
"applyPolicies": "[\"PolicyA\"]"
}
},
{
"id": "3b71972f-3754-464d-99b6-f5c11e5962cc",
"name": "PermissionB",
"type": "resource",
"logic": "POSITIVE",
"decisionStrategy": "AFFIRMATIVE",
"config": {
"resources": "[\"ResourceB\"]",
"applyPolicies": "[\"PolicyB\"]"
}
},
{
"id": "3f21cf9d-f0b4-4b54-a3b9-5cbc4c1f060c",
"name": "Default Permission",
"description": "A permission that applies to the default resource type",
"type": "resource",
"logic": "POSITIVE",
"decisionStrategy": "AFFIRMATIVE",
"config": {
"defaultResourceType": "urn:employee-rest-api:resources:default",
"applyPolicies": "[\"Default Policy\"]"
}
}
],
"scopes": [
{
"id": "c14170f6-0ef7-416e-bde8-c394f6f3ed13",
"name": "urn:generic-rest-api:scopes:employees:attribute:admindescription:view"
},
{
"id": "1d88b211-7961-4a2b-bfae-696566366f1f",
"name": "urn:generic-rest-api:scopes:employees:attribute:adusertype:view"
},
{
"id": "bfa6a52a-46de-4e86-8c72-4ea738752cd0",
"name": "urn:generic-rest-api:scopes:employees:attribute:accessRole:view"
},
{
"id": "f646b5af-bcc4-42a5-b135-de5d0c78ac5b",
"name": "urn:generic-rest-api:scopes:employees:attribute:address:view"
},
{
"id": "50286c2a-b0b4-423b-ac99-9b7b64bdebee",
"name": "urn:generic-rest-api:scopes:employees:attribute:authorizedAmount:view"
},
{
"id": "08140d1b-1c51-4563-96c4-4f6a628e9933",
"name": "urn:generic-rest-api:scopes:employees:attribute:companyid:view"
}
],
"decisionStrategy": "UNANIMOUS"
}
How to create n number of instances in GCP using rest api.
in AWS java SDK, there is a method withMaxCount where we specify number of ec2 instances.
Similarly is there anything for GCP compute.
You can use REST API in loop to create instances.
Example request will look something like this:
{
"kind": "compute#instance",
"name": "INSTANCE-NAME",
"zone": "projects/PROJECT-NAME/zones/us-central1-a",
"machineType": "projects/PROJECT-NAME/zones/us-central1-a/machineTypes/e2-medium",
"displayDevice": {
"enableDisplay": false
},
"metadata": {
"kind": "compute#metadata",
"items": []
},
"tags": {
"items": []
},
"disks": [
{
"kind": "compute#attachedDisk",
"type": "PERSISTENT",
"boot": true,
"mode": "READ_WRITE",
"autoDelete": true,
"deviceName": "INSTANCE-NAME",
"initializeParams": {
"sourceImage": "projects/debian-cloud/global/images/debian-10-buster-v20210122",
"diskType": "projects/PROJECT-NAME/zones/us-central1-a/diskTypes/pd-standard",
"diskSizeGb": "10",
"labels": {}
},
"diskEncryptionKey": {}
}
],
"canIpForward": false,
"networkInterfaces": [
{
"kind": "compute#networkInterface",
"subnetwork": "regions/us-central1/subnetworks/default",
"accessConfigs": [
{
"kind": "compute#accessConfig",
"name": "External NAT",
"type": "ONE_TO_ONE_NAT",
"networkTier": "PREMIUM"
}
],
"aliasIpRanges": []
}
],
"description": "",
"labels": {},
"scheduling": {
"preemptible": false,
"onHostMaintenance": "MIGRATE",
"automaticRestart": true,
"nodeAffinities": []
},
"deletionProtection": false,
"reservationAffinity": {
"consumeReservationType": "ANY_RESERVATION"
},
"serviceAccounts": [
{
"email": "111111111111-compute#developer.gserviceaccount.com",
"scopes": [
"https://www.googleapis.com/auth/devstorage.read_only",
"https://www.googleapis.com/auth/logging.write",
"https://www.googleapis.com/auth/monitoring.write",
"https://www.googleapis.com/auth/servicecontrol",
"https://www.googleapis.com/auth/service.management.readonly",
"https://www.googleapis.com/auth/trace.append"
]
}
],
"shieldedInstanceConfig": {
"enableSecureBoot": false,
"enableVtpm": true,
"enableIntegrityMonitoring": true
},
"confidentialInstanceConfig": {
"enableConfidentialCompute": false
}
}
Replace it with your own project name and service account, and you can test it here.
I'm trying to add some tests with Rest-Assured to my application, but I can't figure out how to assert some nested values. The error message is :
Expected: (a collection containing "json")
Actual: [[json, spring, gulp, path etc...]]
Here is the code :
when().
get("/api/personsByID/{id}/{count}", 262, 2).
then().
statusCode(200).
body("personDependencies.name", hasItems("json"));
And here is the JSON file that is returned by rest controller:
[
{
"id": 346,
"verified": true,
"displayName": "eda656a2c3cb59ae840e40a28ba4ab50bfb9de0185abcb901c6af6dc59d6668f",
"emails": [
{
"email": "16a23f2e5477df0bbcad718c3abc235b2cb8a1b6648d14f58d42a7be13df2b6e"
}
],
"personDependencies": [
{
"name": "json"
},
{
"name": "spring"
},
{
"name": "gulp"
},
{
"name": "path"
},
{
"name": "junit"
},
{
"name": "activemq"
},
{
"name": "hibernate"
},
{
"name": "jstl"
},
{
"name": "phantomjs"
},
{
"name": "activiti"
},
{
"name": "commons"
},
{
"name": "h2"
},
{
"name": "joda"
},
{
"name": "log4j"
},
{
"name": "exec"
},
{
"name": "admin"
},
{
"name": "coveralls"
},
{
"name": "cxf"
},
{
"name": "cglib"
},
{
"name": "camel"
},
{
"name": "sugaronrest"
},
{
"name": "tslint"
},
{
"name": "httpclient"
},
{
"name": "guava"
},
{
"name": "inventory"
},
{
"name": "jackson"
},
{
"name": "gson"
},
{
"name": "event"
},
{
"name": "OTRS"
},
{
"name": "maven"
},
{
"name": "karma"
},
{
"name": "slf4j"
},
{
"name": "postgresql"
},
{
"name": "typescript"
},
{
"name": "jasmine"
},
{
"name": "spa"
},
{
"name": "javax.servlet"
}
],
"countries": [],
"member_of": [],
"projects": [],
"employee_type": [],
"languages": [
{
"language": "reStructuredText",
"sum": 575
},
{
"language": "JSON",
"sum": 21
},
{
"language": "JavaScript",
"sum": 4467
},
{
"language": "Java",
"sum": 7958
},
{
"language": "Python",
"sum": 2
},
{
"language": "XML",
"sum": 477
},
{
"language": "Plain Text",
"sum": 41
}
],
"distance": 0.6028837702084446
}
]
I have no idea how to make proper assertions, any help would be great. Thanks!
If I am reading your question right you need to check if a certain values are present in a list that is returned for a particular ID
The below should work for you
given().when().get().then().body("find {it.id == 346}.personDependencies.name", hasItems("json", "jackson"));
The first problem you don't need to check the presence of an item with hasItems, you should use hasItem
when().
get("/api/personsByID/{id}/{count}", 262, 2).
then().
statusCode(200).
body("personDependencies.name", hasItem("json"));
Then if you need to add more message to the assertion when the test fails you can do such way:
when().
get("/api/personsByID/{id}/{count}", 262, 2).
then().
statusCode(200).
body("personDependencies.name", describedAs("Array not containing the provided item",hasItem("json")));
In your case you can validate such a way:
when().
get("/api/personsByID/{id}/{count}", 262, 2).
then().
statusCode(200).
body("personDependencies[*].name", hasItem("json"));
I have a micro-service with spring-boot-2.2.4 with actuator. When I hit the /actuator/metrics/http.server.requests url, I'm getting below response.
{
"name": "http.server.requests",
"description": null,
"baseUnit": "milliseconds",
"measurements": [
{
"statistic": "COUNT",
"value": 77.0
},
{
"statistic": "TOTAL_TIME",
"value": 2272.891396
},
{
"statistic": "MAX",
"value": 0.0
}
],
"availableTags": [
{
"tag": "exception",
"values": [
"None"
]
},
{
"tag": "method",
"values": [
"GET"
]
},
{
"tag": "uri",
"values": [
"/queue/{queueId}",
"/actuator/metrics/{requiredMetricName}"
]
},
{
"tag": "outcome",
"values": [
"SUCCESS"
]
},
{
"tag": "status",
"values": [
"200"
]
}
]
}
In graphite server I'm able to get the count details but I'm unable to get the TOTAL_TIME metrics details. Can someone please help me to figure out this issue. Where can I get this TOTAL_TIME in graphite server UI.
I have an endpoint of my API, now I want to track that whenever a request is being made to my endpoint then how many were returning 200 and how many were returning 400 or other HTTP status code please advise how can I achieve the same in spring boot project, I am using spring boot actuators 2.
Suppose my endpoint is:
https://localhost:9090/users
So what I want is:
{
"404": 1,
"200": 6,
"409": 1
}
You can use /actuator/metrics/ to get all endPoints which are executed/called with their count, exception, outcome, status, total time, etc. For this case count, status is useful
For that, you have to add a dependency in pom.xml or similar in grade
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
<version>2.1.3.RELEASE</version>
</dependency>
Get Dependency
To get details of all endPoint which are executed/called (see outcome all endPoint)
localhost:8889/actuator/metrics/http.server.requests
To get details of particular endPoint (see outcome particular
endPoint)
localhost:8889/actuator/metrics/http.server.requests?tag=uri:<endPoint>
localhost:8889/actuator/metrics/http.server.requests?tag=uri:/users
To get the count of particular endPoint with Status Code (see outcome
particular endPoint with Status code)
localhost:8889/actuator/metrics/http.server.requests?tag=uri:/users&tag=status:200
outcome all endPoint
{
"name": "http.server.requests",
"description": null,
"baseUnit": "seconds",
"measurements": [
{
"statistic": "COUNT",
"value": 13
},
{
"statistic": "TOTAL_TIME",
"value": 0.42338
},
{
"statistic": "MAX",
"value": 0
}
],
"availableTags": [
{
"tag": "exception",
"values": [
"None"
]
},
{
"tag": "method",
"values": [
"GET"
]
},
{
"tag": "uri",
"values": [
"/actuator/metrics/{requiredMetricName}",
"/getCountByStatus"
]
},
{
"tag": "outcome",
"values": [
"CLIENT_ERROR",
"SUCCESS"
]
},
{
"tag": "status",
"values": [
"404",
"200"
]
}
]
}
see outcome particular endPoint
{
"name": "http.server.requests",
"description": null,
"baseUnit": "seconds",
"measurements": [
{
"statistic": "COUNT",
"value": 5
},
{
"statistic": "TOTAL_TIME",
"value": 0.1830878
},
{
"statistic": "MAX",
"value": 0
}
],
"availableTags": [
{
"tag": "exception",
"values": [
"None"
]
},
{
"tag": "method",
"values": [
"GET"
]
},
{
"tag": "outcome",
"values": [
"CLIENT_ERROR",
"SUCCESS"
]
},
{
"tag": "status",
"values": [
"404",
"200"
]
}
]
}
outcome particular endPoint with Status code
{
"name": "http.server.requests",
"description": null,
"baseUnit": "seconds",
"measurements": [
{
"statistic": "COUNT",
"value": 3
},
{
"statistic": "TOTAL_TIME",
"value": 0.034849
},
{
"statistic": "MAX",
"value": 0
}
],
"availableTags": [
{
"tag": "exception",
"values": [
"None"
]
},
{
"tag": "method",
"values": [
"GET"
]
},
{
"tag": "outcome",
"values": [
"SUCCESS"
]
}
]
}
Configure Spring boot actuator dependencies in your pom.xml file.
The metrics you required can be accessible by this endpoint localhost:8080/actuato/metrics
It will contains statistics like threads, heap memory, remaining memory etc.
For your case these two keys in that JSON response will be useful.
Guage and counter
Guage - will gives each API response time.
Counter - this will full fill your requirement like {
200:4,
400:1
}
Explaination :
Your API returns 200 status code 4 times and 400 status code 1 time.