Develop Reference

Multiple property files for same Spring Boot Application based on request

I have a Spring Boot application for UI test automation using Cucumber and Selenium.
The application is expected to test multiple environments.
To begin with I have created json files with required properties like URLs, credentials etc. and load it(pass the file path as a property and use it) while triggering the test (mvn test -DconfigFile=config/env1_config.json).
I see that using profiles while running test is an option-Dspring.profiles.active=client1 but as i will configure multiple pipelines in Jenkins for testing multiple environments to use same project with different configuration files, it will clone the entire project and run tests in workspace corresponding to the pipeline. To avoid keeping multiple copies of the project, i am planning to use Rest API to trigger Selenium tests and have configuration files in Git.
Is it possible to create multiple application properties files with custom properties, place them in Git and use required property file in a Spring Boot application(inside a Rest API impl) based on a property or RequestParam using Spring Cloud Config or something?

you can use spring boot profiles and pass it as maven argument
just pass -Dspring.profiles.active=test1 as command line argument
you can read more here

You can do it by setting spring.profiles.active environment variable. Like if you are using property file for every environment by convention application-dev.properties, application-qa.properties. You can define you spring.profiles.active as dev and qa.
You can refer to the below link for more in site on same.
https://stackabuse.com/spring-boot-configuring-properties/

You can use Spring cloud server and client.
Make A project With Dependency Spring cloud server and make config file(appilcation.properties) there. For use that config file in other microservice
just add spring cloud cilent dependency in other projects and add this line in application.properties
spring.cloud.config.uri:[your spring cloud server project url]
refrence:-
spring cloud server:-
https://www.youtube.com/watch?v=gb1i4WyWNK4&list=PLqq-6Pq4lTTaoaVoQVfRJPqvNTCjcTvJB&index=11
spring cloud client:- https://www.youtube.com/watch?v=E2HkL766VHs&list=PLqq-6Pq4lTTaoaVoQVfRJPqvNTCjcTvJB&index=12

Sharing the approach I ended up with as it might help someone someday.
As i wanted to create a Rest API and use properties from different config files in API implementation, based on the api request,
I created a Spring Cloud Config Server application that connects to application properties repo in git and I consumed Rest APIs exposed by Spring Cloud Config Server (host:port/app/profile) in the service layer of my Rest API implementation.

Gobblin job metrics not publishing data to InfluxDB

I have configured .pull file to produce and send metrics to InfluxDb for source, extractor and converter jobs. I tried with the example wikipedia job.
metrics.enabled=true
metrics.report.interval=30000
metrics.reporting.influxdb.metrics.enabled=true
metrics.reporting.influxdb.events.enabled=true
metrics.reporting.influxdb.database=****
metrics.reporting.influxdb.url=http://**.**.**.**:8086
metrics.reporting.influxdb.user=********
metrics.reporting.influxdb.password=****
metrics.reporting.influxdb.sending.type=TCP
But the job is not sending any data. I could not find any example with metrics in Gobblin

I found the problem here. The gobblin uses config file as a source for metrics configuration. Instead of adding the properties to *.pull or *.job file, I had to add those to *.conf file.
Once added, it will send metrics to the platform whichever is addded to the gobblin application.

Using Prometheus to monitor Spring Boot Applications in Kubernetes Cluster

I have spring boot powered microservices deployed in my local kubernetes cluster. The microservices are using micrometer and prometheus registry but due to our company policy the actuator is available on another port:
8080 for "business" http requests
8081/manage for actuator. So, I can access http://host:8081/manage/prometheus and see the metrics when running the process locally (without kubernetes).
Now, I'm a beginner in Prometheus and have a rather limited knowledge in kubernetes (I'm coming with a Java developer background).
I've created a POD with my application and succesfully run it in kubernetes. It works and I can access it (for 8080 I've created a service to map the ports) and I can execute "business" level http requests it from the same PC.
But I haven't find any examples of adding a prometheus into the picture. Prometheus is supposed to be deployed in the same kubernetes cluster just as another pod. So I've started with:
FROM #docker.registry.address#/prom/prometheus:v2.15.2
COPY entrypoint.sh /
USER root
RUN chmod 755 /entrypoint.sh
ADD ./prometheus.yml /etc/prometheus/
ENTRYPOINT ["/entrypoint.sh"]
entrypoint.sh looks like:
#!/bin/sh
echo "About to run prometheus"
/bin/prometheus --config.file=/etc/prometheus/prometheus.yml \
--storage.tsdb.path=/prometheus \
--storage.tsdb.retention.time=3d \
--web.console.libraries=/etc/prometheus/console_libraries \
--web.console.templates=/etc/prometheus/consoles
My question is about how exactly I should define prometheus.yml so that it will get the metrics from my spring boot pod (and other microservices that I have, all spring boot driven with the same actuator setup).
I've started with (prometheus.yml):
global:
scrape_interval: 10s
evaluation_interval: 10s
scrape_configs:
- job_name: 'prometheus'
metrics_path: /manage/prometheus
kubernetes_sd_configs:
- role: pod
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
relabel_configs:
- source_labels: [__meta_kubernetes_pod_label_app]
action: keep
regex: sample-pod-app(.*)|another-pod-app(.*)
But apparently it doesn't work, so I've asking for the advices:
If someone has a working example it would be the best :)
Intuitively I understand I need to specify the port mapping for my 8081 port but I'm not exactly know how
Since prometheus is supposed to run on another port am I supposed to expose a kubernetes service for port 8081 at the kubernetes level?
Do I need any security related resources in kubernetes to be defined?
As a side note. At this point I don't care about scalability issues, I believe one prometheus server will do the job, but I'll have to add Grafana into the picture.

Rather than hardcoding it in prometheus config you need to make use of annotations on your pods to to tell prometheus which pods, what path and which port Prometheus should scrape.
prometheus.io/scrape: "true"
prometheus.io/path=/manage/prometheus
prometheus.io/port=8081
prometheus.io/scheme=http
Spring boot micrometer example with Prometheus on kubernetes.
Prometheus deployment guide.

In order to let Prometheus collect metrics from your Spring Boot Application you need to add a specific dependencies to it. Here you can find a guide showing how to make it done: Spring Boot metrics monitoring using Prometheus & Grafana. Here is an example:
<dependency>
<groupId>io.prometheus</groupId>
<artifactId>simpleclient_spring_boot</artifactId>
<version>0.1.0</version>
</dependency>
<dependency>
<groupId>io.prometheus</groupId>
<artifactId>simpleclient_hotspot</artifactId>
<version>0.1.0</version>
</dependency>
If you would like to use a bit different strategy you can also check out this one: Monitoring Spring Boot applications with Prometheus and Grafana:
In order to compare the performance of different JDKs for reactive
Spring Boot services, I made a setup in which a Spring Boot
application is wrapped in a Docker container. This makes it easy to
create different containers for different JDKs with the same Spring
Boot application running in it. The Spring Boot application exposes
metrics to Prometheus. Grafana can read these metrics and allows to
make nice visualizations from it. This blog post describes a setup to
get you up and running in minutes.
Please let me know if that helped.

Access config properties with Python from Java Spring Cloud Config Server

I have a repository containing config properties that a Java Spring Cloud Config Server accesses whenever another Java Spring App requests to pull the properties. I want to have the same concept, but with a Python app that pulls similar properties. I know there's Python Spring libraries out there, but I can't find an example of Python pulling these properties from a similar config server.

In my case I have achieved this using spring-config-client library. I am sharing the code sample.
from spring_config.client import SpringConfigClient
config = (
ClientConfigurationBuilder()
.app_name("dashboard-service-python11") #config file
.address("http://localhost:3051")
.build()
)
c = SpringConfigClient(config)
c.get_config()
print(c.get_config())

How to use Prometheus' JMX exporter java agent to collect custom metrics

I'd like to use Prometheus' JMX exporter to collect custom application metrics using Prometheus. I have an application that I've packaged into a jar file ApplicationToMonitor.jar-- it exposes port 8989 and declares Prometheus metrics, but doesn't expose an end-point for prometheus to scrape (from what I've read, the prometheus javaagent takes care of this).
I'm not sure what the configuration.yaml file should look like. Also, why is it recommended that one use the shaded.io.prometheus library (and register new metric variables under the default registry) as opposed to the regular io.prometheus library and not using a registry at all?
I'm referencing the Prometheus JMX exporter documentation, just simply not understanding the aforementioned components.

You would only use the JMX exporter for code you don't control that's exposing JMX metrics. In this case you need to add some exposition per https://github.com/prometheus/client_java#http. The HTTPServer is simplest.