Develop Reference

How to disable some of the metrics of the actuator for prometheus

I am using spring boot actuator (Spring Boot 2) to monitor the state of the application. Prometheus collects metrics. It calls endpoint actuator/prometheus. But there are quite a lot of metrics there. Is there any way to disable some of the metrics for prometheus?

Spring boot Actuator vs prometheus Jmx exporter

For exporting the metrics (to Prometheus) from the spring boot micro service, we can use the spring boot actuator and one more option is to use the Prometheus JMX exporter(https://github.com/prometheus/jmx_exporter) as a javaAgent when running the service. Though both of the options serve the same purpose, I do see that the JMX exporter is exporting way lot more metrics than the spring boot actuator. I was scouting through some spring boot documentations to see if there is any option to enable more metrics with spring boot actuator, looks like all the JMX metrics are enabled by default. So the questions is, is there a way to expose more metrics from spring boot actuator? Is there any recommendation or comparison study available for both the options mentioned above?
Any help here is greatly appreciated. Thanks!

If you are using Spring boot 2.x, then it works like this:
In Spring Boot 2.0, the in-house metrics were replaced with Micrometer support, so we can expect breaking changes. If our application was using metric services such as GaugeService or CounterService, they will no longer be available.
Instead, we're expected to interact with Micrometer directly. In Spring Boot 2.0, we'll get a bean of type MeterRegistry autoconfigured for us.
for Spring boot 1.x:
The metrics endpoint publishes information about OS and JVM as well as application-level metrics. Once enabled, we get information such as memory, heap, processors, threads, classes loaded, classes unloaded, and thread pools along with some HTTP metrics as well.
and this seems to work like Prometheus JMX

Using Prometheus to monitor Spring Boot Applications in Kubernetes Cluster

I have spring boot powered microservices deployed in my local kubernetes cluster. The microservices are using micrometer and prometheus registry but due to our company policy the actuator is available on another port:
8080 for "business" http requests
8081/manage for actuator. So, I can access http://host:8081/manage/prometheus and see the metrics when running the process locally (without kubernetes).
Now, I'm a beginner in Prometheus and have a rather limited knowledge in kubernetes (I'm coming with a Java developer background).
I've created a POD with my application and succesfully run it in kubernetes. It works and I can access it (for 8080 I've created a service to map the ports) and I can execute "business" level http requests it from the same PC.
But I haven't find any examples of adding a prometheus into the picture. Prometheus is supposed to be deployed in the same kubernetes cluster just as another pod. So I've started with:
FROM #docker.registry.address#/prom/prometheus:v2.15.2
COPY entrypoint.sh /
USER root
RUN chmod 755 /entrypoint.sh
ADD ./prometheus.yml /etc/prometheus/
ENTRYPOINT ["/entrypoint.sh"]
entrypoint.sh looks like:
#!/bin/sh
echo "About to run prometheus"
/bin/prometheus --config.file=/etc/prometheus/prometheus.yml \
--storage.tsdb.path=/prometheus \
--storage.tsdb.retention.time=3d \
--web.console.libraries=/etc/prometheus/console_libraries \
--web.console.templates=/etc/prometheus/consoles
My question is about how exactly I should define prometheus.yml so that it will get the metrics from my spring boot pod (and other microservices that I have, all spring boot driven with the same actuator setup).
I've started with (prometheus.yml):
global:
scrape_interval: 10s
evaluation_interval: 10s
scrape_configs:
- job_name: 'prometheus'
metrics_path: /manage/prometheus
kubernetes_sd_configs:
- role: pod
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
relabel_configs:
- source_labels: [__meta_kubernetes_pod_label_app]
action: keep
regex: sample-pod-app(.*)|another-pod-app(.*)
But apparently it doesn't work, so I've asking for the advices:
If someone has a working example it would be the best :)
Intuitively I understand I need to specify the port mapping for my 8081 port but I'm not exactly know how
Since prometheus is supposed to run on another port am I supposed to expose a kubernetes service for port 8081 at the kubernetes level?
Do I need any security related resources in kubernetes to be defined?
As a side note. At this point I don't care about scalability issues, I believe one prometheus server will do the job, but I'll have to add Grafana into the picture.

Rather than hardcoding it in prometheus config you need to make use of annotations on your pods to to tell prometheus which pods, what path and which port Prometheus should scrape.
prometheus.io/scrape: "true"
prometheus.io/path=/manage/prometheus
prometheus.io/port=8081
prometheus.io/scheme=http
Spring boot micrometer example with Prometheus on kubernetes.
Prometheus deployment guide.

In order to let Prometheus collect metrics from your Spring Boot Application you need to add a specific dependencies to it. Here you can find a guide showing how to make it done: Spring Boot metrics monitoring using Prometheus & Grafana. Here is an example:
<dependency>
<groupId>io.prometheus</groupId>
<artifactId>simpleclient_spring_boot</artifactId>
<version>0.1.0</version>
</dependency>
<dependency>
<groupId>io.prometheus</groupId>
<artifactId>simpleclient_hotspot</artifactId>
<version>0.1.0</version>
</dependency>
If you would like to use a bit different strategy you can also check out this one: Monitoring Spring Boot applications with Prometheus and Grafana:
In order to compare the performance of different JDKs for reactive
Spring Boot services, I made a setup in which a Spring Boot
application is wrapped in a Docker container. This makes it easy to
create different containers for different JDKs with the same Spring
Boot application running in it. The Spring Boot application exposes
metrics to Prometheus. Grafana can read these metrics and allows to
make nice visualizations from it. This blog post describes a setup to
get you up and running in minutes.
Please let me know if that helped.

spring cloud consul high cpu in a spring boot application

I setup a Spring boot application with #EnableDiscoveryClient and using Consul as the cloud provider. I noticed that this generates high cpu consumption due to ConsulCatalogWatch.catalogServicesWatch. Looking at the code, I noticed that the annotation
#Scheduled(fixedDelayString = "${spring.cloud.consul.discovery.catalogServicesWatchDelay:10}")
will cause this method to be call at a very high rate.
Does anyone have met this and am I missing something ?

As a workaround, set spring.cloud.consul.discovery.catalogServicesWatchDelay=30000.
Fixed via issue 94.

Memcached with Spring

I have a Spring based application that uses Spring core, Spring MVC and Spring Data (Mongo) on the server side, and designed as a typical 3-tier application.
I have a list of services, that are typical Spring services/beans that I wish you integrate with memcached to cache some of my service results.
Can someone guide me to the steps to integrate memcached with spring for such an application?
A sample/tutorial/blog that gives a step by step process would be just great.
Thanks.

If you use Spring 3.1 take a look at Spring Cache abstraction. It's the easiest way to integrate caching in Spring application. Unfortunately Spring doesn't support memcached out of the box, there's only support for ehcache.
As far as I know there isn't available any provider that can store data to memcached through Spring Cache. In few days next version 3.0.0 of Simple Spring Memcached is going to be released with such support. In mean time you may try one of the latest snapshot or use Simple Spring Memcached 2.0.0 directly without Spring Cache abstraction.
UPDATE: Simple Spring Memcached 3.0.0 with Spring Cache integration is already available.

It's very trivial to do. You can look at 3levelmemcache project as an example at github its Spring based abstraction.