I have successfully deployed a local docker registry and implemented an listener endpoint to receive event notifications following the documentation for configuration using a sample insecure configuration file. Pushing, pulling and listing images work well. However, i still receive no event notification. The registry logs are throwing some errors i do not really understand:
level=error msg="retryingsink: error writing events: httpSink{http://localhost:5050/event}: error posting: Post http://localhost:5050/event: dial tcp 127.0.0.1:5050: getsockopt: connection refused, retrying"
I will appreciate any info.
The endpoint listener is implemented in java
#RequestMapping(value="/event",method = RequestMethod.POST,consumes = "application/json")
public void listener(#RequestBody Events event) {
Event[] e = event.getEvents();
Event ee = new Event();
for (int i = 0; i < e.length; i++) {
System.out.println(e.length);
System.out.println(e[i].toString());
}
So after several hours of research i.e. inspecting the private registry logs, i realized the media-type of the messages posted by registry listener to notification endpoints is either application/octet stream or application/vnd.docker.distribution.manifest.v2+json. Hence my solution was to permit all media types using the annotation consumes = "*/*" as specified in this Spring documentation i.e.
#RequestMapping(value="/event",method = RequestMethod.POST,consumes = "*/*")
Related
In my current Java project, it's easy to track server-side user events in the "old" Google Analytics Universal Project with simple REST calls to Google Analytics. So that location tracking was working, i could override the server ip with the user ip, according to the parameter "&uip=1.2.3.4" (https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters?hl=de#uip).
As upgrading to GA4 is recommended, I was able to change all the REST parameters in my project and show my events in the new dashboard, except for the user location. I can't find any information about such a parameter. I tried using still "uip" but now all my requests are located to the country of my server.
Unfortunately it's not possible to track the event client side, because my project is a simple REST API, returning only JSON data.
Does anyone have an idea, if there's such a parameter like "uip" for ga4 or if this isn't possible anymore?
In the following way I setup my parameters:
private String getQueryParameters(MeasurementEvent event) {
StringBuilder body = new StringBuilder();
body.append("?v=").append(version);
body.append("&tid=").append(trackingId);
body.append("&cid=").append(event.getClientId());
body.append("&en=").append(eventName);
body.append("&aip=1");
if (StringUtils.hasText(event.getAction())) {
body.append("&ep.useraction=").append(event.getAction());
}
if (StringUtils.hasText(event.getCategory())) {
body.append("&ep.awsregion=").append(event.getCategory());
}
if (StringUtils.hasText(event.getLabel())) {
body.append("&ep.softwarename=").append(event.getLabel());
}
if (StringUtils.hasText(event.getRemoteAddress())) {
body.append("&uip=").append(event.getRemoteAddress());
}
if (StringUtils.hasText(event.getUrl())) {
body.append("&dl=").append(event.getUrl());
}
return body.toString();
}
I have an Azure Java Function App (Java 11, gradle, azure-functions-java-library 1.4.0) that is tied to an event hub trigger. There are parameters that I can inject into the annotation by surrounding with % as per https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-expressions-patterns. The connection isn't using the % since it's a special param that is always taken from the app properties.
When I run my function locally, using ./gradlew azureFunctionsRun it runs as expected. But once it's deployed to an Azure Function App, it complains that it can't resolve the params.
The error in Azure:
2021-05-27T18:25:37.522 [Error] Microsoft.Azure.WebJobs.Host: Error indexing method 'Functions.EventHubTrigger'. Microsoft.Azure.WebJobs.Host: '%app.eventhub.name%' does not resolve to a value.
The Trigger annotation looks like:
#FunctionName("EventHubTrigger")
public void run(
#EventHubTrigger(name = "event",
connection = "app.eventhub.connectionString",
eventHubName = "%app.eventhub.name%",
consumerGroup = "%app.eventhub.consumerGroup%",
cardinality = Cardinality.MANY)
List<Event> event,
final ExecutionContext context
) {
// logic
}
Locally in local.settings.json I have values for:
"app.eventhub.connectionString": "Endpoint=XXXX",
"app.eventhub.name": "hubName",
"app.eventhub.consumerGroup": "consumerName"
And in Azure for the function app, I have Configuration (under Settings) for each of the above.
Any ideas what I'm missing?
After some further investigation, I managed to get things working in Azure Function Apps by changing my naming convention, from using . as separators to _.
This ended up working both locally and when deployed:
#FunctionName("EventHubTrigger")
public void run(
#EventHubTrigger(name = "event",
connection = "app_eventhub_connectionString",
eventHubName = "%app_eventhub_name%",
consumerGroup = "%app_eventhub_consumerGroup%",
cardinality = Cardinality.MANY)
List<Event> event,
final ExecutionContext context
) {
// logic
}
With configuration settings in local.settings.json as:
"app_eventhub_connectionString": "Endpoint=XXXX",
"app_eventhub_name": "hubName",
"app_eventhub_consumerGroup": "consumerName"
And corresponding updates made to the App configuration.
I want to send a scheduled message to the Azure Service Bus with JMS.
My code is based on org.apache.qpid.jms.message.JmsMessage. I've found one solution for the given problem, but it uses org.apache.qpid.proton.message.Message which has .getMessageAnnotations(), which allows to edit message annotations and add some properties that are correctly recognized and processed by Azure Service Bus. My message impl is missing that method.
What I've found in offical docs and implementations in node.js, to schedule a message with Azure Service Bus, you need to send header BrokerProperties/brokerProperties which has valid json.
Other headers/properties will be marked as Customer properties and ignored by Azure Service Bus.
official azure docs about JMS says that setting ScheduledEnqueueTimeUtc is not officialy supported by JMS API. But it can be achieved manually by setting property.
So when I send message to the queue, then I can post process it in lambda and set some properties:
jmsTemplate.convertAndSend(queue, payload, message -> {
var date = Date.from(ZonedDateTime.now(ZoneId.of("UTC")).plus(delay, ChronoUnit.MILLIS).toInstant());
var brokerProps = Map.of("ScheduledEnqueueTimeUtc", date.toGMTString());
message.setStringProperty(
"brokerProperties",
objectMapper.writeValueAsString(brokerProps)
);
return message;
});
And it doesn't work. The message arrives at the queue, but when I try to peek it on the Service Bus Explorer on Azure it throws error in the browser console and the operation lasts forever. I guess setting that property brokerProperties does some impact for Service Bus.
I have also tried to send a map with date as a string (with date format that is used by the Azure) like "ScheduledEnqueueTimeUtc", "Thu, 25 Mar 2021 12:54:00 GMT", but it also is recognized as an error by Service Bus (peeking lasts forever and error in the browser console is thrown).
I've tried to set string properties like x-opt-scheduled-enqueue-time or x-ms-scheduled-enqueue-time which I've found in other threads on SO, but none of them works with my example.
I saw that Microsoft gives some library for Java to communicate with Azure Service Bus, but I need to maintain independency from the Cloud provider in my code and don't include any additional libs.
Is there any example of using JMS message implementation from the package org.apache.qpid.jms.message.JmsMessage to set BrokerProperties for Azure Service Bus?
My team is currently facing the same issue.
We found that the ScheduledEnqueueTimeUtc property is set in the MessageAnnotationsMap. Unfortunately, the org.apache.qpid.jms.provider.amqp.message.AmqpJmsMessageFacade, which is used by JMS, has set the getter and setter to package private. But we found out that you can use the setTracingAnnotation(String key, Object value) Method.
Example:
public void sendDelayedMessage() {
final var now = ZonedDateTime.now();
jmsTemplate.send("test-queue", session -> {
final var tenMinutesFromNow = now.plusMinutes(10);
final var textMessage = session.createTextMessage("Hello Service Bus!");
((JmsTextMessage) textMessage).getFacade().setTracingAnnotation("x-opt-scheduled-enqueue-time", Date.from(tenMinutesFromNow.toInstant()));
return textMessage;
});
log.info("Sent at: " + now);
}
Proof:
Big Thanks to my teammate!!
I need to implement a basic snmp agent into my application. This agent should be able to answer get and set requests from snmp browsers on my own OID's. Since most examples for snmp4j are quite old and use the "extend the org.snmp4j.agent.BaseAgent.classapproach" there are of little use.
Therefore tried to understand and strip down the org.snmp4j.agent.example.SampleAgentincluded inside the snmp4j-agent-3.2.1.jar. I managed to get an example working with just the mandatory MIB's ( i at least think they are required because the source of the AgentConfigManager states in a comment they are mandatory snmpv2MIB, snmpMpdMib, targetMIB, communityMIB, notificationMIB, frameworkMIB, usmMIB, vacmMIB, tlsTmMib ) provided by the sample config file.
I than added a custom OID to query to the agend using the following code
MOServer server = new DefaultMOServer();
MOServer[] moServers = new MOServer[]{server};
...
AgentConfigManager agent = new AgentConfigManager(new OctetString(MPv3.createLocalEngineID()),
messageDispatcher,
null,
moServers,
ThreadPool.create("SampleAgent", 3),
configurationFactory,
new DefaultMOPersistenceProvider(moServers, configFile),
new EngineBootsCounterFile(bootCounterFile), null, dhKickstartParameters);
...
agent.initialize();
...
server.register(new MOScalar(new OID("1.someIODshere.0"), MOAccessImpl.ACCESS_READ_ONLY, new OctetString("object 1")), null);
Now i'm missing a way to intercept get or set requests from snmp managers / browsers to return a dynamic value or change the program behavior according to the request.
I kind of expect to be able to bind a kind of event listener to this OID to get notified on requests / changes.
How to watch an OID?
I found it. I was looking at the wrong place. I thought a had to add a listener to the agent, but instead i have to add the listener to the server.
server.addLookupListener(new MOServerLookupListener() {
#Override
public void lookupEvent(MOServerLookupEvent event) {
System.out.println();
((MOScalar)event.getLookupResult()).setValue(new OctetString(new Date().toString()));
}
#Override
public void queryEvent(MOServerLookupEvent event) {
System.out.println();
}
},moScalar);
I can now react to the requests. YAY!
I have problem with vertx HttpClient.
Here's code which shows that tests GET using vertx and plain java.
Vertx vertx = Vertx.vertx();
HttpClientOptions options = new HttpClientOptions()
.setTrustAll(true)
.setSsl(false)
.setDefaultPort(80)
.setProtocolVersion(HttpVersion.HTTP_1_1)
.setLogActivity(true);
HttpClient client = vertx.createHttpClient(options);
client.getNow("google.com", "/", response -> {
System.out.println("Received response with status code " + response.statusCode());
});
System.out.println(getHTML("http://google.com"));
Where getHTML() is from here: How do I do a HTTP GET in Java?
This is my output:
<!doctype html><html... etc <- correct output from plain java
Feb 08, 2017 11:31:21 AM io.vertx.core.http.impl.HttpClientRequestImpl
SEVERE: java.net.UnknownHostException: failed to resolve 'google.com'. Exceeded max queries per resolve 3
But vertx can't connect. What's wrong here? I'm not using any proxy.
For reference: a solution, as described in this question and in tsegismont's comment here, is to set the flag vertx.disableDnsResolver to true:
-Dvertx.disableDnsResolver=true
in order to fall back to the JVM DNS resolver as explained here:
sometimes it can be desirable to use the JVM built-in resolver, the JVM system property -Dvertx.disableDnsResolver=true activates this behavior
I observed this DNS resolution issue with a redis client in a kubernetes environment.
I had this issue, what caused it for me was stale DNS servers being picked up by the Java runtime, i.e. servers registered for a network the machine was no longer connected to. The issue is first in the Sun JNDI implementation, it also exists in Netty which uses JNDI to bootstrap its list of name servers on most platforms, then finally shows up in VertX.
I think a good place to fix this would be in the Netty layer where the set of default DNS servers is bootstrapped. I have raised a ticket with the Netty project so we'll see if they agree with me! Here is the Netty ticket
In the mean time a fairly basic workaround is to filter the default DNS servers detected by Netty, based on whether they are reachable or not. Here is a code Sample in Kotlin to apply before constructing the main VertX instance.
// The default set of name servers provided by JNDI can contain stale entries
// This default set is picked up by Netty and in turn by VertX
// To work around this, we filter for only reachable name servers on startup
val nameServers = DefaultDnsServerAddressStreamProvider.defaultAddressList()
val reachableNameServers = nameServers.stream()
.filter {ns -> ns.address.isReachable(NS_REACHABLE_TIMEOUT)}
.map {ns -> ns.address.hostAddress}
.collect(Collectors.toList())
if (reachableNameServers.size == 0)
throw StartupException("There are no reachable name servers available")
val opts = VertxOptions()
opts.addressResolverOptions.servers = reachableNameServers
// The primary Vertx instance
val vertx = Vertx.vertx(opts)
A little more detail in case it is helpful. I have a company machine, which at some point was connected to the company network by a physical cable. Details of the company's internal name servers were set up by DHCP on the physical interface. Using the wireless interface at home, DNS for the wireless interface gets set to my home DNS while the config for the physical interface is not updated. This is fine since that device is not active, ipconfig /all does not show the internal company DNS servers. However, looking in the registry they are still there:
Computer\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\Interfaces
They get picked up by the JNDI mechanism, which feeds Netty and in turn VertX. Since they are not reachable from my home location, DNS resolution fails. I can imagine this home/office situation is not unique to me! I don't know whether something similar could occur with multiple virtual interfaces on containers or VMs, it could be worth looking at if you are having problems.
Here is the sample code which works for me.
public class TemplVerticle extends HttpVerticle {
public static void main(String[] args) {
Vertx vertx = Vertx.vertx();
// Create the web client and enable SSL/TLS with a trust store
WebClient client = WebClient.create(vertx,
new WebClientOptions()
.setSsl(true)
.setTrustAll(true)
.setDefaultPort(443)
.setKeepAlive(true)
.setDefaultHost("www.w3schools.com")
);
client.get("www.w3schools.com")
.as(BodyCodec.string())
.send(ar -> {
if (ar.succeeded()) {
HttpResponse<String> response = ar.result();
System.out.println("Got HTTP response body");
System.out.println(response.body().toString());
} else {
ar.cause().printStackTrace();
}
});
}
}
Try using web client instead of httpclient, here you have an example (with rx):
private val client: WebClient = WebClient.create(vertx, WebClientOptions()
.setSsl(true)
.setTrustAll(true)
.setDefaultPort(443)
.setKeepAlive(true)
)
open fun <T> get(uri: String, marshaller: Class<T>): Single<T> {
return client.getAbs(host + uri).rxSend()
.map { extractJson(it, uri, marshaller) }
}
Another option is to use getAbs.