How to hide unessesary Selenium debug messages from console in java - java

I'm using Intellij with jdk 14 to build a seleninum test. I just want the console output as test report, result or error. I've tried many ways found on google or stackoverflow such as setting the loglevel, grep console but these messages still appear. Is there any way to resolve this issue? Here is the output:
16:44:14.551 [main] DEBUG io.netty.buffer.PooledByteBufAllocator - -Dio.netty.allocator.maxCachedByteBuffersPerChunk: 1023
16:44:14.559 [main] DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.allocator.type: pooled
16:44:14.560 [main] DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.threadLocalDirectBufferSize: 0
16:44:14.560 [main] DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.maxThreadLocalCharBufferSize: 16384
Starting ChromeDriver 83.0.4103.39 (ccbf011cb2d2b19b506d844400483861342c20cd-refs/branch-heads/4103#{#416}) on port 7431
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
16:44:15.681 [Forwarding newSession on session null to remote] DEBUG io.netty.channel.DefaultChannelId - -Dio.netty.processId: 17484 (auto-detected)
16:44:15.683 [Forwarding newSession on session null to remote] DEBUG io.netty.util.NetUtil - -Djava.net.preferIPv4Stack: false
16:44:15.683 [Forwarding newSession on session null to remote] DEBUG io.netty.util.NetUtil - -Djava.net.preferIPv6Addresses: false
16:44:15.689 [Forwarding newSession on session null to remote] DEBUG io.netty.util.NetUtil - Loopback interface: lo (Software Loopback Interface 1, 127.0.0.1)
16:44:15.689 [Forwarding newSession on session null to remote] DEBUG io.netty.util.NetUtil - Failed to get SOMAXCONN from sysctl and file \proc\sys\net\core\somaxconn. Default: 200
16:44:15.696 [Forwarding newSession on session null to remote] DEBUG io.netty.channel.DefaultChannelId - -Dio.netty.machineId: fc:aa:14:ff:fe:e6:1d:11 (auto-detected)
16:44:15.758 [AsyncHttpClient-3-1] DEBUG io.netty.buffer.AbstractByteBuf - -Dio.netty.buffer.checkAccessible: true
16:44:15.758 [AsyncHttpClient-3-1] DEBUG io.netty.buffer.AbstractByteBuf - -Dio.netty.buffer.checkBounds: true
16:44:15.759 [AsyncHttpClient-3-1] DEBUG io.netty.util.ResourceLeakDetectorFactory - Loaded default ResourceLeakDetector: io.netty.util.ResourceLeakDetector#3d005e97
16:44:15.777 [AsyncHttpClient-3-1] DEBUG org.asynchttpclient.netty.channel.NettyConnectListener - Using new Channel '[id: 0x09b55496, L:/127.0.0.1:54163 - R:localhost/127.0.0.1:7431]' for 'POST' to '/session'
16:44:15.829 [AsyncHttpClient-3-1] DEBUG io.netty.util.Recycler - -Dio.netty.recycler.maxCapacityPerThread: 4096
16:44:15.829 [AsyncHttpClient-3-1] DEBUG io.netty.util.Recycler - -Dio.netty.recycler.maxSharedCapacityFactor: 2
16:44:15.829 [AsyncHttpClient-3-1] DEBUG io.netty.util.Recycler - -Dio.netty.recycler.linkCapacity: 16
16:44:15.830 [AsyncHttpClient-3-1] DEBUG io.netty.util.Recycler - -Dio.netty.recycler.ratio: 8
16:44:20.511 [AsyncHttpClient-3-1] DEBUG org.asynchttpclient.netty.handler.HttpHandler -
Request DefaultHttpRequest(decodeResult: success, version: HTTP/1.1)
POST /session HTTP/1.1
User-Agent: selenium/4.0.0-alpha-6 (java windows)
Content-Length: 365
Content-Type: application/json; charset=utf-8
host: localhost:7431
accept: */*


What you really need to do is to set the log level on the console log handler (which is responsible for printing all log messages to the console). However doing this whilst using BasicConfigurator will be quite tricky to do, you are better off moving to having your logging configuration specified in a properties file (which is much more flexible).
I would advise working through the log4j logging configuration tutorial - this will help you put together exactly the configuration you want, and should prove a worthwhile investment of your time. However, if you want to get this done quickly, try adding the content below to the file log4j.properties (example taken from here):
# Root logger option
log4j.rootLogger=INFO, stdout
# Direct log messages to stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Target=System.out
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n
And then changing your code to do the following:
Properties props = new Properties();
props.load(new FileInputStream("/my/path/to/log4j.properties"));
PropertyConfigurator.configure(props);


For anyone with a similar request but using log4j, you could also use code to set up logging. The following removes the verbose netty debug messages:
BasicConfigurator.resetConfiguration();
Logger.getRootLogger().setLevel(Level.INFO);
String pattern = "%p: %m %n";
BasicConfigurator.configure(new ConsoleAppender(new PatternLayout(pattern)));

Related

How to limit log4j messages from a 3rd-party library

I am using apache's pdfbox in a Java Swing application; it is evidently configured to give many log4j messages while running (no evidence that log4j2 is in use). I would like to limit those messages to, say, WARNING and above so that I can see my own logging messages. I'm sure what it outputs is useful in some context, but not my current one.
07:03:51.053 [SwingWorker-pool-2-thread-1] INFO org.apache.pdfbox.rendering.PDFRenderer - Your current java version is: 1.8.0_181
07:03:51.058 [SwingWorker-pool-2-thread-1] INFO org.apache.pdfbox.rendering.PDFRenderer - To get higher rendering speed on old java 1.8 or 9 versions,
07:03:51.058 [SwingWorker-pool-2-thread-1] INFO org.apache.pdfbox.rendering.PDFRenderer - update to the latest 1.8 or 9 version (>= 1.8.0_191 or >= 9.0.4),
07:03:51.059 [SwingWorker-pool-2-thread-1] INFO org.apache.pdfbox.rendering.PDFRenderer - or
07:03:51.059 [SwingWorker-pool-2-thread-1] INFO org.apache.pdfbox.rendering.PDFRenderer - use the option -Dsun.java2d.cmm=sun.java2d.cmm.kcms.KcmsServiceProvider
07:03:51.059 [SwingWorker-pool-2-thread-1] INFO org.apache.pdfbox.rendering.PDFRenderer - or call System.setProperty("sun.java2d.cmm", "sun.java2d.cmm.kcms.KcmsServiceProvider")
07:03:52.851 [Finalizer] DEBUG org.apache.pdfbox.io.ScratchFileBuffer - ScratchFileBuffer not closed!
07:03:57.514 [Finalizer] DEBUG org.apache.pdfbox.io.ScratchFileBuffer - ScratchFileBuffer not closed!
07:04:01.340 [Finalizer] DEBUG org.apache.pdfbox.io.ScratchFileBuffer - ScratchFileBuffer not closed!
I have created the log4j.properties file:
log4j.rootLogger=INFO, theConsoleAppender
log4j.appender.theConsoleAppender=org.apache.log4j.ConsoleAppender
log4j.appender.theConsoleAppender.layout=org.apache.log4j.PatternLayout
log4j.appender.theConsoleAppender.layout.ConversionPattern=%-4r [%t] %-5p %c %x --- %m%n
log4j.logger.org.apache.pdfbox.type=warn
Hoping that the 'warn' would be used for the pdfbox messages, but that didn't work.
I added the following line to my app startup:
Logger.getRootLogger().setLevel(Level.WARN);
hoping that it would affect the loggers evidently producing the above messages, but that didn't work.
How can I, either through configuration or program call, limit the log4j messages output by the pdfbox library?

Testcontainer junit test skipped in azure build pipeline after moving to ubuntu 20

I have junit test with testcontainer in my project.
#Testcontainers(disabledWithoutDocker = true)
public class AutoHedgeDaoTest {
#Container
GenericContainer<?> mongoContainer = new GenericContainer<>("mongo:3.6.9").withExposedPorts(27017);
MongoTemplate mongoTemplate;
...
}
I use azure devops pipeline to build my project
- task: Maven#3
displayName: maven build
inputs:
mavenPomFile: 'fixByPassWeb/pom.xml'
goals: 'clean install'
options: ' -DbuildNumber=$(Build.BuildNumber)'
publishJUnitResults: true
testResultsFiles: '**/surefire-reports/TEST-*.xml'
javaHomeOption: 'JDKVersion'
jdkVersionOption: 1.11
mavenVersionOption: 'Default'
mavenAuthenticateFeed: false
effectivePomSkip: false
sonarQubeRunAnalysis: false
On November azure devops uses ubuntu 18 and all was good.
This is part of log
14:58:14.857 [main] DEBUG org.testcontainers.utility.TestcontainersConfiguration - Testcontainers configuration overrides will be loaded from file:/home/vsts/.testcontainers.properties
14:58:15.309 [ducttape-0] DEBUG org.testcontainers.dockerclient.DockerClientProviderStrategy - Pinging docker daemon...
14:58:15.332 [ducttape-0] DEBUG com.github.dockerjava.core.command.AbstrDockerCmd - Cmd:
14:58:15.433 [main] INFO org.testcontainers.dockerclient.EnvironmentAndSystemPropertyClientProviderStrategy - Found docker client settings from environment
14:58:15.434 [main] INFO org.testcontainers.dockerclient.DockerClientProviderStrategy - Found Docker environment with Environment variables, system properties and defaults. Resolved dockerHost=unix:///var/run/docker.sock
14:58:15.434 [main] DEBUG org.testcontainers.dockerclient.DockerClientProviderStrategy - Checking Docker OS type for Environment variables, system properties and defaults. Resolved dockerHost=unix:///var/run/docker.sock
14:58:15.436 [main] DEBUG com.github.dockerjava.core.command.AbstrDockerCmd - Cmd:
14:58:15.623 [main] INFO org.testcontainers.DockerClientFactory - Docker host IP address is localhost
14:58:15.624 [main] DEBUG com.github.dockerjava.core.command.AbstrDockerCmd - Cmd:
14:58:15.643 [main] DEBUG com.github.dockerjava.core.command.AbstrDockerCmd - Cmd:
14:58:15.662 [main] INFO org.testcontainers.DockerClientFactory - Connected to docker:
Server Version: 19.03.13+azure
API Version: 1.40
Operating System: Ubuntu 18.04.5 LTS
Total Memory: 6927 MB
14:58:15.662 [main] DEBUG org.testcontainers.DockerClientFactory - Ryuk is enabled
14:58:15.670 [main] DEBUG com.github.dockerjava.core.command.AbstrDockerCmd - Cmd: ListImagesCmdImpl[imageNameFilter=testcontainersofficial/ryuk:0.3.0,showAll=false,filters=com.github.dockerjava.core.util.FiltersBuilder#0]
14:58:15.726 [main] DEBUG org.testcontainers.utility.RegistryAuthLocator - Looking up auth config for image: testcontainersofficial/ryuk:0.3.0 at registry: index.docker.io
14:58:15.728 [main] DEBUG org.testcontainers.utility.RegistryAuthLocator - RegistryAuthLocator has configFile: /home/vsts/.docker/config.json (exists) and commandPathPrefix:
14:58:15.737 [main] DEBUG org.testcontainers.utility.RegistryAuthLocator - registryName [index.docker.io] for dockerImageName [testcontainersofficial/ryuk:0.3.0]
14:58:15.748 [main] DEBUG org.testcontainers.utility.RegistryAuthLocator - found existing auth config [AuthConfig{username=githubactions, password=hidden non-blank value, auth=hidden non-blank value, email=null, registryAddress=https://index.docker.io/v1/, registryToken=blank}]
14:58:15.750 [main] DEBUG org.testcontainers.utility.RegistryAuthLocator - Cached auth found: [AuthConfig{username=githubactions, password=hidden non-blank value, auth=hidden non-blank value, email=null, registryAddress=https://index.docker.io/v1/, registryToken=blank}]
14:58:15.751 [main] DEBUG org.testcontainers.dockerclient.auth.AuthDelegatingDockerClientConfig - Effective auth config [AuthConfig{username=githubactions, password=hidden non-blank value, auth=hidden non-blank value, email=null, registryAddress=https://index.docker.io/v1/, registryToken=blank}]
14:58:16.929 [tc-okhttp-stream-559050604] INFO org.testcontainers.DockerClientFactory - Starting to pull image
14:58:16.942 [tc-okhttp-stream-559050604] DEBUG com.github.dockerjava.core.command.PullImageResultCallback - ResponseItem(stream=null, status=Pulling from testcontainersofficial/ryuk, progressDetail=null, progress=null, id=0.3.0, from=null, time=null, errorDetail=null, error=null, aux=null)
14:58:16.950 [tc-okhttp-stream-559050604] INFO org.testcontainers.DockerClientFactory - Pulling image layers: 0 pending, 0 downloaded, 0 extracted, (0 bytes/0 bytes)
…………….
[INFO] Running com.AutoHedgeDaoTest
………………
14:58:32.725 [main] INFO com.rencap.fo.fixbypass.web.AutoHedgeDaoTest - Connecting to mongodb://localhost:32769
……………………..
14:58:33.592 [main] DEBUG org.testcontainers.utility.ResourceReaper - Removed container and associated volume(s): mongo:3.6.9
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.047 s - in com.AutoHedgeDaoTest
But now azure devops uses ubuntu 20. And currently test skipped.
18:16:28.143 [main] DEBUG com.github.dockerjava.core.command.AbstrDockerCmd - Cmd:
18:16:28.295 [main] INFO org.testcontainers.DockerClientFactory - Docker host IP address is localhost
18:16:28.297 [main] DEBUG com.github.dockerjava.core.command.AbstrDockerCmd - Cmd:
18:16:28.315 [main] DEBUG com.github.dockerjava.core.command.AbstrDockerCmd - Cmd:
18:16:28.333 [main] INFO org.testcontainers.DockerClientFactory - Connected to docker:
Server Version: 20.10.2+azure
API Version: 1.41
Operating System: Ubuntu 20.04.2 LTS
Total Memory: 6954 MB
18:16:28.334 [main] DEBUG org.testcontainers.DockerClientFactory - Ryuk is enabled
18:16:28.338 [main] DEBUG com.github.dockerjava.core.command.AbstrDockerCmd - Cmd: ListImagesCmdImpl[imageNameFilter=testcontainersofficial/ryuk:0.3.0,showAll=false,filters=com.github.dockerjava.core.util.FiltersBuilder#0]
18:16:28.374 [main] DEBUG org.testcontainers.utility.RegistryAuthLocator - Looking up auth config for image: testcontainersofficial/ryuk:0.3.0 at registry: index.docker.io
18:16:28.376 [main] DEBUG org.testcontainers.utility.RegistryAuthLocator - RegistryAuthLocator has configFile: /home/vsts/.docker/config.json (exists) and commandPathPrefix:
18:16:28.381 [main] DEBUG org.testcontainers.utility.RegistryAuthLocator - registryName [index.docker.io] for dockerImageName [testcontainersofficial/ryuk:0.3.0]
18:16:28.392 [main] DEBUG org.testcontainers.utility.RegistryAuthLocator - found existing auth config [AuthConfig{username=githubactions, password=hidden non-blank value, auth=hidden non-blank value, email=null, registryAddress=https://index.docker.io/v1/, registryToken=blank}]
18:16:28.392 [main] DEBUG org.testcontainers.utility.RegistryAuthLocator - Cached auth found: [AuthConfig{username=githubactions, password=hidden non-blank value, auth=hidden non-blank value, email=null, registryAddress=https://index.docker.io/v1/, registryToken=blank}]
18:16:28.392 [main] DEBUG org.testcontainers.dockerclient.auth.AuthDelegatingDockerClientConfig - Effective auth config [AuthConfig{username=githubactions, password=hidden non-blank value, auth=hidden non-blank value, email=null, registryAddress=https://index.docker.io/v1/, registryToken=blank}]
18:16:28.419 [main] DEBUG com.github.dockerjava.core.command.AbstrDockerCmd - Cmd:
com.github.dockerjava.core.command.CreateContainerCmdImpl#70925b45[name=testcontainers-ryuk-0dd3967c-ded1-45bb-9174-5710ca968be4,hostName=,domainName=,user=,attachStdin=,attachStdout=,attachStderr=,portSpecs=,tty=,stdinOpen=,stdInOnce=,env=,cmd=,healthcheck=,argsEscaped=,entrypoint=,image=testcontainersofficial/ryuk:0.3.0,volumes=com.github.dockerjava.api.model.Volumes#4263b080,workingDir=,macAddress=,onBuild=,networkDisabled=,exposedPorts=com.github.dockerjava.api.model.ExposedPorts#2af616d3,stopSignal=,stopTimeout=,hostConfig=HostConfig(binds=[//var/run/docker.sock:/var/run/docker.sock:rw], blkioWeight=null, blkioWeightDevice=null, blkioDeviceReadBps=null, blkioDeviceWriteBps=null, blkioDeviceReadIOps=null, blkioDeviceWriteIOps=null, memorySwappiness=null, nanoCPUs=null, capAdd=null, capDrop=null, containerIDFile=null, cpuPeriod=null, cpuRealtimePeriod=null, cpuRealtimeRuntime=null, cpuShares=null, cpuQuota=null, cpusetCpus=null, cpusetMems=null, devices=null, deviceCgroupRules=null, deviceRequests=null, diskQuota=null, dns=null, dnsOptions=null, dnsSearch=null, extraHosts=null, groupAdd=null, ipcMode=null, cgroup=null, links=[], logConfig=com.github.dockerjava.api.model.LogConfig#5910de75, lxcConf=null, memory=null, memorySwap=null, memoryReservation=null, kernelMemory=null, networkMode=null, oomKillDisable=null, init=null, autoRemove=true, oomScoreAdj=null, portBindings=null, privileged=false, publishAllPorts=true, readonlyRootfs=null, restartPolicy=null, ulimits=null, cpuCount=null, cpuPercent=null, ioMaximumIOps=null, ioMaximumBandwidth=null, volumesFrom=null, mounts=null, pidMode=null, isolation=null, securityOpts=null, storageOpt=null, cgroupParent=null, volumeDriver=null, shmSize=null, pidsLimit=null, runtime=null, tmpFs=null, utSMode=null, usernsMode=null, sysctls=null, consoleSize=null),labels={org.testcontainers=true},shell=,networkingConfig=,ipv4Address=,ipv6Address=,aliases=,authConfig=AuthConfig(username=githubactions, password=3d6472b9-3d49-4d17-9fc9-90d24258043b, email=null, registryAddress=https://index.docker.io/v1/, auth=Z2l0aHViYWN0aW9uczozZDY0NzJiOS0zZDQ5LTRkMTctOWZjOS05MGQyNDI1ODA0M2I=, registrytoken=null, identitytoken=null, stackOrchestrator=null)]
[INFO] Running com.AutoHedgeDaoTest
[WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.006 s - in com.AutoHedgeDaoTest

From above error log, an old version of testcontainers was in use.
Upgrading the testcontainers dependency to the latest version 1.15.2 fixed above issue.

Application throws error on Ubuntu 19 with Java 11

Could one please help me figure out the reason why my application written in Kotlinfails to run after it was built with Intellij Idea and deployed to Ubuntu 19.10 virtual machine ?
I use the following components:
Ktor 1.3.0
Netty for Ktor
Launch logs:
15:22:08.488 [main] DEBUG io.netty.util.internal.logging.InternalLoggerFactory - Using SLF4J as the default logging framework
15:22:08.530 [main] DEBUG io.netty.util.internal.PlatformDependent0 - -Dio.netty.noUnsafe: false
15:22:08.530 [main] DEBUG io.netty.util.internal.PlatformDependent0 - Java version: 11
15:22:08.568 [main] DEBUG io.netty.util.internal.PlatformDependent0 - sun.misc.Unsafe.theUnsafe: available
15:22:08.569 [main] DEBUG io.netty.util.internal.PlatformDependent0 - sun.misc.Unsafe.copyMemory: available
15:22:08.569 [main] DEBUG io.netty.util.internal.PlatformDependent0 - java.nio.Buffer.address: available
15:22:08.577 [main] DEBUG io.netty.util.internal.PlatformDependent0 - direct buffer constructor: unavailable
java.lang.UnsupportedOperationException: Reflective setAccessible(true) disabled
at io.netty.util.internal.ReflectionUtil.trySetAccessible(ReflectionUtil.java:31)
at io.netty.util.internal.PlatformDependent0$4.run(PlatformDependent0.java:225)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at io.netty.util.internal.PlatformDependent0.<clinit>(PlatformDependent0.java:219)
at io.netty.util.internal.PlatformDependent.isAndroid(PlatformDependent.java:273)
at io.netty.util.internal.PlatformDependent.<clinit>(PlatformDependent.java:92)
at io.netty.channel.epoll.Native.loadNativeLibrary(Native.java:225)
at io.netty.channel.epoll.Native.<clinit>(Native.java:57)
at io.netty.channel.epoll.Epoll.<clinit>(Epoll.java:39)
at io.ktor.server.netty.EventLoopGroupProxy$Companion.create(NettyApplicationEngine.kt:189)
at io.ktor.server.netty.NettyApplicationEngine.<init>(NettyApplicationEngine.kt:74)
at io.ktor.server.netty.Netty.create(Embedded.kt:14)
at io.ktor.server.netty.Netty.create(Embedded.kt:12)
at io.ktor.server.engine.EmbeddedServerKt.embeddedServer(EmbeddedServer.kt:79)
at io.ktor.server.engine.EmbeddedServerKt.embeddedServer$default(EmbeddedServer.kt:77)
at app.common.ApplicationKt.main(Application.kt:96)
15:22:08.581 [main] DEBUG io.netty.util.internal.PlatformDependent0 - java.nio.Bits.unaligned: available, true
15:22:08.582 [main] DEBUG io.netty.util.internal.PlatformDependent0 - jdk.internal.misc.Unsafe.allocateUninitializedArray(int): unavailable
java.lang.IllegalAccessException: class io.netty.util.internal.PlatformDependent0$6 cannot access class jdk.internal.misc.Unsafe (in module java.base) because module java.base does not export jdk.internal.misc to unnamed module #6d3af739
at java.base/jdk.internal.reflect.Reflection.newIllegalAccessException(Reflection.java:361)
at java.base/java.lang.reflect.AccessibleObject.checkAccess(AccessibleObject.java:591)
at java.base/java.lang.reflect.Method.invoke(Method.java:558)
at io.netty.util.internal.PlatformDependent0$6.run(PlatformDependent0.java:335)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at io.netty.util.internal.PlatformDependent0.<clinit>(PlatformDependent0.java:326)
at io.netty.util.internal.PlatformDependent.isAndroid(PlatformDependent.java:273)
at io.netty.util.internal.PlatformDependent.<clinit>(PlatformDependent.java:92)
at io.netty.channel.epoll.Native.loadNativeLibrary(Native.java:225)
at io.netty.channel.epoll.Native.<clinit>(Native.java:57)
at io.netty.channel.epoll.Epoll.<clinit>(Epoll.java:39)
at io.ktor.server.netty.EventLoopGroupProxy$Companion.create(NettyApplicationEngine.kt:189)
at io.ktor.server.netty.NettyApplicationEngine.<init>(NettyApplicationEngine.kt:74)
at io.ktor.server.netty.Netty.create(Embedded.kt:14)
at io.ktor.server.netty.Netty.create(Embedded.kt:12)
at io.ktor.server.engine.EmbeddedServerKt.embeddedServer(EmbeddedServer.kt:79)
at io.ktor.server.engine.EmbeddedServerKt.embeddedServer$default(EmbeddedServer.kt:77)
at app.common.ApplicationKt.main(Application.kt:96)
15:22:08.586 [main] DEBUG io.netty.util.internal.PlatformDependent0 - java.nio.DirectByteBuffer.<init>(long, int): unavailable
15:22:08.587 [main] DEBUG io.netty.util.internal.PlatformDependent - sun.misc.Unsafe: available
15:22:08.710 [main] DEBUG io.netty.util.internal.PlatformDependent - maxDirectMemory: 249364480 bytes (maybe)
15:22:08.711 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.tmpdir: /tmp (java.io.tmpdir)
15:22:08.711 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.bitMode: 64 (sun.arch.data.model)
15:22:08.713 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.maxDirectMemory: -1 bytes
15:22:08.713 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.uninitializedArrayAllocationThreshold: -1
15:22:08.714 [main] DEBUG io.netty.util.internal.CleanerJava9 - java.nio.ByteBuffer.cleaner(): available
15:22:08.714 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.noPreferDirect: false
15:22:08.716 [main] DEBUG io.netty.util.internal.NativeLibraryLoader - -Dio.netty.native.workdir: /tmp (io.netty.tmpdir)
15:22:08.717 [main] DEBUG io.netty.util.internal.NativeLibraryLoader - -Dio.netty.native.deleteLibAfterLoading: true
15:22:08.717 [main] DEBUG io.netty.util.internal.NativeLibraryLoader - -Dio.netty.native.tryPatchShadedId: true
15:22:08.718 [main] DEBUG io.netty.util.internal.NativeLibraryLoader - Unable to load the library 'netty_transport_native_epoll_x86_64', trying other loading mechanism.

Error parsing JSON, POST containing \r\n at the begining of JSON

I have created an API to create notes in my app with a contract and its client. The client is basically a Factory to manage Api instances and a helper to inject configurations and some interceptors for metrics and Basic Auth.
I created an easy and small java application to test the client.
Everything looks find, the client creates the instance correctly and use the api objects to create the request (it is a POST) and sent it. The server receives the request and process authentication correctly but then it throws this error:
org.grails.web.json.JSONException: Unterminated string at character
311 of \r\n{"noteUser":"user.user","subject":"Flight ticket issued
onlydescription this booking has the flight issued only, please check
if any ancillery needs to be booked","description":"GONZALO TEST:
ok?this booking has the flight issued only , please check if any
ancillery needs to be booked","bookingId":"433301392
Debuging I have found these:
It seems like non \r\n is added in client - log:
2018-09-04 09:48:52 DEBUG RestDefaultHttpClient:643 - Attempt 1 to execute request
2018-09-04 09:48:52 DEBUG DefaultClientConnection:264 - Sending request: POST /uno/v3/notes/addNote HTTP/1.1
2018-09-04 09:48:52 DEBUG wire:63 - >> "POST /uno/v3/notes/addNote HTTP/1.1[\r][\n]"
2018-09-04 09:48:52 DEBUG wire:63 - >> "Accept: application/json;charset="UTF-8"[\r][\n]"
2018-09-04 09:48:52 DEBUG wire:63 - >> "Accept-Encoding: gzip, deflate[\r][\n]"
2018-09-04 09:48:52 DEBUG wire:63 - >> "Content-Length: 311[\r][\n]"
2018-09-04 09:48:52 DEBUG wire:63 - >> "Content-Type: application/json;charset="UTF-8"[\r][\n]"
2018-09-04 09:48:52 DEBUG wire:63 - >> "Host: localhost:8080[\r][\n]"
2018-09-04 09:48:52 DEBUG wire:63 - >> "Connection: Keep-Alive[\r][\n]"
2018-09-04 09:48:52 DEBUG wire:63 - >> "User-Agent: Apache-HttpClient/4.1.2 (java 1.5)[\r][\n]"
2018-09-04 09:48:52 DEBUG wire:63 - >> "Authorization: Basic bW9yZ2FuLmZyZWVtYW46UGFuY2V0YTE=[\r][\n]"
2018-09-04 09:48:52 DEBUG wire:63 - >> "[\r][\n]"
2018-09-04 09:48:52 DEBUG wire:63 - >> "[\r][\n]"
2018-09-04 09:48:52 DEBUG headers:268 - >> POST /uno/v3/notes/addNote HTTP/1.1
2018-09-04 09:48:52 DEBUG headers:271 - >> Accept: application/json;charset="UTF-8"
2018-09-04 09:48:52 DEBUG headers:271 - >> Accept-Encoding: gzip, deflate
2018-09-04 09:48:52 DEBUG headers:271 - >> Content-Length: 311
2018-09-04 09:48:52 DEBUG headers:271 - >> Content-Type: application/json;charset="UTF-8"
2018-09-04 09:48:52 DEBUG headers:271 - >> Host: localhost:8080
2018-09-04 09:48:52 DEBUG headers:271 - >> Connection: Keep-Alive
2018-09-04 09:48:52 DEBUG headers:271 - >> User-Agent: Apache-HttpClient/4.1.2 (java 1.5)
2018-09-04 09:48:52 DEBUG headers:271 - >> Authorization: Basic bW9yZ2FuLmZyZWVrtetUGFuY2V0YTE=
2018-09-04 09:48:52 DEBUG wire:77 - >> "{"noteUser":"user.user","subject":"Flight ticket issued onlydescription this booking has the flight issued only, please check if any ancillery needs to be booked","description":"GON TEST: ok?this booking has the flight issued only , please check if any ancillery needs to be booked","bookingId":"113301392"}"
2018-09-04 09:49:53 DEBUG DefaultClientConnection:165 - Connection closed
And these is the request.JSON in the server:
\r\n{"noteUser":"user.user","subject":"Flight ticket issued onlydescription this booking has the flight issued only, please check if any ancillery needs to be booked","description":"GONZALO TEST: ok?this booking has the flight issued only , please check if any ancillery needs to be booked","bookingId":"113301392
As you can see in the server, the request has a \r\n at the begining of the JSON and it is missing "} at the end of it. I thing this is because the Content-Length is set to 311 and due to somehow 2 characters are added at the begining of the JSON (\r\n), 2 char are missing at the end.
I have debug until the request is sent to the server an I cannot find the \r\n added:
I am using grails 3.3.2 and linux in the server (with linux line separator LF)
Client is done in java 1.8 and windows (with windows line separator CRLF)
jackson it is used to handle JSON.
Basic Auth is manage by spring.
I have squeezed my brain and I have run out of ideas. Any help will be really appreciated.
Thanks in advance.

delayed socket communication when sending mail with java

When sending a test email on my Windows 7 system there is a very long delay of 20 seconds in total. I already tried different smtp server implementations (jsmtpd, subethasmtp) but both show the same delay. Running the same test on a linux system has no delay. Is this a windows localhost networking issue?
My sample code:
import javax.mail.Message;
import javax.mail.internet.InternetAddress;
import javax.mail.internet.MimeMessage;
import org.apache.logging.log4j.Level;
import org.apache.logging.log4j.core.config.DefaultConfiguration;
import org.springframework.mail.javamail.JavaMailSenderImpl;
import org.springframework.mail.javamail.MimeMessagePreparator;
import org.subethamail.wiser.Wiser;
public class C_MailerTest {
public static void main( String[] args ) {
System.setProperty( DefaultConfiguration.DEFAULT_LEVEL, Level.DEBUG.name() );
Wiser wiser = new Wiser();
wiser.start();
MimeMessagePreparator preparator = new MimeMessagePreparator() {
#Override
public void prepare(MimeMessage mimeMessage) throws Exception {
mimeMessage.setRecipient( Message.RecipientType.TO, new InternetAddress( "mail#yourcompany.com" ) );
mimeMessage.setFrom( new InternetAddress( "mail#mycompany.com" ) );
mimeMessage.setText( "test" );
}
};
JavaMailSenderImpl sender = new JavaMailSenderImpl();
sender.setHost( "127.0.0.1" );
System.out.println( "sending mail" );
sender.send(preparator);
System.out.println( "mails received: " + wiser.getMessages().size() );
wiser.stop();
}
}
And here are the logs (windows 7 enterprise, service pack 1):
09:36:04.340 [main] DEBUG org.subethamail.smtp.server.CommandHandler - Added command: AUTH
09:36:04.341 [main] DEBUG org.subethamail.smtp.server.CommandHandler - Added command: DATA
09:36:04.341 [main] DEBUG org.subethamail.smtp.server.CommandHandler - Added command: EHLO
09:36:04.341 [main] DEBUG org.subethamail.smtp.server.CommandHandler - Added command: HELO
09:36:04.341 [main] DEBUG org.subethamail.smtp.server.CommandHandler - Added command: HELP
09:36:04.341 [main] DEBUG org.subethamail.smtp.server.CommandHandler - Added command: MAIL
09:36:04.341 [main] DEBUG org.subethamail.smtp.server.CommandHandler - Added command: NOOP
09:36:04.341 [main] DEBUG org.subethamail.smtp.server.CommandHandler - Added command: QUIT
09:36:04.341 [main] DEBUG org.subethamail.smtp.server.CommandHandler - Added command: RCPT
09:36:04.341 [main] DEBUG org.subethamail.smtp.server.CommandHandler - Added command: RSET
09:36:04.341 [main] DEBUG org.subethamail.smtp.server.CommandHandler - Added command: STARTTLS
09:36:04.341 [main] DEBUG org.subethamail.smtp.server.CommandHandler - Added command: VRFY
sending mail
09:36:04.444 [org.subethamail.smtp.server.ConnectionHandler] DEBUG org.subethamail.smtp.server.ConnectionHandler - SMTP connection count: 1
09:36:04.451 [org.subethamail.smtp.server.ConnectionHandler] DEBUG org.subethamail.smtp.server.ConnectionHandler - Server: 220 SWE-JPW.intern.src ESMTP SubEthaSMTP
09:36:04.455 [org.subethamail.smtp.server.ConnectionHandler] DEBUG org.subethamail.smtp.server.ConnectionHandler - Client: EHLO SWE-JPW.intern.src
09:36:04.461 [org.subethamail.smtp.server.ConnectionHandler] DEBUG org.subethamail.smtp.server.ConnectionHandler - Server: 250-SWE-JPW.intern.src
250-8BITMIME
250-STARTTLS
250-AUTH PLAIN LOGIN
250 Ok
09:36:04.481 [org.subethamail.smtp.server.ConnectionHandler] DEBUG org.subethamail.smtp.server.ConnectionHandler - Client: MAIL FROM:<mail#mycompany.com>
09:36:04.481 [org.subethamail.smtp.server.ConnectionHandler] DEBUG org.subethamail.smtp.server.ConnectionHandler - Server: 250 Ok
09:36:04.484 [org.subethamail.smtp.server.ConnectionHandler] DEBUG org.subethamail.smtp.server.ConnectionHandler - Client: RCPT TO:<mail#yourcompany.com>
09:36:04.484 [org.subethamail.smtp.server.ConnectionHandler] DEBUG org.subethamail.smtp.server.ConnectionHandler - Server: 250 Ok
09:36:09.495 [org.subethamail.smtp.server.ConnectionHandler] DEBUG org.subethamail.smtp.server.ConnectionHandler - Client: NOOP
09:36:09.495 [org.subethamail.smtp.server.ConnectionHandler] DEBUG org.subethamail.smtp.server.ConnectionHandler - Server: 250 Ok
09:36:14.495 [org.subethamail.smtp.server.ConnectionHandler] DEBUG org.subethamail.smtp.server.ConnectionHandler - Client: NOOP
09:36:14.495 [org.subethamail.smtp.server.ConnectionHandler] DEBUG org.subethamail.smtp.server.ConnectionHandler - Server: 250 Ok
09:36:19.495 [org.subethamail.smtp.server.ConnectionHandler] DEBUG org.subethamail.smtp.server.ConnectionHandler - Client: NOOP
09:36:19.495 [org.subethamail.smtp.server.ConnectionHandler] DEBUG org.subethamail.smtp.server.ConnectionHandler - Server: 250 Ok
09:36:24.049 [org.subethamail.smtp.server.ConnectionHandler] DEBUG org.subethamail.smtp.server.ConnectionHandler - Client: DATA
09:36:24.049 [org.subethamail.smtp.server.ConnectionHandler] DEBUG org.subethamail.smtp.server.ConnectionHandler - Server: 354 End data with <CR><LF>.<CR><LF>
09:36:24.051 [org.subethamail.smtp.server.ConnectionHandler] DEBUG org.subethamail.smtp.server.ConnectionHandler - Server: 250 Ok
09:36:24.055 [org.subethamail.smtp.server.ConnectionHandler] DEBUG org.subethamail.smtp.server.ConnectionHandler - Client: QUIT
09:36:24.055 [org.subethamail.smtp.server.ConnectionHandler] DEBUG org.subethamail.smtp.server.ConnectionHandler - Server: 221 Bye
mails received: 1
09:36:24.058 [org.subethamail.smtp.server.SMTPServer] INFO org.subethamail.smtp.server.SMTPServer - SMTP Server socket shut down.
There are a lot of NOOP log entries delayed excactly 5 seconds.
The logs running the same test on a linux machine (ubuntu):
09:53:55.312 [main] DEBUG org.subethamail.smtp.server.CommandHandler - Added command: AUTH
09:53:55.313 [main] DEBUG org.subethamail.smtp.server.CommandHandler - Added command: DATA
09:53:55.313 [main] DEBUG org.subethamail.smtp.server.CommandHandler - Added command: EHLO
09:53:55.313 [main] DEBUG org.subethamail.smtp.server.CommandHandler - Added command: HELO
09:53:55.313 [main] DEBUG org.subethamail.smtp.server.CommandHandler - Added command: HELP
09:53:55.314 [main] DEBUG org.subethamail.smtp.server.CommandHandler - Added command: MAIL
09:53:55.314 [main] DEBUG org.subethamail.smtp.server.CommandHandler - Added command: NOOP
09:53:55.314 [main] DEBUG org.subethamail.smtp.server.CommandHandler - Added command: QUIT
09:53:55.314 [main] DEBUG org.subethamail.smtp.server.CommandHandler - Added command: RCPT
09:53:55.314 [main] DEBUG org.subethamail.smtp.server.CommandHandler - Added command: RSET
09:53:55.314 [main] DEBUG org.subethamail.smtp.server.CommandHandler - Added command: STARTTLS
09:53:55.314 [main] DEBUG org.subethamail.smtp.server.CommandHandler - Added command: VRFY
sending mail
09:53:55.395 [org.subethamail.smtp.server.ConnectionHandler] DEBUG org.subethamail.smtp.server.ConnectionHandler - SMTP connection count: 1
09:53:55.397 [org.subethamail.smtp.server.ConnectionHandler] DEBUG org.subethamail.smtp.server.ConnectionHandler - Server: 220 vm-gkls ESMTP SubEthaSMTP
09:53:55.399 [org.subethamail.smtp.server.ConnectionHandler] DEBUG org.subethamail.smtp.server.ConnectionHandler - Client: EHLO vm-gkls
09:53:55.402 [org.subethamail.smtp.server.ConnectionHandler] DEBUG org.subethamail.smtp.server.ConnectionHandler - Server: 250-vm-gkls
250-8BITMIME
250-STARTTLS
250-AUTH PLAIN LOGIN
250 Ok
09:53:55.422 [org.subethamail.smtp.server.ConnectionHandler] DEBUG org.subethamail.smtp.server.ConnectionHandler - Client: MAIL FROM:<mail#mycompany.com>
09:53:55.422 [org.subethamail.smtp.server.ConnectionHandler] DEBUG org.subethamail.smtp.server.ConnectionHandler - Server: 250 Ok
09:53:55.423 [org.subethamail.smtp.server.ConnectionHandler] DEBUG org.subethamail.smtp.server.ConnectionHandler - Client: RCPT TO:<mail#yourcompany.com>
09:53:55.423 [org.subethamail.smtp.server.ConnectionHandler] DEBUG org.subethamail.smtp.server.ConnectionHandler - Server: 250 Ok
09:53:55.423 [org.subethamail.smtp.server.ConnectionHandler] DEBUG org.subethamail.smtp.server.ConnectionHandler - Client: DATA
09:53:55.423 [org.subethamail.smtp.server.ConnectionHandler] DEBUG org.subethamail.smtp.server.ConnectionHandler - Server: 354 End data with <CR><LF>.<CR><LF>
09:53:55.426 [org.subethamail.smtp.server.ConnectionHandler] DEBUG org.subethamail.smtp.server.ConnectionHandler - Server: 250 Ok
09:53:55.427 [org.subethamail.smtp.server.ConnectionHandler] DEBUG org.subethamail.smtp.server.ConnectionHandler - Client: QUIT
09:53:55.427 [org.subethamail.smtp.server.ConnectionHandler] DEBUG org.subethamail.smtp.server.ConnectionHandler - Server: 221 Bye
mails received: 1
09:53:55.429 [org.subethamail.smtp.server.SMTPServer] INFO org.subethamail.smtp.server.SMTPServer - SMTP Server socket shut down.
On a linux system there is absolutely no delay. The program sends and receives the mail immediately.
Environment:
java 8 (1.8.0-91)
spring 4.2.6
javax.mail 1.5.5
subethasmtp-wiser 1.2

Categories

Resources