I have a long list of JVM parameter values:
-XX:+UseSerialGC -XX:+ResizePLAB -XX:-ResizeOldPLAB -XX:-AlwaysPreTouch -XX:-ParallelRefProcEnabled -XX:+ParallelRefProcBalancingEnabled -XX:+UseTLAB -XX:-ResizeTLAB -XX:-ZeroTLAB -XX:-FastTLABRefill -XX:+NeverActAsServerClassMachine -XX:-AlwaysActAsServerClassMachine -XX:+UseAutoGCSelectPolicy -XX:+UseAdaptiveSizePolicy -XX:+UsePSAdaptiveSurvivorSizePolicy -XX:-UseAdaptiveGenerationSizePolicyAtMinorCollection -XX:+UseAdaptiveGenerationSizePolicyAtMajorCollection -XX:+UseAdaptiveSizePolicyWithSystemGC -XX:+UseAdaptiveGCBoundary -XX:+UseAdaptiveSizePolicyFootprintGoal -XX:-UseAdaptiveSizeDecayMajorGCCost -XX:+UseGCOverheadLimit -XX:+DisableExplicitGC -XX:-CollectGen0First -XX:+BindGCTaskThreadsToCPUs -XX:+UseGCTaskAffinity -XX:YoungPLABSize=3397 -XX:OldPLABSize=1123 -XX:GCTaskTimeStampEntries=240 -XX:TargetPLABWastePct=6 -XX:PLABWeight=75 -XX:OldPLABWeight=46 -XX:MarkStackSize=4617021 -XX:MarkStackSizeMax=713160576 -XX:RefDiscoveryPolicy=0 -XX:InitiatingHeapOccupancyPercent=48 -XX:MaxRAM=139765086242 -XX:ErgoHeapSizeLimit=0 -XX:MaxRAMFraction=4 -XX:DefaultMaxRAMFraction=4 -XX:MinRAMFraction=2 -XX:InitialRAMFraction=61 -XX:AutoGCSelectPauseMillis=5557 -XX:AdaptiveSizeThroughPutPolicy=0 -XX:AdaptiveSizePausePolicy=0 -XX:AdaptiveSizePolicyInitializingSteps=28 -XX:AdaptiveSizePolicyOutputInterval=0 -XX:AdaptiveSizePolicyWeight=12 -XX:AdaptiveTimeWeight=19 -XX:PausePadding=0 -XX:PromotedPadding=3 -XX:SurvivorPadding=3 -XX:ThresholdTolerance=10 -XX:AdaptiveSizePolicyCollectionCostMargin=49 -XX:YoungGenerationSizeIncrement=16 -XX:YoungGenerationSizeSupplement=104 -XX:YoungGenerationSizeSupplementDecay=9 -XX:TenuredGenerationSizeIncrement=22 -XX:TenuredGenerationSizeSupplement=117 -XX:TenuredGenerationSizeSupplementDecay=2 -XX:MaxGCPauseMillis=13557897735059052544 -XX:GCPauseIntervalMillis=0 -XX:MaxGCMinorPauseMillis=16119267456708329472 -XX:GCTimeRatio=73 -XX:AdaptiveSizeDecrementScaleFactor=4 -XX:AdaptiveSizeMajorGCDecayTimeScale=11 -XX:MinSurvivorRatio=1 -XX:InitialSurvivorRatio=6 -XX:BaseFootPrintEstimate=272901592 -XX:GCHeapFreeLimit=2 -XX:PrefetchCopyIntervalInBytes=654 -XX:PrefetchScanIntervalInBytes=748 -XX:PrefetchFieldsAhead=1 -XX:ProcessDistributionStride=3
that I need to change before running a program. The values for these parameters will be dynamically determined, thus I need to be able to change these values repeatedly. The program is executed inside a docker container and is a REST endpoint developed using springboot.
Is there some kind of configuration file where I can dynamically set these parameters or how can this be done?
Is there some kind of configuration file where I can dynamically set these parameters
No.
or how can this be done?
You just write some code to do it. Possibly a shell script or a Windows batch file. Possibly in in some other scripting language. Possibly even in Java.
For example, this uses a helper Java program to do the "dynamic" stuff and generate some JVM options. These are then supplied when launching the JVM for the real application.
#!/bin/sh
OPTS=`java com.acme.GenerateJVMOptions some parameters`
java $OPTS com.acme.TheRealApplication some more parameters
Related
I have a long list of JVM parameter values:
-XX:+UseSerialGC -XX:+ResizePLAB -XX:-ResizeOldPLAB -XX:-AlwaysPreTouch -XX:-ParallelRefProcEnabled -XX:+ParallelRefProcBalancingEnabled -XX:+UseTLAB -XX:-ResizeTLAB -XX:-ZeroTLAB -XX:-FastTLABRefill -XX:+NeverActAsServerClassMachine -XX:-AlwaysActAsServerClassMachine -XX:+UseAutoGCSelectPolicy -XX:+UseAdaptiveSizePolicy -XX:+UsePSAdaptiveSurvivorSizePolicy -XX:-UseAdaptiveGenerationSizePolicyAtMinorCollection -XX:+UseAdaptiveGenerationSizePolicyAtMajorCollection -XX:+UseAdaptiveSizePolicyWithSystemGC -XX:+UseAdaptiveGCBoundary -XX:+UseAdaptiveSizePolicyFootprintGoal -XX:-UseAdaptiveSizeDecayMajorGCCost -XX:+UseGCOverheadLimit -XX:+DisableExplicitGC -XX:-CollectGen0First -XX:+BindGCTaskThreadsToCPUs -XX:+UseGCTaskAffinity -XX:YoungPLABSize=3397 -XX:OldPLABSize=1123 -XX:GCTaskTimeStampEntries=240 -XX:TargetPLABWastePct=6 -XX:PLABWeight=75 -XX:OldPLABWeight=46 -XX:MarkStackSize=4617021 -XX:MarkStackSizeMax=713160576 -XX:RefDiscoveryPolicy=0 -XX:InitiatingHeapOccupancyPercent=48 -XX:MaxRAM=139765086242 -XX:ErgoHeapSizeLimit=0 -XX:MaxRAMFraction=4 -XX:DefaultMaxRAMFraction=4 -XX:MinRAMFraction=2 -XX:InitialRAMFraction=61 -XX:AutoGCSelectPauseMillis=5557 -XX:AdaptiveSizeThroughPutPolicy=0 -XX:AdaptiveSizePausePolicy=0 -XX:AdaptiveSizePolicyInitializingSteps=28 -XX:AdaptiveSizePolicyOutputInterval=0 -XX:AdaptiveSizePolicyWeight=12 -XX:AdaptiveTimeWeight=19 -XX:PausePadding=0 -XX:PromotedPadding=3 -XX:SurvivorPadding=3 -XX:ThresholdTolerance=10 -XX:AdaptiveSizePolicyCollectionCostMargin=49 -XX:YoungGenerationSizeIncrement=16 -XX:YoungGenerationSizeSupplement=104 -XX:YoungGenerationSizeSupplementDecay=9 -XX:TenuredGenerationSizeIncrement=22 -XX:TenuredGenerationSizeSupplement=117 -XX:TenuredGenerationSizeSupplementDecay=2 -XX:MaxGCPauseMillis=13557897735059052544 -XX:GCPauseIntervalMillis=0 -XX:MaxGCMinorPauseMillis=16119267456708329472 -XX:GCTimeRatio=73 -XX:AdaptiveSizeDecrementScaleFactor=4 -XX:AdaptiveSizeMajorGCDecayTimeScale=11 -XX:MinSurvivorRatio=1 -XX:InitialSurvivorRatio=6 -XX:BaseFootPrintEstimate=272901592 -XX:GCHeapFreeLimit=2 -XX:PrefetchCopyIntervalInBytes=654 -XX:PrefetchScanIntervalInBytes=748 -XX:PrefetchFieldsAhead=1 -XX:ProcessDistributionStride=3
that I need to change before running a program. The values for these parameters will be dynamically determined, thus I need to be able to change these values repeatedly. The program is executed inside a docker container and is a REST endpoint developed using springboot.
Is there some kind of configuration file where I can dynamically set these parameters or how can this be done?
Is there some kind of configuration file where I can dynamically set these parameters
No.
or how can this be done?
You just write some code to do it. Possibly a shell script or a Windows batch file. Possibly in in some other scripting language. Possibly even in Java.
For example, this uses a helper Java program to do the "dynamic" stuff and generate some JVM options. These are then supplied when launching the JVM for the real application.
#!/bin/sh
OPTS=`java com.acme.GenerateJVMOptions some parameters`
java $OPTS com.acme.TheRealApplication some more parameters
I am trying to remotely invoke an MBean via a commandline. Right now, I am able to list attributes and operations. For example, I can list all the attributes and operations for HotspotDiagnostic using this command:
java -jar cmdline-jmxclient-0.10.3.jar admin:P#sSw0rd 10.11.12.13:1111 com.sun.management:type=HotSpotDiagnostic
Which gives me this list of Attributes and Operations
Attributes:
DiagnosticOptions: DiagnosticOptions (type=[Ljavax.management.openmbean.CompositeData;)
ObjectName: ObjectName (type=javax.management.ObjectName)
Operations:
dumpHeap: dumpHeap
Parameters 2, return type=void
name=p0 type=java.lang.String p0
name=p1 type=boolean p1
getVMOption: getVMOption
Parameters 1, return type=javax.management.openmbean.CompositeData
name=p0 type=java.lang.String p0
setVMOption: setVMOption
Parameters 2, return type=void
name=p0 type=java.lang.String p0
name=p1 type=java.lang.String p1
But now lets say I want to invoke the dumpHeap operation which takes two parameters p0 and p1 of type string and boolean, respectively. How would I pass those arguments in?
I've tried these:
java -jar cmdline-jmxclient-0.10.3.jar admin:P#sSw0rd10.11.12.13:1111 com.sun.management:type=HotSpotDiagnostic dumpHeap p0=aaa p1=true
java -jar cmdline-jmxclient-0.10.3.jar admin:P#sSw0rd10.11.12.13:1111 com.sun.management:type=HotSpotDiagnostic dumpHeap aaa true
But I'm not sure what the syntax is, or even what I'm supposed to pass for the string parameter. This isn't for anything specific btw. Merely want to learn and understand more about how to leverage these operations from the command line. Any docs and assistance much appreciated.
EDIT: I'm naive. Oracle docs indicate the string param is an output file per this link. But still uncertain about how to pass the parameters into my command.
According to the cmdline-jmxclient documentation:
http://crawler.archive.org/cmdline-jmxclient/ you have to use comma-delimited parameters to pass to your operation.
So in your case if would be:
java -jar cmdline-jmxclient-0.10.3.jar admin:P#sSw0rd10.11.12.13:1111 com.sun.management:type=HotSpotDiagnostic dumpHeap test,true
Take note that there is an present bug in the cmdline jar file that doesn't take into account Java primitives(int, booelean, byte, etc.) and will throw a ClassNotFoundException because it can't find by the primitive name.
If you find yourself coming across this issue you can either apply the patch to the jar code that is documented here: https://webarchive.jira.com/browse/HER-1630. Or simply change the type field in the jmx endpoint code from it's primitive type to it's Wrapper object type (int -> Integer)
I am using com.cloudera.crunch version: '0.3.0-3-cdh-5.2.1'.
I have a small program that reads some AVROs and filters out invalid data based on some criteria. I am using pipeline.write(PCollection, AvroFileTarget) to write the invalid data output. It works fine in production run.
For unit testing this piece of code, I use MemPipeline instance.
But, it fails while writing the output in that case.
I get error:
java.lang.UnsatisfiedLinkError: org.apache.hadoop.util.NativeCrc32.nativeComputeChunkedSumsByteArray(II[BI[BIILjava/lang/String;JZ)V
at org.apache.hadoop.util.NativeCrc32.nativeComputeChunkedSumsByteArray(Native Method)
at org.apache.hadoop.util.NativeCrc32.calculateChunkedSumsByteArray(NativeCrc32.java:86)
at org.apache.hadoop.util.DataChecksum.calculateChunkedSums(DataChecksum.java:428)
at org.apache.hadoop.fs.FSOutputSummer.writeChecksumChunks(FSOutputSummer.java:197)
at org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:163)
at org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:144)
at org.apache.hadoop.fs.FSOutputSummer.write(FSOutputSummer.java:78)
at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:50)
at java.io.DataOutputStream.writeBytes(DataOutputStream.java:276)
at com.cloudera.crunch.impl.mem.MemPipeline.write(MemPipeline.java:159)
Any idea what's wrong?
Hadoop environment variable should be configured properly along with hadoop.dll and winutils.exe.
Also pass the JVM argument while executing MR job/application
-Djava.library.path=HADOOP_HOME/lib/native
I have some classes that implement interfaces, some of which have methods whose parameters are by definition unused in the particular class implementation. e.g. A "Shape" interface may define a "contains(point)" method, but my particular class defines a line, which cannot contain anything since it's 1-dimensional, so it always returns false and never uses point.
However, when I compile with GCJ, I'm assaulted with hundreds of "warning: parameter x is unused" messages.
I tried using the -Wno-all flag to disable warnings, as well as the others documented in gcj's manpage, but these have no effect. How do I instruct GCJ to not bother me with these trivial warnings?
I've managed to disable all warnings affecting my source code using:
gcj -Wno-all -Wno-unchecked -Wno-raw *.java
You may want to add more -Wno-... flags to disable more warnings. To figure out the possible flags, I examined the body of the methods org.eclipse.jdt.internal.compiler.batch.Main.handleWarningToken and org.eclipse.jdt.internal.compiler.batch.Main.handleErrorOrWarningToken in the Eclipse Batch Compiler ecjsrc-3.5.2.zip and ecjsrc-3.8.zip.
Specify all these flags to get all warnings disabled:
-Wno-all
-Wno-allDeadCode
-Wno-allDeprecation
-Wno-allJavadoc
-Wno-allOver-ann
-Wno-all-static-method
-Wno-assertIdentifier
-Wno-boxing
-Wno-charConcat
-Wno-compareIdentical
-Wno-conditionAssign
-Wno-constructorName
-Wno-deadCode
-Wno-dep-ann
-Wno-deprecation
-Wno-discouraged
-Wno-emptyBlock
-Wno-enumIdentifier
-Wno-enumSwitch
-Wno-enumSwitchPedantic
-Wno-fallthrough
-Wno-fieldHiding
-Wno-finalBound
-Wno-finally
-Wno-forbidden
-Wno-hashCode
-Wno-hiding
-Wno-includeAssertNull
-Wno-incomplete-switch
-Wno-indirectStatic
-Wno-interfaceNonInherited
-Wno-intfAnnotation
-Wno-intfNonInherited
-Wno-intfRedundant
-Wno-javadoc
-Wno-localHiding
-Wno-maskedCatchBlock
-Wno-maskedCatchBlocks
-Wno-nls
-Wno-noEffectAssign
-Wno-noImplicitStringConversion
-Wno-null
-Wno-nullDereference
-Wno-over-ann
-Wno-over-sync
-Wno-packageDefaultMethod
-Wno-paramAssign
-Wno-pkgDefaultMethod
-Wno-raw
-Wno-redundantSuperinterface
-Wno-resource
-Wno-semicolon
-Wno-serial
-Wno-specialParamHiding
-Wno-static-access
-Wno-static-method
-Wno-staticReceiver
-Wno-super
-Wno-suppress
-Wno-switchDefault
-Wno-syncOverride
-Wno-synthetic-access
-Wno-syntheticAccess
-Wno-typeHiding
-Wno-unavoidableGenericProblems
-Wno-unchecked
-Wno-unnecessaryElse
-Wno-unqualifiedField
-Wno-unqualified-field-access
-Wno-unsafe
-Wno-unused
-Wno-unusedAllocation
-Wno-unusedArgument
-Wno-unusedArguments
-Wno-unusedImport
-Wno-unusedImports
-Wno-unusedLabel
-Wno-unusedLocal
-Wno-unusedLocals
-Wno-unusedPrivate
-Wno-unusedThrown
-Wno-unusedTypeArgs
-Wno-uselessTypeCheck
-Wno-varargsCast
-Wno-warningToken
Although I haven't found an option to do this directly with gcj, one workaround is to pipe the output into grep and look for the pattern "error:", and then only show that line and a few surrounding lines.
e.g.
javac *.java 2>&1 | grep -B 3 -A 2 "error:"
Is there any way in Java (1.6+) to retrieve the partition disk structure? (For example: NTFS, FAT32, HFS+, or EXT3.)
External libraries are permitted.
Thanks,
Gianni
Under OS X the output of "mount" includes the file system:
ravn:~ ravn$ mount
/dev/disk0s2 on / (hfs, local, journaled)
devfs on /dev (devfs, local, nobrowse)
map -hosts on /net (autofs, nosuid, automounted, nobrowse)
map auto_home on /home (autofs, automounted, nobrowse)
Here / is of type hfs
If this doesn't help then - no. You'll need a native library and some OS dependent code for it.
You could use Runtime.getRuntime().exec() to execute a command like sfdisk and then parse the output.
sfdisk -l /dev/hdc
Unfortunately, it isn't very platform independent.