I'm trying to run a Java application as a Windows service with WinRun4J.
I copied WinRun4J64c.exe in my application directory and placed the following service.ini file beside:
service.class=org.boris.winrun4j.MainService
service.id=MyAPP
service.name=MyAPP
service.description=some description
classpath.1=./lib/*
classpath.2=WinRun4J.jar
[MainService]
class=play.core.server.NettyServer
But if I start the service with: WinRun4J64c.exe --WinRun4J:RegisterService I get:
Service control dispatcher error: 1063
What is wrong?
I didn't get it working, so my workaround is to use Apache Commons Deamon. I executed the included prunsrv.exe with the following parameters:
prunsrv.exe install "MeineAnwendung" \
--Install="C:/pfad/zu/prunsrv.exe" \
--JvmOptions=-Dpidfile.path=NUL
--Jvm=auto \
--Startup=auto \
--StartMode=jvm \
--Classpath="c:/irgendwo/anwendung/lib/*;" \
--StartClass=play.core.server.NettyServer
Related
I am getting the below error even though I generated the partner.jar correctly.
I have generated the partner.jar in my windows machine using the below jars.
antlr-runtime-3.5.3.jar,
force-wsc-56.1.0.jar,
js-1.7R2.jar,
partner.wsdl,
ST4-4.3.4.jar,
tools.jar,
java -classpath tools.jar;force-wsc-56.1.0.jar;ST4-4.3.4.jar;js-1.7R2.jar;antlr-runtime-3.5.3.jar com.sforce.ws.tools.wsdlc partner.wsdl partner.jar
I have set the class path correctly before generating the partner.jar.
And I copied this partner.jar to AWS EC2 machine and try executing the below code. But its still failing with the connection error.
df = spark.read.format("com.springml.spark.salesforce") \ .option("username", "dinesh123#force.com") \
.option("password", "passwordtoken) \
.option("login", "https://dev-yh.develop.my.salesforce.com/")\ .option("soql", soql) \
.option("inferSchema", True) \
.load()
Error:
py4j.protocol.Py4JJavaError: An error occurred while calling o31.load.
: java.lang.NoClassDefFoundError: com/sforce/ws/ConnectionException
at com.springml.salesforce.wave.api.APIFactory.forceAPI(APIFactory.java:49)
at com.springml.spark.salesforce.DefaultSource.createRelation(DefaultSource.scala:102)
at com.springml.spark.salesforce.DefaultSource.createRelation(DefaultSource.scala:50)
at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:350)
at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:274)
at org.apache.spark.sql.DataFrameReader.$anonfun$load$3(DataFrameReader.scala:245)
at scala.Option.getOrElse(Option.scala:189)
I am trying to setup the Apache Druid on a single machine following quickstart guide here. When I start historical server, it shows io.druid.java.util.common.IOE: No known server exception on screen.
Command:
java `cat conf-quickstart/druid/historical/jvm.config xargs` \
-cp "conf-quickstart/druid/_common:conf-quickstart/druid/historical:lib/*" \
io.druid.cli.Main server historical
Full stack-trace-
2018-04-07T18:23:40,234 WARN [main]
io.druid.java.util.common.RetryUtils - Failed on try 1, retrying in
1,246ms. io.druid.java.util.common.IOE: No known server at
io.druid.discovery.DruidLeaderClient.getCurrentKnownLeader(DruidLeaderClient.java:276)
~[druid-server-0.12.0.jar:0.12.0] at
io.druid.discovery.DruidLeaderClient.makeRequest(DruidLeaderClient.java:128)
~[druid-server-0.12.0.jar:0.12.0] at
io.druid.query.lookup.LookupReferencesManager.fetchLookupsForTier(LookupReferencesManager.java:569)
~[druid-server-0.12.0.jar:0.12.0] at
io.druid.query.lookup.LookupReferencesManager.tryGetLookupListFromCoordinator(LookupReferencesManager.java:420)
~[druid-server-0.12.0.jar:0.12.0] at
io.druid.query.lookup.LookupReferencesManager.lambda$getLookupListFromCoordinator$4(LookupReferencesManager.java:398)
~[druid-server-0.12.0.jar:0.12.0] at
io.druid.java.util.common.RetryUtils.retry(RetryUtils.java:63)
[java-util-0.12.0.jar:0.12.0] at
io.druid.java.util.common.RetryUtils.retry(RetryUtils.java:81)
[java-util-0.12.0.jar:0.12.0] at
io.druid.query.lookup.LookupReferencesManager.getLookupListFromCoordinator(LookupReferencesManager.java:388)
[druid-server-0.12.0.jar:0.12.0]
I have tried to setup from scratch many times with exactly the same steps mentioned on quick-start guide, but I am unable to resolve this error. How to resolve this error?
If you already tried to start druid, then delete the druid-X.Y.Z/log and druid-X.Y.Z/var folders.
Start zookeeper ./zookeeper-X.Y.Z/bin/zkServer.sh start
Recreate those folders you erased with druid-X.Y.Z/bin/init
Run each command in a new tab in this order
java `cat conf-quickstart/druid/coordinator/jvm.config | xargs` -cp "conf-quickstart/druid/_common:conf-quickstart/druid/coordinator:lib/*" io.druid.cli.Main server coordinator
java `cat conf-quickstart/druid/overlord/jvm.config | xargs` -cp "conf-quickstart/druid/_common:conf-quickstart/druid/overlord:lib/*" io.druid.cli.Main server overlord
java `cat conf-quickstart/druid/broker/jvm.config | xargs` -cp "conf-quickstart/druid/_common:conf-quickstart/druid/broker:lib/*" io.druid.cli.Main server broker
java `cat conf-quickstart/druid/historical/jvm.config | xargs` -cp "conf-quickstart/druid/_common:conf-quickstart/druid/historical:lib/*" io.druid.cli.Main server historical
java `cat conf-quickstart/druid/middleManager/jvm.config | xargs` -cp "conf-quickstart/druid/_common:conf-quickstart/druid/middleManager:lib/*" io.druid.cli.Main server middleManager
You should now have 1 tab open for each of those commands (so 5).
Insert the data curl -X 'POST' -H 'Content-Type:application/json' -d #quickstart/wikiticker-index.json localhost:8090/druid/indexer/v1/task
You will then see {"task":"index_hadoop_wikiticker_2018-06-06T19:17:51.900Z"}
I'm having problem running Java written spark application on AWS EMR.
Locally, everything runs fine. When I submit a job to EMR, I always get "Completed" withing 20 seconds even though job should take minutes. There is no output being produced, no log messages are being printed.
I'm still confused as weather it should be ran as Spark application or CUSTOM_JAR type.
Look of my main method:
public static void main(String[] args) throws Exception {
SparkSession spark = SparkSession
.builder()
.appName("RandomName")
.getOrCreate();
//process stuff
String from_path = args[0];
String to_path = args[1];
Dataset<String> dataInput = spark.read().json(from_path).toJSON();
JavaRDD<ResultingClass> map = dataInput.toJavaRDD().map(row -> convertData(row)); //provided function didn't include here
Dataset<Row> dataFrame = spark.createDataFrame(map, ResultingClass.class);
dataFrame
.repartition(1)
.write()
.mode(SaveMode.Append)
.partitionBy("year", "month", "day", "hour")
.parquet(to_path);
spark.stop();
}
I've tried these:
1)
aws emr add-steps --cluster-id j-XXXXXXXXX --steps \
Type=Spark,Name=MyApp,Args=[--deploy-mode,cluster,--master,yarn, \
--conf,spark.yarn.submit.waitAppCompletion=false, \
--class,com.my.class.with.main.Foo,s3://mybucket/script.jar, \
s3://partitioned-input-data/*/*/*/*/*.txt, \
s3://output-bucket/table-name], \
ActionOnFailure=CONTINUE --region us-west-2 --profile default
Completes in 15 sec without error, output result or logs I've added.
2)
aws emr add-steps --cluster-id j-XXXXXXXXX --steps \
Type=CUSTOM_JAR, \
Jar=s3://mybucket/script.jar, \
MainClass=com.my.class.with.main.Foo, \
Name=MyApp, \
Args=[--deploy-mode,cluster, \
--conf,spark.yarn.submit.waitAppCompletion=true, \
s3://partitioned-input-data/*/*/*/*/*.txt, \
s3://output-bucket/table-name], \
ActionOnFailure=CONTINUE \
--region us-west-2 --profile default
Reads parameters wrongly, sees --deploy-mode as first parameter and cluster as second instead of buckets
3)
aws emr add-steps --cluster-id j-XXXXXXXXX --steps \
Type=CUSTOM_JAR, \
Jar=s3://mybucket/script.jar, \
MainClass=com.my.class.with.main.Foo, \
Name=MyApp, \
Args=[s3://partitioned-input-data/*/*/*/*/*.txt, \
s3://output-bucket/table-name], \
ActionOnFailure=CONTINUE \
--region us-west-2 --profile default
I get this: Caused by: java.lang.ClassNotFoundException: org.apache.spark.sql.SparkSession
When I include all dependencies (which I do not need to locally)
I get: Exception in thread "main" org.apache.spark.SparkException: A master URL must be set in your configuration
I do not want to hardcode the "yarn" into the app.
I find AWS documentation very confusing as to what is the proper way to run this.
Update:
Running command directly on the server does work. So the problem must be in the way I'm defining a cli command.
spark-submit --class com.my.class.with.main.Foo \
s3://mybucket/script.jar \
"s3://partitioned-input-data/*/*/*/*/*.txt" \
"s3://output-bucket/table-name"
The 1) was working.
The step overview on the aws console said that the task was finished within 15 seconds, but in reality it was still running on the cluster. It took him an hour to do the work and I can see the result.
I do not know why the step is misreporting the result. I'm using emr-5.9.0 with Ganglia 3.7.2, Spark 2.2.0, Zeppelin 0.7.2.
I am trying to connect jconsole to a specified port for a local process. I can connect to the local process using the PID but not using the remote option.
I am using ubuntu 14.04 and JDK 1.7
This is what I am doing to run my app.
grails \
-Dcom.sun.management.jmxremote=true \
-Dcom.sun.management.jmxremote.port=9999 \
-Dcom.sun.management.jmxremote.authenticate=false \
-Dcom.sun.management.jmxremote.ssl=false \
-Djava.rmi.server.hostname=xxx.xxx.xxx.xxx \
-Dserver.port=8090 \
run-app
hostname -i also gives me xxx.xxx.xxx.xxx
Grails 2.3 and later uses "forked mode" by default, where the JVM running run-app spawns a separate process to run the target application. Therefore, rather than passing the -D options to grails you should configure them in BuildConfig.groovy. Find the grails.project.fork option and add jvmArgs:
grails.project.fork = [
run:[...., jvmArgs:['-Dcom.sun.management.jmxremote=true',
'-Dcom.sun.management.jmxremote.port=9999',
// etc.
]]
]
Using the -D options on the command line as you are currently doing will set up the JMX connector in the grails process, not in your application.
Adding the below code to resources.groovy resolved the issue for me.
String serverURL = grailsApplication.config.grails.serverURL
URL url = new URL(serverURL)
System.setProperty("java.rmi.server.hostname", "${url.host}")
rmiRegistry(org.springframework.remoting.rmi.RmiRegistryFactoryBean) {
port = 9999
alwaysCreate: true
}
serverConnector(org.springframework.jmx.support.ConnectorServerFactoryBean) { bean ->
bean.dependsOn = ['rmiRegistry']
objectName = "connector:name=rmi"
serviceUrl = "service:jmx:rmi://${url.host}/jndi/rmi://${url.host}:9999/jmxrmi"
environment = ['java.rmi.server.hostname' : "${url.host}",
'jmx.remote.x.password.file' : "${grailsApplication.parentContext.getResource('/WEB-INF/jmx/jmxremote.password').file.absolutePath}",
'jmx.remote.x.access.file' : "${grailsApplication.parentContext.getResource('/WEB-INF/jmx/jmxremote.access').file.absolutePath}",
'com.sun.management.jmxremote.authenticate': true,
'com.sun.management.jmxremote.local.only' : false,
'com.sun.management.jmxremote' : true]
}
I have a problem with configuration of proxy to connect to WebService on Weblogic 10.3.5. I cannot use System.setProperty() because it has to be per connection aproach.
Proxy which I have to use has to have "UserAgent" header.
I have tried two options:
ClientProxyFeature but it doesn't work because it generate request
to proxy without "UserAgent header".
ProxySelector: this approach work great on JUnit test but when I run
it on Weblogic I receive following error
error: java.net.SocketException: Unknown proxy type : HTTP at...
Is there any way to work this around?
Many Thanks in advance
Adam
We are able to do this with ProxySelector. The only thing which we have changed was server start parameter: -DUseSunHttpHandler=true. Now it works and send User-Agent header.
To configure WebLogic proxy file is modified setDomainEnv.cmd (Windows) that is located in the bin folder of the domain.
For example for the integrated server is here
C: \ Users \ [YOUT USER NAME] \ AppData \ Roaming \ JDeveloper \ system11.1.1.4.37.59.23 \ DefaultDomain \ bin
The following property must be placed on file with the details of your proxy
#REM Estableciendo proxi
set JAVA_OPTIONS=%JAVA_OPTIONS% -Dhttp.proxySet=true -Dhttp.proxyHost=192.168.101.11 -Dhttp.proxyPort=8080 -Dhttp.nonProxyHosts=localhost
See the next link
Proxi in WebLogic