Develop Reference

ONOS Service Start FrameworkEvent Error and GUI not ready yet

I have installed ONOS 2.3.0 on an Ubuntu Server 18.04.4 virtual machine running on Hyper-V following this steps (taken from here and here):
Firstly, I have installed Java 11 (openjdk-11-jdk and openjdk-11-jre), maven and curl;
then I have downloaded ONOS 2.3.0 from here and extracted it with tar xzf onos-2.3.0.tar.gz;
lastly, I exported the required environment variable export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64.
When I try to launch it using the command ./onos-service start (tested both from a normal user and sudo), it gives me the following errors:
21:54:57.869 ERROR [onos-core-net] FrameworkEvent ERROR - org.onosproject.onos-core-net
org.osgi.framework.ServiceException: Service factory returned null. (Component: org.onosproject.store.cfg.DistributedComponentConfigStore (6))
at org.apache.felix.framework.ServiceRegistrationImpl.getFactoryUnchecked(ServiceRegistrationImpl.java:380)
at org.apache.felix.framework.ServiceRegistrationImpl.getService(ServiceRegistrationImpl.java:247) org.apache.felix.framework.EventDispatcher.fireEventImmediately(EventDispatcher.java:834)
[...]
at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1373)
at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:308) at java.base/java.lang.Thread.run(Thread.java:834)
[...]
21:54:57.881 WARN [NettyMessagingService] Failed to bind TCP server to port 0.0.0.0:9876 due to {}
java.net.BindException: Address already in use
at java.base/sun.nio.ch.Net.bind0(Native Method)
[...]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:906)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at java.base/java.lang.Thread.run(Thread.java:834)
21:54:57.899 ERROR [onos-core-primitives] bundle org.onosproject.onos-core-primitives:2.3.0 (192)[org.onosproject.store.atomix.impl.AtomixManager(115)] : The activate method has thrown an exception
java.util.concurrent.CompletionException: java.net.BindException: Address already in use
at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:331)
[...]
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.net.BindException: Address already in use
at java.base/sun.nio.ch.Net.bind0(Native Method)
at java.base/sun.nio.ch.Net.bind(Net.java:455)
at java.base/sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:227)
at io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:132)
at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:563)
... 12 more
Connecting to karaf instance with ssh -p 8101 karaf#localhost confirm that ONOS is working (at least partially), the web interface login loads, but after login it hangs saying that ONOS GUI not ready yet... please stand by....
Does anyone has an idea about how to solve this problem?
Thanks in advance.
UPDATE 19-03-2020: I have prepared another virtual machine following exactly the same steps on another PC using VirtualBox and lower virtual resources assigned, and it works. Honestly i don't understand why it fails on the Hyper-V configuration.
UPDATE 20-03-2020: I have reinstalled Ubuntu configuring the network directly from the installer, and prerequisites and dependecies of ONOS offline (downloaded on another machine via sudo apt install --download-only <package-name>) and it worked. I think the problem was related to something in the network configuration that didn't let him recognize its own process on port 9876 (see the WARN above).
Hope this can be helpful for others.

I had this problem. ONOS is locked to the IP at first install. I grepped for my IP in the /onos folder and was able to reset the binding by deleting the following files that contained the IP. They were rebuilt at next ONOS run.
grep -rl 192.168. --exclude=*.log ~/onos
rm ~/onos/apache-karaf-4.2.9/data/db/partitions/data/partitions/1/raft-partition-1.conf
rm ~/onos/apache-karaf-4.2.9/data/db/partitions/data/partitions/1/raft-partition-1.meta
rm ~/onos/apache-karaf-4.2.9/data/db/partitions/data/partitions/1/.raft-partition-1.lock
rm ~/onos/apache-karaf-4.2.9/data/db/partitions/system/partitions/1/.system-partition-1.lock
rm ~/onos/apache-karaf-4.2.9/data/db/partitions/system/partitions/1/system-partition-1.conf
rm ~/onos/apache-karaf-4.2.9/data/db/partitions/system/partitions/1/system-partition-1.meta

I have faced this issue after changing the IP address of the controller (Host machine).
The quick way to solve it is to set the IP controller as it was (Static)
then reboot your machine
after putting the URL (YourIP:8181/onos/ui/index.html)
Karaf will ask you for login in credentials, use (username:karaf/password:karaf)
then on ONOS's login page, use onos/rocks as credentials.
Good luck..

How to remote debug an enterprise application running on web logic server in eclipse IDE(same machine)

I am new to weblogic application server and remote debugging & have gone through several post to set up remote debugging. Some post suggest to edit setDomainEnv.cmd file while others suggest to edit startWeblogic.cmd file in my WEBLOGIC_HOME\user_projects\domains\my_domain\bin.
But neither of the solutions worked for me. Listed below are solutions which I tried :
1) Edit setDomainEnv.cmd file
set JAVA_DEBUG=-Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,address=%DEBUG_PORT%,server=y,suspend=n -Djava.compiler=NONE
set JAVA_OPTIONS=%JAVA_OPTIONS% %enableHotswapFlag% -ea -da:com.bea... -da:javelin... -da:weblogic... -ea:com.bea.wli... -ea:com.bea.broker... -ea:com.bea.sbconsole...
The port number is set to 8543 in the file
if "%DEBUG_PORT%"=="" (
set DEBUG_PORT=8453
)
2)Edit startWeblogic.cmd file
I added the following line at the top of the file
-Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,address=8543,server=y,suspend=n
Then in eclipse,when i run debug configuration(port number : 8543), I get Failed to connect to remote VM. Connection refused.
Connection refused: connect
Please let me know
1) How remote debugging works?
2) How to set up remote debugging in eclipse with weblogic server ?
3) What is the difference between above 2 methods ?
4) Where do I need to add the debug command(-Xdebug....) in the startWeblogic.cmd file(at the top)?
5) What is the purpose of setDomainEnv.cmd file in weblogic server ?
Thanks in advance

I think it should be sufficient to just set environment variable before starting Weblogic. Start cmd and before starting weblogic set debugFlag=true. This should make weblogic open debug port which should be set to some default like 8453. You can also set DEBUG_PORT=8888 or any other free port, if you want to change it.
Start Weblogic and verify that the port has been opened. You can use tools like cports or ProcessExplorer for that (or event netstat).
In case debug port isn't open, check if you run Weblogic in development mode, because debugFlag can be ignored in production mode.
In Eclipse create a remote debug configuration (and remember that this option is not available in Run configuration that is next to it on the toolbar):
In point 3 on the screenshot select the project you deploy to weblogic that you want to debug.
It that does not work post the problems you have.

WebLogic 12c - Destination unreachable exception

First, I had installed jdk 1.6.0_43 and oracle weblogic 12.1.1, I was successfully able to deploy my application.
I then upgraded both my jdk (1.7.0_60) and weblogic (12.1.2), but was unable to deploy my application.
Now, I downgraded my weblogic (12.1.1) but retained my jdk 1.7.0_60, but i was still not able to deploy my application successfully.
In both the failure cases, I got the same error with the following message. Is there something with respect to java 7 I should be aware of? I tried searching for this a lot, but in vain..
[exec] javax.naming.CommunicationException [Root exception is java.net.ConnectException: t3://localhost:9991: Destination unreachable; nested exception is:
[exec] java.net.ConnectException: Connection refused: connect; No available router to destination]

No available router to destination -- Means you do not have a service running to listen on localhost:9991
Go to admin console, check the servers which should be listening on 9991 is up and running.
I am sure it is not running.

a few possibilities
Not listening - check with netstat
No tunneling - check Summary of Servers -> Configure server (admin) -> Protocols tab -> HTTP sub tab-> Enable Tunneling
JDeveloper does not trust the SSL certificate - check root CA e.g.
$ cd /oracle/Middleware-12.2.1.2/oracle_common
$ jdk/bin/keytool -import -v -file ca.crt -storepass changeit -keystore jdk/jre/lib/security/cacerts -alias MyCA

oracle's blog points that you have to enable tunneling:
access the administration console, click servers-> server you want to
reach-> protocols -> http -> enable Tunneling.

Follow these steps (Must be done on Admin only):
Login to Weblogic Console
Click Lock and Edit
Go to Servers
Go to Admin
Go to protocols
Go to HTTP
Scroll down and click on Enable Tunneling
Save
Activate (No restart needed)

I had a similar issue with my Weblogic server while building with maven
If you're using maven, make sure in your maven settings.xml file, the value for: <weblogic.admin.url></weblogic.admin.url> is the same what you have for listen-address></listen-address> your weblogic config.xml
My problem was that my settings.xml had my local IP address where as weblogic config.xml had localhost.

I had the similar problem in WebLogic Server Version: 12.2.1.3.0 and it was resolved by changing the SSL port. On the same server, Admin server and managed server had similar ports for the SSL even though both the ports were disabled yet there was conflict. After changing the ssl port number i was able to solve the problem.
Go to Weblogic Console
Click on the managed server
Under configuration -> General enable ssl and ensure the port mentioned is unique and is not being used by AdminServer or any of the managed servers.

Starting Apache Tomcat v7 at localhost has encountered a p‌r‌o‌b‌l‌e‌m and failed to start in eclipse

I'm using Eclipse kepler and Tomcat v7 64 bit on my windows 7 PC. My apache tomcat was running fine but suddenly it stopped working and shows the following error
"Starting Apache Tomcat v7 at localhost has encountered a problem.
Server Apache Tomcat v7 has failed to start."
I have tried deleting the .snap file and temp0 folder in the workspace folder.
I have also tried uninstalling and re-installing apache.
i referred to the question "Server Tomcat v7.0 Server at localhost failed to start" without stack trace while it works in terminal and tried all solutions.
But all this in vain.
There is no other process running on port number 8080 or other ports.
Tomcat doesn't start only in Eclipse

Try checking the Console tab of Eclipse to get more detail of what caused the error. There are several potential problems that could produce this symptom.
On the Console tab, there will be plenty of red text for "INFO" entries in the log, but likely there will be a "SEVERE" entry that explains what happened. You may have to look down a little further because the first one may list that it is "caused by" something else.
In my case, I had refactored the name of a Servlet, but accidentally left the mapping to the old name (e.g., #WebServlet("/SavePost"). This resulted in two servlets being mapped to the same url-pattern, which caused the error:
Caused by: java.lang.IllegalArgumentException: The servlets named [forum.SavePost] and [forum.SavePost_old] are both mapped to the url-pattern [/SavePost] which is not permitted

If eclipse shows like this:
starting tomcat v7.0 server at localhost has encountered a problem port 8080. default port number of tomcat is 8080. if oracle install in your system then you need to change oracle port number. connect with user sysdba and change HTTP port number of oracle SQL command.
SQL> select dbms_xdb.gethttpport as "HTTP-Port", dbms_xdb.getftpport as "FTP-Port" from dual;
HTTP-Port FTP-Port
8080 0
Change Port HTTP and FTP.
SQL> begin
2 dbms_xdb.sethttpport('80');
3 dbms_xdb.setftpport('2100');
4 end;
5 /
SQL> select dbms_xdb.gethttpport as "HTTP-Port"
, dbms_xdb.getftpport as "FTP-Port" from dual;
HTTP-Port FTP-Port
80 2100

I used to get this error and were very frustrated because none of the answer were solving problem.
Just see the mappings of servlet this error can be because mapping are
not correct
Try this it worked for me

Unable to open debugger port in IntelliJ

Unable to open debugger port in intellij.
The port number 9009 matches the one which has been set in the configuration file for the application.
<java-config debug-options="-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=9009" system-classpath="" native-library-path-prefix="D:\Project\lib\windows\64bit" classpath-suffix="">
<jvm-options>-XX:MaxPermSize=192m</jvm-options>
<jvm-options>-client</jvm-options>
<jvm-options>-XX:+UnlockDiagnosticVMOptions</jvm-options>
<jvm-options>-XX:+LogVMOutput</jvm-options>
<jvm-options>-XX:LogFile=${com.sun.aas.instanceRoot}/logs/jvm.log</jvm-options>
<jvm-options>-Djava.endorsed.dirs=${com.sun.aas.installRoot}/modules/endorsed${path.separator}${com.sun.aas.installRoot}/lib/endorsed</jvm-options>
<jvm-options>-Djava.security.policy=${com.sun.aas.instanceRoot}/config/server.policy</jvm-options>
<jvm-options>-Djava.security.auth.login.config=${com.sun.aas.instanceRoot}/config/login.conf</jvm-options>
<jvm-options>-Dcom.sun.enterprise.security.httpsOutboundKeyAlias=s1as</jvm-options>
<jvm-options>-Djavax.net.ssl.keyStore=${com.sun.aas.instanceRoot}/config/keystore.jks</jvm-options>
<jvm-options>-Djavax.net.ssl.trustStore=${com.sun.aas.instanceRoot}/config/cacerts.jks</jvm-options>
<jvm-options>-Djava.ext.dirs=${com.sun.aas.javaRoot}/lib/ext${path.separator}${com.sun.aas.javaRoot}/jre/lib/ext${path.separator}${com.sun.aas.instanceRoot}/lib/ext</jvm-options>
<jvm-options>-Djdbc.drivers=org.apache.derby.jdbc.ClientDriver</jvm-options>
<jvm-options>-DANTLR_USE_DIRECT_CLASS_LOADING=true</jvm-options>
<jvm-options>-Dcom.sun.enterprise.config.config_environment_factory_class=com.sun.enterprise.config.serverbeans.AppserverConfigEnvironmentFactory</jvm-options>
<jvm-options>-Dosgi.shell.telnet.port=4766</jvm-options>
<jvm-options>-Dosgi.shell.telnet.maxconn=1</jvm-options>
<jvm-options>-Dosgi.shell.telnet.ip=127.0.0.1</jvm-options>
<jvm-options>-Dfelix.fileinstall.dir=${com.sun.aas.installRoot}/modules/autostart/</jvm-options>
<jvm-options>-Dfelix.fileinstall.poll=5000</jvm-options>
<jvm-options>-Dfelix.fileinstall.debug=1</jvm-options>
<jvm-options>-Dfelix.fileinstall.bundles.new.start=true</jvm-options>
<jvm-options>-Dorg.glassfish.web.rfc2109_cookie_names_enforced=false</jvm-options>
<jvm-options>-XX:NewRatio=2</jvm-options>
<jvm-options>-Xmx2048m</jvm-options>
</java-config>
Configuration in IntelliJ:
When I try and enable the remote debugging in for this application it comes up with the following error:

You may have to change the debugger port if your port is already used by another program. To do so:
Run
Edit Configurations
Startup/Connection tab
Debug
Change the port here
Or, maybe in other versions:
Run
Edit Configurations
Remote > Remote debug in the list on the left
Configuration tab, Settings section
Port: change the port here

Add the following parameter debug-enabled="true" to this line in the glassfish configuration.
Example:
<java-config debug-options="-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=9009" debug-enabled="true"
system-classpath="" native-library-path-prefix="D:\Project\lib\windows\64bit" classpath-suffix="">
Start and stop the glassfish domain or service which was using this configuration.

I had the same problem and this solution also did the trick for me: Provide the IP 127.0.0.1 in the Intellij Debug configuration instead of the host name "localhost", in case you're using this hostname.

You must set CHMOD +x (execute for *.sh or *.bat files). For example, I am using macOS
cd /Users/donhuvy/Documents/tools/apache-tomcat-9.0.12/bin
sudo chmod +x *.sh
Then IntelliJ IDEA, and Apache Tomcat running or debugging just good.

In glassfish\domains\domain1\config\domain.xml set before start server
<java-config classpath-suffix="" debug-options="-agentlib:jdwp=transport=dt_socket,address=9009,server=y,suspend=n" java-home="C:\Program Files\Java\jdk1.8.0_162" debug-enabled="true" system-classpath="">
or set debug-enabled="true" server=y,suspend=n in http://localhost:4848/common/index.jsf
In current Idea 2018 - Server Run Configuration - Debug - Port - address

I'm hoping your problem has been solved by now. If not, try this... It looks like you have server=y for both your app and IDEA. IDEA should probably be server=n. Also, the (IDEA) client should have an address that includes both the host name and the port, e.g., address=127.0.0.1:9009.

This one worked for me--
If the issue still persists (in case you are not using a glassFish server), then close your JIdea and stop the server. This will disable the ports connectivity. Then start your server and JIdea, this will start fresh connectivity with the ports, resolving the issue.

For me, the problem was that catalina.sh didnt have execute permissions. The "Unable to open debugger port in intellij" message appeared in Intellij, but it sort of masked the 'could not execute catalina.sh' error that appeared in the logs immediately prior.

This error can happen Tomcat is already running. So make sure Tomcat isn't running in the background if you've asked Intellij to start it up ( default ).
Also, check the full output window for more errors. As a more useful error may have preceded this one ( as was the case with my configuration just now )

Answer is pretty simple,
I also faced the problem finally I got perfect solution.
Create Debug
Create Remote debug with following configuration
Firstly run by debug.
It gives you waitng for socket 5005
then run with remote debug

Try to connect with telnet , if it connects then it shows below:
$telnet 10.238.136.165 9999
Trying 10.238.136.165...
Connected to 10.238.136.165.
Escape character is '^]'.
Connection closed by foreign host.
If port is not available (either because someone else is already connected to it or the port is not open etc) then it shows something like it shows like below:
$telnet 10.238.136.165 9999
Trying 10.238.136.165...
telnet: connect to address 10.238.136.165: Connection refused
telnet: Unable to connect to remote host
So I think one needs to see whether:
the application is property listening to port or not
or someone else has already connected to it
Also try to connect on that m/c itself first like
$telnet localhost 9999

Set the MAVEN_OPTS. It should work !!
export MAVEN_OPTS="-Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,address=4000,server=y,suspend=n"
mvn spring-boot:run -Dserver.port=8090

Run your Spring Boot application with the given command to enable debugging on port 6006 while the server is up on port 8090:
mvn spring-boot:run -Drun.jvmArguments='-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=6006' -Dserver.port=8090

Your Service/ Application might already be running. In Search enter Services and you will get list of services. Stop yours and then try again.

I had the same issue, I just have to remove the HTTP protocol from the URL. That's it.
I hope it works for you.

I once have this problem too.
My solution is to work around this problem by kill the application which is using the port.
Here is a article to teach us how to check which application is using which port, find it and kill/close it.

In my case, I was not setting the debug port while starting the application.
I am using tomcat to deploy 3 war files, and I forgot to configure the debug port.
Tomcat allows us to configure this via setenv.sh.
Here are the commands to create setenv.sh file in the bin directory of my tomcat installation and provide the debug arguments/port.
tee /usr/share/tomcat9/bin/setenv.sh << EOF
export CATALINA_OPTS="$CATALINA_OPTS -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005"
EOF
service tomcat9 restart

Merely hitting the debug icon again fixed my problem in a few seconds.

Make sure to specify an SDK and Project SDK for your app under File --> Project Structure (Project | SDKs)