Develop Reference

Error "io.questdb.cairo.CairoException: [2] could not open read-write [file=<dir>/_tab_index.d]"

Currently, I am testing QuestDB in a Apache Camel / Spring Boot scenario for our project. I set up a custom Camel component and a configuration bean holding the connection properties. As far as I can see, my custom Camel component properly connects to the server where a test instance of QuestDB is running. But when sending data over the Camel route, I get error messages:
io.questdb.cairo.CairoException: [2] could not open read-write [file=<dir>/_tab_index.d]
The exception is thrown when creating the CairoEngine like (taken from QuestDB API documentation:
try (CairoEngine engine = new CairoEngine(this.configuration)) {
... other code ...
} catch (Exception e) {
e.printStackTrace();
...
}
where this.configuration is of type CairoConfiguration and contains the "data_dir" and is instantiated like this:
configuration = new DefaultCairoConfiguration(<quest db directory (String)>);
Currently, I am passing the fully qualified path my database directory: /srv/questdb/db. I confirmed that the file _tab_index.d is available at this location.
What am I going wrong? Maybe I should mention, that I set the access rights to the questdb directory to 777, the owner was set to chown root:questdb ...

Indeed, the embedded API is not suitable for what I want to do. I need to one of the other APIs. I tested my scenario withe the InfluxDB line protocol (see Line protocol documentation) and the data gets written to the server without problems.
The doInsert method in my custom component look like this (just for testing) which is called when building a route with the custom QuestDB "to" end point:
public class QuestDbProducer extends DefaultProducer {
... other code ...
private void doInsert(Exchange exchange, String tableName) throws InvalidPayloadException {
try (Sender sender = Sender.builder().address("lxyrpc01.gsi.de:9009").build()) {
sender.table("inventors")
.symbol("born", "Austrian Empire")
.longColumn("id", 0)
.stringColumn("name", "Nicola Tesla")
.atNow();
sender.table("inventors")
.symbol("born", "USA")
.longColumn("id", 1)
.stringColumn("name", "Thomas Alva Edison")
.atNow();
}
}

Websphere MQ unable to create initial JNDI context

I am trying to run a demo class (JMSJNDIProducer.java) that comes with Websphere MQ 7.0 installation at 'websphere installation location'\tools\jms\samples. I have Websphere up and running, below are my websphere configurations...
Queue Manager Name: JMSDEMO
Queue Manager Port: 1414
Channel (default): SYSTEM.DEF.CLNTCONN
Queue Name: JMSDEMO.QL
let me know if more info needed...
My code is failing during the initial context creation, I'm very new to Websphere MQ and not sure what the initialContextUrl needs to be?
public class JMS_JNDI_Websphere_Sample {
private static String initialContextUrl = "tcp://localhost:1414";
public static void main(String args[]) {
// Instantiate the initial context
String contextFactory = "com.sun.jndi.fscontext.RefFSContextFactory";
Hashtable<String, Object> environment = new Hashtable<String, Object>();
environment.put(Context.INITIAL_CONTEXT_FACTORY, contextFactory);
environment.put(Context.PROVIDER_URL, initialContextUrl);
try {
Context context = new InitialDirContext(environment);
} catch (NamingException e) {
e.printStackTrace();
}
System.out.println("Initial context found!");
}
}
I am getting the below exception.
javax.naming.InvalidNameException: tcp://localhost:1414 [Root exception is java.net.MalformedURLException: unknown protocol: tcp]
at com.sun.jndi.fscontext.FSContextFactory.getFileNameFromURLString(FSContextFactory.java:119)
at com.sun.jndi.fscontext.RefFSContextFactory.createContext(RefFSContextFactory.java:41)
at com.sun.jndi.fscontext.RefFSContextFactory.createContextAux(RefFSContextFactory.java:47)
at com.sun.jndi.fscontext.FSContextFactory.getInitialContext(FSContextFactory.java:49)
at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:667)
at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:288)
at javax.naming.InitialContext.init(InitialContext.java:223)
at javax.naming.InitialContext.<init>(InitialContext.java:197)
at javax.naming.directory.InitialDirContext.<init>(InitialDirContext.java:82)
at com.hcsc.jms.websphere.jndi.JMS_JNDI_Websphere_Sample.main(JMS_JNDI_Websphere_Sample.java:32)
Caused by: java.net.MalformedURLException: unknown protocol: tcp
at java.net.URL.<init>(URL.java:574)
at java.net.URL.<init>(URL.java:464)
at java.net.URL.<init>(URL.java:413)
at com.sun.jndi.fscontext.FSContextFactory.getFileNameFromURLString(FSContextFactory.java:117)
... 9 more

private static String initialContextUrl = "tcp://localhost:1414";
First off, I don't think "tcp" is a valid value and if it is, do you have something running on port 1414 to reply JNDI lookup requests?
Secondly, I think you are confusing MQ port 1414 with JNDI lookup.
Third, why don't you just follow the example in JmsJndiProducer.java and use a file-based JNDI.
i.e. Use MQ Explorer and select "JMS Administered Objects" then do file-based JNDI.
Once you create your file-based JNDI then that value for your initial context.

You need to separate out the concept of messaing as provided by the JMS API, and the lookup of an object via JNDI. As Roger said the issue is confusion between the MQ listener and the JNDI URL
JNDI is split into the interface used to bind and lookup objects in the directory and the 'service providers' that take the object and put into some persistent store. The com.sun.jndi.fscontext.RefFSContextFactory is a service provider that uses the file system, so the URL for this needs to be an EXISTING directory. When objects are 'bound' into that the .bindings file is created (or updated if objects are already there). You don't need to create the .bindings file; that file is created for you by the File System Context. Also don't modify this by hand.
Typically you would use a LDAP backed JNDI service provider for production usage. The 'lookup' APIs used in the application are the same; what would change are the provider URL (as the location of the LDAP server would be provided) and possible the object name.
Would suggest reviewing Oracle's JNDI tutorial ..

Documentum DFS: Timeout for service calls

I'm working with the DFS Java API and was wondering whether anyone knows a simple way to configure a client-side timeout for service-calls that can be configured on the service context, for example?
I have experienced some rare occasions where a Documentum repository was not responding, that's why I am considering a general timeout for all DFS calls.
For testing a hanging service call, I created a dummy TBO implementation that simply blocks the thread for 10 minutes when updating the document:
#Override
public void saveEx(boolean keepLock, String versionLabels) throws DfException {
if (isNew() == false) {
try {
Thread.sleep(1000*60*10);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
super.saveEx(keepLock, versionLabels);
}
I'm not sure if this behaves exactly like a hanging service call, but at least in my tests it worked as expected - my invocations of the update method of the Object Service took about 10minutes.
Is there any configuration I have not yet found, or maybe a runtime-property to pass to the service context to configure the timeout?
I would prefer using existing features of DFS for this instead of implementing my own mechanism.

Have you tried editing the value in dfs-runtime.properties? I don't think the timeout can be context-specific, but you should be able to change it for the client as a whole.
Reposted from https://community.emc.com/message/3249#3249
"Please see the Server runtime startup settings section of the Deployment guide.
The following list describes the precedence that dfs-runtime.properties files take depending on their location:
local-dfs‑runtime.properties file in the local classpath
runtime properties file specified with ‑Ddfs.runtime.properties.file
dfs‑runtime.properties packaged with emc‑dfs‑rt.jar
For example, settings in the local-dfs‑runtime.properties file on the local classpath will take precedence of identical settings in the dfs‑runtime.properties file that is located in emc‑dfs‑rt.jar or the one specified with the ‑D parameter. The DFS application must be restarted after any changes to the configuration. As a best practice, use the provided configuration file that is deployed in the emc‑dfs‑rt.jar file for your base settings and use an external file to override settings that you specifically wish to change."

Determine that application is running under application server

Some code may be reused in various environments including Java EE Application server. Sometimes it is nice to know whether the code is running under application server and which application server is it.
I prefer to do it by checking some system property typical for the application server.
For example it may be
jboss.server.name for JBoss
catalina.base for Tomcat
Does somebody know appropriate property name for other servers?
Weblogic, Websphere, Oracle IAS, others?
It is very easy to check if you have the specific application server installed. Just add line
System.getProperties() to any JSP, Servlet, EJB and print the result.
I can do it myself but it will take a lot of time to install server and make it working.
I have read this discussion: How to determine type of Application Server an application is running on?
But I prefer to use system property. It is easier and absolutely portable solution. The code does not depend on any other API like Servlet, EJBContext or JMX.

JBoss AS sets a lot of diffrent system properties:
jboss.home.dir
jboss.server.name
You can check other properties using for example VisualVM or other tools.
I don't know other servers but I think you can find some kind of properties for each of them.

This is not a 'standard' way but what I did was to try to load a Class of the AppServer.
For WAS:
try{
Class cl = Thread.getContextClassLoader().loadClass("com.ibm.websphere.runtime.ServerName");
// found
}
// not Found
catch(Throwable)
{
}
// For Tomcat: "org.apache.catalina.xxx"
Etc.
Let me know what you think

//for Tomcat
try {
MBeanServer mBeanServer = ManagementFactory.getPlatformMBeanServer();
ObjectName name = new ObjectName("Catalina", "type", "Server");
StandardServer server = (StandardServer) mBeanServer.getAttribute(name,"managedResource");
if (server != null) {
//its a TOMCAT application server
}
} catch (Exception e) {
//its not a TOMCAT Application server
}
//for wildfly
try {
ObjectName http = new ObjectName("jboss.as:socket-binding-group=standard-sockets,socket- binding=http");
String jbossHttpAddress = (String) mBeanServer.getAttribute(http, "boundAddress");
int jbossHttpPort = (Integer) mBeanServer.getAttribute(http, "boundPort");
String url = jbossHttpAddress + ":" + jbossHttpPort;
if(jbossHttpAddress != null){
//its a JBOSS/WILDFLY Application server
}
} catch (Exception e) {
//its not a JBOSS/WILDFLY Application server
}

Tomcat: Change the Virtual hosts programmatically?

Tomcat offers a build in "Virtual Hosting" Support: An Engine/Web-Application can be configured to be responsible for a list of Domains. These Domains have to be put into the server.xml/context.xml files with a special xml directive.
=> Is there any possibility to change the Tomcat Configuration (in general) and especially the "Virtual Hosts" of a Web-Application/Engine programmatically?
For example if a new user signs up, I have to add his domain to the list of "accepted virtual hosts/domains". The only way I currently think of is changing the xml files via a script and then restart Tomcat.
Is there any way to add them add runtime via some Java-Methods programmatically?
Thank you very much!
Jan

Tomcat provides APIs to create new virtual host. To get access to the wrapper object needed for this, you need to implement a ContainerServlet. You can create virtual host like this,
Context context = (Context) wrapper.getParent();
Host currentHost = (Host) context.getParent();
Engine engine = (Engine) currentHost.getParent();
StandardHost host = new StandardHost();
host.setAppBase(appBase);
host.setName(domainName);
engine.addChild(host);
You need to make sure appBase directory exist and you have to find ways to persist the new host to the server.xml or you lose the host on restart.
However, this approach rarely works. If your users run their own apps, you really want run separate instances of Tomcat so you can sandbox the apps better. e.g. One app running out of memory doesn't kill all other apps.
If you provide the app, you can just use one host (defaultHost). You can get the domain name from Host header and do whatever domain-specific stuff in your code.

You shouldn't change the server environment programmatically and there are no reliable and standard ways to do this. Best is to do and keep it all on the webapp side. To start, a Filter is perfectly suitable for this. Store the names somewhere in a database table or a properties file which you cache in the application scope. Check the HttpServletRequest#getRequestURI() (or the getServerName() if it is a subdomain instead of pathinfo) and do the forwarding task accordingly.
Hope this helps.

Use JMX
ArrayList serverList = MBeanServerFactory.findMBeanServer(null);
MBeanServer server = (MBeanServer) serverList.get(0);
Object[] params = { "org.apache.catalina.core.StandardHost", hostName };
String[] signature = { "java.lang.String", "java.lang.String" };
server.invoke(new ObjectName("Catalina:type=Engine"), "addChild", params, signature);
If needed, retrieve the host object and work with it:
ObjectName host = new ObjectName("Catalina:type=Host,host=" + hostName);
server.setAttribute(host, new Attribute("autoDeploy", false));
server.invoke(host, "start", null, null);

I would suggest you set your application to be the default virtual host in server.xml so your single virtual host can respond to requests addressed to any host name. Tomcat ships with the localhost application set as the default virtual host. So you can see how to do this by simply inspecting the server.xml file of a vanilla tomcat installation. You can programatically determine the host name the user sent the request to using the ServletRequest.getServerName() method.
Tomcat used to ship with a web application called "host-manager". Note: this is different than the "manager" web application that still comes with Tomcat. Host manager allowed for changing configuration or adding new virtual hosts on the fly without restarting the server. You could interact with the host-manager over HTTP (programmatically if desired). However, it had the unfortunate flaw of not committing its changes to server.xml so they were all lost on a web server restart. For whatever reason, starting with version 6, Tomcat no longer ships with the host-manager application. So it doesn't appear to be supported anymore.

To sum up ZZ Coder answer which guided me a lot:
You have to create a servlet that implements ContainerServlet and override setWrapper method to get the org.apache.catalina.Wrapper object.
For doing that you have to have privileged="true" in your context.xml Context tag or it will throw an exception. Then you can use the Wrapper object and:
StandardContext context = (StandardContext) wrapper.getParent();
StandardHost currentHost = (StandardHost) context.getParent();
StandardEngine engine = (StandardEngine) currentHost.getParent();
StandardHost host = new StandardHost();
host.setAppBase(currentHost.getAppBase()); //in my case I created another instance of the same application
host.setDomain(currentHost.getDomain());
host.setAutoDeploy(false); // not restarting app whenever changes happen
host.setName("domain.com");
host.setThrowOnFailure(true);// tell it to throw an exception here if it fails to create the host
host.setDeployOnStartup(true);
host.setStartChildren(true);
host.setParent(engine);
// you can add multiple aliases
host.addAlias(alias);
StandardContext ctx = new StandardContext();
ctx.setDocBase(context.getDocBase()); //again I reused my same application setting
ctx.setPath("");
if(currentHost.getWorkDir() != null)
{//create a working directory based on your new host's name
ctx.setWorkDir(currentHost.getWorkDir().replace(currentHost.getName(), host.getName()));
}
ctx.setName(host.getDomain());
//some extra config that you can use
ctx.setUseHttpOnly(false);
ctx.setReloadable(false);
ctx.setXmlValidation(false);
ctx.setXmlNamespaceAware(false);
ctx.setCrossContext(false);
ctx.setParent(host);
// you have to have this or it will not work!!
ctx.addLifecycleListener(new ContextConfig());
//you can also create resources and add it to the context like so:
final ContextResource res = new ContextResource();
res.setName("name");
res.setAuth("Container");
res.setType("javax.sql.DataSource");
ctx.getNamingResources().addResource(res);
host.addChild(ctx);
engine.addChild(host);
You can add properties to your resource by calling res.setProperty("name", "value")
Some properties that you can use are:
initialSize,maxTotal,maxIdle,maxWaitMillis,removeAbandonedOnBorrow,removeAbandonedTimeout,validationQuery,timeBetweenEvictionRunsMillis,driverClassName,url,username,password.
Another exciting thing to is to get the host from the tomcat engine by calling engine.findChild(domain) and use stop(), start(), getStateName() and have your own Tomcat Admin panel!