Develop Reference

Jmeter and MemCached Sets and Gets

So apologies if what I ask is trivial but I am experimenting with Memcached and Jmeter. I have a Memcached server setup (as far as I can tell) and am able to make telnet requests to it via telnet IP PORT and additionally set and get using commands set and get appropriately.
Now point me to a different application if perhaps this is the wrong choice; but my understanding was that Jmeter should allow me to pound the server with equivalent Set and Get requests.
Unfortunately the experimental platform is a remote linux PC running Rockylinux which is similar to CentOS / RedHat to my understanding (I didn't set this part up); and as a result I do not have a GUI to launch while on the Linux PC. I have however opened Jmeter up on my local PC on windows and understand I should be able to send the test file over and run it.
I followed these instructions to try to setup a TCP sampler and set the "text to send" field as below; after doing the additional step in the link regarding the precompiler.
set tutorialspoint 0 900 9${CR}${LF}
memcached${CR}${LF}
quit${CR}${LF}
Running the above as a headless jmeter session doesn't generate any errors called [./jmeter -n -t "Sample.jmx" -l testresults.jtl"] but when I connect via telnet I'm also not seeing the value for the key "tutorialspoint" get updated. When manually doing the get and set I am seeing updates. Any ideas what I could be doing wrong? Checking the log indicates ResponseCode 200 OK as expected. Is there a good method to debug something in a Headless setup?
Thanks for your time.

I believe the easiest way is using Memcached Java Client library
Download spymemcached-2.12.3.jar and drop it to "lib" folder of your JMeter installation (or any other location in JMeter Classpath)
Restart JMeter to pick the .jar up
Add JSR223 Sampler to your test plan and use the following code snippets:
def client = new net.spy.memcached.MemcachedClient(new InetSocketAddress('your-memcached-host', your-memcached-port)) - for connecting to the server
client.set('tutorialspoint', 900, 'memcached').done to write memcached to the tutorialspoint key for 15 minutes
client.get('tutorialspoint') - to read the value of tutorialspoint key
client.shutdown() - to disconnect
More information on Groovy scripting in JMeter: Apache Groovy - Why and How You Should Use It
Demo:

Apiman does not restore gateway

Could you please assist me with the following issue.
I use Apiman version 1.2.1
FROM jboss/wildfly:9.0.2.Final
ENV APIMAN_VERSION 1.2.1.Final
I expose this version via kubernetes, as a persistent volume I use postgres in the same container. Once I create it at the first time, after this in the apiman I have added Organization/ API/.... and all necessary staff.
I press on button to publish api, and may check that it works perfect, so I use kubectl port-forward pod-name 8080:8080 and may check my gateway via browser http:localhost:8080/apiman-gateway/ORgId/bla/bla/bla/bla?givemedescriptionbyid=1.
After this one I go to console and kill apiman pod, as a result of pod restart, the same operation kubectl port-forward new-pod-name 8080:8080, and I can see that the all data of apiman like organization, apis and all other staff is already there.
But one big problem if you try to call gateway again, it tells you that:
{"responseCode":500,"message":"API not
found.","trace":"io.apiman.gateway.engine.beans.exceptions.InvalidApiException:
API not found.\n\tat
io.apiman.gateway.engine.impl.ApiRequestExecutorImpl$3.handle(ApiRequestExecutorImpl.java:278)\n\tat
io.apiman.gateway.engine.impl.ApiRequestExecutorImpl$3.handle(ApiRequestExecutorImpl.java:271)\n\tat
io.apiman.gateway.engine.impl.SecureRegistryWrapper$1.handle(SecureRegistryWrapper.java:122)\n\tat
io.apiman.gateway.engine.impl.SecureRegistryWrapper$1.handle(SecureRegistryWrapper.java:111)\n\tat
io.apiman.gateway.engine.es.CachingESRegistry.getApi(CachingESRegistry.java:116)\n\tat
io.apiman.gateway.engine.impl.SecureRegistryWrapper.getApi(SecureRegistryWrapper.java:111)\n\tat
io.apiman.gateway.engine.impl.ApiRequestExecutorImpl.execute(ApiRequestExecutorImpl.java:270)\n\tat
io.apiman.gateway.platforms.servlet.GatewayServlet.doAction(GatewayServlet.java:232)\n\tat
io.apiman.gateway.platforms.servlet.GatewayServlet.doGet(GatewayServlet.java:77)\n\tat
javax.servlet.http.HttpServlet.service(HttpServlet.java:687)\n\tat
javax.servlet.http.HttpServlet.service(HttpServlet.java:790)\n\tat
io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:86)\n\tat
io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)\n\tat
io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)\n\tat
org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)\n\tat
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)\n\tat
io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)\n\tat
io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)\n\tat
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)\n\tat
io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)\n\tat
io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)\n\tat
io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:58)\n\tat
io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:72)\n\tat
io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)\n\tat
io.undertow.security.handlers.SecurityInitialHandler.handleRequest(SecurityInitialHandler.java:76)\n\tat
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)\n\tat
org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)\n\tat
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)\n\tat
io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)\n\tat
io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:282)\n\tat
io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:261)\n\tat
io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:80)\n\tat
io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:172)\n\tat
io.undertow.server.Connectors.executeRootHandler(Connectors.java:199)\n\tat
io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:774)\n\tat
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)\n\tat
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)\n\tat
java.lang.Thread.run(Thread.java:745)\n"}
I can see, that by metrics of calls in api manager, all my calls can reach apiman-gateway, but I get 500 response code.

If anybody will met such error, so, I have solved it with the following steps, first of all we need to use only new version of APIMAN, list of fixes already implemented. And also apiman-gateway cannot live without elastic search to restore itself. Hence elasticsearch configuration should to be provided.

Elasticsearch - No node exception after few minutes intermittently

I am using elastic search with java transport client .
Elastic search : 1.1.0
Java client : 1.1.0
I am making search query using transport client which works pretty smoothly for sometime. But the problem i am facing is that after some time or few minutes it starts giving me no node available exception on all requests from my java client while the same machine gives me response from curl.
Settings settings = ImmutableSettings.settingsBuilder().put("client.transport.ping_timeout","50s").build();
client = new TransportClient(settings).addTransportAddress(new InetSocketTransportAddress(host, port));
This is how i am connecting with the node .
Problem: I am getting no node available exception after some time when we hit it starts giving exception
Also after few mins it again starts giving me response properly
Also I have integrated elasticsearch with couchbase with transport plugin so it also keeps on updating in parallel not heavily but sometimes , can it also be a reason as elastic search also indexes some data from couchbase

Java client can't find master node: MasterNotDiscoveredException waited for [1m]

I'm using vagrant and I installed ES on it using the debian package:
elasticsearch-1.1.1.deb
In my web app, I am using the jar:
org.elasticsearch elasticsearch 1.1.1
I am creating my client like:
val node = nodeBuilder.client(true).node
val client: Client = node.client
When I try and index I get the error:
val response = client.prepareIndex("articles", "article", article.id.toString).setSource(json).execute.actionGet
The error I get is:
[MasterNotDiscoveredException: waited for [1m]]
I can see my ES instance is working fine by going to:
http://localhost:9200
I ran some test queries from the README file and they worked fine, but now for some reason it isn't working either:
http://localhost:9200/twitter/user/kimchy?pretty=true
I get the error:
{
"error" : "ClusterBlockException[blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];[SERVICE_UNAVAILABLE/2/no master];]",
"status" : 503
}
My vagrant file 2 ports open for elastic search:
config.vm.network "forwarded_port", guest: 9200, host: 9200 # ES
config.vm.network "forwarded_port", guest: 9300, host: 9300 # ES
What seems to be the problem?
Note: my web application isn't using a elasticsearch.yml file because it is just connecting to the default localhost:9200 from what I understand.

Normally you have to connect to ES from outside through http (normally, but there are also others protocols available) and then talk REST/JSON. So your webapp should use a scala/java ES client (see http://www.elasticsearch.org/guide/en/elasticsearch/client/community/current/clients.html) and then connect via http to your host which is running ES on port 9200. Port 9300 is only for internode communication (ES is a distributed clustered system). But there is another way to talk remotely to ES: Powerup a node which joins the cluster and then query this node through the internal client. But:
In your above question you try to connect to ES through the internal Java client (internal transport) which starts a node and then try to joins the cluster. That fails because the master node could to be found. Maybe due to networking issues. Try to include elasticsearch.yml in the classpath or use REST like described above. There is also a third option: TransportClient - look http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/current/client.html#transport-client
See also http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-transport.html and http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-http.html and http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-memcached.html

Since you are generating your client node with .client(true), that disables both data-storage and master-eligibility on your node, if I understand the docs correctly. (the source is not very helpful either)
Note that any ES cluster needs at least 1 master node.
First, to clarify the config situation, your main elasticsearch.yml (see reference config) configuration is under /etc/elasticsearch/. You can also configure a second elasticsearch.yml in your src/main/resources folder, which will apply to the nodes you create in your app. I'd recommend doing this as it's way clearer compared to using the mysterious nodeBuilder methods.
Can you show what is the response when you query, right after starting es up, http://localhost:9200/_nodes ?
Specifically, if you have
"attributes": {
"master": "true"
},
set on one of the nodes. If so, then it looks like a networking problem as your client node is unable to contact the master node. I actually had a similar issue when I was setting up, and the solution was to set network.host: 127.0.0.1 in the app's elasticsearch.yml (wish I knew why)

uncomment discovery.zen.ping.multicast.enabled: false in /etc/elasticsearch/elasticsearch.yml

Python script won't read() after urlopen from localhost gae dev server -- Connection Reset By Peer

So I'm running the Google App Engine development server (Java) on localhost. I'm trying to retrieve a URL using Python 2.7 urllib.urlopen. The initial retrieve works, but then when I try to call read() or readlines() I get:
Traceback (most recent call last):
File "./getMap.py", line 6, in <module>
lst = f.readlines()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 513, in readlines
line = self.readline()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 445, in readline
data = self._sock.recv(self._rbufsize)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 552, in read
s = self.fp.read(amt)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 378, in read
data = self._sock.recv(left)
socket.error: [Errno 54] Connection reset by peer
The browser works, wget works. Problem occurs with both urllib and urllib2. Here's the code:
import urllib2
f = urllib2.urlopen("http://localhost:8080/default.jsp")
lst = f.readlines()
for a in lst:
print a
Strangely, I can print out the first line of the file using readline() -- I just can't get the whole file. I get the sense that maybe Python is "lazily" not requesting the entire contents of the URL until I request it via readlines(), and by then the app engine dev server has overzealously closed the connection. But I could be totally wrong about that.
I tried researching this problem but I didn't see anything applicable. Most of the Google hits I'm seeing surround random, intermittant timing issues (this isn't an intermittant problem, it's reliable) or proxy/firewall issues (nothing like that going on here).
Assuming my theory is correct -- is there a way to tell urlopen to get the whole response right away, as wget and the browser seem to be doing? Or is there a way to tell the GAE dev server to chill out and not close the connection so quickly? I'd rather not dive into lower-level python socket stuff if I don't have to.
thanks
p.s. clarification: the python script is just running from the command line and trying to make a connection to the GAE dev server, which is running on the same box. I'm NOT trying to connect to the GAE dev server from itself or something weird like that, the GAE server is running Java, not Python. What I'm actually trying to do here is this: my GAE web app has some web services and I'm writing a batch script to get/post to those webservices, so that when I need to reset/clear the data store (example: data gets corrupted) I can use this python script to back up the data first, then I erase the data store, and then use the script again to load that data back in.
UPDATE: so I tried a few more tests. Python has no trouble reading any HTML file served by the GAE dev server. However any JSP, even the simplest "hello world" JSP, fails to read with the same "connection reset by peer" error. I'll try updating to the 1.6.1 version of the GAE SDK, I have to do that anyway at some point, might as well be now. Hopefully it will fix this.

While I cannot see anything wrong with your python code and have no idea what might be wrong with your Java GAE setup I instead propose a different take on the problem.
You mention that you basically want to send GET/POST requests to your server and save/later read the content and that command line tools like wget works. I suggest you use a bash script and curl and python for the cases when you need to do more advanced text editing.
curl http://localhost:8080/default.jsp > default.bak
... wipe db ...
data = $(cat default.bak)
curl -X "POST" -d "backup=$data" http://localhost:8080/default_restore.jsp
If you need to edit the data before sending it you can use python to either read from default.bak or by piping it to stdin
data = $(cat default.bak)
python your_script.py $data
curl http://localhost:8080/default.jsp | python yourscript.py > default.bak

Obviously a bit late to the party, but I had exactly the same issue, and I solved it by swapping out urllib for httplib:
import httplib
conn = httplib.HTTPConnection('localhost:8080')
# get the current image and save to file
url = 'default.jsp'
conn.request("GET", url)
response = conn.getresponse()
if response.status == 404:
return None
img_file = open("out.jpg",'wb')
img_file.write(response.read())
img_file.close()
response.close()
conn.close()
I don't know why this works, I can only assume that httplib is slightly better behaved than urllib