Develop Reference

Couchbase view pagination with java client

I am trying to pull records from a view which emits in following way
DefaultViewRow{id=0329a6ac-84cb-403e-9d1d, key=[“X”,“Y”,“0329a6ac-84cb-403e-9d1d”,“31816552700”], value=1}
As we have millions of record, we are trying to implement pagination to pull 500 records per page and do some process then get next 500 records
i have implemented following code with java client
def cluster = CouchbaseCluster.create(host)
def placesBucket = cluster.openBucket("pass", "pass")
def startKey = JsonArray.from(X, "Y")
def endKey = JsonArray.from(X, "Y", new JsonObject())
hasRow = true
rowPerPage = 500
page = 0
currentStartkey=""
startDocId=""
def viewResult
def counter = 0
while (hasRow) {
hasRow = false
def skip = page == 0 ?0: 1
page = page + 1
viewResult = placesBucket.query(ViewQuery.from("design", "view")
.startKey(startKey)
.endKey(endKey)
.reduce(false)
.inclusiveEnd()
.limit(rowPerPage)
.stale(Stale.FALSE)
.skip(skip).startKeyDocId(startDocId)
)
def runResult = viewResult.allRows()
for(ViewRow row: runResult){
hasRow = true
println(row)
counter++
startDocId = row.document().id()
}
println("Page NUMBER "+ page)
}
println("total "+ counter)
Post execution, i am getting few repetitive rows and even though the total records is around 1000 for particular small scenario i get around 3000+ rows in response and it keeps going.
can someone please tell me if i am doing something wrong ?? PS: My start key value will be same for each run as i am trying to get each unique doc _id.
please help.

How to update remaining days of all user in firestore from server side?

I just want to know that how we can update all remaining days of all users in firestore from server side.Is there any need to use firebase CLI and node.js.

To run periodic updates (on a daily basis) I would suggest a Firebase function which queries the collection and updated the data according to your needs. You can create a http function which can be triggered via URL (you can schedule a CRON job to trigger it) or pubsub function which has it's own scheduling mechanism (It needs a paid Firebase plan).
PubSub example:
export scheduledFunctionCrontab =
functions.pubsub.schedule('5 11 * * *').onRun((context) => {
console.log('This will be run every day at 11:05 AM UTC!');
//query and update database collection
});
Http example:
exports.functionName = functions.https.onRequest((req, res) => {
//query and update database collection
});
Javascript for updating Firestore in Node.js would be something like this:
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp(functions.config().firebase);
var db = admin.firestore();
async function getDocuments() {
let docRef = db.collection('<collectionName>');
const result = await docRef.where(admin.firestore.FieldPath.documentId(), '>=', 'condition')
.get();
return result.docs;
}
exports.updateData = functions.pubsub.schedule('every 1 days').onRun(async context => {
const documents = await getDocuments();
for(i = 0; i < documents.length; i++) {
documents[i].update({
timestamp: firebase.firestore.FieldValue.serverTimestamp()
});
}
});

ZMQ missing events being propagated in jeromq scala

I am new to ZeroMQ and seem to be losing messages in a loop in my begin() method.
I'm wondering if I am missing a piece where I am not queuing messages or something?
When I cause an event on my publisher, that sends two messages to my subscriber with a small gap in between, I seem not to be getting the second message that is relayed. What am I missing?
class ZMQSubscriber[T <: Transaction, B <: Block](
socket: InetSocketAddress,
hashTxListener: Option[HashDigest => Future[Unit]],
hashBlockListener: Option[HashDigest => Future[Unit]],
rawTxListener: Option[Transaction => Future[Unit]],
rawBlockListener: Option[Block => Future[Unit]]) {
private val logger = BitcoinSLogger.logger
def begin()(implicit ec: ExecutionContext) = {
val context = ZMQ.context(1)
// First, connect our subscriber socket
val subscriber = context.socket(ZMQ.SUB)
val uri = socket.getHostString + ":" + socket.getPort
//subscribe to the appropriate feed
hashTxListener.map { _ =>
subscriber.subscribe(HashTx.topic.getBytes(ZMQ.CHARSET))
logger.debug("subscribed to the transaction hashes from zmq")
}
rawTxListener.map { _ =>
subscriber.subscribe(RawTx.topic.getBytes(ZMQ.CHARSET))
logger.debug("subscribed to raw transactions from zmq")
}
hashBlockListener.map { _ =>
subscriber.subscribe(HashBlock.topic.getBytes(ZMQ.CHARSET))
logger.debug("subscribed to the hashblock stream from zmq")
}
rawBlockListener.map { _ =>
subscriber.subscribe(RawBlock.topic.getBytes(ZMQ.CHARSET))
logger.debug("subscribed to raw block")
}
subscriber.connect(uri)
subscriber.setRcvHWM(0)
logger.info("Connection to zmq client successful")
while (true) {
val notificationTypeStr = subscriber.recvStr(ZMQ.DONTWAIT)
val body = subscriber.recv(ZMQ.DONTWAIT)
Future(processMsg(notificationTypeStr, body))
}
}
private def processMsg(topic: String, body: Seq[Byte])(implicit ec: ExecutionContext): Future[Unit] = Future {
val notification = ZMQNotification.fromString(topic)
val res: Option[Future[Unit]] = notification.flatMap {
case HashTx =>
hashTxListener.map { f =>
val hash = Future(DoubleSha256Digest.fromBytes(body))
hash.flatMap(f(_))
}
case RawTx =>
rawTxListener.map { f =>
val tx = Future(Transaction.fromBytes(body))
tx.flatMap(f(_))
}
case HashBlock =>
hashBlockListener.map { f =>
val hash = Future(DoubleSha256Digest.fromBytes(body))
hash.flatMap(f(_))
}
case RawBlock =>
rawBlockListener.map { f =>
val block = Future(Block.fromBytes(body))
block.flatMap(f(_))
}
}
}
}

So this seems to have been solved by using a ZMsg.recvMsg() in the while-loop instead of
val notificationTypeStr = subscriber.recvStr(ZMQ.DONTWAIT)
val body = subscriber.recv(ZMQ.DONTWAIT)
I'm not sure why this works, but it does. So here is what my begin method looks like now
while (run) {
val zmsg = ZMsg.recvMsg(subscriber)
val notificationTypeStr = zmsg.pop().getString(ZMQ.CHARSET)
val body = zmsg.pop().getData
Future(processMsg(notificationTypeStr, body))
}
Future.successful(Unit)
}

What am I missing?
How the blocking v/s non-blocking modus operandi work :
The trick is in the (non-)blocking mode of the respective call to the .recv() method.
A second call to the subscriber.recv( ZMQ.DONTWAIT )-method thus returns immediately, so your second part, ( the body ) may and will legally contain nothing, even though your promise stated a pair of messages was indeed dispached from the publisher-side ( a pair of .send() method calls - one may also object, there are chances the sender was actually sending just one message, in a multi-part fashion - MCVE-code is not specific on this part ).
So, once you have moved your code from non-blocking mode ( in the O/P ) into a principally blocking-mode ( which locked / sync-ed the further flow of the code with the external event of an arrival of any plausibly formatted message, not returning earlier ), in:
val zmsg = ZMsg.recvMsg(subscriber) // which BLOCKS-till-a-1st-zmsg-arrived
both the further processed .pop()-ed parts just unload the components ( ref. the remark on actual ZMsg multi-part structure actually sent by the published-side, presented above )
Safety next :unlimited alloc-s v/s a mandatory blocking / dropping messages ?
the code surprised me on several points. Besides a rather very "late" call to the .connect()-method, compared to all the previous socket-archetype detailed settings ( that normally get arranged "after" a request to setup a connection ). While this may work fine, as intended, yet it exposes even tighter ( smaller ) time-window for the .Context()-instance to setup and (re-)negotiate all the relevant connection-details so as to become RTO.
One particular line attracted my attention: subscriber.setRcvHWM( 0 ) this is a version-archetype dependent trick. Yet, the value of zero causes an application to become vulnerable and I would not advise doing so in any production-grade application.

Digest JSON message payload

I'm not familiar to JSON and not sure if this is possible,
Background:
I'm using AWS cloudwatch and uses AWS SNS to send a HTTPS Endpoint which is Twilio. Notification are Good, Working as Expected in Hipchat, It was concise and summarize
[ALARM] saas-ajtest-mem-gt-80-5min Threshold Crossed: 2 datapoints [47.9 (07/08/17 01:10:00), 47.9 (07/08/17 01:05:00)] were greater than or equal to the threshold (30.0).
but when it was sent to my Twilio number, I will receive all the message way too long.
Type: "Notification"
MessageId" : "27684767-bf13-5e0f-b02d-15667f98e595"
TopicArn: "arn:aws:sns:us-east-1:00000000:critical_twilio"
Subject: "ALARM: "saas-ajtest-mem-gt-80-5min" in US East (N. Virginia)"
Message: {"AlarmName":"saas-ajtest-mem-gt-80-5min""AlarmDescription":null"AWSAccountId":"00000000""NewStateValue":"ALARM""NewStateReason":"Threshold Crossed: 2 datapoints [47.9 (07/08/17 01:10:00) 47.9 (07/08/17 01:05:00)] were greater than or equal to the threshold (30.0).""StateChangeTime":"2017-08-07T01:11:39.708+0000""Region":"US East (N. Virginia)""OldStateValue":"OK""Trigger":{"MetricName":"MemoryUtilization""Namespace":"test""StatisticType":"Statistic""Statistic":"AVERAGE""Unit":null"Dimensions":[{"name":"ContainerName""value":"ajtest"}]"Period":60"EvaluationPeriods":5"ComparisonOperator":"GreaterThanOrEqualToThreshold""Threshold":30.0"TreatMissingData":"""EvaluateLowSampleCountPercentile":""}}"
What I want to do is to Get only the AlarmName, Threshold Crossed,Dimensions name. Basically I want a output similar to the hipchat.
This my sample groovy script
def slurper = new groovy.json.JsonSlurper()
def result = slurper.parseText( body )
def type = result.Type
def messageID = result.MessageId
def subject = result.Subject
def message = result.Message
def alldata = type + messageID + subject + message
Tried using
def message = result.Message.AlarmName
def message = result.Message."AlarmName"
But it will return null only.
Any Inputs will be appreciated. Thanking you ahead of time :)

Actually found a solution, I've used map
Map<String, String> mappedResult = (Map<String, String>) slurper.parseText( message )
This will get the other json :)

Warn user before session time out in spring mvc

I have a web application implemented in Spring MVC, JSP having default timeout is 30 minutes.
I need to show alert in UI saying "Your session is going to end in 5 minutes. Please click OK to continue" if the session is going to expire in another 5 minutes.
How to achieve this in better way?
I found few answers here
Just interested to know if there is any other better way to do this.

try this out,below code works me fine
var cookieName = 'sessionMsg';
var message = 'Your session is Expires 5 min, click OK to continue';
function getCookie(name)
{
var name = name + "=";
var ca = document.cookie.split(';');
for(var i=0; i<ca.length; i++)
{
var c = ca[i].trim();
if (c.indexOf(name)==0) return c.substring(name.length,c.length);
}
return "";
}
function setSessionPrompt() {
var timeout = getCookie(cookieName) - new Date().getTime();
setTimeout(function(){
if (new Date().getTime() < getCookie(cookieName)) {
setSessionPrompt();
} else {
if(confirm(message)) {
// do your action here
}
}
}, timeout);
}
setSessionPrompt();