I have a lucene index that i build and update using raw lucene indexers. I was wondering if there is a way to force solr to re-read the index without restarting the solr instance. Ive tried the update?commit=true but it doesnt seem to matter. The only way i can be sure solr -re-reads the index is by a total restart, which of course is not ideal in a production environment.
If you are using a multi-core setup, you can just reload that single core. AFAIK, while the core is being reloaded, the requests to that core are queued.
Not sure if there is another better way to do it.
But I wrote this script to perform full or delta imports.
#!/bin/bash
# script to index Solr
# by Felipe Ferreira Sept 2013
TYPE=$1
DATE=`date +%d_%m_%y`
DATEFULL=`date +%H:%M:%S_%d_%m_%y`
LOG="/var/log/solr/solr_import_${TYPE}_${DATE}.log"
LOGTMP="/var/log/solr/solr_status_${DATE}.log"
URL="http://<SERVER>:8080/solr/dataimport?command=status"
pt() {
echo -e $1
echo -e $1 >> $LOG
}
if [ $TYPE == "full" ]; then
pt "$DATEFULL - Starting full import"
URL="http://<SERVER>:8080/solr/dataimport?command=full-import"
# CMD="curl --location --no-buffer --fail --silent --output ${LOG} '${URL}'"
CMD="curl --location --silent --no-buffer '${URL}' >> $LOG"
pt "Executing $CMD"
CMD_E=`eval $CMD`
pt $CMD_E
elif [ $TYPE == "delta" ]; then
pt "$DATEFULL - Starting delta import"
URL="http://<SERVER>:8080/solr/dataimport?command=delta-import"
#CMD="curl --location --no-buffer --fail --silent --output ${LOG} '${URL}'"
CMD="curl --location --silent --no-buffer '${URL}' >> $LOG"
pt "Executing $CMD"
CMD_E=`eval $CMD`
pt $CMD_E
else
pt "ERROR - $TYPE not found, only delta or full,\n Usage: $0 delta/full"
fi
sleep 3
#check status after command
pt "$DATEFULL - Checking $TYPE status"
URL="http://infofish2:9080/solrcadin/cadin/dataimport?command=status"
CMD="curl --location --silent --no-buffer '${URL}' > $LOGTMP"
pt "Executing $CMD"
CMD_E=`eval $CMD`
pt $CMD_E
CHECK=0
CHECK=`grep -c failed $LOGTMP`
if [ $CHECK -eq 0 ]; then
pt "OK - Command $TYPE executed with success!"
exit 0
else
pt "CRITICAL - Command $TYPE failed, solr did not index!"
pt "grep -c failed $LOGTMP"
pt "CHECK $CHECK"
exit 0
fi
Related
I have a part in init.pp, that starts my app:
systemd::service {'app':
user => 'java',
exec => "${java_home}/bin/java \
-server \
'-XX:OnOutOfMemoryError=\\'kill -9 %%p\\'' \
-Dmw.config=/etc/app/app.config \
-cp /usr/share/app/app.jar ru.app.main.Main app",
}
The newer version of the app.jar starts differently. There's a way, stop the puppet on the nodes, deregister the node from the consul, start the puppet with the new init.pp, and put that node back to the consul. But that just doesn't feel right.
Is there a way to put some "if" in there so I'd be able to make smooth transitions back and forth depending on my app's version or any other field?
found it
if $app_v2 == 'true' {
systemd::service {'app':
user => 'java',
exec => "${java_home}/bin/java \
-server \
'-XX:OnOutOfMemoryError=\\'kill -9 %%p\\'' \
-Dmw.config=/etc/app/app_new.config \
-cp /usr/share/app/app.jar ru.app.main.App app",
}
} else {
systemd::service {'app':
user => 'java',
exec => "${java_home}/bin/java \
-server \
'-XX:OnOutOfMemoryError=\\'kill -9 %%p\\'' \
-Dmw.config=/etc/app/app.config \
-cp /usr/share/app/app.jar ru.app.main.Main app",
}
}
and for getting app_v2 flag, add to /lib/facter/app_v2.rb
require 'facter'
Facter.add(:app_v2) do
confine :kernel => :Linux
setcode do
result = false
if Facter::Core::Execution.exec('dpkg -l | grep app|awk \'{print$3}\'| cut -c1') == '2'
result = true
end
result
end
end
I am trying to execute a perl script via a cron job that uses the system command to execute a java wrapper for boilerpipe. It fails when executed via cron. The script (simplified) is
my $link = "http://foo.bar"; # with a real link
my $cmdstring = 'java -jar boilerpipe-wrapper.jar url '.$link.' out.txt';
my $result = system($cmdstring);
if ($? == -1) {
print LOG "failed to execute system command: $!\n";
die;
}
elsif ($? & 127) {
printf LOG "child died with signal %d, %s coredump\n",
($? & 127), ($? & 128) ? 'with' : 'without';
die;
}
else {
printf LOG "child exited with value %d\n", $? >> 8;
die;
}
The die commands are in there for debugging purposes. What appears in LOG is
child exited with value 1
The crontab line is
24 7 * * * perl /home/retrievetext.pl >> /home/cron-msgs 2>&1
What appears in cron-msgs is
Error: unable to access jarfile boilerpipe-wrapper.jar
Died at /home/retrievetext.pl line 15
The ls -la for boilerpipe-wrapper.jar is
-rwxr-xr-x 1 steve sudo 1945275 Apr 29 03:53 boilerpipe-wrapper.jar
The Perl script and the system call work as expected when executed from the terminal. Anyone know what the problem is?
While using java "setOptions" function with the bellow properties I am getting an error : "Quote parse error."...
"weka.classifiers.meta.OneClassClassifier -num \"weka.classifiers.meta.generators.GaussianGenerator -S 1 -M 0.0 -SD 1.0\" -nom \"weka.classifiers.meta.generators.NominalGenerator -S 1\" -trr 0.1 -tcl F&B -cvr 10 -cvf 10.0 -P 0.5 -S 1 -W weka.classifiers.meta.Bagging -- -P 100 -S 1 -num-slots 1 -I 10 -W weka.classifiers.trees.REPTree -- -M 2 -V 0.001 -N 3 -S 1 -L -1 -I 0.0"
I understand that this is occurring because of error in applying quotes, but I am not sure as to where they should be applied.
I thought of applying them before calling REPTree but that doesn't seem to work.
Whenever using a Weka classifier with Java code, use the weka.core.Utils.splitOptions method to parse the config string without worrying about quoting and escaping characters.
// set your configurations parameters here, e.g., "-S 1"
String options = "whatever configuration you want";
// instantiate the classifier object
REPTree tree = new REPTree();
// parse and set the classifier's configuration
tree.setOptions(Utils.splitOptions(options));
Also, remember to provide your current code and to format your question.
First time I'm doing an hook like this..
I need a pre-commit hook that scan all the java classes to commit, it should check for the presence of some character into the class and avoid the commit if found some of them, chars like † or ¥ and so on, i think a good way to make this dynamically change could be put all these invalid chars into a plan file in order to change it easily if we need to...
I'm starting from a simple hook that i wrote long time ago..
Now the BIG problem is getting the location of the working copy files..
The one I should scan the content.
I tried many svnlook commands, but I'm really unable to catch this information into the pre-commit hook....
Getting a lot of information but not the local path of the file. I'm using this to scan for content...
OUTPUT="$($SVNLOOK changed -t $TXN $REPOS)"
echo $SVNLOOK changed -t $TXN $REPOS 1>&2
echo "$BASEDIR" 1>&2
echo "${OUTPUT}" 1>&2
echo "$TXN $REPOS" 1>&2
Maybe it is my approach that is wrong?
Thanks a lot!
UPDATED
Thanks "CaffeineAddiction", you know it is always a "BIG QUESTION" when you do something for the first time.
In reality, the real issue in the end, after one day of attempts was another one, a SVN Bug related to the client char coding:
Error output could not be translated from the native locale to UTF-8
Now also this last issue is solved and the script works as well, you can see it below, it just need to be beautified, by the way thanks for yours, i'll get some ideas from yours:
REPOS="$1"
TXN="$2"
SVNLOOK=/usr/bin/svnlook
OUTPUT="$($SVNLOOK changed -t $TXN $REPOS | awk '{print $2}')"
for LINE in $OUTPUT
do
FILE=`echo $LINE`
MESSAGE=`$SVNLOOK cat -t "$TXN" "$REPOS" "${FILE}"`
echo "File is: $FILE" 1>&2
echo "${MESSAGE}" > /tmp/app.txt
grep -P -n '[\x80-\xFF]' /tmp/app.txt | cut -f1 -d: 1>&2
done
This is not a full answer, but it might be enough to get you headed in the right direction. A while back I was asked to run gjslint against javascript files before allowing them to be checked into SVN. Here is the pre-hook I used for the task:
#!/bin/sh
SVNLOOK=/usr/bin/svnlook
GJSLINT=/usr/local/bin/gjslint
ECHO=$(which echo)
GREP=$(which grep)
SED=$(which sed)
## Used for Debug
#MYRUNLOG=/run/svn-pre-commit/pre-commit.log
#touch $MYRUNLOG
#echo "" > $MYRUNLOG
MYTEMPJS=/run/svn-pre-commit/temp.js
touch $MYTEMPJS
echo "" > $MYTEMPJS
MYTEMPLOG=/run/svn-pre-commit/gjslint.log
touch $MYTEMPLOG
echo "" > $MYTEMPLOG
REPOS="$1"
TXN="$2"
FILES_CHANGED=`$SVNLOOK changed -t$TXN $REPOS | $SED -e "s/^....//g"`
LINTERROR=0
for FILE in $FILES_CHANGED
do
if $ECHO $FILE | $GREP "\.js$"
then
if ! $ECHO "$REPOS/$FILE" | $GREP "/paweb5/\|/pamc/"; then exit 0; fi
if $ECHO "$REPOS/$FILE" | $GREP "/doc/"; then exit 0; fi
if $ECHO "$REPOS/$FILE" | $GREP "/docs/"; then exit 0; fi
$SVNLOOK cat -t$TXN $REPOS $FILE > $MYTEMPJS
$ECHO "$REPO/$FILE" >> $MYTEMPLOG
$GJSLINT --strict --disable 0001 $MYTEMPJS >> $MYTEMPLOG
GJSL_ERROR_CODE=$?
if [ $GJSL_ERROR_CODE != 0 ]
then
LINTERROR=1
fi
$ECHO "~~~" >> $MYTEMPLOG
fi
done
if [ $LINTERROR != 0 ]
then
echo "..........................................................................." >&2
while read line; do
if $ECHO $line | $GREP "Line\|no errors\|new errors\|paweb5\|~~~"
then
echo $line >&2
fi
done < $MYTEMPLOG
echo "..........................................................................." >&2
exit 1
fi
# If we got here, nothing is wrong.
exit 0
I believe the answer to your "BIG problem" getting the location of the working copy files might lie within $SVNLOOK cat -t$TXN $REPOS $FILE > $MYTEMPJS
If you have questions about the script feel free to ask
I'm working on a configuration script for a JNI wrapper. One of the configuration parameters is the path to jni.h. What's a good quick-and-dirty Autoconf test for whether this parameter is set correctly for C++ compilation? You can assume you're running on Linux and g++ is available.
Alternatively, is there a way to get javah (or a supporting tool) to give me this path directly?
Then there is the easy way: http://www.gnu.org/software/autoconf-archive/ax_jni_include_dir.html
Sometimes it is best to just use the standard recipies.
Checking for headers is easy; just use AC_CHECK_HEADER. If it's in a weird place (i.e., one the compiler doesn't know about), it's entirely reasonable to expect users to set CPPFLAGS.
The hard part is actually locating libjvm. You typically don't want to link with this; but you may want to default to a location to dlopen it from if JAVA_HOME is not set at run time.
But I don't have a better solution than requiring that JAVA_HOME be set at configure time. There's just too much variation in how this stuff is deployed across various OSes (even just Linux distributions). This is what I do:
AC_CHECK_HEADER([jni.h], [have_jni=yes])
AC_ARG_VAR([JAVA_HOME], [Java Runtime Environment (JRE) location])
AC_ARG_ENABLE([java-feature],
[AC_HELP_STRING([--disable-java-feature],
[disable Java feature])])
case $target_cpu in
x86_64) JVM_ARCH=amd64 ;;
i?86) JVM_ARCH=i386 ;;
*) JVM_ARCH=$target_cpu ;;
esac
AC_SUBST([JVM_ARCH])
AS_IF([test X$enable_java_feature != Xno],
[AS_IF([test X$have_jni != Xyes],
[AC_MSG_FAILURE([The Java Native Interface is required for Java feature.])])
AS_IF([test -z "$JAVA_HOME"],
[AC_MSG_WARN([JAVA_HOME has not been set. JAVA_HOME must be set at run time to locate libjvm.])],
[save_LDFLAGS=$LDFLAGS
LDFLAGS="-L$JAVA_HOME/lib/$JVM_ARCH/client -L$JAVA_HOME/lib/$JVM_ARCH/server $LDFLAGS"
AC_CHECK_LIB([jvm], [JNI_CreateJavaVM], [LIBS=$LIBS],
[AC_MSG_WARN([no libjvm found at JAVA_HOME])])
LDFLAGS=$save_LDFLAGS
])])
FYI - the patch below against the latest ax_jni_include_dir.m4 works for me on Macos 11.1.
--- a/m4/ax_jni_include_dir.m4
+++ b/m4/ax_jni_include_dir.m4
## -73,13 +73,19 ## fi
case "$host_os" in
darwin*) # Apple Java headers are inside the Xcode bundle.
- macos_version=$(sw_vers -productVersion | sed -n -e 's/^#<:#0-9#:>#
*.\(#<:#0-9#:>#*\).#<:#0-9#:>#*/\1/p')
- if #<:# "$macos_version" -gt "7" #:>#; then
- _JTOPDIR="$(xcrun --show-sdk-path)/System/Library/Frameworks/JavaVM.framework"
- _JINC="$_JTOPDIR/Headers"
+ major_macos_version=$(sw_vers -productVersion | sed -n -e 's/^\(#<:#0-9#:>#*\).#<:#0-9#:>#*.#<:#0-9#:>#*/\1/p')
+ if #<:# "$major_macos_version" -gt "10" #:>#; then
+ _JTOPDIR="$(/usr/libexec/java_home)"
+ _JINC="$_JTOPDIR/include"
else
- _JTOPDIR="/System/Library/Frameworks/JavaVM.framework"
- _JINC="$_JTOPDIR/Headers"
+ macos_version=$(sw_vers -productVersion | sed -n -e 's/^#<:#0-9#:>#*.\(#<:#0-9#:>#*\).#<:#0-9#:>#*/\1/p')
+ if #<:# "$macos_version" -gt "7" #:>#; then
+ _JTOPDIR="$(xcrun --show-sdk-path)/System/Library/Frameworks/JavaVM.framework"
+ _JINC="$_JTOPDIR/Headers"
+ else
+ _JTOPDIR="/System/Library/Frameworks/JavaVM.framework"
+ _JINC="$_JTOPDIR/Headers"
+ fi
fi
;;
*) _JINC="$_JTOPDIR/include";;