I'm looking for a way to start up a subprocess in clojure (java is fine to) and have the output of it get sent directly to stdout in real-time. The closest I've been able to find is the Conch library for clojure, which allows you to send the output to *out*, but it doesn't actually display the output until the process is done running.
Not sure if there is a convenient Clojure wrapper for this:
(->> (.. Runtime getRuntime (exec "ls") getInputStream)
java.io.InputStreamReader.
java.io.BufferedReader.
line-seq
(map println))
It's worth noting in practice that you need to read both stdin and stderr
regularly or the process can hang when one of the buffers fill.
I marked Arthur's answer as correct since it led me to the solution, and it basically is what someone would want in the basic case. I ended up building out a larger method which does more though, and figured I'd put it here in case anyone else found it useful
(defn- print-return-stream
[stream]
(let [stream-seq (->> stream
(java.io.InputStreamReader.)
(java.io.BufferedReader.)
line-seq)]
(doall (reduce
(fn [acc line]
(println line)
(if (empty? acc) line (str acc "\n" line)))
""
stream-seq))))
(defn exec-stream
"Executes a command in the given dir, streaming stdout and stderr to stdout,
and once the exec is finished returns a vector of the return code, a string of
all the stdout output, and a string of all the stderr output"
[dir command & args]
(let [runtime (Runtime/getRuntime)
proc (.exec runtime (into-array (cons command args)) nil (File. dir))
stdout (.getInputStream proc)
stderr (.getErrorStream proc)
outfut (future (print-return-stream stdout))
errfut (future (print-return-stream stderr))
proc-ret (.waitFor proc)]
[proc-ret #outfut #errfut]
))
Related
I'm trying to read a file in a macro in Clojure.
I'm launching my macro with that line :
(def result (rd [s (FileReader. (File. "myFile.txt"))] (.read s)))
where "rd" is the name of my macro.
The prototype of my macro is like that :
(defmacro rd
([] nil)
([arg] arg)
([[variable val] expr]
)
)
The thing is that I can "execute" the FileReader, but when I'm trying to "execute" expr (.read s), it's not working because s is not known.
So I'm trying to link my elements of a vector to made s known, so I want "variable" pointed by val.
I'm not sure I'm in what I want to do, so if you see other ways, I'm up to it.
Thanks in advance guys.
if you need to read the file at runtime, as you said, you need to introduce the var.. something like this:
(defmacro rd [[variable val] expr]
`(let [~variable ~val]
~expr))
and then your macro call would expand to this:
(let [s (FileReader. (File. "myFile.txt"))] (.read s))
I am trying to read events from the event log of two windows machines. One is Windows 10 Enterprise (works perfectly) and one is Small Business Server 2008. On the SBS2008 machine output is truncated at the 78th character but I can’t see why.
This is the powershell command that I am running:
get-eventlog Application -After (Get-Date).AddDays(-10) |
Where-Object {$_.EntryType -like 'Error' -or $_.EntryType -like
'Information'} | Format-Table -Property TimeGenerated, Index,
EventID, EntryType, Source, Message -autosize | Out-String -Width 4000
It performs fine in a powershell edit window on both machines, no truncation.
If I run it in a command window using powershell -file GetEvents.ps1 the output is truncated:
C:\BITS>powershell -file GetEvents.ps1
I cannot work out what is doing this. This is the Java code:
String command = "powershell.exe " + Cmd;
Process powerShellProcess = Runtime.getRuntime().exec(command);
powerShellProcess.getOutputStream().close();
String line;
BufferedReader stdout = new BufferedReader(new InputStreamReader(powerShellProcess.getInputStream()));
while ((line = stdout.readLine()) != null) {
if (line.length() > 0) {
System.out.println(line);
}
}
Can anyone suggest a better way to get the events out of the log or suggest how I read the output from the powershell script without it being broken up by carriage returns? This was quite disappointing as I had tested it extensively on my machine (the W10 one) only to find it fails on the SBS2008 customer machine!
I have checked the libraries and java versions used on the different machines and they are the same. It’s not the println statement because I do some parsing of the string within the final ‘While’ block and the incoming line is definitely truncated.
I have subsequently tried using Get-winevent but when I put a filterhashtable flag on the command it fails when run on SBS2008 saying 'the parameter is incorrect' (works fine on W10).
My ideal would be able to get all the events from a specific logfile since a given eventID (because I can store that) but it seems to be virtually impossible to get the same output across all windows operating systems in the same format. Any suggestions welcome.
The answer for SBS2008 is here; http://www.mcbsys.com/blog/2011/04/powershell-get-winevent-vs-get-eventlog/
SBS2008 cannot use a hashtable, must use filterxml. Unfortunately when I use filterxml on SBS2008 it does not return the error message, everything else, just no the message. This is using the prescribed method of cutting and pasting the XML query from Event Viewer (https://blogs.msdn.microsoft.com/powershell/2011/04/14/using-get-winevent-filterxml-to-process-windows-events/).
After more research I have come up with a script which does (sort of) what I want. It lacks the eventindex (which is a shame) but it consistently returns the events from the System & Application eventlogs:
$fx = '<QueryList>
<Query Id="0" Path="Application">
<Select Path="Application">*[System[(Level=1 or Level=2 or Level=3) and TimeCreated[timediff(#SystemTime) <= 43200000]]]</Select>
</Query>
</QueryList>'
function Set-Culture([System.Globalization.CultureInfo] $culture) { [System.Threading.Thread]::CurrentThread.CurrentUICulture = $culture ; [System.Threading.Thread]::CurrentThread.CurrentCulture = $culture } ; Set-Culture en-US ; get-winevent -FilterXml $fx | out-string -width 470
$fx = '<QueryList>
<Query Id="0" Path="System">
<Select Path="System">*[System[(Level=1 or Level=2 or Level=3) and TimeCreated[timediff(#SystemTime) <= 43200000]]]</Select>
</Query>
</QueryList>'
Set-Culture en-US ; get-winevent -FilterXml $fx | out-string -width 470
I hope that this is useful to someone else!
I am getting the following error when I try to run my program: Exception in thread "main" java.io.FileNotFoundException: Could not locate apply
/clojure/core/vector__init.class or apply/clojure/core/vector.clj on classpath:
, compiling:(erbium/compile.clj:1:1). It seems to be pointing to file below and suggests that I need to put clojure.core/vector in my dependencies. Is it not included by default?
(ns erbium.compile
(require `[clojure.string :as string])
)
(defn produce-out "Convert 'command %1 %2 %3' with stack [5 6 7] to 'command 5 6 7'" [word stack definitions]
(let [
code (definitions word) ; dictionary/hash lookup. eg. "println" -> "echo $1"
replacement (fn [match] (-> match second Long/parseLong dec stack str))
]
; evaluate arguments. eg. "echo %1"
; stack=["blah"]
; -> "echo blah"
(string/replace code #"%(\d)" replacement)
)
)
(defn parse-word "Verifies that word is in defintitions and then returns (produce-out word stack)" [word stack definitions]
(if (some #{word} (keys definitions))
(produce-out word stack)
)
)
(defn compile "Main compile function" [code]
(let [
split-code (string/split code #"\s")
definitions {
"println" "echo %1"
"+" "%1 + %2"
"-" "%1 - %2"
}
stack []
]
(for [word [split-code]]
(if (integer? (read-string word))
(do
(println "Found integer" word)
(def stack (conj stack (read-string word)))
(println "Adding to argument stack:" stack)
)
; else
(do
(parse-word word stack definitions)
(def stack [])
)
)
)
)
)
This file is loaded by the core file via (load "compile") if that makes a difference.
the first error I see is this:
(require `[clojure.string :as string])
It should be like this:
(:require [clojure.string :as string])
in a regular clojure source file. This fixed it for me.
That said, here comes some general advices:
There are a lot of formatting "mistakes". Of course you can format your code as you would like to, however, it's easier for others to follow if you stick to basic formatting principles. Here is a nice collection of them: https://github.com/bbatsov/clojure-style-guide Most editors implement some formatting tool.
split-code (str/split code #"\s") this won't work as you required [clojure.string :as string] So change it to: split-code (string/split code #"\s")
I am not sure about (load ...) and in which context one uses that generally. However, for starting to learn clojure I recommend Lighttable as it has instant feedback built in which is very valuable when learning something new.
To expand on the answer, the "require" appears in the namespace declaration "ns". ns is actually a macro that expands to a series of statements to create the namespace, and do some setup.
This macro treats statements like (:require ...) as calls to a function with the name require, and automatically quotes any following arguments. Since you had specified a quote yourself:
(ns erbium.compile
(require '[clojure.string :as string]))
Then the result ends up being double-quoted, and the call to require ends up being:
... (require (quote (quote [clojure.string :as string])))
So it ends up trying to load a namespace called "quote" followed by a vector that was syntactically in the wrong place. :)
The ns macro is a standard and convenient way to set up namespaces, but it took me a long time to learn it properly. I found the best way was to copy other people's setup code until I learnt how to do it right.
Incidentally, the use of require instead of :require does not matter, though the standard is to use :require so it does not look like a direct call to that function.
If I try to evaluate the following code in my emacs cider-repl, nil is returned, as expected, but none of the printing takes place in the repl buffer or console. How can I make this print out as intended?
(dotimes [i 5]
(.start
(Thread.
(fn []
(Thread/sleep (rand 500))
(println (format "Finished %d on %s" i (Thread/currentThread)))))))
;=> nil
This works fine, however:
(println (format "Finished 1 on %s" (Thread/currentThread)))
;=> Finished 1 on Thread[nREPL-worker-18,5,main]
----------- mini-buffer -----------------
nil
The behavior of println is to use a dynamically bound var called *out* as its output stream. emacs dynamically binds *out* to go to the repl buffer for code evaluated in the repl buffer, but if you create a thread, that thread's *out* gets the root binding of *out*, which in the case of cider will not be the repl buffer.
If you started the repl using cider-jack-in, when you look at you buffer list there should be a buffer with a name like *nrepl-server* which contains the output of the root *out* binding. Here is the contents of mine after running your code:
nREPL server started on port 52034 on host 127.0.0.1 - nrepl://127.0.0.1:52034
Finished 1 on Thread[Thread-9,5,main]
Finished 0 on Thread[Thread-8,5,main]
Finished 2 on Thread[Thread-10,5,main]
Finished 3 on Thread[Thread-11,5,main]
Finished 4 on Thread[Thread-12,5,main]
If you did not use cider-jack-in, the output will print to the terminal where you started the nrepl process.
*out* is the dynamic variable determining where output from println and similar functions goes. It is thread-bound to someplace that causes stuff to be sent back to emacs for display by cider; if you start a new thread, that binding is not present, and the output goes elsewhere (probably to the stdout of the nrepl server emacs/leiningen started in the background).
You can address this in a few ways. You could capture the value of *out* from the parent thread, and then pass it along to the child thread in a closure, and rebind *out* to it:
(let [out *out*]
(.start (Thread. (fn []
(binding [*out* out]
(println "test"))))))
Or you can use a future instead of starting the thread yourself: Clojure automatically conveys relevant thread-local bindings to new threads started for a future.
Execute the following expression in the repl, then all output will end up in the repl:
(alter-var-root #'*out* (constantly *out*))
original answer:
https://groups.google.com/d/msg/cider-emacs/bIVBvRnGO-U/nDszDbGoVzgJ
If you are using Figwheel, then doing prn/println in ring handlers (which are actually similar to the Threads example shown above) can also be swallowed by Fighweel itself. Check your project's project.clj (look for the key :server-logfile inside the :figwheel map), where you can control if out should go to the repl or to a logfile. Please note, this only applies if you are using figwheel, otherwise the printing to the REPL of course works fine.
See my answer on this question for more details: Output compojure server print statements into figwheel terminal?
I am calling a function in a java library from jython which prints to stdout. I would like to suppress this output from the jython script. I attempt the python idiom replacing sys.stdout with a file like object (StringIO), but this does not capture the output of the java library. I'm guessing sys.stdout does not affect the java program. Is there a standard convention for redirecting or suppressing this output programatically in jython? If not what ways can I accomplish this?
You can use System.setOut, like this:
>>> from java.lang import System
>>> from java.io import PrintStream, OutputStream
>>> oldOut = System.out
>>> class NoOutputStream(OutputStream):
... def write(self, b, off, len): pass
...
>>> System.setOut(PrintStream(NoOutputStream()))
>>> System.out.println('foo')
>>> System.setOut(oldOut)
>>> System.out.println('foo')
foo
Note that this won't affect Python output, because Jython grabs System.out when it starts up so you can reassign sys.stdout as you'd expect.
I've created a context manager to mimic (Python3's) contextlib's redirect_stdout (gist here):
'''Wouldn't it be nice if sys.stdout knew how to redirect the JVM's stdout? Shooting star.
Author: Sean Summers <seansummers#gmail.com> 2015-09-28 v0.1
Permalink: https://gist.githubusercontent.com/seansummers/bbfe021e83935b3db01d/raw/redirect_java_stdout.py
'''
from java import io, lang
from contextlib import contextmanager
#contextmanager
def redirect_stdout(new_target):
''' Context manager for temporarily redirecting sys.stdout to another file or file-like object
see contextlib.redirect_stdout documentation for usage
'''
# file objects aren't java.io.File objects...
if isinstance(new_target, file):
new_target.close()
new_target = io.PrintStream(new_target.name)
old_target, target = lang.System.out, new_target
try:
lang.System.setOut(target)
yield None
finally:
lang.System.setOut(old_target)