Develop Reference

Why can't Nextflow handle this awk phrase?

Background:
Using a csv as input, I want to combine the first two columns into a new one (separated by an underscore) and add that new column to the end of a new csv.
Input:
column1,column2,column3
1,2,3
a,b,c
Desired output:
column1,column2,column3,column1_column2
1,2,3,1_2
a,b,c,a_b
The below awk phrase works from the command line:
awk 'BEGIN{FS=OFS=","} {print \$0, (NR>1 ? \$1"_"\$2 : "column1_column2")}' file.csv > full_template.csv
However, when placed within a nextflow script (below) it gives an error.
#!/usr/bin/env nextflow
params.input = '/file/location/here/file.csv'
process unique {
input:
path input from params.input
output:
path 'full_template.csv' into template
"""
awk 'BEGIN{FS=OFS=","} {print \$0, (NR>1 ? \$1"_"\$2 : "combined_header")}' $input > full_template.csv
"""
}
Here is the error:
N E X T F L O W ~ version 21.10.0
Launching `file.nf` [awesome_pike] - revision: 1b63d4b438
class groovyx.gpars.dataflow.expression.DataflowInvocationExpression cannot be cast to class java.nio.file.FileSystem (groovyx.gpars.dataflow.expression.Dclass groovyx.gpars.dataflow.expression.DataflowInvocationExpression cannot be cast to class java.nio.file.FileSystem (groovyx.gpars.dataflow.expression.DataflowInvocationExpression is in unnamed module of loader 'app'; java.nio.file.FileSystem is in module java.base of loader 'bootstrap')
I'm not sure what is causing this, and any help would be appreciated.
Thanks!
Edit:
Yes it seems this was not the source of the error (sorry!). I'm trying to use splitCsv on the resulting csv and this appears to be what's causing the error. Like so:
Channel
.fromPath(template)
.splitCsv(header:true, sep:',')
.map{ row -> tuple(row.column1, file(row.column2), file(row.column3)) }
.set { split }
I expect my issue is it's not acceptable to use .fromPath on a channel, but I can't figure out how else to do it.
Edit 2:
So this was a stupid mistake. I simply needed to add the .splitCsv option directly after the input line where I invoked the channel. Hardly elegant, but appears to be working great now.
process blah {
input:
what_you_want from template.splitCsv(header:true, sep:',').map{ row -> tuple(row.column1, file(row.column2), file(row.column3)) }

I was unable to reproduce the error you're seeing with your example code and Nextflow version. In fact, I get the expected output. This shouldn't be much of a surprise though, because you have correctly escaped the special dollar variables in your AWK command. The cause of the error is likely somewhere else in your code.
If escaping the special characters gets tedious, another way is to use a shell block instead:
It is an alternative to the Script definition with an important
difference, it uses the exclamation mark ! character as the variable
placeholder for Nextflow variables in place of the usual dollar
character.
The example becomes:
params.input_csv = '/file/location/here/file.csv'
input_csv = file( params.input_csv)
process unique {
input:
path input_csv
output:
path 'full_template.csv' into template
shell:
'''
awk 'BEGIN { FS=OFS="," } { print $0, (NR>1 ? $1 "_" $2 : "combined_header") }' \\
"!{input_csv}" > "full_template.csv"
'''
}
template.view { it.text }
Results:
$ nextflow run file.nf
N E X T F L O W ~ version 20.10.0
Launching `file.nf` [wise_hamilton] - revision: b71ff1eb03
executor > local (1)
[76/ddbb87] process > unique [100%] 1 of 1 ✔
column1,column2,column3,combined_header
1,2,3,1_2
a,b,c,a_b

Get specific java version with powershell

I have some issues with getting the java version out as a string.
In a batch script I have done it like this:
for /f tokens^=2-5^ delims^=.-_^" %%j in ('%EXTRACTPATH%\Java\jdk_extract\bin\java -fullversion 2^>^&1') do set "JAVAVER=%%j.%%k.%%l_%%m"
The output is: 1.8.0_121
Now I want to do this for PowerShell, but my output is: 1.8.0_12, I miss one "1" in the end Now I have tried it with trim and split but nothing gives me the right output can someone help me out?
This is what I've got so var with PowerShell
$javaVersion = (& $extractPath\Java\jdk_extract\bin\java.exe -fullversion 2>&1)
$javaVersion = "$javaVersion".Trim("java full version """).TrimEnd("-b13")
The full output is: java full version "1.8.0_121-b13"

TrimEnd() works a little different, than you might expect:
'1.8.0_191-b12'.TrimEnd('-b12')
results in: 1.8.0_19 and so does:
'1.8.0_191-b12'.TrimEnd('1-b2')
The reason is, that TrimEnd() removes a trailing set of characters, not a substring. So .TrimEnd('-b12') means: remove all occurrences of any character of the set '-b12' from the end of the string. And that includes the last '1' before the '-'.
A better solution in your case would be -replace:
'java full version "1.8.0_191-b12"' -replace 'java full version "(.+)-b\d+"','$1'

Use a regular expression for matching and extracting the version number:
$javaVersion = if (& java -fullversion 2>&1) -match '\d+\.\d+\.\d+_\d+') {
$matches[0]
}
or
$javaVersion = (& java -fullversion 2>&1 | Select-String '\d+\.\d+\.\d+_\d+').Matches[0].Groups[0].Value

Groovy remove beginning of path

I'm trying to delete the beginning of a path that has '\' and ' ' in it. I seem to be getting the some issues saying escape character issue at character 3.
Example:
SomePath: C:\Users\ADMINISTRATOR\App Play\blah\blah
SomePath.replaceFirst('C:\\Users\\ADMINISTRATOR\\App Play\\', '');
Path should be blah\blah
I've tried:
SomePath.replaceFirst("C:\Users\ADMINISTRATOR\App Play\", "");
SomePath.replaceFirst("C:\\Users\\ADMINISTRATOR\\App Play\\", "");
SomePath.replaceFirst("C:\\\\Users\\\\ADMINISTRATOR\\\\App Play\\\\", "");
SomePath.replaceAll("C:\Users\ADMINISTRATOR\App Play\", "");
SomePath.replaceAll("C:\\Users\\ADMINISTRATOR\\App Play\\", "");
SomePath.replaceAll("C:\\\\Users\\\\ADMINISTRATOR\\\\App Play\\\\", "");

Just gave it a try... the examples with four backslashes work for me:
def somePath = "C:\\Users\\ADMINISTRATOR\\App Play\\blah\\blah"
println somePath
somePath.replaceFirst("C:\\\\Users\\\\ADMINISTRATOR\\\\App Play\\\\", "");
The problem is that the string needs one escaping \ and since the replaceFirst uses a regexp, the regexp-engine needs another \ to escape the \. The result are four backslashes.
Btw: you can use string operations to get your path, but you could also try file operations like this:
def root= new File("C:\\Users\\ADMINISTRATOR\\App Play\\")
def full= new File("C:\\Users\\ADMINISTRATOR\\App Play\\blah\\blah")
def relPath = root.toPath().relativize( full.toPath() ).toFile()
println relPath
(taken from https://gist.github.com/ysb33r/5804364)

You can tackle this problem differently. You could tokenize your input path using \ as a delimiter and then you could pick the last 2 elements (blah and blah) or skip first 4 elements (C:, Users, ADMINISTRATOR, App Play). It depends which assumption is easier to deduct for you. Consider following example:
def somePath = 'C:\\Users\\ADMINISTRATOR\\App Play\\blah\\blah'
// Build a new path by accepting the last 2 parts of the initial path
assert 'blah\\blah' == somePath.tokenize('\\')[-2..-1].join('\\')
// Build a new path by skipping the first 4 parts from initial path
assert 'blah\\blah' == somePath.tokenize('\\').drop(4).join('\\')
First option works better if you want only two last parts from the initial path. Second option works better if you can expect final path like blah\blah\blahhhh because you don't know how many nested children initial path contains and you want to start building a new path right after \App Play\ .

Using map functionality in shell script

I have below shellscript
MYMAP12=$(java -jar hello-0.0.1-SNAPSHOT.jar)
echo "==="
echo ${MYMAP12}
The output of java -jar hello-0.0.1-SNAPSHOT.jar will be map {one=one, two=two, three=three}
how to get each element from the key in shell script
I tried echo ${MYMAP12{one}} but it gave me an error

As #chepner implied, the Java code is just outputting a text string which has to be parsed and manipulated in bash to make it useful. There are no doubt several ways to do this, here is one which uses pure bash (i.e. no external programs):
# This is the text string supplied by Java
MYMAP12='{one=one, two=two, three=three}'
# Create an associative array called 'map'
declare -A map
# Remove first and last characters ( { and } )
MYMAP12=${MYMAP12#?}
MYMAP12=${MYMAP12%?}
# Remove ,
MYMAP12=${MYMAP12//,/}
# The list is now delimited by spaces, the default in a shell
for item in $MYMAP12
do
# This splits around '='
IFS='=' read key val <<< $item
map[$key]=$val
done
echo "keys: ${!map[#]}"
echo "values: ${map[#]}"
Gives:
keys: two three one
values: two three one
EDIT:
You should to use the correct tool for the job, if you need an associative array (map, hash table, dictionary) then you need a language with that feature. These include bash, ksh, awk, perl, ruby, python and C++.
You can extract the keys and values using a POSIX shell (sh) but you cannot store them in an associative array since sh does not have that feature. The best you can do is a generic list, which is just a text string of whitespace separated values. What you can do is to write a lookup function which emulates it:
get_value() {
map="$1"
key="$2"
for pair in $MYMAP12
do
if [ "$key" = "${pair%=*}" ]
then
value="${pair#*=}"
# Remove last character ( , or } )
value=${value%?}
echo "$value"
return 0
fi
done
return 1
}
MYMAP12='{kone=one, ktwo=two, kthree=three}'
# Remove first character ( { )
MYMAP12=${MYMAP12#?}
val=$(get_value "$MYMAP12" "ktwo")
echo "value for 'ktwo' is $val"
Gives:
value for 'ktwo' is two
Using this function you can also test for the presence of a key, for example:
if get_value "$MYMAP12" "kfour"
then
echo "key kfour exists"
else
echo "key kfour does not exist"
fi
Gives:
key kfour does not exist
Note that this is inefficient compared to an associative array since we are sequentially searching a list, although with a short list of only three keys you won't see any difference.

if you change your output format to the right hand side
$ x="( [one]=foo [two]=bar [three]=baz )"
then, you can use bash associative arrays
$ declare -A map="$x"
$ echo "${map[one]}"
foo

JMeter - Groovy script concatenation of variables

Groovy is the preferred scripting in JMeter
We advise using Apache Groovy or any language that supports the Compilable interface of JSR223.
The following code in JSR233 Sampler works in Java but not in Groovy
String a= "0"+"1" +
"2"
+"3";
log.info(a);
I found the reasons for + operator not to work as expected,
but what is the solution is I want to concatenate several variables to a script?
I failed to use answer of using three quotes """The row Id is: ${row.id}..."""
Currently I use Java as script language and use JMeter ${variable} although is also not recommended:
In this case, ensure the script does not use any variable using ${varName} as caching would take only first value of ${varName}
String text ="...<id>${id}</id><id2>${id2}</id2>...";
What's a better approach in groovy in such case?
EDIT:
Try using << but different error where it split to new line
String text ="<id>" <<vars["id1"] << "<id><id2>"
<< vars["id2"] << "<id2>";
Receives an error:
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
Script12.groovy: 2: unexpected token: << # line 2, column 1.
<< vars["id2"] << "<id2>";

Groovy uses the new line character to indicate end of statement except in cases where it knows the next line must extend the current one. Numerous binary operators on the start of the next line are supported. The '+' and '-' operators have binary and unary variants and currently (Groovy versions at least up to 2.5.x) don't support those operators at the start of the next line. You can place the operator on the end of the previous line (as in your first line) or use the line continuation character at the end of the previous line:
String a = "0" + "1" +
"2" \
+ "3"
log.info(a)

Why don't you use :
String text ="<id>" <<vars["id1"] << "<id><id2>" << vars["id2"] << "<id2>";
It works for me

If I had a hashmap to concat like yours, I would try:
def vars = ["id": "value", "id2": "value2", "id3": "value3"]
String text = ""
vars.each { k, v ->
text += "<${k}>${v}</${k}>"
}
println text