You are not logged in.
I have a bash script that expects three tokens that I'd like to push through parallel. The problem I am experiencing is that when I cat a text file and pipe it into parallel, the spaces in the tokens are not treated as such; they are treated as a single token.
Here is a simplified script: ~/bin/action
#!/bin/bash
echo "first = $1"
echo "second = $2"
echo "third = $3"
If I simply run it like so, I get the expected output:
% ~/bin/action a b c
first = a
second = b
third = c
So to use parallel, I place the "a b c" into a text file that I cat and pipe into parallel but all of the tokens become $1 as you can see:
% echo "a b c" > worklist
% cat worklist | parallel ~/bin/action
first = a b c
second =
third =
Please help me understand the error of my way
Last edited by graysky (2012-11-24 14:38:40)
CPU-optimized Linux-ck packages @ Repo-ck • AUR packages • Zsh and other configs
Offline
Did you not find the manpage?
--delimiter delim
-d delim
Input items are terminated by the specified character. Quotes and backslash are not special; every character in the input is taken literally. Disables the end-of-file string, which is treated like any other argument. This can be used when the input consists of simply newline-separated items, although it is almost always better to design your program to use --null where this is possible. The specified delimiter may be a single character, a C-style character escape such as \n, or an octal or hexadecimal escape code. Octal and hexadecimal escape codes are understood as for the printf command. Multibyte characters are not supported.
Last edited by falconindy (2012-11-22 16:03:07)
Offline
@falconindy - I saw that but am unclear how to make use of it to treat items in a line as tokens... for example: my goal is to have a whole list of items in the worklist and have them treated as $1 $2 and $3 on a per-line basis.
% cat worklist
a b c
d e f
g h i
j k l
m n o
p q r
s t u
v w x
y z 1
So the first command should be processed as:
~/bin/action a b c
The 2nd as:
~/bin/action d e f
The 3rd as:
~/bin/action g h i
...you get the idea.
Last edited by graysky (2012-11-22 16:12:52)
CPU-optimized Linux-ck packages @ Repo-ck • AUR packages • Zsh and other configs
Offline
I would think this would be easier just using bash, then...
#!/bin/bash
worker() {
dostuff "$@"
}
maxjobs=4
while read -ra v; do
# ratelimit max inflight jobs
while (( $(jobs | wc -l) >= maxjobs )); do
sleep 0.1
jobs >/dev/null
done
# dispatch in the background
worker "${v[@]}" &
done <"$inputfile"
# let children finish before the script exits
wait
Offline
@falconindy - Thank you for the suggestion. It seems that the correct way to invoke parallel in the fashion I want is to use the -a switch so you were close
parallel -a worklist --colsep ' ' ~/bin/action
first = a
second = b
third = c
first = d
second = e
third = f
first = g
second = h
third = i
first = j
second = k
third = l
first = m
second = n
third = o
first = p
second = q
third = r
first = s
second = t
third = u
first = v
second = w
third = x
first = y
second = z
third = 1
http://www.gnu.org/software/parallel/ma … e_as_input
Powerful stuff once you learn how to invoke it
Last edited by graysky (2012-11-24 15:11:40)
CPU-optimized Linux-ck packages @ Repo-ck • AUR packages • Zsh and other configs
Offline