Arch Linux

tethys

Member

Registered: 2019-08-13

Posts: 121

[SOLVED] Search for string & replace it with a function of that string

I have a text file where I want to search for strings and replace them with other strings:

string -> $(myscript $string)

where "myscript" is a bash script.

How is it possible to do that in the easiest way?

The obvious choice is sed and searching I found this: https://www.tecmint.com/sed-number-replacement/
There is given the following example:

$ echo "Item 1 costs 100, item 2 costs 200." | sed -E 's/[0-9]+/echo $((\0 + 10))/ge'

where the option /e for sed is used.

The problem is that this does not seem to work in Arch Linux where I get the following output:

sh: line 1: Item: command not found

Is it possible to pass the found string (\0 or &) as argument to a script to be executed on the replacement side of sed?

Last edited by tethys (2024-12-30 18:25:07)

lmn

Member

Registered: 2021-05-09

Posts: 75

Re: [SOLVED] Search for string & replace it with a function of that string

Are the replacement strings know beforehand or are they dynamically computed? In other words can you precompute the substitution list and use sed like this

#!/bin/sh

replacements=""

for r in string string2 string3
do
	output=$(myscript $r)
# The separator (/ in this case) is not allowed to be part of the output. 
	replacements="$replacements -e s/$r/$output/g"
done

sed $replacements input

NB: The example you give (arithmetic on numbers) is not something I would recommend sed for.

echo "These are numbers 1 2 3 0.5 -10.0" | awk '{ for(k=1; k<=NF; k++) if($k+0 == $k){$k=$k+10}; print $0 }'

Could you provide more information on what exactly you trying to achieve? I suspect that can be done in an easier way.

Edit: Fixed awk

Last edited by lmn (2024-12-29 21:37:47)

tethys

Member

Registered: 2019-08-13

Posts: 121

Re: [SOLVED] Search for string & replace it with a function of that string

The strings are not known beforehand; they are actually numbers on which I have to perform a transformation.
Of course, one ideea would be to search for these numbers first with grep, write them in file, then do replacement with sed as you indicate in your bash script. But I was wondering if there was a simple "on the fly" solution, perhaps with sed.

dimich

Member

From: Kharkiv, Ukraine

Registered: 2009-11-03

Posts: 313

Re: [SOLVED] Search for string & replace it with a function of that string

The problem is that this does not seem to work in Arch Linux

It works in different way than you think. See this answer on stackexchange.

Is it possible to pass the found string (\0 or &) as argument to a script to be executed on the replacement side of sed?

echo "Item 1 costs 100, item 2 costs 200." | perl -pe 's/([0-9]+)/`echo -n \$(("$1" + 1))`/ge'

or with your script:

echo "Item 1 costs 100, item 2 costs 200." | perl -pe 's/([0-9]+)/`/path/to/myscript "$1"`/ge'

How is it possible to do that in the easiest way?

Probably there are simpler ways if you give an example of actual input string and expected output.

Last edited by dimich (2024-12-30 01:36:15)

tethys

Member

Registered: 2019-08-13

Posts: 121

Re: [SOLVED] Search for string & replace it with a function of that string

Thank you dimich, that was what I was looking for !
I did not find the /e option in the GNU sed, so I was suspecting that it was someting wrong with that option.

I have one more question though as I am not a perl expert.
I am trying to call the perl search & replace from a bash script, so what syntax should I use for:

perl -pe 's/([0-9]+)/`myscript "$1"`/ge'

if I want to use a local function (say "myfunc") defined in my bash script instead of "myscript" ?

Last edited by tethys (2024-12-30 10:15:03)

dimich

Member

From: Kharkiv, Ukraine

Registered: 2009-11-03

Posts: 313

Re: [SOLVED] Search for string & replace it with a function of that string

if I want to use a local function (say "myfunc") defined in my bash script instead of "myscript" ?

I don't think there is a simple solution for such case. Perl (as well as any other program) can't know whether its caller has any "functions" and how to invoke them.

One option is to move your function to a separate file.
Second option is to call your script from Perl with some extra parameter indicating that it should behave as "substitutor" (too curly, as for me).
Third option is to implement "myfunc" natively in Perl.
...

The further we go, the more it seems to me like a XY problem.

PS: neat: if actual regex is "[0-9]+" indeed, for Perl it can be shorter:

perl -pe 's/(\d+)/`myscript "$1"`/ge'

tethys

Member

Registered: 2019-08-13

Posts: 121

Re: [SOLVED] Search for string & replace it with a function of that string

Thank you dimich for your reply. I indeed did not find any solution to call a local function in the "perl -pe s///ge" expression inside a bash script. So I decided to write the whole script in perl and I was happy at the end: the perl script is more than 100 times faster than the bash script that uses sed.

For the bash script I first search for numbers using grep; then I write "s///" sed expressions in a file which at the end I call using "sed -f". The file with sed expressions is quite big, which explains the long execution time of about one minute. The perl script executes in about 200ms.

I will mark this post as solved.

mountaintrek

Member

Registered: 2024-02-01

Posts: 40

Re: [SOLVED] Search for string & replace it with a function of that string

I'm not 100% sure of the input/outout in this entry, but if I were to replace numbers using bash, I'd either use pattern substitution that support "&" (requires bash v 5.2) or a regular expression.

#!/bin/bash
### Concept Example

# Option 1: pattern substitution
shopt -s patsub_replacement  # bash v5.2
shopt -s extglob

# Option 2: Regular expression
# declare -r reDigits='([[:digit:]]+)'
# Use with [[ "${input}" =~ $reDigits ]]; then use BASH_REMATCH[n]...

while read -re input; do

  # skip comments in input
  #
  [[ "${input}" == '#'* ]] && continue

  # replace "digits" with "mystring digits"
  #
  output="${input//+([[:digit:]])/mystring &}"

  # output - write to std or redirect to file
  #
  printf -- '%s\n' "${output}"

done <<"_EOF_"
# Sample input
#
$(string 12)
$(string $string)
$(string 3)
$(string 456)
$(string 12) $(string 456)
_EOF_