Arch Linux

w1ll1am

Member

Registered: 2013-04-07

Posts: 19

bash script for finding URL's IP address failing

Hello.

So I have a script that takes a list of URLs and converts them to IP address. I am trying to use the host command to do this and it keeps failing with the below error

host: convert UTF-8 textname to IDN encoding: prohibited character found 

I did a google search on that an found this https://bbs.archlinux.org/viewtopic.php?id=130097 I ran the same script on an ubuntu box and didn't get the error. I am assuming that I have a problem.

Another strange thing is the command works find when I run it in the terminal so for example

host some_url_here

but when I do it in a script in a while loop it fails with the error

 while read u; do
  host $u | grep -Eo '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' >> full_ip_list.txt
done < hosts.txt

Anyone have any thoughts? Thanks

Trilby

Inspector Parrot

Registered: 2011-11-29

Posts: 30,335

Website

Re: bash script for finding URL's IP address failing

Which url does it give that error on?  You can identify this by adding an "echo $u" before the host line in your script.

Does quoting $u in the host command help?

---

"UNIX is simple and coherent" - Dennis Ritchie; "GNU's Not Unix" - Richard Stallman

w1ll1am

Member

Registered: 2013-04-07

Posts: 19

Re: bash script for finding URL's IP address failing

Which url does it give that error on?  You can identify this by adding an "echo $u" before the host line in your script.

Does quoting $u in the host command help?

Thanks for the help, I figured it out. I thought it was happening on every url becuase there were so many errors. Turns out all I needed to do was run dos2unix on the url list file and now it works. Thanks for the quick reply.

karol

Archivist

Registered: 2009-05-06

Posts: 25,440

Re: bash script for finding URL's IP address failing

Please remember to mark the thread as solved https://bbs.archlinux.org/viewtopic.php?id=130309