[solved] parsing and displaying data with conky

Docbroke · 2016-01-30 03:19:03

Hi all,

I know that what I am asking is probably too much of spoon feeding, but any help will be greatly appreciated.

I am trying to get stock data (nifty index) to display on my desktop with conky. Source url is nse data.
This link gives index data from nseindia.com about all indices. I want to display 'last price' and 'change' from following line of above link.

{"name":"NIFTY 50","lastPrice":"7,563.55","change":"138.90",

I have almost 0 knowledge of coding, only thing I can do right now is to get this data in text file with

w3m "https://www1.nseindia.com//homepage/Indices1.json" > 123.txt

or put it into conky with

${execpi 60 w3m "https://www1.nseindia.com//homepage/Indices1.json" }

thanks for reading

Last edited by Docbroke (2016-01-30 06:06:24)

jasonwryan · 2016-01-30 04:14:23

You want jshon for that.

The excerpt you posted is not a valid json file, so I can't provide further detail. And the website is giving me a market closed message. If you paste the complete output that you want parsed then I, or someone else, can give you a hand with it. jshon has good documentation, though: it should be straightforward.

Docbroke · 2016-01-30 04:48:38

thanks for reply, I will look into jshon and post here what I can understand/do from that,

https://www1.nseindia.com//homepage/Indices1.json

Below is the full output received from above link

{"preOpen":900,"time":"Jan 29, 2016","mktStatusCode":"5","corrOpen":1540,"status":"MARKET CLOSED","haltedStatus":"","mktOpen":915,"data":[{"name":"NIFTY 50 Pre Open","lastPrice":"7,413.35","change":"-11.30","pChange":"-0.15","imgFileName":"NIFTY_50_Pre_Open_open.png"},{"name":"NIFTY 50","lastPrice":"7,563.55","change":"138.90","pChange":"1.87","imgFileName":"NIFTY_50_open.png"},{"name":"NIFTY NEXT 50","lastPrice":"18,399.00","change":"338.60","pChange":"1.87","imgFileName":"NIFTY_NEXT_50_open.png"},{"name":"NIFTY100 LIQ 15","lastPrice":"2,794.80","change":"26.85","pChange":"0.97","imgFileName":"NIFTY100_LIQ_15_open.png"},{"name":"NIFTY BANK","lastPrice":"15,522.40","change":"140.95","pChange":"0.92","imgFileName":"NIFTY_BANK_open.png"},{"name":"INDIA VIX","lastPrice":"17.2400","change":"-0.66","pChange":"-3.69","imgFileName":"INDIA_VIX_open.png"},{"name":"NIFTY 100","lastPrice":"7,659.65","change":"140.70","pChange":"1.87","imgFileName":"NIFTY_100_open.png"},{"name":"NIFTY 500","lastPrice":"6,339.45","change":"109.45","pChange":"1.76","imgFileName":"NIFTY_500_open.png"},{"name":"NIFTY MIDCAP 100","lastPrice":"12,469.10","change":"166.25","pChange":"1.35","imgFileName":"NIFTY_MIDCAP_100_open.png"},{"name":"NIFTY MIDCAP 50","lastPrice":"3,093.25","change":"30.50","pChange":"1.00","imgFileName":"NIFTY_MIDCAP_50_open.png"},{"name":"NIFTY INFRA","lastPrice":"2,424.40","change":"30.60","pChange":"1.28","imgFileName":"NIFTY_INFRA_open.png"},{"name":"NIFTY REALTY","lastPrice":"153.55","change":"2.35","pChange":"1.55","imgFileName":"NIFTY_REALTY_open.png"},{"name":"NIFTY ENERGY","lastPrice":"8,516.85","change":"135.55","pChange":"1.62","imgFileName":"NIFTY_ENERGY_open.png"},{"name":"NIFTY FMCG","lastPrice":"19,132.65","change":"233.15","pChange":"1.23","imgFileName":"NIFTY_FMCG_open.png"},{"name":"NIFTY MNC","lastPrice":"8,681.75","change":"111.85","pChange":"1.31","imgFileName":"NIFTY_MNC_open.png"},{"name":"NIFTY PHARMA","lastPrice":"11,755.85","change":"315.15","pChange":"2.75","imgFileName":"NIFTY_PHARMA_open.png"},{"name":"NIFTY PSE","lastPrice":"2,973.85","change":"57.30","pChange":"1.96","imgFileName":"NIFTY_PSE_open.png"},{"name":"NIFTY PSU BANK","lastPrice":"2,292.60","change":"-51.30","pChange":"-2.19","imgFileName":"NIFTY_PSU_BANK_open.png"},{"name":"NIFTY SERV SECTOR","lastPrice":"9,765.60","change":"167.70","pChange":"1.75","imgFileName":"NIFTY_SERV_SECTOR_open.png"},{"name":"NIFTY IT","lastPrice":"11,236.15","change":"233.45","pChange":"2.12","imgFileName":"NIFTY_IT_open.png"},{"name":"NIFTY SMALL 100","lastPrice":"5,030.45","change":"63.55","pChange":"1.28","imgFileName":"NIFTY_SMALL_100_open.png"},{"name":"NIFTY 200","lastPrice":"3,969.20","change":"70.95","pChange":"1.82","imgFileName":"NIFTY_200_open.png"},{"name":"NIFTY AUTO","lastPrice":"7,605.05","change":"147.05","pChange":"1.97","imgFileName":"NIFTY_AUTO_open.png"},{"name":"NIFTY MEDIA","lastPrice":"2,468.40","change":"77.50","pChange":"3.24","imgFileName":"NIFTY_MEDIA_open.png"},{"name":"NIFTY METAL","lastPrice":"1,658.45","change":"17.80","pChange":"1.08","imgFileName":"NIFTY_METAL_open.png"},{"name":"NIFTY DIV OPPS 50","lastPrice":"1,789.80","change":"32.80","pChange":"1.87","imgFileName":"NIFTY_DIV_OPPS_50_open.png"},{"name":"NIFTY COMMODITIES","lastPrice":"2,315.75","change":"41.10","pChange":"1.81","imgFileName":"NIFTY_COMMODITIES_open.png"},{"name":"NIFTY CONSUMPTION","lastPrice":"3,337.95","change":"60.90","pChange":"1.86","imgFileName":"NIFTY_CONSUMPTION_open.png"},{"name":"NIFTY CPSE","lastPrice":"1,943.60","change":"47.35","pChange":"2.50","imgFileName":"NIFTY_CPSE_open.png"},{"name":"NIFTY FIN SERVICE","lastPrice":"6,507.80","change":"85.75","pChange":"1.34","imgFileName":"NIFTY_FIN_SERVICE_open.png"},{"name":"NIFTY GROWSECT 15","lastPrice":"5,075.40","change":"111.20","pChange":"2.24","imgFileName":"NIFTY_GROWSECT_15_open.png"},{"name":"NIFTY50 VALUE 20","lastPrice":"3,429.25","change":"52.85","pChange":"1.57","imgFileName":"NIFTY50_VALUE_20_open.png"},{"name":"NIFTY50 TR 2X LEV","lastPrice":"3,222.60","change":"115.70","pChange":"3.72","imgFileName":"NIFTY50_TR_2X_LEV_open.png"},{"name":"NIFTY50 PR 2X LEV","lastPrice":"2,754.75","change":"98.90","pChange":"3.72","imgFileName":"NIFTY50_PR_2X_LEV_open.png"},{"name":"NIFTY50 TR 1X INV","lastPrice":"480.30","change":"-9.05","pChange":"-1.85","imgFileName":"NIFTY50_TR_1X_INV_open.png"},{"name":"NIFTY50 PR 1X INV","lastPrice":"519.55","change":"-9.80","pChange":"-1.85","imgFileName":"NIFTY50_PR_1X_INV_open.png"}],"corrClose":1600,"preClose":908,"mktClose":1530}

jasonwryan · 2016-01-30 05:20:52

jshon -e data -e 0 -e lastPrice -u -p -e change -u < file.json
7,413.35
-11.30

# edit: note, that is for the entry 0: "NIFTY 50 Pre Open" - adjust for the actual stock you want (jshon -e data < file.json will print all the entries) I think you are after 1, but there are a heap there...

For conky, something like:

printf "Price: %s Change: %s" $(jshon -e data -e 0 -e lastPrice -u -p -e change -u < file.json)
Price: 7,413.35 Change: -11.30

Docbroke · 2016-01-30 06:02:23

thanks a lot, got it working with this

${execpi 60 w3m https://www1.nseindia.com//homepage/Indices1.json | jshon -e data -e 1 -e lastPrice -u -p -e change -u}

output

7,563.55
138.90

Is that correct way (I mean it still works, but can I improve it)

Anyway marking as solved, thanks again

jasonwryan · 2016-01-30 06:05:30

I would probably run a script outside conky (as a cron job or systemd-timer) and write the result to a file in /tmp, then have conky read from that periodically. But I have no idea how efficient conky is at those execs, so it may not matter, performance wise...

Please remember to mark your thread as [Solved] by editing your first post and prepending it to the title.

Docbroke · 2016-01-30 06:15:15

Yes, that may be better idea, but beyond my capacity at present. Still need to learn a lot
Another small question, how can I get output like yours in conky, ( in single line with price: *** change: ***)

jasonwryan · 2016-01-30 06:16:27

You would need to pipe it to another utility, jshon can't do it (see the man page), which is why I used printf.

Docbroke · 2016-01-30 06:28:46

I tried to put it like this

${execpi 60 w3m https://www1.nseindia.com//homepage/Indices1.json | jshon -e data -e 1 -e lastPrice -u -p -e change -u | printf "Price: %s Change: %s" }

but it didn't work

jasonwryan · 2016-01-30 06:35:25

I don't know how conky works; you may have one too many pipes. You could try using command substitution as I did above, or just write it all to a file and read from that.

Docbroke · 2016-01-30 06:38:52

this didn't work in terminal also, so probably conky is not at fault

~ > w3m https://www1.nseindia.com//homepage/Indices1.json | jshon -e data -e 1 -e lastPrice -u -p -e change -u | printf "Price: %s Change: %s"
Price:  Change:

EDIT: Crude solution, but I think this makes me download same page twice

${color #999999}NIFTY:${color white}${execpi 60 w3m https://www1.nseindia.com//homepage/Indices1.json | jshon -e data -e 1 -e lastPrice -u}, ${color #999999}${execpi 60 w3m https://www1.nseindia.com//homepage/Indices1.json | jshon -e data -e 1 -e change -u}

Last edited by Docbroke (2016-01-30 07:20:02)

x33a · 2016-01-30 07:21:34

As Jason said, it's better to write it to a file.

Here's one way to do it:

$ w3m https://www1.nseindia.com//homepage/Indices1.json | jshon -e data -e 1 -e lastPrice -u -p -e change -u > parsed_output.txt
echo "Price: $(head -1 parsed_output.txt) Change: $(tail -1 parsed_output.txt)"

Put this inside a script and call that script from conky.

PS: I got banned from the website for making too many requests

Edit: Removed the -F switch of jshon, as it's not needed with the pipe.
Edit 2: Fixed missing double quote.

Last edited by x33a (2016-01-30 09:49:12)

Docbroke · 2016-01-30 08:19:36

Thanks, that worked perfect

EDIT: you missed ["] at the end of echo line.

Last edited by Docbroke (2016-01-30 08:29:14)

Roken · 2016-01-30 08:20:55

jasonwryan wrote:

I would probably run a script outside conky (as a cron job or systemd-timer) and write the result to a file in /tmp, then have conky read from that periodically. But I have no idea how efficient conky is at those execs, so it may not matter, performance wise...
Please remember to mark your thread as [Solved] by editing your first post and prepending it to the title.

Conky can get inefficient when running external scripts. The way around it is to get your script going, either cron, or with built in waits, format the output for conky and write it to a file in /tmp, and then simply read the /tmp file for conky. That way, conky is never waiting for a script and only has to output the text.

Docbroke · 2016-01-30 09:30:01

Final solution,
systemd service started and enabled,

[Unit]
Description=Download Nifty Value
After=network.target

[Service]
Type=simple
ExecStart=/home/sharad/bin/nifty.sh
Restart=always
RestartSec=30

nifty.sh file

#!/bin/bash

w3m https://www1.nseindia.com//homepage/Indices1.json | jshon -e data -e 1 -e lastPrice -u -p -e change -u > /tmp/parsed_output.txt
echo "NIFTY: $(head -1 /tmp/parsed_output.txt) Change: $(tail -1 /tmp/parsed_output.txt)" > /tmp/nifty.txt

and from conky config

${color #999999}${execpi 30 cat /tmp/nifty.txt}

EDIT: is there any difference in this and creating .timer file separately?

Last edited by Docbroke (2016-01-30 09:31:44)

jasonwryan · 2016-01-30 09:46:27

A timer would be the correct approach.

You could remove an additional process, like so:

printf "%s " $(awk 'FNR==1{printf "%s ", "NIFTY: "$1}; FNR==2{printf "%s", "Change: "$1}' /tmp/parsed_output.txt) > /tmp/nifty.txt

Docbroke · 2016-01-30 10:09:33

I have not enabled the service, I intend to start it during market hours and stop on the end. Therefor enabling timer doesn't seem useful. I have also created alias for easy start and stop.

Raynman · 2016-01-30 10:43:56

Docbroke wrote:

Final solution,
systemd service started and enabled,

Docbroke wrote:

I have not enabled the service

Hmm.

A better description for that service would be "repeatedly download nifty value". The only reason I can think of to do this, is to desperately avoid a second unit file for the timer, but it is also less flexible. If you make it a oneshot service without the restart part, you can start the service to run it once or you can start the timer to have it executed every 30 seconds until you stop the timer (and you can choose to really make it every 30s instead of restarting 30s after the previous run ends).

Slithery · 2016-01-30 14:19:32

Docbroke wrote:

I have not enabled the service, I intend to start it during market hours and stop on the end. Therefor enabling timer doesn't seem useful. I have also created alias for easy start and stop.

With systemd timers you can have the script run every n minutes during a specified range, for example every 5 minutes between 0900 and 1700.

Docbroke · 2016-01-30 14:35:14

Raynman wrote:

Docbroke wrote:
Final solution,
systemd service started and enabled,
Docbroke wrote:
I have not enabled the service
Hmm.

well, ideas can change in half an hour! I disabled the service as I didn't wanted it to run 24x7, when market is closed, and another issue is listed below.

Raynman wrote:

A better description for that service would be "repeatedly download nifty value". The only reason I can think of to do this, is to desperately avoid a second unit file for the timer, but it is also less flexible. If you make it a oneshot service without the restart part, you can start the service to run it once or you can start the timer to have it executed every 30 seconds until you stop the timer (and you can choose to really make it every 30s instead of restarting 30s after the previous run ends).

Thanks, understood the difference now.

slithery wrote:

With systemd timers you can have the script run every n minutes during a specified range, for example every 5 minutes between 0900 and 1700.

Thanks, will try to implement that after solving below issue.

One issue I found by enabling this service is that it conflicts with starting of my netctl profile, but running it after start of netctl is harmless. So manual start/stop is current solution.

jasonwryan · 2016-01-30 19:11:40

jasonwryan wrote:

You could remove an additional process, like so:

printf "%s " $(awk 'FNR==1{printf "%s ", "NIFTY: "$1}; FNR==2{printf "%s", "Change: "$1}' /tmp/parsed_output.txt) > /tmp/nifty.txt

For the record, this is just embarassing. With some sleep:

printf "NIFTY: %s Change: %s" $(</tmp/parsed_output.txt) > /tmp/nifty.txt

Docbroke · 2016-01-31 01:56:02

jasonwryan wrote:

For the record, this is just embarassing. With some sleep:
printf "NIFTY: %s Change: %s" $(</tmp/parsed_output.txt) > /tmp/nifty.txt

Changed to this line. I didn't tried your earlier version as it was too much geeky (read I didn't understand anything of it)! This one is in use now and confirmed working
I am having two processes running one for displaying nifty-preopen price, and second for regular price, check the screenshot!
BTW: This script is getting better and better, Candidate of 'Ooh Nice' thread !

Raynman wrote:

A better description for that service would be "repeatedly download nifty value". The only reason I can think of to do this, is to desperately avoid a second unit file for the timer, but it is also less flexible. If you make it a oneshot service without the restart part, you can start the service to run it once or you can start the timer to have it executed every 30 seconds until you stop the timer (and you can choose to really make it every 30s instead of restarting 30s after the previous run ends).

On little more thinking, I feel that using single script with manual start/stop and nice alias is better then timer. Isn't it better to start next instance of script only after exit of first instance (restart), as opposed to starting every n sec, which carries risk of running multiple instances of the same script. Though I may be wrong, but still I didn't feel any big difference!

Last edited by Docbroke (2016-01-31 02:59:47)

Raynman · 2016-01-31 09:46:23

Docbroke wrote:

On little more thinking, I feel that using single script with manual start/stop and nice alias is better then timer. Isn't it better to start next instance of script only after exit of first instance (restart), as opposed to starting every n sec, which carries risk of running multiple instances of the same script. Though I may be wrong, but still I didn't feel any big difference!

I don't see a big difference either. The nice aliases can also be used to manually start/stop the timer and with the timer you can also schedule the next run 30s after the previous one ends. As I said, using the unit types as intended provides more flexibility and the only downside I see is that you need a second unit file.

Docbroke · 2016-01-31 11:03:54

Did few more changes to the script to get different colors,

now nifty.sh file looks like this

#!/bin/bash

w3m https://www1.nseindia.com//homepage/Indices1.json | jshon -e data -e 1 -e lastPrice -u -p -e change -u > /tmp/parsed_output.txt

echo "$(head -1 /tmp/parsed_output.txt)" > /tmp/nifty1.txt
echo "$(tail -1 /tmp/parsed_output.txt)" > /tmp/nifty2.txt

and conky.conf

${color #777799}${execpi 5 cat /tmp/nifty-pre.txt}
${color white}NIFTY: ${color #11ff11}${execpi 5 cat /tmp/nifty1.txt}  ${color white}Change: ${color #ffff11}${execpi 5 cat /tmp/nifty2.txt}

Thanks to all archers

Last edited by Docbroke (2016-01-31 11:06:05)

Arch Linux

#1 2016-01-30 03:19:03

[solved] parsing and displaying data with conky

#2 2016-01-30 04:14:23

Re: [solved] parsing and displaying data with conky

#3 2016-01-30 04:48:38

Re: [solved] parsing and displaying data with conky

#4 2016-01-30 05:20:52

Re: [solved] parsing and displaying data with conky

#5 2016-01-30 06:02:23

Re: [solved] parsing and displaying data with conky

#6 2016-01-30 06:05:30

Re: [solved] parsing and displaying data with conky

#7 2016-01-30 06:15:15

Re: [solved] parsing and displaying data with conky

#8 2016-01-30 06:16:27

Re: [solved] parsing and displaying data with conky

#9 2016-01-30 06:28:46

Re: [solved] parsing and displaying data with conky

#10 2016-01-30 06:35:25

Re: [solved] parsing and displaying data with conky

#11 2016-01-30 06:38:52

Re: [solved] parsing and displaying data with conky

#12 2016-01-30 07:21:34

Re: [solved] parsing and displaying data with conky

#13 2016-01-30 08:19:36

Re: [solved] parsing and displaying data with conky

#14 2016-01-30 08:20:55

Re: [solved] parsing and displaying data with conky

#15 2016-01-30 09:30:01

Re: [solved] parsing and displaying data with conky

#16 2016-01-30 09:46:27

Re: [solved] parsing and displaying data with conky

#17 2016-01-30 10:09:33

Re: [solved] parsing and displaying data with conky

#18 2016-01-30 10:43:56

Re: [solved] parsing and displaying data with conky

#19 2016-01-30 14:19:32

Re: [solved] parsing and displaying data with conky

#20 2016-01-30 14:35:14

Re: [solved] parsing and displaying data with conky

#21 2016-01-30 19:11:40

Re: [solved] parsing and displaying data with conky

#22 2016-01-31 01:56:02

Re: [solved] parsing and displaying data with conky

#23 2016-01-31 09:46:23

Re: [solved] parsing and displaying data with conky

#24 2016-01-31 11:03:54

Re: [solved] parsing and displaying data with conky

Board footer