You are not logged in.

#1 2016-02-01 04:46:39

Docbroke
Member
From: India
Registered: 2015-06-13
Posts: 1,433

[solved] How to parse this json data?

Below is the data I am trying to parse with jshon.

indicereq([{"BValue":"24940.79","Bchg":"70.10","Bpchg":"0.28","Bdt":"20160201100414"},{"NValue":"7589.70","Nchg":"26.15","Npchg":"0.35","Ndt":"20160201100415"}])

error I am getting

~ > jshon -e data < 123.txt
json read error: line 1 column 9: '[' or '{' expected near 'indicereq'

source of data

Note: this source gives faster updates as compared to the source in my previous post, so this new request!

thanks for reading

Last edited by Docbroke (2016-02-01 10:12:08)

Offline

#2 2016-02-01 06:04:22

jasonwryan
Anarchist
From: .nz
Registered: 2009-05-09
Posts: 30,424
Website

Re: [solved] How to parse this json data?

That isn't valid json...


Arch + dwm   •   Mercurial repos  •   Surfraw

Registered Linux User #482438

Offline

#3 2016-02-01 06:08:31

Docbroke
Member
From: India
Registered: 2015-06-13
Posts: 1,433

Re: [solved] How to parse this json data?

thanks, but that is the data provided by website, (link in first post)

Is it some other type of data? can it be parsed some other way or converted to json?

Last edited by Docbroke (2016-02-01 06:09:06)

Offline

#4 2016-02-01 06:14:54

jasonwryan
Anarchist
From: .nz
Registered: 2009-05-09
Posts: 30,424
Website

Re: [solved] How to parse this json data?

I don't know what data type it is. You could parse it with awk or sed, but it would be fragile if it changes over time.


Arch + dwm   •   Mercurial repos  •   Surfraw

Registered Linux User #482438

Offline

#5 2016-02-01 07:02:12

Docbroke
Member
From: India
Registered: 2015-06-13
Posts: 1,433

Re: [solved] How to parse this json data?

it becomes valid json if we can only keep data from "[" to "]"

invalid json

indicereq([{"BValue":"24940.79","Bchg":"70.10","Bpchg":"0.28","Bdt":"20160201100414"},{"NValue":"7589.70","Nchg":"26.15","Npchg":"0.35","Ndt":"20160201100415"}])

Valid json

[{"BValue":"24940.79","Bchg":"70.10","Bpchg":"0.28","Bdt":"20160201100414"},{"NValue":"7589.70","Nchg":"26.15","Npchg":"0.35","Ndt":"20160201100415"}]

Offline

#6 2016-02-01 07:30:13

jasonwryan
Anarchist
From: .nz
Registered: 2009-05-09
Posts: 30,424
Website

Re: [solved] How to parse this json data?

Strip it off before processing then:

sed 's/indicereq//;s/[()]//g'

Arch + dwm   •   Mercurial repos  •   Surfraw

Registered Linux User #482438

Offline

#7 2016-02-01 07:40:34

Docbroke
Member
From: India
Registered: 2015-06-13
Posts: 1,433

Re: [solved] How to parse this json data?

worked great

w3m http://indices.moneycontrol.co.in/sensex_nifty/indtick.json?callback=jsonp1454300599978 | sed 's/indicereq//;s/[()]//g' |jshon -e 1 -e NValue -u -p -e Nchg -u > /tmp/parsed_output.txt

I was reading 'info sed' and sed - tutorial but reache only upto "sed -n '/PATTERN/p'", where I was not able to define the pattern.

thanks

BTW: is your name related with 'json'?

Offline

#8 2016-02-01 07:44:15

jasonwryan
Anarchist
From: .nz
Registered: 2009-05-09
Posts: 30,424
Website

Re: [solved] How to parse this json data?

It is pronounced the same, which given I work with web developers, is extremely annoying...

Also stop using w3m to retrieve that string: use curl.


Arch + dwm   •   Mercurial repos  •   Surfraw

Registered Linux User #482438

Offline

#9 2016-02-01 08:43:40

Raynman
Member
Registered: 2011-10-22
Posts: 1,539

Re: [solved] How to parse this json data?

Docbroke wrote:

thanks, but that is the data provided by website, (link in first post)

Is it some other type of data? can it be parsed some other way or converted to json?

It's a form of JSONP.

Offline

#10 2016-02-01 09:09:49

Docbroke
Member
From: India
Registered: 2015-06-13
Posts: 1,433

Re: [solved] How to parse this json data?

jasonwryan wrote:

It is pronounced the same, which given I work with web developers, is extremely annoying...

Also stop using w3m to retrieve that string: use curl.

thanks, changed to curl from w3m.

Offline

#11 2016-02-01 12:59:11

Docbroke
Member
From: India
Registered: 2015-06-13
Posts: 1,433

Re: [solved] How to parse this json data?

curl doesn't work on every site

~ > curl https://www1.nseindia.com//homepage/Indices1.json
<HTML><HEAD>
<TITLE>Access Denied</TITLE>
</HEAD><BODY>
<H1>Access Denied</H1>
 
You don't have permission to access "http&#58;&#47;&#47;www1&#46;nseindia&#46;com&#47;homepage&#47;Indices1&#46;json" on this server.<P>
Reference&#32;&#35;18&#46;5187d317&#46;1454331129&#46;c6a8455
</BODY>
</HTML>

same link works with w3m

However curl works with another json-data site

~ > curl http://indices.moneycontrol.co.in/sensex_nifty/indtick.json?callback=jsonp1454300599978
indicereq([{"BValue":"24824.83","Bchg":"-45.86","Bpchg":"-0.18","Bdt":"20160201153044"},{"NValue":"7555.95","Nchg":"-7.60","Npchg":"-0.10","Ndt":"20160201153048"}])

Not an issue, posting to understand the difference if someone can throw light on this.

Offline

#12 2016-02-01 13:30:33

Raynman
Member
Registered: 2011-10-22
Posts: 1,539

Re: [solved] How to parse this json data?

Works with curl if I add "--user-agent Mozilla/4.0".

Offline

#13 2016-02-02 07:11:02

x33a
Forum Fellow
Registered: 2009-08-15
Posts: 4,587

Re: [solved] How to parse this json data?

@Docbroke,

some sites will deliberately block user agents such as curl or wget to prevent programmatic access to their resources. What they don't realize is that user agents can be easily spoofed, as Raynman demonstrated. To be on a safer side, try with an even newer user agent (not specific to this site, but in general).

Offline

Board footer

Powered by FluxBB