You are not logged in.

#126 2013-02-20 16:23:44

schultzter
Member
From: Montreal, QC
Registered: 2012-01-18
Posts: 15
Website

Re: pkgstats round two: take your vote and help improving Arch

Where I work we often use the boot disk's UUID or the first network card's MAC address to identify a machine. That's not perfect, they can be played with too, but they are typically more stable than a machine's IP address.

Is the data anonymized before it is sent? Or only once it arrives on the server? Is it sent over a secure connection? Just some questions that come to mind, I'll try to look at the code later.


Headed for the second star to the right and straight on 'til morning...

  Schultzter

Offline

#127 2013-05-09 21:01:45

maggie
Member
Registered: 2011-02-12
Posts: 255

Re: pkgstats round two: take your vote and help improving Arch

So it runs /etc/cron.weekly/pkgstats. Does that mean it runs Sunday at 00:00? What happens if my machine is switched off then (laptop)?

Offline

#128 2013-05-10 07:43:50

x33a
Forum Moderator
Registered: 2009-08-15
Posts: 4,180
Website

Re: pkgstats round two: take your vote and help improving Arch

@ maggie, Anacron?

Basically it depends on which cron implementation you are using. But most usually have mechanisms in place to run *missed jobs*.

Offline

#129 2016-01-14 22:14:34

glider
Member
Registered: 2016-01-14
Posts: 6

Re: pkgstats round two: take your vote and help improving Arch

@schultzter you miss some important points which was mentioned by @Pierre

Pierre wrote:

Not at all.

See:
1) I don't want track individual users for privacy reasons.
2) Everything that is sent by pkgstats can easily manipulated by the user without us noticing. So any idea based on sending data is flawed.
3) The IP hash is only used to prevent too easy flooding; not to track users or make the stats any more accurate.
4) There is no way to get exact values, but over time if more and more people use pkgstats some single variations (e.g. when someone sends garbage) wont matter.

Offline

#130 2016-01-15 04:52:14

ewaller
Administrator
From: Pasadena, CA
Registered: 2009-07-13
Posts: 15,062

Re: pkgstats round two: take your vote and help improving Arch

He glider, Welcome to Arch Linux.

Be aware that you just responded to a post that is coming up on three years old.


Nothing is too wonderful to be true, if it be consistent with the laws of nature -- Michael Faraday
Sometimes it is the people no one can imagine anything of who do the things no one can imagine. -- Alan Turing
----
How to Ask Questions the Smart Way

Offline

#131 2016-09-26 19:28:34

SirCmpwn
Member
Registered: 2013-09-18
Posts: 89

Re: pkgstats round two: take your vote and help improving Arch

Sorry to necropost, but this looks like the right place to ask.

Is there anywhere to get the raw data from pkgstats? The web page is a bit hard to analyze.

Offline

#132 2016-11-28 08:56:43

psycho_tea_drinker
Member
From: West Sussex, United Kingdom
Registered: 2013-07-02
Posts: 30

Re: pkgstats round two: take your vote and help improving Arch

SirCmpwn wrote:

Sorry to necropost, but this looks like the right place to ask.

Is there anywhere to get the raw data from pkgstats? The web page is a bit hard to analyze.

Yes!!! big_smile I recently made a project which scrapes the site and outputs the result as a JSON file. (Have updated the wiki at: https://wiki.archlinux.org/index.php/pkgstats with a link to the project.)

You will need to install Haskell, alternatively I've got the JSON file hosted here: http://trycatchchris.co.uk/files/packagestatistics.json. I could look into getting this dockerized, or even hosted and updated accordingly.

Do you have any project in mind for the data?

Last edited by psycho_tea_drinker (2016-11-28 09:01:29)

Offline

#133 2017-08-30 07:19:02

Pierre
Developer
From: Bonn
Registered: 2004-07-05
Posts: 1,954
Website

Re: pkgstats round two: take your vote and help improving Arch

I did a little polish of the stats page. I might extract it into its own service some day. There is a JSON export now:
* https://www.archlinux.de/statistics
* https://www.archlinux.de/statistics/package
* https://www.archlinux.de/statistics/package.json
* https://www.archlinux.de/statistics/module.json

Expect some things to change though.

Offline

#134 2017-08-30 11:14:31

WorMzy
Forum Moderator
From: Scotland
Registered: 2010-06-16
Posts: 7,260
Website

Re: pkgstats round two: take your vote and help improving Arch

Pierre wrote:

I did a little polish of the stats page. I might extract it into its own service some day. There is a JSON export now:
* https://www.archlinux.de/statistics/

This page is 404. sad


Sakura:-
Mobo: ASUS P8Z77-V PRO // Processor: Intel Core i7-3770K 3.4GHz // GFX: nVidia GeForce GTX 970 // RAM: 32GB (4x 8GB) Corsair DDR3 (@ 2133MHz) // Storage: 1x 3TB Seagate SATAII 5x 1TB Samsung SATAII, 2x 120GB Corsair SSD

Making lemonade from lemons since 2015.

Offline

#135 2017-08-30 11:17:30

Pierre
Developer
From: Bonn
Registered: 2004-07-05
Posts: 1,954
Website

Re: pkgstats round two: take your vote and help improving Arch

Yep, fixed that. Remove the trailing /.

Offline

#136 2017-09-01 00:57:30

Xavion
Member
From: Australia
Registered: 2010-03-13
Posts: 26

Re: pkgstats round two: take your vote and help improving Arch

@Pierre
With the "package.json" file:
1) How do we convert the 'count' to a percentage?  Is there a maximum value somewhere, or do we just use 'filesystem' (count = 29901) as the reference point?
2) The file is about 2.1 MiB in size.  Can you also provide a "package.json.tar.bz2"?  This would get it down to about 308 KiB.

Last edited by Xavion (2017-09-01 03:12:33)

Offline

#137 2017-09-01 10:05:43

Pierre
Developer
From: Bonn
Registered: 2004-07-05
Posts: 1,954
Website

Re: pkgstats round two: take your vote and help improving Arch

It's already gzip'ed and about 380KByte. But this is just a complete dump. Probably not that useful. I plan to decouple this API from the rest of the website and start improving on it. E.g. it would be great to asko for stats of a single package.

I still need to figure out what is going on with the numbers. Using some package as base sounds fine to me for now.

Offline

#138 2017-09-01 21:19:12

Xavion
Member
From: Australia
Registered: 2010-03-13
Posts: 26

Re: pkgstats round two: take your vote and help improving Arch

Pierre wrote:

It's already gzip'ed and about 380KByte.

What's the URL?  I want to start using it right away.

Pierre wrote:

But this is just a complete dump. Probably not that useful.

How often is the dump updated (e.g. daily or weekly)?

Offline

Board footer

Powered by FluxBB