You are not logged in.

#126 2013-02-20 16:23:44

schultzter
Member
From: Montreal, QC
Registered: 2012-01-18
Posts: 15
Website

Re: pkgstats round two: take your vote and help improving Arch

Where I work we often use the boot disk's UUID or the first network card's MAC address to identify a machine. That's not perfect, they can be played with too, but they are typically more stable than a machine's IP address.

Is the data anonymized before it is sent? Or only once it arrives on the server? Is it sent over a secure connection? Just some questions that come to mind, I'll try to look at the code later.


Headed for the second star to the right and straight on 'til morning...

  Schultzter

Offline

#127 2013-05-09 21:01:45

maggie
Member
Registered: 2011-02-12
Posts: 255

Re: pkgstats round two: take your vote and help improving Arch

So it runs /etc/cron.weekly/pkgstats. Does that mean it runs Sunday at 00:00? What happens if my machine is switched off then (laptop)?

Offline

#128 2013-05-10 07:43:50

x33a
Forum Moderator
Registered: 2009-08-15
Posts: 4,450
Website

Re: pkgstats round two: take your vote and help improving Arch

@ maggie, Anacron?

Basically it depends on which cron implementation you are using. But most usually have mechanisms in place to run *missed jobs*.

Offline

#129 2016-01-14 22:14:34

glider
Member
Registered: 2016-01-14
Posts: 6

Re: pkgstats round two: take your vote and help improving Arch

@schultzter you miss some important points which was mentioned by @Pierre

Pierre wrote:

Not at all.

See:
1) I don't want track individual users for privacy reasons.
2) Everything that is sent by pkgstats can easily manipulated by the user without us noticing. So any idea based on sending data is flawed.
3) The IP hash is only used to prevent too easy flooding; not to track users or make the stats any more accurate.
4) There is no way to get exact values, but over time if more and more people use pkgstats some single variations (e.g. when someone sends garbage) wont matter.

Offline

#130 2016-01-15 04:52:14

ewaller
Administrator
From: Pasadena, CA
Registered: 2009-07-13
Posts: 15,687

Re: pkgstats round two: take your vote and help improving Arch

He glider, Welcome to Arch Linux.

Be aware that you just responded to a post that is coming up on three years old.


Nothing is too wonderful to be true, if it be consistent with the laws of nature -- Michael Faraday
Sometimes it is the people no one can imagine anything of who do the things no one can imagine. -- Alan Turing
---
How to Ask Questions the Smart Way

Offline

#131 2016-09-26 19:28:34

SirCmpwn
Member
Registered: 2013-09-18
Posts: 89

Re: pkgstats round two: take your vote and help improving Arch

Sorry to necropost, but this looks like the right place to ask.

Is there anywhere to get the raw data from pkgstats? The web page is a bit hard to analyze.

Offline

#132 2016-11-28 08:56:43

psycho_tea_drinker
Member
From: West Sussex, United Kingdom
Registered: 2013-07-02
Posts: 37

Re: pkgstats round two: take your vote and help improving Arch

SirCmpwn wrote:

Sorry to necropost, but this looks like the right place to ask.

Is there anywhere to get the raw data from pkgstats? The web page is a bit hard to analyze.

Yes!!! big_smile I recently made a project which scrapes the site and outputs the result as a JSON file. (Have updated the wiki at: https://wiki.archlinux.org/index.php/pkgstats with a link to the project.)

You will need to install Haskell, alternatively I've got the JSON file hosted here: http://trycatchchris.co.uk/files/packagestatistics.json. I could look into getting this dockerized, or even hosted and updated accordingly.

Do you have any project in mind for the data?

Last edited by psycho_tea_drinker (2016-11-28 09:01:29)

Offline

#133 2017-08-30 07:19:02

Pierre
Developer
From: Bonn
Registered: 2004-07-05
Posts: 1,954
Website

Re: pkgstats round two: take your vote and help improving Arch

I did a little polish of the stats page. I might extract it into its own service some day. There is a JSON export now:
* https://www.archlinux.de/statistics
* https://www.archlinux.de/statistics/package
* https://www.archlinux.de/statistics/package.json
* https://www.archlinux.de/statistics/module.json

Expect some things to change though.

Offline

#134 2017-08-30 11:14:31

WorMzy
Forum Moderator
From: Scotland
Registered: 2010-06-16
Posts: 7,915
Website

Re: pkgstats round two: take your vote and help improving Arch

Pierre wrote:

I did a little polish of the stats page. I might extract it into its own service some day. There is a JSON export now:
* https://www.archlinux.de/statistics/

This page is 404. sad


Sakura:-
Mobo: MSI X299 TOMAHAWK ARCTIC // Processor: Intel Core i7-7820X 3.6GHz // GFX: nVidia GeForce GTX 970 // RAM: 32GB (4x 8GB) Corsair DDR4 (@ 3000MHz) // Storage: 1x 3TB HDD, 5x 1TB HDD, 2x 120GB SSD, 1x 275GB M2 SSD

Making lemonade from lemons since 2015.

Offline

#135 2017-08-30 11:17:30

Pierre
Developer
From: Bonn
Registered: 2004-07-05
Posts: 1,954
Website

Re: pkgstats round two: take your vote and help improving Arch

Yep, fixed that. Remove the trailing /.

Offline

#136 2017-09-01 00:57:30

Xavion
Member
From: Australia
Registered: 2010-03-13
Posts: 28

Re: pkgstats round two: take your vote and help improving Arch

@Pierre
With the "package.json" file:
1) How do we convert the 'count' to a percentage?  Is there a maximum value somewhere, or do we just use 'filesystem' (count = 29901) as the reference point?
2) The file is about 2.1 MiB in size.  Can you also provide a "package.json.tar.bz2"?  This would get it down to about 308 KiB.

Last edited by Xavion (2017-09-01 03:12:33)

Offline

#137 2017-09-01 10:05:43

Pierre
Developer
From: Bonn
Registered: 2004-07-05
Posts: 1,954
Website

Re: pkgstats round two: take your vote and help improving Arch

It's already gzip'ed and about 380KByte. But this is just a complete dump. Probably not that useful. I plan to decouple this API from the rest of the website and start improving on it. E.g. it would be great to asko for stats of a single package.

I still need to figure out what is going on with the numbers. Using some package as base sounds fine to me for now.

Offline

#138 2017-09-01 21:19:12

Xavion
Member
From: Australia
Registered: 2010-03-13
Posts: 28

Re: pkgstats round two: take your vote and help improving Arch

Pierre wrote:

It's already gzip'ed and about 380KByte.

What's the URL?  I want to start using it right away.

Pierre wrote:

But this is just a complete dump. Probably not that useful.

How often is the dump updated (e.g. daily or weekly)?

Offline

#139 2017-12-27 14:09:36

ua4000
Member
Registered: 2015-10-14
Posts: 96

Re: pkgstats round two: take your vote and help improving Arch

archlinux32.org project also offers pkgstats package and seems to report it's summary also to https://pkgstats.archlinux.de/post

The data can be distinguish by arch="$(uname -m)" -  is there a way on https://pkgstats.archlinux.de/package to filter for i686 usage ?
Or should archlinux32.org project don't provide the current pkgstats package and implement a fork with another post server ?

Thanks.

Offline

#140 2017-12-27 14:15:00

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 19,089
Website

Re: pkgstats round two: take your vote and help improving Arch

ua4000 wrote:

is there a way on https://pkgstats.archlinux.de/package to filter for i686 usage?

What do you mean filter for it?  In old data maybe, but in current data: yes all i686 is already filtered out!


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#141 2017-12-28 08:42:32

ua4000
Member
Registered: 2015-10-14
Posts: 96

Re: pkgstats round two: take your vote and help improving Arch

filtered = discarded - i686 statistics are now not available anymore ?

I rewrite my question: does it make sense to use and run pkgstats package (same as on archlinux.org) in archlinux32 ?

Offline

#142 2017-12-28 12:14:18

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 19,089
Website

Re: pkgstats round two: take your vote and help improving Arch

I suppose it does collect the architecture along with everything else, and right now it does still have i686 included.


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#143 2018-01-22 06:48:11

Jristz
Member
From: America/Santiago
Registered: 2011-06-11
Posts: 1,020

Re: pkgstats round two: take your vote and help improving Arch

Trilby wrote:

I suppose it does collect the architecture along with everything else, and right now it does still have i686 included.

I'm more concerned that still on archlinux.de and not on archlinux.org and the move to .org was due like since it conception I remember


Well, I suppose that this is somekind of signature, no?

Offline

Board footer

Powered by FluxBB