Ruby PKGD Clone

jshield · 2010-02-15 16:34:10

Note: by clone I refer to sharing packages over a network from a number of ad-hoc machines, this is not actually functionally compatible with Xyne's pkgD and other tools

Hi all, I have written a Package Daemon "clone" in ruby.
It relies on having avahi installed to provide "auto-sensing" of other ruby Package Daemons on the network, on my machines it seems to be faster than the other one by Xyne, I have no idea about memory usage or anything else to that effect.
gems required are sinatra and dnssd.

How it works is relatively straight forward.
1. start dbus and avahi.
2. start the daemon however one sees fit.
3. modify the mirrorlist to include
Server = http://127.0.0.1:14686/$repo
4. repeat on other machines

if it does not find a package either on it self or on the network it returns a 404.
if it finds a server with the packages, it returns a 302 with the server and package url.

going to http://127.0.0.1:14686/ will produce a simple list of found servers apart from itself.

current limitations include:
must all be running on the same port, dnssd does not seem to want to actually resolve anything.
it relies on mDNS being in nsswitch.conf, same reason.

require 'rubygems'
require 'sinatra'
require 'dnssd'
require 'net/http'

configure do
  
  Hostname = `hostname`.strip
  CacheDir = "/var/cache/pacman/pkg/"
  Port = 14686
  Domain = "local"
  Info = true
  Colleagues = {}

  set :port, Port

  register_service =
    DNSSD::register(
      Hostname,
      "_pkgd._tcp.", Domain,
      Port) do

    puts "* Registering the Package Daemon" if Info
    puts "* Scanning network for Package Daemons" if Info

    service = DNSSD.browse('_pkgd._tcp.') do |reply|
      if (reply.flags == DNSSD::Flags::Add)
        unless reply.name == Hostname
          puts "adding: #{reply.name}" if Info
          Colleagues[reply.name] = 1 
        end
      else
        unless reply.name == Hostname
          puts "removing: #{reply.name}" if Info
          Colleagues.delete reply.name 
        end
      end
    end
  end  
 
end

helpers do

  def cache_check?(file)
    if File.exist? CacheDir + file
      return true
    else
      return false
    end
    return false
  end
  
  def ask_colleague?(colleague,uri)
    resp = Net::HTTP.get_response(URI.parse "http://#{colleague}:#{Port}#{uri}/ask")
    case resp
      when Net::HTTPSuccess
        return true
      else
        return false
    end
  end  

end

get '/' do
  redirect '/colleagues'
end

get '/colleagues' do
  html = "<html>"
  Colleagues.each_key do |colleague|
    html += "<b> #{colleague} </b><br>"
  end
  html += "</html>"
end

get '/:repo/:file' do
  file = params[:file]
  if cache_check?(file)
    send_file CacheDir + file    
  end
  Colleagues.each_key do |colleague|
    if ask_colleague?(colleague,request.path_info)
     redirect("http://#{colleague}:#{Port}#{request.path_info}")
    end
  end
  404  
end

get '/:repo/:file/ask' do
  file = params[:file]
  if cache_check?(file)
    200
  else
    404
  end  
end

Last edited by jshield (2010-02-18 19:53:47)

jshield · 2010-02-18 16:59:18

Future version may drop the avahi requirement. Perhaps using a 1.9.1 compatible version of journeta instead. thoughts?

Xyne · 2010-02-18 18:23:53

Hi jshield, and welcome to the forum!

What stability issues have you encountered with PkgD?

Also, is this able to handle the non-HTTP requests that PkgD uses internally? Those are used for Powerpill|Bauerbill integration and some users might expect that functionality if you refer to this as a PkgD "clone". Perhaps you should mention that in the OP.

jshield · 2010-02-18 18:56:43

My issues are more speed related. I have 5 archlinux machines and pkgD was taking several seconds to do its thing over the network, so I wrote this.
It is just http only and it does not interface with bauerbill or powerpill in anyway beyond being a repo, so I will make that clear.

Xyne · 2010-02-18 19:39:23

jshield wrote:

My issues are more speed related. I have 5 archlinux machines and pkgD was taking several seconds to do its thing over the network, so I wrote this.

Ok, but why did you make this claim then:

jshield wrote:

and generally all around more stable than the other one by Xyne

That implies that PkgD is not stable, and I take issue with the accusation if there is no basis for it.

I have nothing to say about speed. You might have been able to improve it with a change to your configuration, but now that you've written this it hardly matters. It will be interesting to see what others find.

jshield · 2010-02-18 19:53:17

ah, I apologise I did not mean it to be an accusation. I did mention prior to it, that it was based on experiences with my machines. I think the issues were related to my network more than your code. My code is meant to be simpler and also smaller in scope than your pkgD and also allow adding machines in an ad-hoc fashion via avahi which is handy because 3 of the machines are laptops.

Once again I was not attempting to discredit your software, it was merely a poor choice of words. I would be happy to remove the inflamatory statement if so desired.

Xyne · 2010-02-18 21:08:25

Thanks for the clarification and changes to the OP.

Arch Linux

#1 2010-02-15 16:34:10

Ruby PKGD Clone

#2 2010-02-18 16:59:18

Re: Ruby PKGD Clone

#3 2010-02-18 18:23:53

Re: Ruby PKGD Clone

#4 2010-02-18 18:56:43

Re: Ruby PKGD Clone

#5 2010-02-18 19:39:23

Re: Ruby PKGD Clone

#6 2010-02-18 19:53:17

Re: Ruby PKGD Clone

#7 2010-02-18 21:08:25

Re: Ruby PKGD Clone

Board footer