You are not logged in.

#1 2013-03-14 15:58:23

joebew42
Member
Registered: 2013-03-14
Posts: 11

cwatch 1.0 experimental is out. Who wants to collaborate?

Hi to all,

I'm an archlinux user, I hope that this is the right forum where post this thread. So, I just released the first experimental version of cwatch[1], is a filesystem activity monitor written in C and it is based on the inotify library of the Linux kernel. The main feature of this program is the handling of symbolic links, and that it has not dependency with external programs (such as sendmail, rsync or inotifytools). It can monitor filesystem activity on a certain type of events and then trigger a user defined command when one of the events specifed occurs. The "man" and the documentation are not yet ready, but the integrated "--help" of the command is complete, and you can read it to know how the program works.

Example of usage:

$ cwatch -d /home/joe/mydir/ -c "echo EVENT %e TRIGGERED ON: %s" -v -r

It will monitor recursively (-r) for default events (see the --help) that occurs in "/home/joe/mydir/ and execute the command specified with the -c option[2] (the %f and %e token will be replaced with the path where the event occurs and the name of event). The -v tell the program to be verbose.

I'm searching for interested people that can test the software, bug discovering, bug fixing, code optimization, write code and write documentation and so on ...

So, if you think to contribute at this project, feel free to do it :D

Thanks,
Joe.

P.S.
The program is in experimental stage. So, don't use it in production environment and please don't distribute it as distro-specific package. It is not yet ready! Is necessary to do some tests, fix some issues and optimize few parts of the code (in particular the memory allocation).

[1] https://github.com/joebew42/cwatch
[2] echo EVENT %e TRIGGERED ON: %s

Last edited by joebew42 (2013-03-18 01:46:46)

Offline

#2 2013-03-14 16:04:45

ewaller
Administrator
From: Pasadena, CA
Registered: 2009-07-13
Posts: 19,739

Re: cwatch 1.0 experimental is out. Who wants to collaborate?

joebew42 wrote:

Hi to all,... I hope that this is the right forum where post this thread.

Moving to Community Contributions...


Nothing is too wonderful to be true, if it be consistent with the laws of nature -- Michael Faraday
Sometimes it is the people no one can imagine anything of who do the things no one can imagine. -- Alan Turing
---
How to Ask Questions the Smart Way

Offline

#3 2013-03-16 18:50:35

fogobogo
Member
Registered: 2008-08-24
Posts: 83

Re: cwatch 1.0 experimental is out. Who wants to collaborate?

this, sir, is a real gem! I always thought writing something like that myself someday but the lazyness was worth it. The possiblities are endless! Really awesome.

Offline

#4 2013-03-16 19:16:49

joebew42
Member
Registered: 2013-03-14
Posts: 11

Re: cwatch 1.0 experimental is out. Who wants to collaborate?

fogobogo wrote:

this, sir, is a real gem! I always thought writing something like that myself someday but the lazyness was worth it. The possiblities are endless! Really awesome.

Hi fogobog,

I'm currently fixing some issues that regards the parsing of command line arguments. In the next hours I'll commit these fixes, If you are interested to collaborate feel free to clone the project and start testing. Remember that we are in "experimental",  I am committing new improvements/fixes hour by hour.

Last edited by joebew42 (2013-03-16 21:26:38)

Offline

#5 2013-03-16 19:46:39

ksira
Member
Registered: 2009-10-27
Posts: 31

Re: cwatch 1.0 experimental is out. Who wants to collaborate?

I like the idea, and I have been wanting something like this as well. It may be good for you to come up with some some interesting use cases for the documentation.

So, would this work for say watching a directory where you are editing some source files, and then automatically compiling and running unit tests on when you write a file? If so, it seems similar to a ruby program I looked at recently called Guard (possibly worth stealing some ideas from), but this looks much more KISS.

Offline

#6 2013-03-16 20:17:04

joebew42
Member
Registered: 2013-03-14
Posts: 11

Re: cwatch 1.0 experimental is out. Who wants to collaborate?

ksira wrote:

...
So, would this work for say watching a directory where you are editing some source files, and then automatically compiling and running unit tests on when you write a file? If so, it seems similar to a ruby program I looked at recently called Guard (possibly worth stealing some ideas from), but this looks much more KISS.

Of course, I already used the experimental release to do that, exactly to automatically run test-case where I've modified a source file in a directory.

Here a simple example to do that:

cwatch -d "path/of/your/sources" -c "ruby %f" -e create,modify -v

When a file in the directory is created or modified, cwatch execute the command "ruby %f", where the %f will be replaced with the full_path of the interested file.

Last edited by joebew42 (2013-03-16 21:46:19)

Offline

#7 2013-03-16 20:19:22

falconindy
Developer
From: New York, USA
Registered: 2009-10-22
Posts: 4,111
Website

Re: cwatch 1.0 experimental is out. Who wants to collaborate?

Curious how this differs from inotifywait (part of inotify-tools in community).

Offline

#8 2013-03-16 20:29:28

joebew42
Member
Registered: 2013-03-14
Posts: 11

Re: cwatch 1.0 experimental is out. Who wants to collaborate?

falconindy wrote:

Curious how this differs from inotifywait (part of inotify-tools in community).

cwatch manages and can traverse symbolic links, you can run cwatch outside from script and you have capability to execute a command when an event occurs.

Last edited by joebew42 (2013-03-16 21:43:58)

Offline

#9 2013-03-16 21:23:06

joebew42
Member
Registered: 2013-03-14
Posts: 11

Re: cwatch 1.0 experimental is out. Who wants to collaborate?

For everyone interested in this project, I've just pushed some significant changes: https://github.com/joebew42/cwatch

Follow the README.md to to some test.

Last edited by joebew42 (2013-03-18 01:47:49)

Offline

#10 2013-03-16 22:56:22

falconindy
Developer
From: New York, USA
Registered: 2009-10-22
Posts: 4,111
Website

Re: cwatch 1.0 experimental is out. Who wants to collaborate?

joebew42 wrote:
falconindy wrote:

Curious how this differs from inotifywait (part of inotify-tools in community).

cwatch manages and can traverse symbolic links, you can run cwatch outside from script and you have capability to execute a command when an event occurs.

Sounds like a simple patch to inotifywait if it can't follow symlinks already.

You can already dispatch whatever you want based on events from inotifywait as well, e.g.

while read -r event path; do
  # do something with $event and $path...
done < <(inotifywait -m -e close_write,modify -r --fmt '%e %w' /some/path)

This also has the distinct advantage that it properly handles commands or arguments which necessarily contain whitespace. Your bsplit function will not manage this.

Last edited by falconindy (2013-03-16 22:58:54)

Offline

#11 2013-03-17 04:19:46

joebew42
Member
Registered: 2013-03-14
Posts: 11

Re: cwatch 1.0 experimental is out. Who wants to collaborate?

falconindy wrote:

You can already dispatch whatever you want based on events from inotifywait as well, e.g.

while read -r event path; do
  # do something with $event and $path...
done < <(inotifywait -m -e close_write,modify -r --fmt '%e %w' /some/path)

As you pointed out, you had to write a bash script to intercept and handle events. The philosophy behind cwatch is diametrically opposed (I do not have to write a script to execute a command): You have an existing command that can be executed by cwatch (as you can see, it is not a task of the command to know how to deal with inotify, or of the code you've posted to encapsulate and "eval" some user specific command). For example, if you have a script for email notification (already written) you can easily connect it to cwatch without writing a new bash script:

cwatch -d /path/to/monitor/ -c "send_email %f" -r

Ready-to-go! Very simple! (no bash scripting needed here).

In more specific case, you can execute a personal script/command that will be executed by cwatch (similarly as inotifywait).

By the way, as I always say, It is merely a question of domain context of application (inotifywait is an excellent command, however it can't handle symbolic links yet, and here we are talking about a ready-to-go/easy-to-use approach).

falconindy wrote:

This also has the distinct advantage that it properly handles commands or arguments which necessarily contain whitespace. Your bsplit function will not manage this.

Thanks for your observation, can you give an example about that? What do you mean?

Joe.

Last edited by joebew42 (2013-03-18 02:05:37)

Offline

#12 2013-03-17 12:57:46

falconindy
Developer
From: New York, USA
Registered: 2009-10-22
Posts: 4,111
Website

Re: cwatch 1.0 experimental is out. Who wants to collaborate?

As you pointed out, you had to write a bash script to intercept and handle events. The philosophy behind cwatch is diametrically opposed: You write a script (command) that can be executed by cwatch

I don't get it. I wrote a loop, you write a script. What's really the difference? I certainly don't see any "diametric opposition" here other than how the command is "passed".

it is not a task of the script/program to know how to deal with inotify

But you've exposed %e, which presumably expands to one to many inotify events which could be acted on... I don't see how this is hiding any implementation details of inotify. Furthermore, you expose the same -e flag which filters events to listen for.

Thanks for your observation, can you give an example about that? What do you mean?

cwatch -d /path/to/watch -c "mv %f "/move/to/this dir"'

This will be split into:

{ "mv", file, "\"move/to/this", "dir\"" }

You're now forced to write a separate script which has the destination path hardcoded. With inotifywait, you haven't left the shell. It's a one liner if you're doing something simple like your mail or my mv example:

inotifywait -rme close_write --fmt '%f' | xargs -d\\n -I{} mv -t "/move to/this dir/" {}

Note that you're essentially making the same mistake that su made (and which sudo did not). If you avoid embedding the command in an argument to a flag and simply accept the non-option arguments as the command to execute, then you avoid this mess entirely. I'm also still suggesting that inotifywait isn't that much different here. I would suspect inotifywait only needs a 5-10 line patch to following symlinks (under a flag, of course).

Offline

#13 2013-03-17 14:05:22

joebew42
Member
Registered: 2013-03-14
Posts: 11

Re: cwatch 1.0 experimental is out. Who wants to collaborate?

falconindy wrote:

But you've exposed %e, which presumably expands to one to many inotify events which could be acted on... I don't see how this is hiding any implementation details of inotify. Furthermore, you expose the same -e flag which filters events to listen for.

I'm sorry but I've no time do continue this discussion with you, sorry. If you don't get the point it's not an ISSUE of the program (you are OT): we are talking about cwatch and not about inotifywait or iwatch or other similar software. There are so many Desktop Environment/Window Manager for GNU/Linux, so what about? Which are the differences between GNOME 3 and Unity, they do basically the same thing, right? Maybe one has a simplest use of the other, or viceversa (depends on the context).

falconindy wrote:
cwatch -d /path/to/watch -c "mv %f "/move/to/this dir"'

This will be split into:

{ "mv", file, "\"move/to/this", "dir\"" }

You're now forced to write a separate script which has the destination path hardcoded. With inotifywait, you haven't left the shell. It's a one liner if you're doing something simple like your mail or my mv example:

inotifywait -rme close_write --fmt '%f' | xargs -d\\n -I{} mv -t "/move to/this dir/" {}

Note that you're essentially making the same mistake that su made (and which sudo did not). If you avoid embedding the command in an argument to a flag and simply accept the non-option arguments as the command to execute, then you avoid this mess entirely. I'm also still suggesting that inotifywait isn't that much different here.

This is one interesting ISSUE, thank you!

falcoindy wrote:

I would suspect inotifywait only needs a 5-10 line patch to following symlinks (under a flag, of course).

If you think that could be a significant improvement, you can write those 5-10 lines code patch. But, here, we are talking about cwatch.

Last edited by joebew42 (2013-03-17 15:59:11)

Offline

#14 2013-03-17 22:45:48

falconindy
Developer
From: New York, USA
Registered: 2009-10-22
Posts: 4,111
Website

Re: cwatch 1.0 experimental is out. Who wants to collaborate?

joebew42 wrote:
falconindy wrote:

But you've exposed %e, which presumably expands to one to many inotify events which could be acted on... I don't see how this is hiding any implementation details of inotify. Furthermore, you expose the same -e flag which filters events to listen for.

I'm sorry but I've no time do continue this discussion with you, sorry. If you don't get the point it's not an ISSUE of the program (you are OT): we are talking about cwatch and not about inotifywait or iwatch or other similar software.

I'm trying to figure out what cwatch does that inotifywait doesn't already do. So far, it appears that cwatch follows symlinks by default and there's no ability to control this. This is scary default behavior since it opens you up to the possibility of infinite loops and possibly unwanted watches (especially since you have a hard limit system wide and every watch consumes kernel resources). I'm sorry if you're feeling defensive towards constructive criticism. I'm trying to understand what you hope to accomplish from writing cwatch.

If you think that could be a significant improvement, you can write those 5-10 lines code patch. But, here, we are talking about cwatch.

Sure thing. Looks like there's a few improvements inotify-tools could benefit from, now that I glance at the code.

Offline

#15 2013-03-17 22:53:29

joebew42
Member
Registered: 2013-03-14
Posts: 11

Re: cwatch 1.0 experimental is out. Who wants to collaborate?

falconindy wrote:

...
So far, it appears that cwatch follows symlinks by default and there's no ability to control this.
...

Are you sure? Look at the -n (--no-symlink) option.

falconindy wrote:

...
This is scary default behavior since it opens you up to the possibility of infinite loops and possibly unwanted watches (especially since you have a hard limit system wide and every watch consumes kernel resources).

cwatch is developed to be aware of cyclic loop generated by symlinks.

falconindy wrote:

I'm trying to understand what you hope to accomplish from writing cwatch

To achieve an easy-to-use / ready-to-go / KISS / real-time filesystem alteration monitor software (and of course, for fun).

Last edited by joebew42 (2013-03-17 23:08:51)

Offline

#16 2013-03-17 23:07:43

falconindy
Developer
From: New York, USA
Registered: 2009-10-22
Posts: 4,111
Website

Re: cwatch 1.0 experimental is out. Who wants to collaborate?

joebew42 wrote:

Are you sure? Look at the -n (--no-symlink) option.

My mistake.

joebew42 wrote:

Achieve an easy-to-use / ready-to-go / KISS / real-time filesystem alteration monitor software (and of course, for fun).

You keep using this word realtime. I don't think it means what you think it means. Inotify is inherently racy (unlike fanotify) because the file that an event occurs on can change (even be deleted) between the time the event occurs and when you react to it. That doesn't seem like realtime to me.

Still not understanding the why. Have fun with your project.

Offline

#17 2013-03-17 23:10:28

joebew42
Member
Registered: 2013-03-14
Posts: 11

Re: cwatch 1.0 experimental is out. Who wants to collaborate?

falconindy wrote:

...
You keep using this word realtime. I don't think it means what you think it means. Inotify is inherently racy (unlike fanotify) because the file that an event occurs on can change (even be deleted) between the time the event occurs and when you react to it. That doesn't seem like realtime to me.

My mistake.

falconindy wrote:

Still not understanding the why. Have fun with your project.

As I said before is merely a your issue. Of course I'll continue to have a lot of fun with this project, I started this thread to find new collaboration and NOT to open a discussion about what software is better than other (as well as inotiywait, exists iwatch, and other and other. But this is NOT the main topic of this thread.)

I want to bring this discussion to the real topic big_smile

If someone wants to contribute to the project, I will be very happy to be able to rely on new contributors to improve, together, even further cwatch. There are a lot of things to do: Write documentation, write usage example, test the software, propose new features, optmize the code, bug discovering, bug fixing, and more.

Joe.

Last edited by joebew42 (2013-03-18 01:55:47)

Offline

#18 2013-04-08 10:12:10

joebew42
Member
Registered: 2013-03-14
Posts: 11

Re: cwatch 1.0 experimental is out. Who wants to collaborate?

Hi to all,

just yesterday I've released the version 1.2.2 of cwatch (it is always to be considered as an experimental software yet).  You can get it directly from here (v1.2.2-experimental tree). However, you have to follow the integrated help of the command as documentation reference (-h --help). Now cwatch can be easly embedded in bash script (as well as inotify-wait):

cwatch in embedded mode:

#!/bin/sh

# This is an example of how is simple to embed
# cwatch into a bash script

cwatch -d /home/user/path/to/monitor/ -F '%e %f' -r | while read -r event filepath; do
  # All code goes here ...
  printf "EVENT OCCURRED: %s IN FILE %s\n" $event $filepath
  echo "OPERATION DONE!!!"
done

Or in traditional inline mode (directly from a terminal):

$ cwatch -d /home/user/path/to/monitor/ -c "echo EVENT %e TRIGGERED ON: %s" -vr

Now it supports interesting features, such as the --exclude option or the --regex-catch.

I hope to commit the first stable release, soon. Maybe for the end of April (a full test coverage, a rich documentation and usage examples).

Every kind of contribution to this project is appreciated smile

Last edited by joebew42 (2013-04-08 10:15:59)

Offline

#19 2013-04-22 10:31:12

ksira
Member
Registered: 2009-10-27
Posts: 31

Re: cwatch 1.0 experimental is out. Who wants to collaborate?

Thanks for the update. Easy embedding is a nice feature.

Offline

#20 2013-04-22 11:06:08

joebew42
Member
Registered: 2013-03-14
Posts: 11

Re: cwatch 1.0 experimental is out. Who wants to collaborate?

ksira wrote:

Thanks for the update. Easy embedding is a nice feature.

At this moment we are working on fixing some bugs, please don't use cwatch for production environment smile I hope soon to release a STABLE and well tested version of cwatch, with a rich documentation.
If you want to propose new features we'll be happy to be aware of them big_smile

Offline

Board footer

Powered by FluxBB