anything-sync-daemon - keep ANYTHING in tmpfs and sync'ed

Mr Green · 2012-09-20 15:20:47

I just think with initscripts you could add a test for asd sync before shutdown just to make sure everything is ok, with systemd I am a little more in the dark. Maybe asd could log any output so we could check it syncs ok prior to shutdown...

graysky · 2012-09-20 15:34:57

@mrg - you can add a logging step yourself if you're volunteering to test... use this patch which should work but I have not tested.

--- anything-sync-daemon  2012-09-20 11:23:27.363527867 -0400
+++ anything-sync-daemon  2012-09-20 11:29:53.350443701 -0400
@@ -71,11 +71,12 @@
      fi

      # sync the tmpfs targets to the disc
+     LOG=/home/YOURUSER/asd.log
      if [[ -e $LINK/.flagged ]]; then
-       su -c "rsync -a --delete --exclude .flagged '$LINK/' '$BACKUP/'" $USER
+       su -c "rsync -av --delete --exclude .flagged '$LINK/' '$BACKUP/' >>$LOG" $USER
      else
-       su -c "rsync -a '$BACKUP/' '$LINK/'" $USER
-       su -c "touch '$LINK/.flagged'" $USER
+       su -c "rsync -a '$BACKUP/' '$LINK/' >>$LOG" $USER
+       su -c "touch '$LINK/.flagged' >>$LOG" $USER
      fi
    fi
  done

Note that you MUST define a username in the patch or in /usr/bin/anything-sync-deamon so that the log file goes somewhere.
LOG=/home/YOURUSER

# patch -i logging.patch /usr/bin/anything-sync-daemon

Last edited by graysky (2012-09-22 12:31:24)

Mr Green · 2012-09-20 15:45:12

I will test it of course, just need a safe directory to sync to ram that is safe presume /var/log is still out of bounds...

graysky · 2012-09-20 15:56:11

Mr Green wrote:

I will test it of course, just need a safe directory to sync to ram that is safe presume /var/log is still out of bounds...

Yeah, use a guinea pig directory for sure. Do populate it w/ a good chunk of data. Maybe 200 MB or so?

Mr Green · 2012-09-20 16:06:50

Ok on it.... Some justin bieber might do the trick

graysky · 2012-09-20 16:10:46

Mr Green wrote:

Ok on it.... Some justin bieber might do the trick

I said guinea pig, not garbage dump

Mr Green · 2012-09-21 04:43:22

Not had any problems yet, checked to make sure service was running and found this

asd.service                loaded active exited

graysky · 2012-09-21 09:06:51

Dunno for sure but that seems normal. Just look to see that your target is symlinked. If it is then it's active. Also, what does the log say?

Mr Green · 2012-09-21 10:27:14

No log nothing naa daa....

graysky · 2012-09-22 12:30:57

Ack... forgot to specify a log file. Edited the patch. All you need to do is add "/asd.log" to the LOG=line in /usr/bin/anything-sync-daemon like this:

LOG=/home/YOURUSER/asd.log

Last edited by graysky (2012-09-22 12:32:00)

Mr Green · 2012-09-22 12:47:51

Thought it looked odd added asd.log

graysky · 2012-09-22 13:33:21

Cool, plz report back your findings with regard to systemd waiting for a sync.

Mr Green · 2012-09-22 14:20:58

At the moment asd.log is empty really need to test it on an active directory....

Mr Green · 2012-09-23 06:55:44

Ok .... what I have this morning

sending incremental file list
./

sent 29 bytes  received 15 bytes  88.00 bytes/sec
total size is 0  speedup is 0.00
sending incremental file list
./

sent 29 bytes  received 15 bytes  88.00 bytes/sec
total size is 0  speedup is 0.00

X-dark · 2012-10-16 16:18:09

Quick question, what is happening if a power failure happens during the rsync? Will my data get corrupted?
Did you consider using the "--delay-updates" rsync option (not sure if it is enough to have a real atomic operation)?

graysky · 2012-10-16 19:09:11

@Xd - Not sure. One could ask the same question for a standard write operation to a HDD as well. I do like your suggestion to add the --delay-updates switch which should make it a safer operation. I will test and implement.

graysky · 2012-10-17 16:24:41

v3.07 is released with this suggestion. Thank you.

fabertawe · 2012-10-18 08:48:34

Won't "--delay-updates" actually write to the disk twice though? Or have I got this wrong?

graysky · 2012-10-18 21:47:40

fabertawe wrote:

Won't "--delay-updates" actually write to the disk twice though? Or have I got this wrong?

No, it just rotates the files from the temp names to the actual names. You can verify this with:

# iotop -Pao

fabertawe · 2012-10-19 08:14:41

Ah, ok, thanks for clarifying. That actually makes a lot more sense. Thanks for all your work, really enjoying your packages

X-dark · 2012-10-19 09:43:56

graysky wrote:

v3.07 is released with this suggestion. Thank you.

Thanks.

mandos · 2012-10-22 17:51:11

Can I use anything-sync-daemon for mysql?

In my development pc I decided to add lots of ram, to use it for local web development.
Everything mysql-related is under /srv/mysql, if I could put the databases in ram my dev-environment would fly!

Is there anything I should check? Will it sync back to the file-system on shutdown?
I'm under systemd, so the order the services start isn't configured by me.

graysky · 2012-10-22 18:49:30

mandos wrote:

Can I use anything-sync-daemon for mysql?
In my development pc I decided to add lots of ram, to use it for local web development.
Everything mysql-related is under /srv/mysql, if I could put the databases in ram my dev-environment would fly!
Is there anything I should check? Will it sync back to the file-system on shutdown?
I'm under systemd, so the order the services start isn't configured by me.

I don't see why it wouldn't work. Backup your databases before attempting would be good advice. Plz report back. And yes, it syncs back on shutdown and once per hour. See the wiki page for more (or the man page).

ndejay · 2012-11-01 01:12:47

Amazing job at anything-sync-daemon and profile-sync-daemon, it works like a charm on my system! However, wouldn't deleting a file off a "RAM-cached" disk re-spawn after reboot (because of rsync and _backup folders)? Is there a solution to this other than deleting the files at both places?

EDIT: Removed a stupid question I figured out after hitting "post"

Last edited by ndejay (2012-11-01 01:13:30)

graysky · 2012-11-01 01:26:03

I'm confused... The daemon effectively makes your system have a single location. If you delete something from tmpfs, so long as you stop the daemon gracefully, the changes are committed to disk.

Arch Linux

#151 2012-09-20 15:20:47

Re: anything-sync-daemon - keep ANYTHING in tmpfs and sync'ed

#152 2012-09-20 15:34:57

Re: anything-sync-daemon - keep ANYTHING in tmpfs and sync'ed

#153 2012-09-20 15:45:12

Re: anything-sync-daemon - keep ANYTHING in tmpfs and sync'ed

#154 2012-09-20 15:56:11

Re: anything-sync-daemon - keep ANYTHING in tmpfs and sync'ed

#155 2012-09-20 16:06:50

Re: anything-sync-daemon - keep ANYTHING in tmpfs and sync'ed

#156 2012-09-20 16:10:46

Re: anything-sync-daemon - keep ANYTHING in tmpfs and sync'ed

#157 2012-09-21 04:43:22

Re: anything-sync-daemon - keep ANYTHING in tmpfs and sync'ed

#158 2012-09-21 09:06:51

Re: anything-sync-daemon - keep ANYTHING in tmpfs and sync'ed

#159 2012-09-21 10:27:14

Re: anything-sync-daemon - keep ANYTHING in tmpfs and sync'ed

#160 2012-09-22 12:30:57

Re: anything-sync-daemon - keep ANYTHING in tmpfs and sync'ed

#161 2012-09-22 12:47:51

Re: anything-sync-daemon - keep ANYTHING in tmpfs and sync'ed

#162 2012-09-22 13:33:21

Re: anything-sync-daemon - keep ANYTHING in tmpfs and sync'ed

#163 2012-09-22 14:20:58

Re: anything-sync-daemon - keep ANYTHING in tmpfs and sync'ed

#164 2012-09-23 06:55:44

Re: anything-sync-daemon - keep ANYTHING in tmpfs and sync'ed

#165 2012-10-16 16:18:09

Re: anything-sync-daemon - keep ANYTHING in tmpfs and sync'ed

#166 2012-10-16 19:09:11

Re: anything-sync-daemon - keep ANYTHING in tmpfs and sync'ed

#167 2012-10-17 16:24:41

Re: anything-sync-daemon - keep ANYTHING in tmpfs and sync'ed

#168 2012-10-18 08:48:34

Re: anything-sync-daemon - keep ANYTHING in tmpfs and sync'ed

#169 2012-10-18 21:47:40

Re: anything-sync-daemon - keep ANYTHING in tmpfs and sync'ed

#170 2012-10-19 08:14:41

Re: anything-sync-daemon - keep ANYTHING in tmpfs and sync'ed

#171 2012-10-19 09:43:56

Re: anything-sync-daemon - keep ANYTHING in tmpfs and sync'ed

#172 2012-10-22 17:51:11

Re: anything-sync-daemon - keep ANYTHING in tmpfs and sync'ed

#173 2012-10-22 18:49:30

Re: anything-sync-daemon - keep ANYTHING in tmpfs and sync'ed

#174 2012-11-01 01:12:47

Re: anything-sync-daemon - keep ANYTHING in tmpfs and sync'ed

#175 2012-11-01 01:26:03

Re: anything-sync-daemon - keep ANYTHING in tmpfs and sync'ed

Board footer