You are not logged in.

#1 2018-07-22 11:11:21

mipi
Member
From: Germany
Registered: 2018-01-03
Posts: 22

smsync: synch & conversion of big music collections

Hi,
I have created a small program to sync and convert big music collections. smsync is available in AUR. The source code is on Github.

smsync is made for use cases where you have a folder structure for your high quality lossless or lossy but high bit rate music that acts as a "master". From this master you replicate your music to "slaves", such as a smartphone or an SD card / hard drive for your car etc. On a smartphone or in the car you either don't have or you don't want to spend that much storage capacity that you might have for you master music storage. smsync takes care of the replication/conversion while keeping the folder structure. Find more details here

First, I tried to use Make for my use case (there is an approach with Make mentioned in the Arch Linux wiki), but there are some problems with that approach. Therefore, I decided to write something new that solves my problem.

Would be great to get some feedback.

Thanks,
-mipi

Offline

#2 2018-07-23 00:01:17

Rasi
Member
From: Germany
Registered: 2007-08-14
Posts: 1,901
Website

Re: smsync: synch & conversion of big music collections

Very nice. Will this also detect changes in master (e.g. renamed/deleted directories) and reflect these in the sync?

Also: please support ogg and opus.


He hoped and prayed that there wasn't an afterlife. Then he realized there was a contradiction involved here and merely hoped that there wasn't an afterlife.

Douglas Adams

Offline

#3 2018-07-23 04:42:22

mipi
Member
From: Germany
Registered: 2018-01-03
Posts: 22

Re: smsync: synch & conversion of big music collections

Yes, these changes will be detected.

Implementing support of ogg shouldn't be a problem. Regarding opus, I have to see.

BTW: You're welcome to contribute.

Offline

#4 2018-08-04 11:25:56

mipi
Member
From: Germany
Registered: 2018-01-03
Posts: 22

Re: smsync: synch & conversion of big music collections

Support of ogg(vorbis) and opus is now implemented :-) Also wav is supported as source format. :-)

Last edited by mipi (2018-08-04 11:26:10)

Offline

#5 2018-10-22 10:42:08

dizwell
Member
Registered: 2018-10-22
Posts: 2

Re: smsync: synch & conversion of big music collections

The documentation is a bit strange. Copying your config file contents as shown at Github, exactly, I get: "Error: Rule #1: 'copy' is not a valid conversion". There is then no help anywhere I could see that would tell you what valid conversion options actually were.

I removed the Rule 1 and Rule 2 shown in the Github doco and the conversion process started well. But it wasn't an encouraging start when the only available doco appears wrong!

Incidentally, the doco contains another error, too. It shows for rule 0 "conversion = abr:192|cl:3" and then goes on to describe "In the example, [rule0] tells smsync to convert FLAC files (i.e. files with the extension '.flac') to MP3, using the conversion vbr:5|cl:3" Which rule 0 clearly doesn't do as written (i.e., the rule itself says "abr", and the commentary says "vbr").

Minor points: it's a very useful tool. But a bit of doco tidying might be in order, I think.

Nevertheless, thanks for the great functionality.

Last edited by dizwell (2018-10-22 10:42:52)

Offline

#6 2018-10-23 07:22:54

dizwell
Member
Registered: 2018-10-22
Posts: 2

Re: smsync: synch & conversion of big music collections

As mentioned, it's a very useful tool, so what follows are just a couple of suggestions.

1. It would be good if the tool was able to be run non-interactively (eg, a command-line option such as --no-confirm). That is, at the moment, when you type 'smsync', it asks, "Start synchronization (Y/n)?", which is fine -but means setting up crontab to periodically ensure a directory is synched to its source is trickier than it ought to be.

2. When I did a manual ffmpeg of one the files that smsync baulked at converting, I saw this in the output:

Stream mapping:
  Stream #0:1 -> #0:0 (mjpeg (native) -> png (native))
  Stream #0:0 -> #0:1 (flac (native) -> mp3 (libmp3lame))
Press [q] to stop, [?] for help
[swscaler @ 0x55a4fed78900] deprecated pixel format used, make sure you did set range correctly
[mp3 @ 0x55a4fecb9340] Frame rate very high for a muxer not efficiently supporting it.
Please consider specifying a lower framerate, a different muxer or -vsync 2

Which I think means that ffmpeg is not fond of the cover art I have embedded in that particular flac. No problem: the manual convert concluded correctly and output a working, listenable mp3 file.

Except smsync doesn't. It instead simply declares an 'error' in the log file, and refuses to convert the file. As a result, I'm missing about 2000 files in my MP3 directory, and I don't feel like altering the cover art in that many source FLACs in order to have another go to get source and destination properly in sync!

If there's a way to configure 'gentler' error handling, such that non-audio-related errors are ignored, that would be useful. Of course, smsync might have been reacting to the 'Frame rate very high' warning -but again, it's only a warning, and didn't stop ffmpeg finishing the job when run manually, so I don't quite think smsync should just quit at the job when receiving it.

3. Feedback: there isn't much of it apart from a simple file count/files processed type thing. If it were able to display the file name it was working on (or something!), that would be helpful when processing a very large source music collection in interactive mode. Otherwise, it's difficult to work out where smsync has reached in order to check on (say) the output quality of a freshly-created MP3.

Again, it's a very useful tool I'm pleased to have found. But #2 in particular is a bit of a show-stopper for me.

Regards
HJR

Offline

#7 2018-10-27 07:59:48

mipi
Member
From: Germany
Registered: 2018-01-03
Posts: 22

Re: smsync: synch & conversion of big music collections

Hi dizwell,
thanks for reporting this. I will check the docu and correct it. Regarding your other points:
#1: Good idea. Adding that is not a big deal. Will do that asap.
#2: This requires some more investigation. I need to figure out how to distinguish (a) between errors and warnings and (b) between the types of errors that ffmpeg throws. But overall your idea makes sense to me. BTW: Your ffmpeg message looks like you want to convert the cover art from JPG to PNG?
#3: Could add another cli option to make it more chatty.

Great that you like the tool overall. I wrote it because I needed something like that and there was nothing else with this functionality.

mipi

Offline

#8 2018-11-24 15:04:33

Rasi
Member
From: Germany
Registered: 2007-08-14
Posts: 1,901
Website

Re: smsync: synch & conversion of big music collections

Hmm, don't know how to get Opus conversion to work.

[general]
source_dir = /mnt/dump2/Audio/Rips/flac
num_cpus   = 4
num_wrkrs  = 4

[rule0]
source = flac
target = opus
conversion = vbr:128                   

gives me:

Rule #0: 'vbr:128' is not a valid conversion

He hoped and prayed that there wasn't an afterlife. Then he realized there was a contradiction involved here and merely hoped that there wasn't an afterlife.

Douglas Adams

Offline

#9 2018-11-25 12:33:10

mipi
Member
From: Germany
Registered: 2018-01-03
Posts: 22

Re: smsync: synch & conversion of big music collections

Will look into it, latest during the Xmas vacation.

Offline

Board footer

Powered by FluxBB