You are not logged in.

#26 2014-11-08 10:13:05

Ambrevar
Member
Registered: 2011-08-14
Posts: 212
Website

Re: Demlo - A dynamic and extensible music library organizer

I've just pushed version 1.4!

It features mostly bug fixes and an improved tagging heuristic. It is getting fairly stable now.
I have also opened an official web page, check it out! (Still a lot of work to do though tongue )

Offline

#27 2014-11-08 19:24:21

hrezaee
Member
From: Tehran, Iran
Registered: 2014-11-08
Posts: 4

Re: Demlo - A dynamic and extensible music library organizer

Hello

This is so amazing. I need to try this.
I'll give this a try very soon and provide feedback.


Cheers
Hadi Rezaee

Offline

#28 2014-11-10 09:38:08

Ambrevar
Member
Registered: 2011-08-14
Posts: 212
Website

Re: Demlo - A dynamic and extensible music library organizer

Thanks! Feedback is very welcome!

Offline

#29 2015-02-12 12:35:24

CjK
Member
Registered: 2008-10-17
Posts: 20

Re: Demlo - A dynamic and extensible music library organizer

Hi,

I tried out demlo today but right after installation I got the following error:

% demlo ~/music/videos/Bad\ Things\ -\ K.flay-DSYopj6Z_Fo.mp3
lua: /usr/share/lua/5.2/demlo/titlecase.lua:141: attempt to index upvalue 'utf8' (a nil value)
stack traceback:
        /usr/share/lua/5.2/demlo/titlecase.lua:141: in function 'constantine'
        /usr/bin/demlo:405: in main chunk
        [C]: in ?

Any idea of what I did wrong?

I was looking for an issue-tracker to report this but it apparently was deactivated.

Thanks,
Claus

Offline

#30 2015-02-16 08:16:19

Ambrevar
Member
Registered: 2011-08-14
Posts: 212
Website

Re: Demlo - A dynamic and extensible music library organizer

Thanks for the notification. Indeed, the issue-tracker was not activated, this is now fixed: issue-tracker.

slnunicode is currently broken, I'll look into it.

Offline

#31 2015-02-16 08:35:23

Ambrevar
Member
Registered: 2011-08-14
Posts: 212
Website

Re: Demlo - A dynamic and extensible music library organizer

Found the bug: slnunicode moved its functions from the 'utf8' table to the global table. I'll push a fixed version soon.

Offline

#32 2015-02-17 10:54:09

Ambrevar
Member
Registered: 2011-08-14
Posts: 212
Website

Re: Demlo - A dynamic and extensible music library organizer

Demlo 1.6 is out!

Main features:

* Fix slnunicode loading.
* Automatically shrinks big covers.
* Huge improvement in Musicbrainz heuristics (mostly thanks to fuzzy matching).
* Fixed most concurrency issues.

I have improved the quality of the Internet tag fetcher a big load. Now, even if some tags are missing, or if the album name is terribly spelled, Demlo should figure out the right album release and tags.
The whole process has been largely optimized thanks to fuzzy memoization.
Same for cover fetching.

As usual, feedback is very welcome! smile

Offline

#33 2015-02-19 16:10:30

CjK
Member
Registered: 2008-10-17
Posts: 20

Re: Demlo - A dynamic and extensible music library organizer

Ambrevar wrote:

Demlo 1.6 is out!

Wow... thanks for the quick turnaround. I'll soon give demlo another spin (not satisfied with Beets' results yet, esp. on Non-Album tracks) and report any feedback here (and bug - if any - in the tracker).

Thanks again for making this nice tool available to the community!

Claus

Offline

#34 2015-02-20 09:18:45

Ambrevar
Member
Registered: 2011-08-14
Posts: 212
Website

Re: Demlo - A dynamic and extensible music library organizer

Thanks for your support!

Regarding non-album tracks, you can handle them pretty much like you want. Default behaviour is, when the 'album' tag is empty:
* Remove 'track' and 'disc' tags.
* Rename file to 'library/artist/title'.

Tell me if there is anything you cannot figure out.

Offline

#35 2016-01-12 02:53:02

Ambrevar
Member
Registered: 2011-08-14
Posts: 212
Website

Re: Demlo - A dynamic and extensible music library organizer

Version 2.0 is out!

I rewrote the core in Go, which fixes plenty of issues related to parallelism (race conditions), unicode, network support, dependencies, etc.
I've added many features in the process:
* FFmpeg encoding can now be parameterized from user scripts.
* Same for covers.
* MusicBrainz heuristic is much better.
* Preview format is slightly improved.

I've updated the home page with a list of changes: http://ambrevar.bitbucket.org/demlo/.

Offline

#36 2016-04-30 18:31:48

CjK
Member
Registered: 2008-10-17
Posts: 20

Re: Demlo - A dynamic and extensible music library organizer

Nice work, Demlo 2.0 now comes pretty close to the ideal swiss-army-knife CLI id3-tagger smile

Installation is now much easier too, though I had to remove "$GOPATH" in the PKGFILE to make it compile without error (my gopath is empty, perhaps this is why? It complained about it being a relative path when an absolute is required).

But otherwise it worked well after install.

I then wanted to try out your code-snippet from the dev-page, i.e. to modify ouput.path to copy processed files into the right ~/music/... hierarchy. So I added another script into the chain in demlorc, created a new script inside the scripts-subdir (under ~/.config/demlo/) and ran "$demlo -t some-audio.mp3". It then complained about not knowing about the #empty() function. It took me a while to figure out I had to copy it from the path.lua-script, then it worked.
Just wondering why I had to do this, since some other scripts in the same directory are also using this #empty() function, but don't provide it locally - strange, but I'm a Lua-rookie after all.

Anyway, after that, demlo is my way to do mp3-tagging again - off goes Picard and friends! ;-)

Thanks for your hard work on this nice tool!

\Claus

Offline

#37 2016-05-01 09:20:19

Ambrevar
Member
Registered: 2011-08-14
Posts: 212
Website

Re: Demlo - A dynamic and extensible music library organizer

Thanks for the feedback!

Agreed, the way GOPATH was set in the PKGBUILD was wrong in my opinion. I used to follow the wiki but I believe it's wrong since it will fail in case the variable is unset, as it did for you. I've commented on this on the wiki.
The PKGBUILD is now fixed.

Regarding the 'empty()' function: this used to be a global function in Demlo 1.x, but since it is so small and only called in path.lua, I've downgraded it to a local function instead in Demlo 2.x. You said other scripts call it: none of the official scripts do, except for path.lua... Can you be more specific?

The documentation mentionned the empty() function, but that was wrong: I've updated it accordingly.
Also, the link to the manual was broken: it now correctly links to the godoc page.

This page contains *a lot* of in-depth details. Among others:

All default functions and variables (excluding 'output') are reset on every script call to enforce consistency. Local variables are lost from one script call to another. Global variables are preserved. Use this feature to pass data like options or new functions.

With this knowledge, removing the "local" keyword at 'empty()'s definition will make it available to all scripts run afterwards.

Hope this helps.

Offline

#38 2016-05-01 13:31:41

Rasi
Member
From: Germany
Registered: 2007-08-14
Posts: 1,912
Website

Re: Demlo - A dynamic and extensible music library organizer

could demolo do these things:

* have a whitelist of tags to be written to files
* flexibly handle sorting of multi-disc albums (if the album has one disc, do not create sub-directories, otherwise do)
* run something to replaygain files that need it (by album)
* offer the possibility to edit tags before writing them
* append chosen release types to album name ("EP" "single")

plus the usual, tag the files, properly sort them into a target directory.

if the answer to all of these is yes, picard might be obsolete for me smile

Last edited by Rasi (2016-05-01 13:34:01)


He hoped and prayed that there wasn't an afterlife. Then he realized there was a contradiction involved here and merely hoped that there wasn't an afterlife.

Douglas Adams

Offline

#39 2016-05-02 01:50:14

Ambrevar
Member
Registered: 2011-08-14
Posts: 212
Website

Re: Demlo - A dynamic and extensible music library organizer

"Demolo" is an interesting nickname, I like it! wink

Rasi wrote:

* have a whitelist of tags to be written to files

Sure, just write the appropriate Lua code. The example script 'tag.lua' is doing that already, by resetting all tags and writing only artist, album, album_artist, title, track, date and, in some cases, genre.

Rasi wrote:

* flexibly handle sorting of multi-disc albums (if the album has one disc, do not create sub-directories, otherwise do)

No problem at all. As a matter of fact, the official script path.lua does something similar: when the album has no disc information, save the track to /$album/, otherwise save it to /$album - Disc $discnumber/.

Rasi wrote:

* run something to replaygain files that need it (by album)

This one is a bit trickier. I do not use ReplayGain myself, so I would need some feedback here. FFmpeg has a volume filter and can read replaygain information. Since Demlo can pass arbitrary parameters to FFmpeg, you can adjust the volume from Demlo.

There is one limitation with Demlo though: it is a batch tool that processes tracks independently. There is currently no way for one track to influence to transformation of another (this is by design). That might change in the future.

Rasi wrote:

* offer the possibility to edit tags before writing them

That's one of the killer features of Demlo I am especially proud of: you can preview the changes in a JSON format which in turn can be edited with your favorite editor. The resulting JSON can be passed back to Demlo. You can wrap up the process in a script to save a few keystrokes, e.g.

'demlo "$@" >> /tmp/demlo-$$.json && $EDITOR /tmp/demlo-$$.json && demlo -p -i /tmp/demlo-$$.json "$@".

Long story short: you can edit the tags (and other transformations) with your favorite editor!

Rasi wrote:

* append chosen release types to album name ("EP" "single")

Demlo can obviously not decide the type of the album by itself. If it this info is already in the tags (or else it can be fetched from MusicBrainz with the -t parameter), then Demlo can act accordingly.

Rasi wrote:

plus the usual, tag the files, properly sort them into a target directory.

Obviously. See the scripts tag.lua and path.lua in the repo.

Rasi wrote:

if the answer to all of these is yes, picard might be obsolete for me smile

Hopefully it is!

Understand that Demlo is customizable in Lua, which is Turing-complete, so it yields you virtually unlimited customization possibilities, as opposed to the very limited "markup" that Picard and friends use.

Last edited by Ambrevar (2016-05-02 01:50:29)

Offline

#40 2016-05-03 04:42:56

Rasi
Member
From: Germany
Registered: 2007-08-14
Posts: 1,912
Website

Re: Demlo - A dynamic and extensible music library organizer

Demlo can obviously not decide the type of the album by itself. If it this info is already in the tags (or else it can be fetched from MusicBrainz with the -t parameter), then Demlo can act accordingly.

Indeed musicbrainz has this information. Damn, now I need time to learn the needed lua stuff smile


Regarding replaygain: Using ffmpeg to read gain information to know if a song already has information might be helpful. It would probably help already, if demlo simply dumped a list with tracks/directories that need to be replaygained.

Last edited by Rasi (2016-05-03 04:47:21)


He hoped and prayed that there wasn't an afterlife. Then he realized there was a contradiction involved here and merely hoped that there wasn't an afterlife.

Douglas Adams

Offline

#41 2016-05-03 06:01:49

Ambrevar
Member
Registered: 2011-08-14
Posts: 212
Website

Re: Demlo - A dynamic and extensible music library organizer

No worries, Lua's learning curve is very short. You don't even have to learn it at all to use Demlo: simply tinker with the official scripts and you should be good to go!
(Well, I believe Lua to be a beautiful language, you should give it a try anyways, you might like it!)

Replaygain: you don't need Demlo to dump audio details, just use mediainfo or ffprobe and filter the result. A one-liner will do.
If you can think of any good use-case with replaygain and Demlo, let me know and give me a concrete example.

Offline

#42 2016-05-04 22:17:41

Rasi
Member
From: Germany
Registered: 2007-08-14
Posts: 1,912
Website

Re: Demlo - A dynamic and extensible music library organizer

Hmm, the absence of any album logic is sad. Musicbrainz supports clustered lookups, This would speed up the whole process by a great margin.

The way I see it, demlo could do the following:

a) Collect all available data (tags)
b) group data (artist-album plus date, if available)
c) lookup musicbrainz releases by grouped data
d) process the rest track-by-track

Last edited by Rasi (2016-05-04 22:27:47)


He hoped and prayed that there wasn't an afterlife. Then he realized there was a contradiction involved here and merely hoped that there wasn't an afterlife.

Douglas Adams

Offline

#43 2016-05-05 08:34:16

Ambrevar
Member
Registered: 2011-08-14
Posts: 212
Website

Re: Demlo - A dynamic and extensible music library organizer

Demlo already optimizes MusicBrainz queries on a per-album basis, although it may query multiple times if the tags are too wrong.
I might have been re-inventing the wheel though. Do you have an example code of MB clustered lookups?

I've been thinking of implementing album-logic for a while, but it's a much harder problem than it looks.
I'd be happy to implement it, but I first need to discuss the benefits and implementation suggestions with other people. (E.g. you tongue)

Benefits:

- Possibility to force one single MB query per album.

- Per-album Replaygain values.

- Per-album offline tags: track numbers, total track-count, disc numbers.

Anything else?

Last edited by Ambrevar (2016-05-05 08:47:15)

Offline

#44 2016-05-06 14:36:54

Rasi
Member
From: Germany
Registered: 2007-08-14
Posts: 1,912
Website

Re: Demlo - A dynamic and extensible music library organizer

I think all clustering does it to group the albums in picards UI by artist/album. if demlo already does album lookup, that part is already covered.


He hoped and prayed that there wasn't an afterlife. Then he realized there was a contradiction involved here and merely hoped that there wasn't an afterlife.

Douglas Adams

Offline

#45 2016-05-06 23:19:25

Ambrevar
Member
Registered: 2011-08-14
Posts: 212
Website

Re: Demlo - A dynamic and extensible music library organizer

Yep, but the main difference between Picard / friends and Demlo is that Picard is a UI, while Demlo is a batch tool. By design, Demlo does not allow for selecting the right MusicBrainz result. 2 tracks from the same album can end up being tagged with 2 different MB album entries.

One way to overcome this would be to add a CLI flag forcing the online tagging to use a single MB entry. Not very pretty...

Offline

#46 2016-06-26 17:34:55

Ambrevar
Member
Registered: 2011-08-14
Posts: 212
Website

Re: Demlo - A dynamic and extensible music library organizer

Version 3.1 it out!

* FFmpeg metadata is fully accesible from the Lua scripts. (Previously only partly accessible.)

* Scripts are executed in lexicographic order. The '-s' flag inserts a script to the list instead of resetting the list. The new '-r' flag can remove scripts matching a regexp.

* The list of available scripts and supported extensions is printed at the beginning of each run.

* Debug messages have been streamlined.

* Consistency in online queries has been enforced: This will fix most of the previously inconsistent album tags when querying MusicBrainz. Network queries are also reduced which speeds up the whole process.

* Previously, TagLib would be used when the stream needn't be reencoded, but TagLib's interface does not allow tagging arbitrary tags. To enforce correct results, TagLib is only used when a specific set of tags is changed, otherwise fallback to using FFmpeg.

Edit: Option names in demlorc are now titlecased, update your config accordingly.

Last edited by Ambrevar (2016-07-01 17:22:25)

Offline

#47 2016-06-27 15:30:52

CjK
Member
Registered: 2008-10-17
Posts: 20

Re: Demlo - A dynamic and extensible music library organizer

That's good news, thanks for this release!

In particular I like the more stable internet-queries. In previous versions, I had to take between 1-5 tries before the Musicbrainz-lookup succeeded.

However, one thing that stopped working for me is the "copy.lua" script. I used a slightly customized version that you described on your website to move a music-file into a [artist]/[album]/[title] structure. Starting with version 3.1 this stopped working an the path is left untouched.

Perhaps this is due to the new lexical script ordering? What shall I do to make this work again?

Here is an example - the "warning" at the end is a symtom that the path-copy is no longer active/working:

% demlo -t Princess\ Chelsea\ -\ We\ Were\ Meant\ 2\ B.mp3
:: Load config: /home/cjk/.config/demlo/demlorc
:: Register extensions: aac ape flac m4a mp3 mp4 mpc ogg wav wv
:: System script folder: /usr/share/demlo/scripts
:: System scripts: ["10-tag.lua" "20-sub.lua" "30-case.lua" "40-punctuation.lua" "50-encoding.lua" "60-path.lua" "70-cover.lua"]
:: User script folder: /home/cjk/.config/demlo/scripts
:: User scripts: ["const.lua" "copy.lua" "ffmpeg.lua" "flac2ogg.lua" "humour.lua" "xray.lua"]
==> Princess Chelsea - We Were Meant 2 B.mp3

                                                                                              === FILE         ===
                                         [Princess Chelsea - We Were Meant 2 B.mp3]             | path         | [/home/cjk/music.raw/incubator/Princess Chelsea - We Were Meant 2 B.mp3]
                                                                                          [mp3] | format       | [mp3]
                                                                               [bitrate=134514] | parameters   | [[-c:a copy]]
                                                                                              === TAGS         ===
                                                                                             [] | album        | [The Great Cybernetic Depression]
                                                                                             [] | album_artist | [Princess Chelsea]
                                                                                             [] | artist       | [Princess Chelsea]
                                                                                             [] | date         | [2015]
                                                                                             [] | title        | [We Were Meant 2 B]
                                                                                             [] | track        | [6]
                                                                                              === COVERS       ===

:: Warning: Destination exists: /home/cjk/music.raw/incubator/Princess Chelsea - We Were Meant 2 B.mp3

Thanks,
Claus

Offline

#48 2016-06-27 20:54:25

Ambrevar
Member
Registered: 2011-08-14
Posts: 212
Website

Re: Demlo - A dynamic and extensible music library organizer

@CjK: No script gets loaded: either specify them in your configuration file, or add them from command-line with the '-s' flag.
Scripts are now run in lexicographic order, so your 'copy.lua' script will be run last. Prefix it with a number if you want to change the execution order.

Edit: Would it still be failing after fixing your demlorc, please post it here.

Last edited by Ambrevar (2016-06-28 13:21:49)

Offline

#49 2016-07-01 08:34:40

CjK
Member
Registered: 2008-10-17
Posts: 20

Re: Demlo - A dynamic and extensible music library organizer

Ah, adding the script on the command-line (-s option) helped. I didn't recognize those scripts where found but not loaded before.

But specifiying the scripts only in my config-file isn't enough - no script get's loaded despite I have the following line in my config file:

[...]
scripts = {'10-tag', '20-sub', '30-case', '40-punctuation', '50-encoding', '60-path', '70-cover'}

Is this the wrong syntax?

Other than that demlo finds and loads my config file just fine.

In the meantime, running demlo like this works fine as a workaround:

demlo -t -s '10-tag' -s '20-sub' -s '30-case' -s '40-punctuation' -s '50-encoding' -s '60-path' -s '70-cover' Princess\ Chelsea\ -\ We\ Were\ Meant\ 2\ B-RT6X1h-S5g0.mp3

thanks,
Claus

Offline

#50 2016-07-01 16:07:05

Ambrevar
Member
Registered: 2011-08-14
Posts: 212
Website

Re: Demlo - A dynamic and extensible music library organizer

Subtle typo: it's 'Scripts', with a capital S! smile

Make sure to update demlorc on every release, looking at the provided config sample.

The reason all the options are now titlecased is because of Go's syntax for exportation. It simplifies the code a bit.

Last edited by Ambrevar (2016-07-06 13:02:35)

Offline

Board footer

Powered by FluxBB