You are not logged in.

#1 2016-02-01 20:05:14

student975
Member
From: Russian Federation
Registered: 2011-03-05
Posts: 613

IMAP <-> Maildir sync with utf7 <-> utf8 support

Hi!

I'm looking for a tool to bidirectionally synchronize IMAP accounts and Maildirs. There is a set of such tools, offlineimap and mbsync (from isync package) are ones of the most used. But there is a problem: IMAP uses modified utf7 encoding for non-ascii folder names. mbsync doesn't support folder names translation (that is utf7 <-> utf8, where utf8 is for file system) at all, and offlineimap supports just one-direction sync IMAP -> Maildir for accounts with such folders [1].

So, are there another ways to bidirectionally synchronize multiple IMAP accounts (with non-ascii folders) and local Maildirs?
What do you think?

[1] https://github.com/OfflineIMAP/offlineimap/issues/299


"I exist" is the best myth I know..

Offline

#2 2016-02-02 23:32:30

student975
Member
From: Russian Federation
Registered: 2011-03-05
Posts: 613

Re: IMAP <-> Maildir sync with utf7 <-> utf8 support

I see, nobody knows how to do smile So, let's try another way.

There is IMAP server dovecot [1] which does support some kind of bidirectional synchronizing [2]  (and dovecot does support utf7 <-> utf8). I have tried to dig in, but quickly became lost inside infinite new world of dovecot concepts and configuration options.

So, I ask you just few start points where to see, where to apply efforts to understand the dovecot way (if it is the way at all).

Basically, there are a list of pairs in hands. A pair consists of:

1. Maildir path
2. IMAP account parameters (host, port, user/pass, auth protocol, optionally a list of IMAP folders to use)

The aim is to bidirectionally synchronize (some or all) pairs the way mbsync or offlineimap do.

[1] http://dovecot.org/
[2] http://wiki2.dovecot.org/Tools/Doveadm/Sync

Last edited by student975 (2016-02-02 23:35:05)


"I exist" is the best myth I know..

Offline

#3 2016-02-03 19:16:12

ghen
Member
From: Belgium
Registered: 2010-08-31
Posts: 121

Re: IMAP <-> Maildir sync with utf7 <-> utf8 support

I think the dovecot sync utilities are designed to replicate dovecot servers, not 3rd party IMAP servers, but you could try it.

Another thing you could try is to run dovecot over your local Maildir, and then use one of the existing imap-to-imap sync tools, ie. without accessing the Maildir directly.  Then dovecot will take care of the utf7 conversion.

Offline

#4 2016-02-03 19:40:44

student975
Member
From: Russian Federation
Registered: 2011-03-05
Posts: 613

Re: IMAP <-> Maildir sync with utf7 <-> utf8 support

@ghen, yes, it seems doveadm sync isn't intended to work with remote independent IMAP accounts.

End user mail client (say, mu4e, notmuch, ...) works with Maildir in both direction. In particular, Maildir modification must be propagated to remote IMAP accounts. If I understand your suggestion correctly, the whole structure looks this way:

end-user-client <-> Maildir <-> dovecot <-> imap2imap <-> net <-> remoteImapServer

That is dovecot must listen Maildir modification. Is it possible? Is my understandib correct?


"I exist" is the best myth I know..

Offline

#5 2016-02-03 21:44:47

ghen
Member
From: Belgium
Registered: 2010-08-31
Posts: 121

Re: IMAP <-> Maildir sync with utf7 <-> utf8 support

Dovecot does not mind others accessing/modifying the Maildir directly.
I've done this for years -- procmail & mutt manipulate my Maildir locally, combined with a dovecot on top for remote IMAP access.

Offline

#6 2016-02-03 22:16:20

student975
Member
From: Russian Federation
Registered: 2011-03-05
Posts: 613

Re: IMAP <-> Maildir sync with utf7 <-> utf8 support

@ghen, can you, please, (partly?) share your configurations if it's possible?


"I exist" is the best myth I know..

Offline

#7 2016-02-05 20:28:58

ghen
Member
From: Belgium
Registered: 2010-08-31
Posts: 121

Re: IMAP <-> Maildir sync with utf7 <-> utf8 support

Quite generic really:

protocols = imap
auth_mechanisms = plain login
ssl = required
[more ssl stuff...]

# system users, separate passwords
userdb {
  driver = passwd
}
passdb {
  args = /etc/dovecot/dovecot.passdb
  driver = passwd-file
}
mail_location = maildir:~/Maildir

Offline

#8 2016-02-05 21:10:07

student975
Member
From: Russian Federation
Registered: 2011-03-05
Posts: 613

Re: IMAP <-> Maildir sync with utf7 <-> utf8 support

Thanks, will dig in.


"I exist" is the best myth I know..

Offline

#9 2018-02-27 21:19:31

roolebo
Member
From: Russia, St.Petersburg
Registered: 2010-01-17
Posts: 3

Re: IMAP <-> Maildir sync with utf7 <-> utf8 support

student975 wrote:

So, are there another ways to bidirectionally synchronize multiple IMAP accounts (with non-ascii folders) and local Maildirs?
What do you think?

It's possible to manually create localized symlinks to mUTF-7 encoded Maildirs. Then you might use the former ones in your MUA and ignore them in mbsync, so mbsync will only use the latter ones. Here's the layout of my ~/.mail/gmail/[Gmail]/ and excerpts from configs for mutt and mbsync:

drwx------  7 roolebo  staff  224 Feb 27 23:39 &BB4EQgQ,BEAEMAQyBDsENQQ9BD0ESwQ1-
drwx------  7 roolebo  staff  224 Feb 27 23:39 &BB8EPgQ8BDUERwQ1BD0EPQRLBDU-
drwx------  7 roolebo  staff  224 Feb 27 23:39 &BBIEMAQ2BD0EPgQ1-
drwx------  7 roolebo  staff  224 Feb 27 23:39 &BBIEQQRP- &BD8EPgRHBEIEMA-
drwx------  7 roolebo  staff  224 Feb 27 23:39 &BBoEPgRABDcEOAQ9BDA-
drwx------  7 roolebo  staff  224 Feb 27 23:39 &BCEEPwQwBDw-
drwx------  7 roolebo  staff  224 Feb 27 23:39 &BCcENQRABD0EPgQyBDgEOgQ4-
lrwxr-xr-x  1 roolebo  staff   27 Feb 26 00:03 Вся почта -> &BBIEQQRP- &BD8EPgRHBEIEMA-
lrwxr-xr-x  1 roolebo  staff   13 Feb 26 00:20 Спам -> &BCEEPwQwBDw-
lrwxr-xr-x  1 roolebo  staff   18 Feb 26 00:21 Важное -> &BBIEMAQ2BD0EPgQ1-
lrwxr-xr-x  1 roolebo  staff   21 Feb 26 00:01 Корзина -> &BBoEPgRABDcEOAQ9BDA-
lrwxr-xr-x  1 roolebo  staff   26 Feb 26 00:04 Черновики -> &BCcENQRABD0EPgQyBDgEOgQ4-
lrwxr-xr-x  1 roolebo  staff   29 Feb 26 00:21 Помеченные -> &BB8EPgQ8BDUERwQ1BD0EPQRLBDU-
lrwxr-xr-x  1 roolebo  staff   34 Feb 26 00:09 Отправленные -> &BB4EQgQ,BEAEMAQyBDsENQQ9BD0ESwQ1-

mutt shows nice localized folder names:

set spoolfile = "~/.mail/gmail/Inbox"
set folder    = "~/.mail/gmail/[Gmail]"
set mbox      = "+Вся почта"
set postponed = "+Черновики"
set record    = "+Отправленные"
set trash     = "+Корзина"
mailboxes ">" "+Важное" "+Помеченные" "+Черновики" "<" "+Корзина" "+Спам"

Instead of specifying UTF-7 encoded nonsense explicitly, I rather sync all folders subfolders but ignore symlinks. I think both approaches are viable though:

# Gmail
IMAPAccount gmail
Host imap.gmail.com
User XXX
PassCmd "XXX"
SSLType IMAPS
AuthMechs LOGIN

IMAPStore gmail-remote
Account gmail
# Required for INBOX-only sync to work
PathDelimiter /

MaildirStore gmail-local
Path ~/.mail/gmail/
Inbox ~/.mail/gmail/Inbox
SubFolders Verbatim

Channel gmail
Master :gmail-remote:
Slave :gmail-local:
Patterns "INBOX" * !"[Gmail]/Вся почта" !"[Gmail]/Черновики" !"[Gmail]/Отправленные" !"[Gmail]/Корзина" !"[Gmail]/Спам" !"[Gmail]/Важное" !"[Gmail]/Помеченные"

MaxMessages 200
Create Both
Expunge Both
SyncState *

A quick and dirty decoding of mUTF-7 might be handy for link creation:

echo '&BBIEQQRP- &BD8EPgRHBEIEMA-' | tr "&" "+" | tr "," "/" | iconv -f UTF-7 -t UTF-8
Вся почта

-Roman

Offline

Board footer

Powered by FluxBB