These enable reading from decompressed mail, but don't you need to also configure to store compressed mail? Have you tested that?

There was an issue raised last year about this feature and it seemed to require enabling for lmtp (which we use to hand mail over from Postfix to Dovecot for storage): #4178 (comment)

the full snippet includes the compression on save, i am not sure what is confusing?

    mail_plugins = $mail_plugins zlib

    # Enable these only if you want compression while saving: <<<<<<<<<<<<<<<<<<<<<<
    plugin {
      # zlib_save = gz
      # zlib_save_level = 1
      zlib_save = zstd
    }

    # Enable Zlib for imap
    protocol imap {
      mail_plugins = $mail_plugins zlib
    }

    # Enable Zlib for pop3
    protocol pop3 {
      mail_plugins = $mail_plugins zlib
    }

    # Increase memory allowed for imap as it costs more to read compressed files
    service imap {
      vsz_limit = 1GB
    }

anyways, I have now included 2 sections for DMS15- and DMS16+

I'm a bit reluctant to merge this as it would become part of our DMS v16 docs, and this table is copied from Dovecot 2.3 zlib_plugin docs, but DMS v16 will be upgraded to Dovecot 2.4 where the equivalent settings have been renamed (zlib_save_level is now split into multiple settings, one per codec).

Your related Github docs link is also the Dovecot 2.4 docs.

we must include some sort of summary for compression options, what harm is it? besides, the values have not changed aside from the keyword names.

So i have rewritten this section to separate between dovecot 2.3-/2.4+ and DMS 15-/16+

These are valid, but more generic and I think our standard debugging/troubleshooting docs page has that mostly covered. To avoid repeating that on all our config guides, we prefer to exclude such and only include when enough users repeatedly encounter these mistakes (which likely means our docs are failing them and that should be addressed instead).

I have encountered this mistake when I imported mailu into dms. Also I got it again when I did not load 12-compress,conf in the right order. I want to keep this in the document.

• There is no equivalent Dovecot setting (like for zstd or others) for LZ4 supported by Dovecot, just a call to the default LZ4 compression level.
• For reference the LZ4 CLI args explain default settings (and the related options to favor faster compression or higher compression levels).

I'm still of the opinion that this isn't that relevant for us to document though? This is a feature where Dovecot is compressing/decompressing from storage, none of it relevant to the client compatibility wise, so the choice is really down to zstd vs lz4 (lacks config flexibility), where LZ4's main advantage is decompression speed AFAIK but without that being tunable while zstd is, it may make zstd the sane choice.

More interested users that care about the difference could do so via browsing upstream docs if they care enough to make a decision there? Majority likely don't need to think much about this and would be fine with the happy path :)

There really shouldn't be a need to document the config filename by Dovecot version with the DMS comment. Additionally the yaml snippet was indented which was unnecessary.

I am a bit iffy with that choice for a 12- prefix, which seems arbitrary?

Reference - Why the numeric prefix (12-) in the config file name?

This PR doesn't document or mention the reasoning for the 12- prefix you chose for the config file (it was however previously explained here).

Without the prefix a warning would be logged like this:

dovecot: doveconf: Warning: /etc/dovecot/conf.d/compression.conf line 1: Global setting mail_plugins won't change the setting inside an earlier filter at /etc/dovecot/conf.d/15-lda.conf line 47 (if this is intentional, avoid this warning by moving the global setting before /etc/dovecot/conf.d/15-lda.conf line 47)

15-lda.conf is a Dovecot supplied config, DMS is not involved in that. It's related snippet triggering that warning is this:

protocol lda {
  # Space separated list of plugins to load (default is global mail_plugins).
  mail_plugins = $mail_plugins sieve
}

/etc/dovecot/dovecot.conf loads all configs in lexicographical order via this snippet:

# Most of the actual configuration gets included below. The filenames are
# first sorted by their ASCII value and parsed in that order. The 00-prefixes
# in filenames are intended to make it easier to understand the ordering.
!include conf.d/*.conf

In this case 15-lda.conf was processed earlier than any non-numeric prefixed config files, where it referenced the global setting $mail_plugins before the later parsed config (compression.conf) was loaded with it's own modification to the mail_plugins global setting. As such, 15-lda.conf (and any other configs using $mail_plugins, since Dovecot only logs the first instance encountered to trigger that warning) the expected change wasn't applied (unless the more explicit protocol override was present which also included the change).

So to avoid that warning, 12- as a prefix does work... but it could still introduce the problem should a custom config (or Dovecot update) introduce similar earlier than that 12- prefix. Alternatively, as per the affected Dovecot snippets, one could just reference the services themselves to avoid the otherwise necessary prefix and warnings concerns.

DMS also supplies custom 20-lmtp.conf + 20-imap.conf configs (there's additionally 20-pop3.conf included internally from Dovecot, which DMS does not provide an override for, where mail_plugins for that protocol/service is unset, and thus only uses whatever was set globally):

Along with DMS supplied 10-mail.conf (which sets the mail_plugins global setting):

Avoiding the global setting, the warning is avoided, but as a result services not explicitly configured would lack the plugin being enabled (which was the concern I raised in my previous review feedback and referenced my comment from an earlier zlib config issue):

services:
  dms:
    image: ghcr.io/docker-mailserver/docker-mailserver:15.1
    hostname: mail.example.test
    configs:
      - source: dms-accounts
        target: /tmp/docker-mailserver/postfix-accounts.cf
      - source: dovecot-compression-2.3
        target: /etc/dovecot/conf.d/compression.conf

# Docker Compose `configs` feature
# NOTE: `$` literals must be escaped as `$$` to avoid being mistaken for ENV interpolation.
configs:
  dovecot-compression-2.3:
    content: |
      # Configure the `zlib` plugin to use `zstd`:
      plugin {
        zlib_save = zstd
      }

      # Globally enable:
      #mail_plugins = $$mail_plugins zlib

      # Enable per protocol (redundant if globally enabled)
      protocol imap {
        mail_plugins = $$mail_plugins zlib
      }

      protocol pop3 {
        mail_plugins = $$mail_plugins zlib
      }

  dms-accounts:
    content: |
      john.doe@example.test|{SHA512-CRYPT}$$6$$sbgFRCmQ.KWS5ryb$$EsWrlYosiadgdUOxCBHY0DQ3qFbeudDhNMqHs6jZt.8gmxUwiLVy738knqkHD4zj4amkb296HFqQ3yDq4UXt8.

# Global setting:
$ doveconf mail_plugins
mail_plugins =  quota

# LMTP missing support as it wasn't explicitly configured,
# without being set globally nor was it inherited.
$ doveconf -f protocol=lmtp mail_plugins
mail_plugins =  quota sieve

# IMAP added support:
$ doveconf -f protocol=imap mail_plugins
mail_plugins =  quota imap_quota zlib

# POP3 originally only used global config:
$ doveconf -f protocol=pop3 mail_plugins
mail_plugins =  quota zlib

So this would enable IMAP/POP3 read support, where Dovecot would decompress such mail upon retrieval, but when mail is delivered (write) to Dovecot it will be via LDA or LMTP, so these are relevant too.

Test mails to the Dovecot LMTP and LDA services can be tried via:

# LMTP:
# `--socket` + `--protocol` options to send via LMTP to Dovecot directly (instead of SMTP via Postfix)
$ swaks --silent --socket /var/run/dovecot/lmtp --protocol lmtp \
  --from hello@not-relevant.test \
  --to john.doe@example.test


# LDA:
# Have swaks dump it's test mail to a file:
$ swaks --from hello@not-relevant.test --to john.doe@example.test --dump-mail > /tmp/example.eml
# Then reference that via `dovecot-lda` with the `-p` option, `-d` must be given for where to deliver:
$ /usr/lib/dovecot/dovecot-lda -d john.doe@example.test -p /tmp/example.eml


# Both delivered as text (no compression):
# Types identified differ as `dovecot-lda` excludes additional mail headers that `file` recognizes
$ file /var/mail/example.test/john.doe/new/*
/var/mail/example.test/john.doe/new/1762066677.M530062P7803.mail.example.test,S=549,W=566: SMTP mail, ASCII text
/var/mail/example.test/john.doe/new/1762066687.M352940P7825.mail.example.test,S=276,W=285: news or mail, ASCII text


# For reference, this would submit mail to Postfix that delivers (via LMTP) to the Dovecot managed mailbox:
swaks --silent --server 0.0.0.0 --port 465 \
  --auth-user john.doe@example.test --auth-pass secret \
  --from hello@not-relevant.test \
  --to john.doe@example.test

Normally mail will go through Postfix which will deliver mail to Dovecot via that LMTP socket, but it's also possible that Getmail/Fetchmail services for example are using dovecot-lda to deliver mail to Dovecot directly without going through LMTP (see this example documented by me a while back). Thus LDA config should be considered when configuring this feature too.

Therefore, just as the Dovecot docs config example shows, we should just keep it simple with a global mail_plugins setting only? (which would require a numeric prefix below 15) This way the user doesn't need to think about each protocol used and their separate syntax to support?

Unresolved for visibility.

This is only considering imap, as noted in other feedback that may be inaccurate. If memory is an issue it could be relevant to consider the cost for compression for inbound mail large enough to warrant this, in addition to the outbound read to clients over IMAP and potentially POP3.

A global upgrade to that setting may be better?