Moovember updates #1

Updated on Moovember 14, 2018: More updates!

Important changes

  • You can now add a subdomain for all existing domains by using ADDITINAL_SAN like ADDITINAL_SAN=mail.* – thanks to @markusg on GitHub!
  • mail_log is enabled, if it is heavy on resources, let us know.
  • There is a new mailbox_format attribute in vars.inc.php, that CAN be changed to mdbox, but you most likely break existing ACLs on mixed-setups (also change dovecot.conf to use mdbox for shared namespace) – unsupported.
  • We use a /var/volatile directory for files that can cause trouble on NFS shares (e.g. locking files)
  • A custom-sogo.js file is now included to SOGo by default. This allows, for example, to set CKEditor attributes.

Summary

Added on 15th Oct

[Web] Add “alias_domains” ACL to prevent alias domains to add alias domains (by default!)
[Web] Edit alias domains: use select menu
[Web] Minor fixes


[Compose] Remove dedicated index (wip)
[Web] mailbox_format maildir
[Dovecot] Enable mail_log (events: delete undelete expunge copy mailbox_delete mailbox_rename)
[Dovecot] Increase vsz_limit for some services to 1 G
[Dovecot] Enable auth_cache
[Dovecot] Remove dedicated index (wip)
[SOGo] Fix sogo_view
[Config] Add info for sub.* records to generate_config.sh
[Compose] New images for Rspamd, SOGo, Dovecot, Postfix, ACME
[Compose] New volume for deduplicated attachments <- only on mdbox; encrypted [Web] Adjust mailbox format
[Web] Include IMAP lib for future use
[Web] Fix default exception handler
[Web] Fix sync job edit forms
[Web] Other minor fixes
[Web] Fix _sogo_static_view creation when parent tables changed order of cols
[Web] Fix details for blind DKIM keys
[SOGo] Include custom-sogo.js to dynamically add JS to SOGo, increase textarea font of CKeditor by default
[Rspamd] Add fuzzy hash to msg
[Rspamd] Add SOGo contacts to whitelist
[SOGo] Adjust SOGo view
[Nginx] Remove Strict-Transport-Security for subdomains (prevented autoconfig from working without TLS)
[Rspamd] Add stopsignal (testing)
[Dovecot] Create crypted mail_attachment_fs to store attachments with a min size of 128k
[Dovecot] Shared location to “auto:” to auto-detect legacy mailbox formats across shared mailboxes <- reverted, wip [Dovecot] Create config service for crypted mail_attachment_fs
[Postfix] Adjust mailbox query
[Dovecot] Dovecot 2.3.3, Pigeonhole 0.5.3
[Dovecot] Use “–enable-hardening” flag
[Dovecot] Fix cronjobs
[Dovecot] Use /var/volatile to prevent locking files from being written to NFS storage (if vmail is on NFS)
[Dovecot] Change userdb query
[Dovecot] Use /var/attachments for mdbox attachment deduplication and /var/index for index files <- index reverted, deduplication only with mdbox [Dovecot] Fix sieve user creation
[Dovecot] Make console writable
[Dovecot] Fix trim_logs.sh
[ACME] Allow for sub.* values in ADDITIONAL_SAN
[Rspamd] Reduce rspamd DNS timeout
[Web] Fix init_db for older mailcow installations, fixes #1961

Close-to-Halloween updates and fixes

Important changes

  • We do now try to reload a service instead of restarting it, when a certificate changed. When reloading fails, we restart the container.
  • A supervisord controlled container will now die when a program it started exits.

Summary

[Web] More mailq fixes
[Compose] Update SOGo, Dovecot and Postfix images
[Web] Fix mailq styles in /admin
[Web] Move ‘get’ method to mailq functions file
[Web] Add overflows in /admin for small devices
[Web] Fix maildir cleanup after deleting mailbox
[Postfix] Use events to kill supervisord when main proc dies
[SOGo] Use events to kill supervisord when main proc dies
[Dockerapi] Some minor changes
[Web] Cleanup _sogo_static_view and memcached
[Compose] Update Docker API and ACME images
[ACME] Try to reload services after certificate changes instead of restarting
[DockerAPI] Add service reload commands
[Postfix] Do not remove user agent


Updated on 27 Oct:

[Rspamd] Change log level to silent (see docs)
[Rspamd] Adjust default values for (perm) failures of DKIM and SPF
[Compose] Update ClamAV, watchdog and Docker API images
[Compose] Remove whitelist mount in ClamAV service
[DockerAPI] Add top and stats
[ClamAV] Do not try to modify cross-mounted file, copy whitelist from conf to lib directory
[ClamAV] Remove AllowSupplementaryGroups from freshclam.conf (deprecated)
[Watchdog] Check if initdb is running and if true skip killing php-fpm-mailcow
[Watchdog] Allow multiple rcpts separated by comma
[Postfix] Add tls_preempt_cipherlist to SMTPS
[Update] Remove obsolete parameters


Updated on 28 Oct:

[Web] Fix sieve validation, fixes #1960
[Update] Change umask for update to ensure its 0022
[Watchdog] Skip container restart if running for less than 120 seconds

Some more features

Important changes

  • No breaking changes, don’t worry… 🙂
  • New: Postqueue manager
  • New: Grant/Disallow SOGo access
  • New: Reset SOGo profiles

Summary

[Compose] Update SOGo and Docker API images
[Web] Queue manager for Postfix
[Web] Add sogo_access mail attribute
[Web] Allow to wipe SOGo profiles
[SOGo] Read .sogo_access attribute when bootstrapping view
[DockerAPI] WIP: change of structure, add some more commands to control mail queue
[Helper] Do not use full network name for MySQL backup

“Almost November” updates

Important changes

  • We do not request autoconfig.* names anymore!
    Who needs to change what?
    – If you are using a HTTP -> HTTPS redirect without reverse proxy, check the updated docs here. The first “server” block is new, you probably already use the second “server” block. 🙂
    – If you are using a reverse proxy, you should check the updated guides here.
  • The default guide for a reverse proxy setup has changed! See here. We use “acme-mailcow” as ACME client in our examples now. It is probably easier for most use-cases. You don’t need to change your current configuration – besides stopping redirecting autoconfig.* to a HTTPS session. Check out the examples for Nginx and Apache.
  • Rspamd 1.8.1

Summary

[Compose] New Rspamd image (1.8.1)
[Compose] Update ACME and Rspamd images
[Watchdog] Remove cert check (wip)
[Watchdog] Append last check loop as attachment to mail
[Watchdog] Print time and date in mail alerts
[ACME] Stop requesting certificates for autoconfig.*
[Rspamd] Upgrade base to Bionic
[Rspamd] Remove deprecated attachments_only in AV module
[Rspamd] Remove old symbol score
[Update] Checkout pcre header check if missing, fixes #1906
[Update] Remove old header check to prevent update failure
[Unbound] Reduce negative max ttl to 60s and min-ttl for all other keys to 5
[Web] Fix API (broken in previous update, still a wip)
[Web] Set new expire date for time limited aliases via actions button, fixes #1903
[Web] Hardening HTTP headers
[Web] Hide autodiscover records on DNS page for alias domains
[Web] Read default actions from Rspamd instead of using/printing “5,15”
[Web] Allow to reset spam score to server default (which deletes the custom spam score from the database and prints the default action values of Rspamd in use)
[Postfix] Change mail_name to Postcow and only replace headers when mail_name matches
[Postfix] Remove headers only when mail_name matches
[PHP-FPM] Disabling more functions inside php-fpm

October updates (more updates!)

Updated on October 15, 2018: More updates!

Important changes

  • New: send system emails to mailboxes hosted on mailcow (via LMTP)
  • API table changes (in case anyone is using it already ;-))
  • Add multiple administrators
  • Database initialization is now run in the entrypoint script, check php-fpm-mailcow logs if it fails to start
  • Removed Bitcoin donation and added liberapay.com/mailcow
  • Support packages are almost here (many thanks to Tim Korves for everything!)

Summary

Added on 15th Oct

[PHP-FPM] Disable some functions by default
[Postfix] Add mailcow_anonymize_headers to default config
[Web] Minor language fix
[Helper] Add MAILCOW_BACKUP_LOCATION as alternative to BACKUP_LOCATION to backup script, fixes #957


Added on 14th Oct

[PHP-FPM] Base on Alpine 3.8
[ACME] Base on Alpine 3.8
[ACME] Do not add alias domains to auto* domains
[Web] Fall back to raw content when mail parsing fails, fixes #1892
[Compose] Add some parameters to watchdog-mailcow
[Compose] New images for ClamAV, ACME and watchdog
[Compose] New PHP-FPM image
[Watchdog] Minor changes
[Watchdog] Base on Alpine 3.8
[Watchdog] Remove some check_ping checks
[Watchdog] Add ClamAV check (if SKIP_CLAMD=n)
[Watchdog] Add Unbound check
[Watchdog] Do not use Docker API by default to determine IP of containers (see “IP_BY_DOCKER_API”)
[Watchdog] Minor changes
[ClamAV] Update to 0.100.2
[Netfilter] Remove duplicate import
[Unbound] Upgrade to Alpine 3.8, fixes #1882


[Compose] Update Postfix and Dovecot images
[Compose] New images: Unbound, PHP-FPM, SOGo, Dovecot, ACME
[Postfix] Proper permissions for sql config files
[Dovecot] Proper permissions for sql config files
[Dovecot] Set imap_max_line_length = 2 M
[Dovecot] Use mysqladmin status instead of ping to determine readiness
[README] Remove Bitcoin donation link, add liberapay.com/mailcow
[Config] Add allowed chars for API key
[Helper] Fix mailcow reset admin to work in multi-admin environment
[Web] Some language updates for sys mails
[Web] Fix require_once to always include document root
[Web] Add system mails (send mails to all mailboxes via LMTP)
[Web] Allow to add more administrators
[Web] Fix domain administrator editing
[Web] Remove some foreign keys
[Web] Remove username from API
[Web] Remove more .php extension from code
[Web] More minor fixes
[Rspamd] Prefix quarantine error_log messages with “QUARANTINE”
[Rspamd] Fix quarantine max size check (it was ignored)
[PHP-FPM] Move max_execution_time and max_input_time to general PHP config, removed as fixed php_admin_value
[PHP-FPM] Use mysqladmin status instead of ping to determine readiness
[PHP-FPM] Init database in entrypoint
[PHP-FPM] Change API credential injection
[ACME] Log acme-client output base64 encoded, use mysqladmin status instead of ping to determine readiness
[SOGo] Use mysqladmin status instead of ping to determine readiness

Updates, again…

Important changes

  • “Better” URLs, edit.php?what=item becomes edit/what/item etc.
  • Define default mailbox attributes for new mailboxes (as of today: “tls_enforce_in”, “tls_enforce_out”, “force_pw_update” – all default to false) =>
    // Force incoming TLS for new mailboxes by default
    $MAILBOX_DEFAULT_ATTRIBUTES['tls_enforce_in'] = false;
    
    // Force outgoing TLS for new mailboxes by default
    $MAILBOX_DEFAULT_ATTRIBUTES['tls_enforce_out'] = false;
    
    // Force password change on next login (only allows login to mailcow UI)
    $MAILBOX_DEFAULT_ATTRIBUTES['force_pw_update'] = false;
    

Summary

[Compose] New Postfix image
[Web] Fix domain admin edit function
[Web] Feature: TLS policy maps
[Web] Avoid php extensions in links
[Web] Minor fixes
[Postfix] Enable/create smtp_tls_policy_maps
[Nginx] Avoid php extensions, use rewrite
[SOGo] SOGoMaximumSyncWindowSize = 99

Updates, updates, updates…

Important changes

  • Maildir encryption is enabled by default! Backup “crypt-vol-1”! You lose/delete this key, you lose your mail. There is no way to recover them.
    bash helper-scripts/backup_and_restore.sh backup crypt
    
  • Deleted mailboxes and domains will be moved to /var/vmail/_garbage and cleaned up after $MAILDIR_GC_TIME minutes, the collector runs hourly
  • Rspamd controller password change commands are now piped to a bash to hide them from process lists
  • Docker API now uses a self-generated key pair
  • Unbound logging is finally fixed
  • “unbound-control” was made available
  • Peer Heinlein allowed us to use their SA rules, many thanks!

Summary

[Update] Add MAILDIR_GC_TIME
[Postfix] Increase default message size limit to 100 MiB
[Rspamd] Add desc to high spam networks
[Rspamd] Ignore custom files, but keep bad asn map
[Rspamd] Fix permissions of controller password file
[Rspamd] Place socket in _rspamd home and fix permissions
[Rspamd] Ignore sa-rules-heinlein file, remove from index
[Unbound] Fix logging, fixes #585
[Unbound] Enable unbound-control
[Docker API] Use TLS encryption for communication with “on-the-fly” created key paris (non-exposed)
[Docker API] Create pipe to pass Rspamd UI worker password
[Dovecot] Do not query gid and uid
[Dovecot] Pull Spamassassin ruleset to be read by Rspamd (MANY THANKS to Peer Heinlein!)
[Dovecot] Garbage collector for deleted maildirs (set keep time via MAILDIR_GC_TIME which defaults to 1440 minutes)
[Dovecot] Encrypt maildir with global key pair in crypt-vol-1 (BACKUP!), also fixes #1791
[Dovecot] Check garbage hourly
[Dovecot] Update SA rules once when container starts
[Web] Flush memcached after mailbox item changes, fixes #1808
[Web] Fix duplicate IDs, fixes #1792
[Web] Fix deletion of spam aliases
[Web] Do not exit loop on fuzzy errors when learning a message as spam
[Compose] Use SQL sockets
[Compose] New images for Rspamd, PHP-FPM, SOGo, Dovecot, Docker API, Watchdog, ACME, Postfix
[Compose] Update Unbound image and set tty true
[Compose] Remove volume for Rspamd socket
[PHP-FPM] Update APCu and Redis libs
[Helper] Add “crypt” to backup script
[Helper] Override file for external SQL socket (not supported!)

ACL and ‘;–have i been pwned?

Hi,

I would love to get some feedback on the ACL implementation. If you find bugs etc., please let us know @ GitHub.

There is some info in the docs => https://mailcow.github.io/mailcow-dockerized-docs/model-acl/ – they still need more updates.

One improvement I see is to hide the divs completely and/or deny access to the functions ‘get’ methods. Let us know on Freenode, #mailcow.

Thanks for the idea to integrate haveibeenpwned.com, I like it! Sorry to haveibeenpwned.com for playing with it and trying a bunch of old passwords, I hope I didn’t hammer your API too much. 🙂

For your information: Your password is never sent to their API!
We only query the API with the first 5 characters of the SHA1 hash of the current input fields value (generated in your browser, not server-side) and check the response for matches of the full hash, still stored in your browser.

André

Updates and two important fixes

We just fixed SOGo theme switching again. There is a chance we did it, I promise…

Knight1 made us aware of a critical bug, that led to mailcow accepting custom X-FORWARDED-FOR headers. This bug was introduced with the last update.

Important change: We disabled “any” and “all authenticated” ACL settings in Dovecot and removed the box in SOGos ACL editor (big thanks to the SOGo devs, please help them, buy a subscription!).
You can find information about how to re-enable it here.

We will add an easy way to enable your SOGo subscription soon.

Please don’t forget to support mailcow. 🙂