MariaDB 10.3 rollout

We just included MariaDB 10.3. If you run ./update.sh, you will encounter some errors in mysql-mailcow, that will be fixed by the upgrade process triggered by php-fpm-mailcow.
A SQL backup is recommended.

EN: If you have a mailcow support subscription, feel free to use the ticket system for assistance or help after a failed update.
DE: Falls ein mailcow Supportpaket gebucht wurde, helfen wir gerne beim Update der mailcow Installation oder bei etwaigen Problemen.

Mootember updates

Sorry for the delay, mailcow was still worked on. Time is running too fast. 🙁

Instead of writing down all commits you can already find on GitHub, I will only mention important changes or fixes dating back from the previous post until today.

Edit: Forgot to say thanks to @irgendwr for fixing XSS!

  • The GAL (Global Address List) is now enabled by default. Various actions like availability in calendars depends on it.
  • Images are prefetched on ./update.sh (`–prefetch` will only prefetch images and exit)
  • Added SOGO_EXPIRE_SESSION variable to mailcow.conf to define when a session in SOGo times out
  • Added a whitelist map for IPs in Rspamd (`data/conf/rspamd/custom/ip_wl.map`, CIDR)
  • SAL was introduced – an **optional** license with some benefits in the future. Think about basic monitoring etc. 🙂
  • Various XSS fixes by @patschi and @irgendwr – thanks!
  • Some services were finally ported to Py3 – thank you @zkryakgul! Also thank you for exposing your policyd password in your commit, we will take care of it.
  • Added a bad word list, that triggers only, when received from a fishy tld (yes, that’s a new map, too)
  • You can now allow a mailbox to send from an external domain or only a defined set of mail addresses (edit a mailbox to find this feature)
  • @christianbur forces me to update the images more regulary, thank you.
  • Various fixes and changes in mailcow UI
  • Solr is now exposed to 127.0.0.1:18389 by default, you can setup a reverse proxy to browse its fancy UI – do not expose it to the internet!
  • I broke the anonymize headers, again. @iiegn and @patschi pointed it out, thanks!

Attention for Ubuntu 18.04 users: Kernel 4.15.0-60 causes kernel panic

This is a short heads-up for users running their mailcow instance on Ubuntu 18.04: Do NOT upgrade to kernel 4.15.0-60 as of now!

Based on various user reports and more extensive testing this specific kernel release triggers a kernel panic and crashes your server. So far we have been able to figure out, this is related when setting a nameserver in the docker-compose.yml file explicitly (which isn’t new in our code). For some strange reason this commit/hotfix fixes the crashes so far.

You can track the progress for this issue here. However please keep in mind that this is a small warning for our Ubuntu users, as there’s currently no final fix available yet. We’ve also filed a Ubuntu bug report already.

Update on 5th September: The issue was confirmed as a valid kernel bug and is already fixed by the Ubuntu team, see the Launchpad issue here. However, there’s no ETA when the new kernel update will be released.

Update on 11th September: The new fixed kernel image 4.15.0-62.69 was released on 9th September.

Security, security and security

So what happened the last few days and weeks? You guessed it! Some important security updates!

To sum it up:
Critical: There was a dovecot security issue fixed which can lead to private information leakage and remote code execution. Read more here.
– ClamAV got updated to v0.101.4 to address zipbomb vulnerability. Read more here.
– A few XSS vulnerabilities in the mailcow UI were fixed.
– Beside the security fixes there were also more mailserver blacklists and spam rules added to improve spam detection.

An update of your mailcow instances is strongly recommended and should be done as soon as possible. Update is possible as usual by executing the update script.

(Telegram users might be wondering: Yes, this is indeed a cross-post from my Telegram News post a few days arlier.)

 

Update your HTTP to HTTPS redirection and mailcow to prevent acme-mailcow HTTP verification failures

Hi,

Please see the updated reverse proxy docs and adjust your site.conf (only if you are not using a reverse proxy!).

An older draft excluded autoconfig from the redirection. Older setups may still have two server blocks at the top of that file. This is obsolete and needs to be changed! This is what it looked like before, it is wrong!

Correct and new: https://mailcow.github.io/mailcow-dockerized-docs/u_e-80_to_443/
Wrong and old: https://github.com/mailcow/mailcow-dockerized-docs/blob/cdf1a436ca904186d78e391bab71f2747af2a1af/docs/u_e-80_to_443.md

Please also update mailcow or at least acme-mailcow (make sure you use at least v1.60).

June updates

For commercially used mailcows, please consider buying a support subscription or help to keep mailcow alive by donating. 🙂

ONCE AGAIN A BIG THANKS TO ALL CONTRIBUTORS!

Important changes

  • Thanks to @c-rosenberg we integrated olefy into mailcow for macro scanning. It is skipped for authenticated users.
  • Quarantine can now be configured to remove old items.
  • *.autoconfig is back as part of default SANs in the certificate acquired by “acme-mailcow”.
    IMPORTANT: The certificate request will fail, if you copied the site config from our previous HTTP -> HTTPS redirect docs! Please update your site config (see here), make sure to remove the autoconfig block (see here).

Summary

[ACME] Autoconfig is back (re-added to SAN list by default for all mail domains)
[ACME] Better HTTP verification
[ClamAV] Fix missing exit code var, fixes #2746
[ClamAV] Update to 0.101.2
[Dovecot] Fix cleanup of old quarantine items, fixes #2721
[Dovecot] Rename sieve_after to global_sieve_after and create a global_sieve_before file
[Dovecot] Run cronjob at 4:15 AM to remove max aged quarantine items (default: 365 days)
[Dovecot] Fix permissions of console
[Helper] Omits the verbose option for gzip in the backup script
[Netfilter] Keep musl-dev, update pip
[Netfilter] Remove unused files after installation
[Netfilter] Set some f2boptions to int
[Olefy] A new container is born, thanks to @c-rosenberg
[PHP-FPM] Add exif module
[PHP-FPM] Add Q_MAX_AGE key and set to 365 days if missing
[PHP-FPM] Update PHP and some modules
[Postfix] Add UA header check, not enabled by default
[Postfix] Create resource maps
[Postfix] Remove authed user from header
[Postfix] Remove duplicate proxy read maps, add resource maps
[Rspamd] Added comment to composite
[Rspamd] Add OLEFY_MACRO symbol
[Rspamd] Add oletools via olefy, big thanks to @c-rosenberg
[Rspamd] Increase OLEFY_MACRO score
[Rspamd] Less aggressive bayes
[Rspamd] Remove authenticated user from auth results header
[Rspamd] Sign ARC inbonud, thanks to @Kraeutergarten
[Update] Add missing WATCHDOG_NOTIFY_BAN update option
[Update] Check if file is tracked before running git rm
[Update] Hide error when running git rm on old worker passwd file
[Update] Remove controller passwd file from index
[update.sh] Rename enable_ipv6 option
[Watchdog, Config] Added WATCHDOG_NOTIFY_BAN to disable IP ban notifications
[Watchdog] Fix broken mail with more than one rcpt in some cases
[Watchdog] Fix for fix for wrong mails
[Watchdog] Minor text changes and send whois report of banned IP
[Watchdog] Send mail when IP was banned
[Web] Allow to set max_age for quarantine items
[Web] Fix BCC error message
[Web] Fix lang strings for sieve pathes and sieve flow
[Web] Fix SPF link
[Web] Generate readable passwords
[Web] increased db version
[Web] Show resource alias

Moo-y? May updates!

For commercially used mailcows, please consider buying a support subscription or help to keep mailcow alive by donating. 🙂

BIGGEST THANKS TO ALL CONTRIBUTORS! I think you guys fixed more bugs than I could add… seriously, you guys did a great job, thanks!

Important changes

  • Quota is no more recalculated in a daily cronjob. If you need to recalc the quota, run “docker-compose exec dovecot-mailcow doveadm quota recalc -A”
  • White and blacklists (IP) are read every minute. You don’t need to restart netfilter-mailcow after adding blacklist records anymore, but you will need to wait about a minute for new entries to apply.
  • A lot of scripts are now compatible with Py3, thank you guys!!!

Summary

[ACME] acme-tiny with python3 (thanks to @christianbur)
[ACME] Add 0 byte check for cert.pem
[ACME] Allow to skip all names but MAILCOW_HOSTNAME
[ACME] Changed the threshold for certificate renewal
[ACME] Register error when no hostname could be validated
[ClamAV] Increase watchdog clamd-mailcow thresholds
[Compose] Remove oom check for compatibility
[Config] Clarification about mailcow_hostname
[DockerAPI] Python3 (big thanks to @christianbur, again)
[Dovecot] Added domain alias handling to quarantine function; Add recipients row to quarantine overview (thanks to @franz.reiter)
[Dovecot] Auto-generate shared namespace
[Dovecot] (Dirty/Workaround) Fix memory leak when quarantine sender has non-ascii chars in mail address
[Dovecot] Fix processing imapsync custom parameters (big thanks to @hunter-nl)
[Dovecot] Removed nightly quota recalc job (too intensive on larger systems)
[Dovecot] Remove shared namespace
[Dovecot] Revert to previous imapsync cron script
[Dovecot] Set default_vsz_limit = 1024 M
[Dovecot] Trim watchdog logs (thanks to @HorayNarea for fixing this commit)
[Dovecot] Update Dovecot to v2.3.6 and Pigeonhole to v0.5.6
[Dovecot] Update imapsync script to 1.937 and added noreleasecheck parameter
[GitHub Templates] Added funding template (thanks to @ntimo)
[Git] Ignore shared namespace file
[Helper] Update Nextcloud helper – todo: fix upgrade…
[Netfilter] Reworked by @Kraeutergarten – thanks! –
[Nextcloud] Always install under subdomain, minor changes to site config and install script
[Postfix] Do not allow DSN for postscreen
[Rspamd] Add SIEVE_HOST map and skip spoof check for these IPs
[rspamd] Allow to easily use custom rspamd lua plugins
[Rspamd] Auto-generate SIEVE_HOST map and add dnsutils
[Rspamd] Change spoofed mail handling
[Rspamd] Do not apply SPOOFED_UNAUTH on ARC_ALLOW
[Dovecot] Set sieve_redirect_envelope_from to rcpt
[Rspamd] Fix spoofing detection
[Rspamd] Improve spoofing detection
[Rspamd] meta_exporter: return false if not matched
[Rspamd] Much higher scores for DMARC failures
[Rspamd] Set to to_ip to_ip_from rate buckets to 100 / 1s
[Rspamd] Update to 1.9.2, minor entrypoint changes
[Solr] Make entrypoint executable
[Update] Increase docker-compose timeout
[Update] Prefetch images, big thanks to everyone in #2637!
[Watchdog] Change error message for acme-mailcow
[Watchdog] Send mail when starting
[Web] Add function to read F2B data via API
[Web] Add only existing domains in table to the filter and removes additional ajax request (thanks to @Kraeutergarten)
[Web] Allow aliases as send-as
[Web] Allow to rename “alias” to “Alias”
[Web] API reads JSON body, big thanks to @feldsam
[Web] Change session timeout handling
[Rspamd] Add missing spamassassin.conf
[Web] Disable refresh button on reload, re-enable after table init
[Web] Fix class for full mailbox
[Web] Fix null ua in debug.js – fixes #2615
[Web] Fix showing domain with disabled sender check (thanks to @foutrelis)
[Web] Fix some breakpoints
[Web] Form cache for user passwd change modal disabled
[Web] Handle mobileconfig display names with special characters (thanks to @emericklaw)
[Web] Minor fixes
[Web] More minor css fixes
[Web] Show error when connection to redis fails (instead of 5xx)
[Web] Updated php-mime-mail-parser library to 5.1 to fix webui html preview (thanks to @Howaner)
[Web] Various UI fixes
[Web] Write API logs when format is data binary

Mid-April updates

For commercially used mailcows, please consider buying a support subscription or help to keep mailcow alive by donating. 🙂

I really, really miss working on mailcow full time. Only because of all amazing contributors, mailcow is still growing and gaining new features. Thanks!!!

Greetings to tinc.gmbh!

Important changes

  • Thanks to @mhofer117 we can now add “ALLOW_ADMIN_EMAIL_LOGIN=y” to mailcow.conf to allow an administrator to login as a mailbox user to SOGo!
  • New setups will now use ~/Maildir as default mail path.
  • A spamassassin.conf for Rspamd was missing, so Peer Heinleins SA rules were not enabled – damn! Thanks, Peer!
  • We added a name for mailcows Docker bridge: br-mailcow

Summary

[ACME] Allow to skip http verification
[ACME] Set mode 600 for key files
[ACME] Write redis key on non-empty exit code
[ClamAV] Increase watchdog clamd-mailcow thresholds
[Compose] Add named volume sogo-web-vol-1 for static web content
[Compose] Add SKIP_HTTP_VERIFICATION defaulting to n
[Compose] IMPORTANT: Added name for mailcow Docker bridge
[Compose] Update Dovecot, PHP-FPM, Solr, watchdog, Rspamd, ClamAV, SOGo and ACME images
[Config] Add MAILDIR_SUB, “Maildir” for new setups by default
[Config] Add SKIP_HTTP_VERIFICATION
[Dovcot] Cleanup random user maildirs
[Dovecot] Read MAILDIR_SUB for mail_home
[Dovecot] Remove auth cache
[Dovecot] Update Dovecot to v2.3.5.1
[Helper] Do not delete updater for Nextcloud
[Helper] Fix nc script, fixes #2484
[Helper] Remove custom_apps from Nextcloud
[Nextcloud] Always install under subdomain, minor changes to site config and install script
[Nginx] Fix site when ALLOW_ADMIN_EMAIL_LOGIN=y and reverse proxy is used, fixes #2489
[PHP-FPM] Fix SQL upgrade script
[Rspamd] Add missing spamassassin.conf
[Rspamd] Improve spoofing detection
[Rspamd] Update to 1.9 stable repository
[SOGo] Adjust sync parameters, revert if you run into problems!
[SOGo] Remove unnamed volume and rsync web content to named volume
[SOGo] Revert to previous settings
[Solr] Bootstrap cannot be omitted and must occur before mounting the data directory
[Solr] Change default configset before bootstrapping
[Solr] Keep EdgeNGramFilterFactory out of query
[Solr] Make entrypoint executable
[Update] Add MAILDIR_SUB= for updated mailcows
[Update, Config] Set mode 600 for mailcow.conf
[Update] Fix MAILDIR_SUB
[Update] Make the update check in update.sh readonly
[Update] Remove obsolete check/replace command
[Watchdog] Check for ACME failures
[Watchdog] Send mail when starting
[Web] Add ACL for unlimited quota (default 0)
[Web] Allow logout with broken session
[Web] Change session timeout handling
[Web] Continue when a check in add_alias fails
[Web, Dovecot] Allow empty/unlimited quota
[Web] Fix class for full mailbox
[Web] Fix js when adding resource
[Web] Fix “null” output in mailbox table when comments are missing
[Web] Fix rejected mails not being quarantized properly if they are tagged
[Web] Fix slow UI by switching QR provider and only generating qr image on demand
[Web] Fix totp qr code, fixes #2490
[Web] Quarantine – Enhanced JS + Show btn fix event binding
[Web] Reload view and memcached when changing a resource
[Web] Show unlimited quota in user view
[Web] Try to set aria hidden to false when a modal opens
[Web] Update bootstrap slider

Almost-spring-updates

Updated on March 10, 2019: Moooore!

For commercially used mailcows, please consider buying a support subscription or help to keep mailcow alive by donating. 🙂

Many under-the-hood changes since Foobruary updates.

Important changes

  • Updated on March 10, 2019: Due to changes in Solr, you should trigger a rescan (not a full reindex!) for all users: docker-compose exec dovecot-mailcow doveadm fts rescan -A – yes, again – sorry!
  • TLSv1.2 is now the min. required protocol for mandatory encryption in Postfix.
    This affects per-user TLS encryption (when a user enforces TLS) or any TLS policy created with “encrypt” policy or higher.
    We only make an exception for authenticated connections over port 587 and 465, where we accept TLSv1 and higher.
    Dovecot remains at TLSv1+ for IMAP, while we require TLSv1.2+ for LMTP connections now.
  • “vacation-seconds” can now be used in sieve filters
  • The IPv6 NAT check in watchdog-mailcow is less CPU intensive
  • A Postfix transport destination “*” now excludes hosts matching /localhost$/
  • Rspamd settings map is checked for changes instead of always re-applying it in Rspamd – saves CPU time

Summary

Added on March 10

[Web] Change core to dovecot-fts
[Dovecot] Use dovecot-fts core
[Solr] Use fixed, recommended schema but add EdgeNGramFilterFactory
[Compose] Update Rspamd, Postfix, Dovecot and Solr images
[Dovecot] v2.3.5 (PH 0.5.5)
[Dovecot] Change Solr cronjob to fit dovecot-fts
[Postfix] Fix sasl_passwd query from alias domain, fixes #2410
[Rspamd] Remove buggy last-modified check


[ClamAV] Create directory before handling whitelist
[ClamAV] More checks and permission fixes
[Compose] Update ClamAV, watchdog, SOGo and Rspamd images
[Dovecot] Add flags and notify to sieve_extensions
[Dovecot] Enable sieve vacation seconds not just for global scripts
[Dovecot] Fix very stupid error in quarantine_notify.py – thanks to @DevTek314
[Dovecot] Remove vacation-seconds from global-only
[Postfix] Fix mandatory encryption protocols and always require at least TLS 1.2 for LMTP
[Postfix] Mandatory encryption protocol is now min. TLS 1.2
[Postfix] Mandatory protocol for authenticated clients over 587/tcp and 465/tcp is now TLSv1.0+
[Postfix] Force route localhost$ over local:
[Postfix] Remove sasl requiring policies from port 25
[Rspamd] Add fuzzy worker with worker-fuzzy.inc
[Rspamd] Drop rspamd.conf.local
[Rspamd] Make upstream an object
[Rspamd] Mime from and rcpt can now be checked by from_mime and rcpt_mime instead of “header { XY }”
[Rspamd] Reduce SOGO_CONTACT score to -99
[Rspamd] Use almost-stable unstable 🙂
[Rspamd] Check if filterconf table was changed and return Last-Modified accordingly
[Update] Add /opt/bin to PATH, fixes #2391
[Watchdog] Do not hammer API too much when running Ipv6 NAT check
[Watchdog] Run IPv6 NAT check more often (300s sleep instead of 3600s)
[Watchdog] Minor fixes, print last log lines on error
[Watchdog] Use ipv6nat-mailcow instead of ipv6nat
[Web] Fix bootstrap pathes
[Web] Fix transport_check over port 465, fixes #2386
[Web] Strip < and > from start/end of full name
[Web] Update bootstrap to 3.4.1, fixes #2381