Some people are afraid of the update process, even though it is a very easy and stable routine.

Even pretty old installations update just fine.

If you think an update may break your installation, contact me, André, at info@servercow.de.

\o/ Update all cows.

By the way: We update the code on a regular basis, you don’t need to wait until we post these overviews. 🙂
 
 

Important changes for all moo cows

  • A mailcow fuzzy storage! Please contact me, if you want to share your spam with mailcow => info@servercow.de – fair-use, please.
  • Netfilter does now log the matched regex (finally).
  • Global sieve filters can be modified using the UI.
  • We score CSA crap relatively high now (X-CSA-* headers).
  • We do now use mariabackup for a fully-consistent backup of the SQL data directory. We will write the data to a tar archive in the future (see open issues).
  • Redis is now exposed to 127.0.0.1:7654 (FYI, has no further use in default setups).
  • We disabled TLS 1.0 and 1.1 for authenticated channels. We made a post about how to re-enable old protocols a few weeks ago.

 
 
A BIG THANK YOU to all supporters! Thank you so much for keeping mailcow alive. 🙂
Another BIG THANK YOU goes out to all contributors!
 
 

Changes (please also see the commit history)

[ACME] Force renewal with force_renew file, docs will follow
[ACME] Restart Postfix, reload seems not work all the time
[ACME] Use redis master for write operations
[ACME, Watchdog] Improve waiting for Redis
[Rspamd] Add mailflowmonitoring.com to no_log in Rspamd
[SOGo] Sort aliases (#3386)
[ClamAV] Add specific db mirrors
[Compose] A few updated images, REDIS_SLAVEOF_IP, REDIS_SLAVEOF_PORT ans MASTER (not yet supported ot documented)
[Compose] Update SOGo and ACME [ACME] SKIP IP check for SNAT’ed setups to workaround race conditions
[Dovecot] Add auth_passdb_lookup to LUA, add default plugins for replicator, check if master, add node to GUID creation, use correct syslog-ng config if Redis write-master is not
[Dovecot] Fix check to determine running imapsync procs, todo: more jobs at the same time
[Dovecot] IMPORTANT: Disabling TLS 1.0 and 1.2 – welcome to 2020
[Dovecot] LUA: Passdb: Reconnect to SQL if connection was lost
[Dovecot] Set replicator options by default – unused, no support or docs as of today
[Dovecot] Show last mail (pop3, imap) login in web interface
[Dovecot] Wait for versions table instead of failing and restarting
[Git] Add last_login to gitignore
[Git] Ignore global sieve scripts (BUT: Scripts may be forcefully overwritten, when new features are added, that depend on a given change on global sieve maps)
[Helper] Use mariabackup for SQL
[Netfilter] Log matched regex
[Netfilter] Use Redis master if set
[Nginx] Add proxy_send_timeout and proxy_read_timeout of 300 to /SOGo
[PHP-FPM] Check if master, write to Redis master only
[PHP-FPM] Do not use Redis for session handling
[PHP-FPM] Fix permissions for global maps
[PHP-FPM] Update libs, add gnupg
[Postfix] Added custom_postscreen_whitelist.cidr for a custom Postscreen wl, fixes #3313
[Postfix] Add hooks
[Postfix] IMPORTANT: Disabling TLS 1.0 and 1.1 for submission and smtps
[Postfix] Remove default rcpt count limit
[Postfix] Remove duplicate COPY from Dockerfile, fixes #3397
[Postfix] Set empty HELO restrictions for quarantine smtpd
[Postfix] Use Redis master if set
[Rspamd] Add fuzzy hashes to headers, if matched
[Rspamd] Add mailcow fuzzy hash store
[Rspamd] Add X-CSA to bulk headers
[Rspamd] Add X-Last-TLS-Session-Version header
[Rspamd] Disable 304 until SOGO_CONTACT triggers an update, needs rework
[Rspamd] Fix neural.lua
[Rspamd] Forced action add header seems to be broken atm, switching to rewrite subject until fixed
[Rspamd] Move monitoring hosts to monitoring_nolog.map file
[Rspamd] Quarantine: Set sender to null@localhost when sender is missing
[Rspamd] Reduce CSA crap to 2.0
[Rspamd] Reduce Sorbs recent score
[Rspamd] Add annoying CSA to bulk symbols and score then with 3.2
[Rspamd] Update to v2.4
[Rspamd] Set fixed name for fuzzy store
[Rspamd] Set Redis slaveof if not master, adjust redis configs automatically
[Rspamd] Use redis master for RL operations in pipe_rl
[Rspamd, Web] Escape monitoring hosts, add regex maps to vars file
[SOGo] Auto-backup user data to sogo-userdata-backup-vol-1 daily, keep one backup
[SOGo] Check if master, only run DB prep if master, use correct syslog-ng config if not master
[SOGo] Cronjob for SOGo user data backup
[Update] Add –skip-start switch, implements #3317
[Update, Config] Add Redis to exposed hosts
[Update] Display git diff save message only when local changes exist (#3351)
[Update] Make sure containers are gone before updating mailcow
[Update] Save git diff only when local changes exist (#3350)
[Watchdog] Define thresholds in docker-compose(.override) file
[Watchdog] Send 10 last applied ratelimits in mail report
[Watchdog] Use Redis master for write operations
[Web] Added hint where api docs can be found (#3335)
[Web] Add icon to indicate relayed domain
[Web] Add latin-ext to PT Sans font #3018 (#3333)
[Web] Add missing maps
[Web] Add slovak language (#3387)
[Web] Allow empty bcc when saving quarantine settings, fixes #3363
[Web] Allow to change page size in table header for /mailbox tables
[Web] Allow to set global sieve filters
[Web] Allow to skip IP check for API
[Web] Check smtp_tls_policy_map destination (more checks should be added)
[Web] Fix button order, thanks to @dragoangel
[Web] Fix cow level, sorry 🙁
[Web] Fix data type for port1 in imapsync
[Web] Fix DNS check for relayed domain
[Web] Fix mail validation for quota sender address
[Web] Fix quarantine view and add missing lang string
[Web] Fix selection bug (reproduce: select an item, select all, deselect all, click an action and find previously selected items)
[Web] Fix sieve example insert
[Web] Fix sv lang
[Web] Fix tooltips in quarantine
[Web] Fix transport validation, thanks to Gideon!
[Web] Further work to improve the swedish translation and sentence structure to improve general quality, in context to Mailcow functions (#3396)
[Web] Implement table size to quarantine, implements #3325
[Web] Keep modal data when adding a sync job
[Web] Add hint to disable TFA instead of deleting last key
[Web] Prefer sieve redirects: adjust lang files
[Web] Replace rtrim by preg_replace to fix transport checks
[Web] Set desc == domain name, when desc is empty, implements #3341
[Web] Some more quarantine lang strings
[Web] Unlearn spam if released from quarantine, implements #3327
[web] Update lang.sk.json
[Web] Use redis master where necessary, hide UI if not master, create replicate quota2 table

I did not check a PR sufficiently and merged a `auto = subscribe` for Swedish folder names.

If you updated between ~ GMT 06:00 AM and GMT 10:00 AM, please update again and delete the new folders, that might have appeared.

I will check PRs more thorough in the future.

How to remove these folders?

# UPDATE! Update your mailcow to make sure, the subscriptions do not return.
docker-compose exec dovecot-mailcow doveadm mailbox unsubscribe -A "Skräp" "Borttagna Meddelanden" "Arkiv" "Arkeverat" "Skickat" "Skickade Meddelanden" "Utkast"
docker-compose exec dovecot-mailcow doveadm mailbox delete -A "Skräp" "Borttagna Meddelanden" "Arkiv" "Arkeverat" "Skickat" "Skickade Meddelanden" "Utkast"

André

Hi,

The fuzzy storage is now enabled in mailcow, so please update your cows.

Please contact me, André, at info@servercow.de, if you want to share your spam mail with us. Old, unused domains with a high spam rate are very welcome!

There are a lot of other cool changes. We will create a new post for these soon!

André

Today we disabled the deprecated protocols TLS 1.0 and 1.1.

Unauthenticated mail via SMTP on port 25/tcp does still accept >= TLS 1.0 . It is better to accept a weak encryption than none at all.

How to re-enable weak protocols?

nano data/conf/postfix/extra.cf

submission_smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3
smtps_smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3

nano data/conf/dovecot/extra.conf

ssl_min_protocol = TLSv1

Restart the affected services:

docker-compose restart postfix-mailcow dovecot-mailcow

Hint: You can enable TLS 1.2 in Windows 7.

Sorry for the lack of december update news.
 
 

Important change for SAL users (see “support development” on the right sidebar)

You can now set WATCHDOG_EXTERNAL_CHECKS=y in mailcow.conf to enable an open relay check. The check is run about every minute.
In the future, you will be able to shut down Postfix whenever watchdog-mailcow detects an open relay.
Your source IP must match your mailcow IP, will only work with unmodified mailcows.
 
 

Important changes for all moos

  • App passwords! They work for IMAP and SMTP connections, not yet for SOGo – but we are working on it. Login as user to find them. You can also restrict access via ACL.
  • Do not reject .doc per se, but reject when any document has a macro assigned to it.
  • SOGo can be built using a subscription:
      sogo-mailcow:
        build:
          context: ./data/Dockerfiles/sogo
          dockerfile: Dockerfile
          args:
            - SOGO_DEBIAN_REPOSITORY=https://user:pass@packages.inverse.ca/SOGo/release/4/debian/
    
  • Sieve and Rspamd presets were improved. Create presets in data/web/inc/presets/rspamd/. Headline can be a lang string. Please feel free to add more useful presets!
  • Mail forwards and rejects were improved. Rejects are now signed, forwards are only ARC_SIGNED and remain SPF and DKIM valid.

A BIG THANK YOU to all supporters! Thank you so much for keeping mailcow alive. 🙂
Another BIG THANK YOU goes out to all contributors!
 
 

Changes (please also see the commit history)

[API] Added DKIM get route to api docs
[API] Added docs for new status api
[API] Added new status route to get some system infos
[API] Fixed api docs not being displayed correctly
[API] Make Solr API return data if Solr is enabled
[API] Update API docs with app password routes
[Rspamd] ARC remains active for forwards. Result: fully signed and trusted forwards and signed rejects in sieve.
[Rspamd] block all Office documents with macros
[CI] Added automated testing using drone (#3278)
[ClamAV] Whitelist JS in PDF – too many false-positives
[Web] Disable refresh button, while refreshing (#3199)
[Dovecot] Add map for app passwds
[Dovecot] Change LUA path
[Dovecot] Delete ham/spam hash if previously learned; Change LUA script pathes
[Dovecot] Drop logs
[Dovecot] Enable editheaders plugin in sieve for all users
[Dovecot] Fix app passwds: allow multiple pass hashes by using LUA construct
[Dovecot] Fix lua error when trying to escape empty domains
[Dovecot] Really strange race condition when reading an untouched LUA file on slower systems
[Dovecot] Remove CONTROL from shared namespace – thanks to @Keessaus
[Dovecot] Set BCC in quarantine notify
[Git] Ignore auto generated Dovecot LUA
[Git] Ignore whitelist.ign2
[IMPORTANT] If you run Ubuntu 16.04, upgrade your kernel to linux-generic-hwe-16.04
[Nginx] Catch case-insensitive /sogo$ request and redirect to /SOGo
[PHP-FPM] Remove useless flag for gd
[Postfix] Add bl.suomispam.net
[Postfix] Client rcpt rate limit set to 50
[Postfix] Set CA path for smtpd
[Postfix] Update Postscreen whitelist
[Rspamd] Add mailcow_networks map
[Rspamd] Allow empty envfrom for system mails, add only Dovecot to sign_networks and sign by header when sign_networks fires.
[Rspamd] allow_hdrfrom_mismatch true, auth_only false (sieve)
[Rspamd] Decrease weight of missed charset
[Rspamd] Do not normalise domains to eSLD for ARC
[Rspamd] Lower map watch interval
[Rspamd] Ratelimit for bounces reduced, max_rcpt for ratelimit increased
[Rspamd] SA trivial converter (wip)
[Rspamd] Set rspamd as trusted host, rspamd is not spoofing
[Rspamd] Split deprecated metrics.conf to actions.conf and groups.conf
[SOGo] Fix for whitespaces in mysql return; Order aliases
[SOGo] Make view more readable
[SOGo] Read build args
[SSL] fix bug with pruning old certificates (#3272)
[Update] Split metrics to actions and groups, warn if metrics is different from repo
[Web] Use main_name in the “Yubico OTP Authentifizierung” modal and in the mailbox edit modal.
[Watchdog] Add external check for open relay, requires SAL
[Watchdog] Fix ipv6 config check
[Watchdog] Retry to get current ACME log status, if empty (may fix watchdog mails on very busy servers – eg while running a backup)
[Watchdog] Revert acme-mailcow threshold to 1
[Watchdog] smtp-cli 3.10 (yay) and a new check for IPv6 configuration problems
[Web] Add “add” button to header of table
[Web] Add missing lang strings for edit
[Web] Add more password generator links
[Web] Add more map types soon; Do not expose private key via API if hidden in vars (fixes #3231)
[Web] Add more sieve presets
[Web] Add new preset for Rspamd settings map: Only allow specific senders to send to a mailbox
[Web] Allow to set BCC for quarantine
[Web] Allow to use data/web/css/build/0081-custom-mailcow.css for ignored overrides
[Web] Better mobileconfig handling
[Web] Complain about non-email email fields
[Web] Deleted hashes previously learned
[Web] Do not show Solr and Clam status when disabled, thanks to Tina
[Web] Feature: Allow app passwords for imap/smtp, allow to set acl permission for app passwords (domain admin [when logged in as user] and user)
[Web] Finally fix solr and clam status…
[Web] Fix global maps
[Web] Fix lang.en.json
[Web] Fix policy map selection for dane
[Web] Fix quarantine for sneaky dots, also fixes #3263
[Web] Fix Solr status and sort containers
[Web] Fix some major errors in app passwds but disable app passwds due to a show stopper… todo: fix asap
[Web] Fix some transport verifications
[Web] Fix transport validation for hostnames
[Web] Generate longer passwords for app passwords
[Web] Generate longer passwords for app passwords (edit was missing)
[Web] Get all app passwd ids for a single user by using get/app-passwd/all/user@domain
[Web] Hide app passwords from logs
[Web] hide echoed var
[Web] Make mobile usage less annoying; anchors for maps; sidebar for maps
[Web] Minor style fix and re-enable app passwds
[Web] Remove “add domain” from table when not admin, fixes #3267
[Web] Remove tracking for custom-mailcow css
[Web] Revert dropup to dropdown
[Web] Revert some style changes, mobile view should be fixes/better with bootstrap 4
[Web, Rspamd] Add bad language map, add map to mailcow UI
[Web] Show hint when SOGo admin login is enabed, fix sieve preset in API
[Web] Small adjustments to presets
[Web] Update languages
[Web] Various fixes for app passwd functions

You may encounter errors with Dovecot or ClamAV (and probably other containers besides mailcow), if you run Ubuntu 16.04 with its default kernel 4.4 and Docker from the official Docker repository.

Please install the HWE kernel from the Ubuntu repository and reboot:

apt-get update
apt-get install --install-recommends -y linux-generic-hwe-16.04
reboot

We just included MariaDB 10.3. If you run ./update.sh, you will encounter some errors in mysql-mailcow, that will be fixed by the upgrade process triggered by php-fpm-mailcow.
A SQL backup is recommended.

EN: If you have a mailcow support subscription, feel free to use the ticket system for assistance or help after a failed update.
DE: Falls ein mailcow Supportpaket gebucht wurde, helfen wir gerne beim Update der mailcow Installation oder bei etwaigen Problemen.

Sorry for the delay, mailcow was still worked on. Time is running too fast. 🙁

Instead of writing down all commits you can already find on GitHub, I will only mention important changes or fixes dating back from the previous post until today.

Edit: Forgot to say thanks to @irgendwr for fixing XSS!

  • The GAL (Global Address List) is now enabled by default. Various actions like availability in calendars depends on it.
  • Images are prefetched on ./update.sh (`–prefetch` will only prefetch images and exit)
  • Added SOGO_EXPIRE_SESSION variable to mailcow.conf to define when a session in SOGo times out
  • Added a whitelist map for IPs in Rspamd (`data/conf/rspamd/custom/ip_wl.map`, CIDR)
  • SAL was introduced – an **optional** license with some benefits in the future. Think about basic monitoring etc. 🙂
  • Various XSS fixes by @patschi and @irgendwr – thanks!
  • Some services were finally ported to Py3 – thank you @zkryakgul! Also thank you for exposing your policyd password in your commit, we will take care of it.
  • Added a bad word list, that triggers only, when received from a fishy tld (yes, that’s a new map, too)
  • You can now allow a mailbox to send from an external domain or only a defined set of mail addresses (edit a mailbox to find this feature)
  • @christianbur forces me to update the images more regulary, thank you.
  • Various fixes and changes in mailcow UI
  • Solr is now exposed to 127.0.0.1:18389 by default, you can setup a reverse proxy to browse its fancy UI – do not expose it to the internet!
  • I broke the anonymize headers, again. @iiegn and @patschi pointed it out, thanks!