🤔🐮 What's up? - ARM64 Integration

Moohoo everyone!

It has been more than a month since you heard from us and today we have a shadowdrop as an update.

Since we have a few things to discuss and want to communicate directly in the future, we came up with a section called What’s up?.

Today we start with the first topic, namely ARM64:

Originally we announced to have mailcow ARM64 ready by June 2023. Obviously that didn’t work out so well. However, we still want to provide it and are still working on it. However, there are some difficulties to make a smooth transition from mailcow to ARM64.

These are not really noticeable for the main operation, but they bother us enough to keep the whole thing on hold for now.

What is it you ask?

Nice of you to ask! It’s about the Hyperscan library or the implementation of Hyperscan or Vectorscan (as it’s called on ARM64) for Rspamd. Said Hyperscan compiles regular regex entries that are dynamically generated by mailcow and used for spam detection. To be more precise, Hyperscan serves as a performance boost here, since the compilation of the regex entries does not have to be done over and over again, but remains compiled.

What is the problem?

The problem is that the said Hyperscan (Vectorscan) does not work properly on ARM64 at the moment, or the compilation is invalid after a reboot because it can no longer load it. But it has to be said that Rspamd just got native ARM64 support with the current version 3.5 and therefore may still have some bugs.

The main reason we are holding back the ARM64 support (even though the actual functionality is already there and has been successfully tested by some testers [THANKS]) is that it comes with some warnings in the console that can confuse or scare inexperienced users. Also, due to this issue, equivalent performance cannot be guaranteed.

In addition, some important core components like Dovecot (especially) have to be compiled by us with the nightly release of ARM64, because the Dovecot team will not provide native ARM64 support and we get the packages in the mailcow stack directly from them and not via the APT repo of Debian like we do with Postfix for example.

This may have unforeseen consequences at this point that need to be clarified in a larger testing round.

We don’t want to just “throw in” ARM64 support to say “hey, mailcow can do ARM64 now, look!!!”, but integrate it into the normal release cycle and the normal mailcow architecture, so that we can maintain not two repos, but one repo with the same content for x86 and ARM64.

So in the end, everybody benefits.

What does this mean for ARM64 support?

We are still working on support for ARM64 and will actually start implementing these changes in the nightly branch soon. However, it will take some time before the whole thing is moved to the normal stable branch and everyone can use it as they like.

So in short, we don’t want to guarantee a full release this year. We hope for it, of course.

We will of course let you know as soon as there is any news.

Now you know what’s going on with ARM64 and why it’s stagnating, even though it didn’t look that way at first.

Maybe some of you are thinking:

“You’re not going to release it for such a small thing?

And you’re probably not entirely wrong, but we don’t want to do any experiments quickly.

We plan to publish more “Ask the Developer” style blog posts in the future to keep you informed about the current state of the project.

For all LDAP fans, I can also announce a similar “Ask the Developer” type post. But that will be done by Patrick when he is ready to tell you something about it.

Otherwise, the usual applies:

Stay healthy and happy mailing!

Your mailcow Team

Niklas aka. DerLinkman