HOWTO Fix Artificially High Load Average on FreeBSD Running on Oracle Cloud

I noticed that my amd64 VPSes on Oracle Cloud all had load averages over 1.0. These are the free tier VM.Standard.E2.1.Micro instances.

# w
6:11PM up 5 days, 1 hr, 2 users, load averages: 1.39, 1.62, 1.41

This was with a basically idle machine. When I checked top I found nothing interesting. No processes were running constantly.

The machines in question are FreeBSD 13.0

# freebsd-version -kru
13.0-RELEASE-p7
13.0-RELEASE-p7
13.0-RELEASE-p7

I dug around in sysctl:

kern.timecounter.choice: i8254(0) ACPI-fast(900) HPET(950) TSC(-100) dummy(-1000000)
kern.timecounter.hardware: HPET

The timecounter clock source was HPET by default. I found that if I switched the clock source to i8254 the load avergaes reverted to nominal levels.

# sysctl kern.timecounter.hardware=i8254

# w
6:32PM  up 5 days,  1:21, 2 users, load averages: 0.23, 0.27, 0.63

This was the solution to fixing the high load average on FreeBSD running on Oracle Cloud VPSes.

In order to make this change permanent add it to your /etc/sysctl.conf file:

# echo 'kern.timecounter.hardware=i8254' >> /etc/sysctl.conf

Interestingly, the aarch64 (Ampere) FreeBSD 14/current machine did not have this issue.

I also tried the ACPI-fast timecounter source and it did not fix the issue. I was only able to fix the load average using the i8254 timecounter clock source.

Thanks to koobs on #freebsd for helping me figure this out.

Posted in Uncategorized | Tagged , , , , , , , | Leave a comment

HOWTO Allow Ping and Traceroute and MTR (ICMP) to Oracle Cloud Vm Instances and Generally Configure the Firewall- A Visual Guide

So you set up your Free Oracle Cloud trial, including the always free tier which give you some free VPS virtual machines ostensibly forever. Or you are migrating your business resources to Oracle Cloud, as it is slightly less evil than some other clouds.

You find that you cannot ping, or traceroute, or mtr to your Oracle Cloud VPS virtual machines. Oracle Cloud by default has a limited set of ingress firewall rules, and blocks ICMP. But you can modify those rules!

Unless you have done something special your vm instances will end up in a single subnet which has a common set of firewall filtering rules.

In order to allow ICMP just follow these steps with pretty highlighted screenshots:

From the “hamburger”menu at the top left choose Networking->Virtual Cloud Networks.

Under the Virtual Cloud Networks listing click on your vcn (which will be named differently, but in the same place as the highlighted link above).

Under the Subnets listing click on your subnet (again, it will be named differently, but in the same place as the highlighted link above).

Under the Security Lists display select your Default Security List (in the highlighted location above).

Click Add Ingress Rules to add a new rule.

Set the Ingress Rule up as displayed above with the Source CIDR set to 0.0.0.0/0 (all hosts– or limit to your own subnet), and the IP Protocol set to ICMP (for ping/traceroute/mtr).

Click Add Ingress Rule when done.

That’s it! Now you should be able to ping or traceroute or mtr to your Oracle cloud server. This same Add Ingress Rule can be used to modify your other firewall settings to open or close specific ports, limit ssh to certain source hosts (or change its port), etc. You can adjust all your Oracle Cloud Firewall settings right here. If you want to get advanced you can create different subnets and make different rules for those subnets. I am not sure how much can be done with the free tier, but it is seems fairly powerful.

NOTE on Port 25 SMTP: By default (link and link) outbound port 25 (SMTP email) is now blocked on Oracle Cloud instances. This is a shame. Also, under the free tier I do not believe you can set up reverse DNS (PTR). So it would be difficult to use to send email anyway. You will have to send through a smart host (on, e.g. port 587) instead.

Posted in Uncategorized | Tagged , , , , , , , , , , , , | Leave a comment

Clock Offset When Running DragonFlyBSD in a VM with Bhyve under FreeBSD

When running a DragonFlyBSD system as a guest OS virtual machine under the Bhyve hpervisor running under FreeBSD as the host I ran into an issue where the clock on the DragonFlyBSD vm was way off and had to do a coarse time adjustment shortly after boot.

Dec 2 11:04:03 dragonflybsdvm dntpd[541]: issuing COARSE offset adjustment: -21598.706669s, 02-Dec-2021 11:04:03.591

This caused things like WireGuard to fail do to the large time adjustment. So WireGuard had to be restarted, which was a pain. WireGuard has trouble if the clock changes while it is running.

It turns out that DragonFlyBSD was assuming that the clock was set to local time, whereas Bhyve was setting it to UTC. Bhyve defaults to setting the clock to UTC in later versions. (This may actually be the vm-bhyve utility that has this default– see below).

The simple solution was to modify the conf file for the vm to set the clock to localtime instead of UTC.

I use vm-bhyve to manage my virtual machines. So the fix is easily achieved in the conf file for the DragonFlyBSD vm:

#other config options like cpu/memory and network settings.
...
utctime="no"
...

That’s all you have to do (do a full shutdown of the vm, and start it back up to see the change, rebooting the vm will not affect the change). Just put that line in the conf file that you use when running the vm utility from vm-bhyve.

From the manpage for vm (part of vm-bhyve):

utctime 

As of version 1.2, vm-bhyve defaults to yes for this option. This causes bhyve to try and set the guests RTC clock to UTC rather than the host's time. I consider this more consistent, and should produce the correct time in the guest as long as the timezone is correctly set. Additionally, some guests actually expect a UTC realtime clock.
  
If you require bhyve to use the host's time, as it would by default, explicitly set this to no.

So there is your solution if the time is not coming up right in your DragonFlyBSD vm. Likely some other guest OSes expect the local time as well. So this may help you fix clock offset issues with other virtual machines under Bhyve on FreeBSD.

Another Solution: Another option would be to tell the guest OS, in this case DragonFlyBSD, that the CMOS clock is set to local time, and not to UTC. This is done simply by creating the file /etc/wall_cmos_clock

root@dragonflybsd# touch /etc/wall_cmos_clock

From the DragonFlyBSD adjkerntz manpage:

...
If the file /etc/wall_cmos_clock exists, it means that CMOS clock keeps local time (MS-DOS and MS-Windows compatible mode). If that file does not exist, it means that the CMOS clock keeps UTC time. The adjkerntz utility passes this state to the machdep.wall_cmos_clock kernel variable.
...

So there you have two options to fix the clock issues on your virtual machines.

Posted in Uncategorized | Tagged , , , , , , , , , , , , , | Leave a comment

HOWTO Fix Slow Networking with a WireGuard server in a FreeBSD Jail on a VPS, and Slow Downloads in a Jail on a VPS

I was setting up WireGuard server on a FreeBSD machine (13.0) on a VPS. In this case the VPS was a Contabo VPS which uses KVM. (Edit: I just encountered the same issue with an Oracle Cloud VM Instance as well (which also used KVM).)

In particular the goal was to run a WireGuard server inside a jail on this VPS. I essentially followed this pretty nice guide on setting things up. This got everything working, but it was far from fast. An OpenBSD machine, running on a much inferior VPS was tons faster.

I determined that simple downloads with the likes of wget from inside the jail on the VPS were terribly slow as well. I was getting about 90K/sec vs. the many MB/sec I should have been getting. When I tried the same download from the VPS itself (outside the jail) I go full speed.

I tried NOT using the bridge interface as discussed in the article above. (Look for where it says,”But if you need only a single jail, it’s not necessary to use the bridge.”) I wondered if perhaps that extra layer was slowing things down. This made no measurable difference. However, since I got all the results I wanted without the bridge, I figured I was indeed better off without this extra layer.

I found that pf (and ipfw too I believe) need tso disabled in the vtnet driver in order to do NAT in kernel. So I added the following to the /boot/loader.conf on the VPS machine (outside the jail):

hw.vtnet.X.tso_disable="1"
hw.vtnet.tso_disable="1"

(Changing this file requires a full reboot of the VPS, not just the jail.) That really didn’t seem to do much either.

Then I found this posting from the mailing list archive. Here the gentleman suggested disabling lro as well. So now my /boot/loader.conf contained the following (along with anything else that had been in there before I started all this).

hw.vtnet.X.tso_disable="1"
hw.vtnet.tso_disable="1"
hw.vtnet.lro_disable="1"
hw.vtnet.X.lro_disable="1"

So this fixed my download speeds when doing a wget directly inside the jail! Everything was back to full speed inside the jail. Good progress!

However, transfers via WireGuard from the WireGuard client machine(s) were still quite slow (like around 1-2Mbps), and seemed to have some high latency like possibly some packets were dropped, though I could never see dropped packets or high latency via tools like ping or mtr.

So I decided, what the heck, let’s try disabling checksum (csum) offloading too! Now the /boot/loader.conf contained:

hw.vtnet.X.tso_disable="1"
hw.vtnet.tso_disable="1"
hw.vtnet.lro_disable="1"
hw.vtnet.X.lro_disable="1"
hw.vtnet.csum_disable="1"
hw.vtnet.X.csum_disable="1"

And with these settings on the vps (non-jail) machine and a reboot I was finally able to realize the full potential speed of the WireGuard VPN on the Contabo (or Oracle Cloud) KVM based VPS.

This is how I fixed the slow networking with a WireGuardVPN server in a FreeBSD Jail. This also fixed slow download speeds with a FreeBSD Jail on a VPS. I hope this helps someone.

Posted in Uncategorized | Tagged , , , , , , , , , , , | Leave a comment

HOWTO Fix obhttpd[1234]: parent: send server: Protocol not supported on FreeBSD

I was putting OpenBSD Httpd (obhttpd as it is known on FreeBSD) in a jail on FreeBSD 13.0. When I fired up the webserver I got the following error in the logs:

obhttpd[1234]: parent: send server: Protocol not supported

The error occurred because FreeBSD jails do not handle localhost 127.0.0.1 the same as a normal non-jailed host.

By default the obhttpd listens on all ip addresses, including localhost.

You can fix the error and get obhttpd running in a jail by adding something like this to your obhttpd.conf file (by default /usr/local/etc/obhttpd.conf ):

#specific your external ip here
ext_ip="11.22.33.44"

chroot "/var/www"

server "www.example.com" {

####### Note the listen on now listening only to the external ip, 
####### and not localhost
listen on $ext_ip port 80

location "/.well-known/acme-challenge/*" {
root "/acme"
request strip 2
}
location * {
block return 302 "https://$HTTP_HOST$REQUEST_URI"
}
}

...

Basically to fix it you configure obhttpd NOT to listen on localhost. That’s it. There are plenty of other examples out on the web if you need to figure out how to make it listen to multiple ip addresses.

Hopefully this got you back up and running quickly when encountering the “parent: send server: Protocol not supported ” error with obhttpd in a FreeBSD jail.

Posted in Uncategorized | Tagged , , , , , , , , | Leave a comment

Broken IPv6 on FreeBSD or mtr: Unexpected mtr-packet error with IPv6 on FreeBSD 13, or Other BSD Systems

This error could be caused by a number of things, but in my case it was a simple error in my ipfw rules. My FreeBSD 13 box would work perfectly for IPv4, but fail with this error for IPv6. IPv6 was broken completely for this FreeBSD 13 machine.

It turns out that I had forgotten to allow ICMP for IPv6 in the ipfw firewall.

Although IPv4 uses ARP to find out which MAC address belongs to a certain IP address. IPv6 instead uses IPv6 Neighbor Discovery. This works via IPv6 ICMP. Also, mtr likes the IPv6 ICMP available as well…

So in order to fix this you have to be sure to pass IPv6 ICMP with ipfw (or pf for that matter, but I don’t have that syntax handy– but now you know what the problem is you should be able to find it on DuckDuckGo, or that EVIL search engine).

So edit your ipfw rules, wherever they, are and include a line to pass the IPv6 ICMP:

$IPF 95 allow icmp6 from any to any

In my case $IPF is a variable set to IPF=”ipfw -q add” at the top of the ipfw script. Look at your script and see how your rules are configured and replace $IPF with whatever your script is using to call ipfw add. If this doesn’t make any sense to you then just copy one of the rule you have and modify only the latter part to match the above. I have faith you can figure out how to make it work!

The constant number 95 in my example is the rule number in my file– You will want to set this for your setup.

So this is at least one way of how to fix the broken IPv6 on FreeBSD, or how to fix mtr: Unexpected mtr-packet error with IPv6 on FreeBSD 13. Note I have also seen the mtr packet error on systems before they are fully booted. So if I mtr to a host that is in the boot up process I will occasionally get this error as well.

Posted in Uncategorized | Tagged , , , , , , | Leave a comment

Custom Routing for FreeBSD Jails – or How To Set a Unique Default Route or Default Gateway for Each Jail

I was recently configuring a FreeBSD 13.0 box to run unbound (caching DNS Resolver) in two separate jails. The machine was connected to two WANs through two interfaces. The goal was to have one unbound jail route queries to WAN A and the other unbound jail route queries to WAN B.

Important Side Note: If you want to configure unbound to do anything other than be a basic local cache you will want to install the unbound FreeBSD package and use the unbound_enable=”YES” in your /etc/rc.conf. This is different from the pre-installed local_unbound and the local_unbound_enable=”YES|NO” setting.

I opted to use FreeBSD fib support for this. See the setfib(1) manpage for more info. Basically, fib’s allow you to have independent routing tables based on the particular fib ids. So each fib can have it’s own default gateway (default route), as well as its own custom routing setups if needed. You can then run any process in the context of a particular fib routing table, or even an entire jail, as we will do here.

Note: Your kernel needs to have fibs enabled with the ROUTETABLES option. This seems to be the default in newer GENERIC kernels. I did not have to change anything on arm64 aarch64 FreeBSD 13.0.

In order to enable fibs you have to set some options in the /boot/loader.conf to load at boot time.

# echo 'net.fibs=4' >> /boot/loader.conf
# echo 'net.add_addr_allfibs=0' >> /boot/loader.conf

The first line sets the number of fib routing tables our host will have after rebooting. We really only need two at the moment, but you will have to reboot again to change later. The second line, ” net.add_addr_allfibs=0 ‘ prevents all your routing commands (and more importantly the default routing commands setup by rc.conf) from affecting ALL fibs by default. You want independent control over each fib; so this should be disabled.

Note: There have been reports that in later versions of FreeBSD (~12+) this option must be set in /etc/sysctl.conf So I added it there as well. You probably don’t need it in the /boot/loader.conf, but it doesn’t seem to hurt.

 # echo 'net.add_addr_allfibs=0' >> /etc/sysctl.conf 

Make sure the ips you wish your jails to use have been configured in /etc/rc.conf as usual (typically as aliases). Set your default router for the parent host there as well, but not for the jails which need special routing.

#/etc/rc.conf

...

# WAN A ips
# Be sure to change the ue0 to YOUR interface name
ifconfig_ue0="inet 192.168.1.10 netmask 255.255.255.0"
ifconfig_ue0_alias0="inet 192.168.1.20 netmask 255.255.255.0"

#WAN A DEFAULT GATEWAY
defaultrouter="192.168.0.1"

# WAN B ips
# Be sure to change ue1 to YOUR interface name
ifconfig_ue1="inet 192.168.2.10 netmask 255.255.255.0"
ifconfig_ue1_alias0="inet 192.168.2.20 netmask 255.255.255.0"

I my case WAN A is on 192.168.1.xxx and interface ue0 and WAN B is on 192.168.2.xxx and interface ue1. I believe that these could in fact be on the same physical interface if you desire them to be. Mine needed to be on separate Ethernets.

192.168.1.10 is the parent host’s IP on the WAN A network.

192.168.1.20 is the jail IP on the WAN A network

192.168.2.10 is the parent host’s IP on the WAN B network.

192.168.2.20 is the jail IP on the WAN B network.

We set up the default route for the parent host with the defaultrouter line. This will also be the default route for the jail on the WAN A network. Nothing extra need be done with this jail. It will work like every other jail and follow the parent host’s network routing.

We will setup the special routing rules for the WAN B jail in /etc/rc.local (or wherever you like to put your local startup scripts).

We might as well set up our routing for the jail on WAN B before we reboot. Edit or create an /etc/rc.local (or put this where ever you like to put such startup script).

# /etc/rc.local

...

# fib Routing 

# fib 0: Default
# fib 1: jail B on WAN B
# fib 2: <reserved for future use>
# fib 3: <reserved for future use>

#
# Set up the jail B on WAN B routing table
#
# Interface route(s)
# Be sure to set the iface to YOUR iface
setfib 1 route add -net 192.168.2.0/24 -iface ue1

# Default route
setfib 1 route add default 192.168.2.1

You will have to change ue1 in the example above to your interface for the second jail. Note that we use the setfib command to first configure which fib the route command will affect. So these routes are added only to fib 1, which is the second fib, and the one we are going to set our jail on WAN B to use.

Speaking of the jail. We might as well set the jail to use the correct fib routing table before we restart as well. Edit your /etc/jail.conf (or use whatever jail manager you prefer to do so) to add the following option to your jail B which is on WAN B and should get the special routing table:

 exec.fib=1; 

This tells jail to run this jail ion the context the fib indicated. For instance, your jail.conf might look something like this sample:

# SAMPLE ONLY-- change all the pertinent data
jailB {
  exec.fib=1;  #Run this jail with fib 1 routing
  host.hostname="jailB.domain.com";
  ip4.addr="192.168.2.20";
  path="/jail/jailB/root";

  # For debugging. Remove for production
  allow.raw_sockets = 1;
}

Make sure your jails are all configured and setup properly to reflect there respective ip addresses. Don’t add any routing settings in the jail /etc/rc.conf files. You would just specify the ip addresses only. In the case of my jail B this would just be something like this:

#/etc/rc.conf

...

# Be sure to change the interface and ip address to your own!
ifconfig_ue1="inet 192.168.2.20 netmask 255.255.255.0"

...

Note we also enabled allow.raw_sockets = 1; This will let us use tools like mtr (install the “nox” version to avoid all the X Windows stuff) or traceroute, etc. to work in the jail.

Note the ifconfig line has the interface name in there as well (ifconfig_ue1). Be sure to change this to your interface.

Now you need to reboot in order to make the loader.conf settings take effect. This will also make all your /etc/rc.conf settings take effect (which could be done without rebooting), but the /boot/loader.conf stuff has to be done at startup.

Assuming your parent host has jail_enable=”YES” set in /etc/rc.conf your jails should automatically start up as well. If not, service jail start should do the trick.

If you enabled raw sockets in the jail.conf you should be able to log in to your jail B on WAN B and do a traceroute or use mtr to see that your packets from this jail use the new routing you set up for fib 1. You can disable raw sockets once you get everything working.

Hopefully everything should be working at this point. You will now have a separate default route / default gateway for each jail you wish to configure.

Obviously if you want to run unbound jails like I have you will have to configure each unbound with the correct ip address, and access lists, etc.

I also put the FreeBSD void-zones package on mine in order to also block ads and telemetry at the dns level (similar to Pi-Hole, but with no GUI). It is working quite well.

If you have difficulty, or spot a mistake, feel free to leave a comment, and I will try to help.

I found the book “FreeBSD Mastery Jails” by Michael W. Lucas to be very helpful for learning about jails. Don’t forget the manpages!

Posted in Uncategorized | Tagged , , , , , , , , , , , , | Leave a comment

FreeBSD 13 on Raspberry Pi Hangs at First Reboot After Install

I have been putting FreeBSD on a couple of Raspberry Pis. One was an old Raspberry Pi Model B (256M of RAM– the first one released). The other was a newer RPI 3B. On the RPI3B I had a 64GB MicroSD card.

I burned the FreeBSD 13.0 disk img with Etcher and it booted up fine. The install process went pretty seamlessly until the first reboot. Then the Pi would hang up right after it showed the Ethernet MAC address:

miibus0: on smsc0 
smscphy0: PHY 1 on miibus0 
smscphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto 
ue0: on smsc0 
ue0: Ethernet address: b8:27:eb:xx:xx:xx

The PI would just sit there with no error message. I re-did the install a couple of times, tried a different SD card, etc. Eventually, I decided to try FreeBSD 12 instead. This one got to the same place, but then eventually dropped into a mountroot prompt.

At this point I knew it had something to do with the root filesystem not working right. I tried to manually load the root filesystem from the mountroot prompt, but it still would not work.

I had an inkling in the back of my mind that this likely had something to do with growfs, the program that gets run on boot in order to expand the size of the root partition to the size of the entire SD card. After all, in my testing I had tried a couple of times where all I did was install FreeBSD, login as root and reboot immediately.

gorwfs is run automatically from the line in /etc/rc.conf

growfs_enable="YES"

I needed to turn this off on the SD card image. So on another FreeBSD machine I unxzipped the Raspberry Pi img:

unxz FreeBSD-13.0-RELEASE-arm64-aarch64-RPI.img.xz

Next I needed to link the img file to a virtual device so I could mount it:

# mdconfig -a -t vnode -f FreeBSD-13.0-RELEASE-arm64-aarch64-RPI.img -u 0

And then mount it– note that the root partition is partition 2, and you can find it at /dev/md0s2a

# mount /dev/md0s2a /mnt

Then I had to disable growfs in /mnt/etc/rc.conf using sed (or a text editor).

# sed -ibkp 's/growfs_enable="YES"/growfs_enable="NO"/' rc.conf

Unmount and disable the mdconfig:

umount /mnt
mdconfig -d -u 0

After this surgery to the img file you can re-flash the SD card and when you run the installer your root filesystem will not be auto expanded, and your install will not hang after the first reboot.

I am not yet sure what the underlying issue is. I tried manually resizing the partition on the SD card to 32 GB and 16GB and 8GB, and all would not boot. It may be a 4GB limit (which is odd since this is arm64 bit).

In my case I ended up moving the root filesystem to a USB drive anyway. So I did not worry too much about all the wasted space on the SD card. I suppose you could also create another partition and and mount it someplace like /home.

This should at least get you started with the Raspberry Pi 3B on FreeBSD 13 or FreeBSD 12. This bug should be reported, though. I hope to soon, but likely should do some more testing first. Feel free to report it!

Posted in Uncategorized | Tagged , , , , , , , , , , , | 2 Comments

How to Fix It: Error connecting to database: SQLSTATE[HY000] [2054] The server requested authentication method unknown to the client with PHP and MySQLd

So you updated MySQLd, or perhaps you just updated your distro, like to Ubuntu 20 LTS or whatever the latest and great flavor of the day is… And now you keep getting these pesky errors every time you try to use your website.

Error connecting to database: SQLSTATE[HY000] [2054] The server requested authentication method unknown to the client

You know you have the right username and password for your database. You can likely even connect to your db via the mysql command line.

Well, MySQL version 8 has some problems with certain versions of PHP. (For more details you can see this post from PHP.) MySQL changed the default password type, and older versions of PHP have not adapted to this yet.

The Fix

So how do you fix the “Error connecting to database: SQLSTATE[HY000] [2054] The server requested authentication method unknown to the client” error? Very easy!

Modify your mysqld.cnf file for your MySQLd daemon. This is typically someplace like /etc/mysql/. In Ubuntu 20 LTS it is in /etc/mysql/mysql.conf.d/mysqld.cnf

In Ubuntu 20 LTS use the following:

# echo 'default_authentication_plugin=mysql_native_password' >>  /etc/mysql/mysql.conf.d/mysqld.cnf 

# systemctl restart mysql

The line above uses your shell to append (>>) the correct line to the mysqld.cnf file.

In other linux or *nix distributions:

Just edit your mysqld.cnf file and add the following line at the end:

 default_authentication_plugin=mysql_native_password 

Then restart the MySQL server using your system’s method for doing so. This is often something like:

# service mysql restart

Or perhaps:

# service mysqld restart

Or sometimes:

 /etc/init.d/mysqld restart

On FreeBSD it would be something like:

# /usr/local/etc/rc.d/mysql-server stop
# /usr/local/etc/rc.d/mysql-server start

You get the idea, and can always web search for your OS.

Basically this tells MySQL to use the old password style by default and all your old PHP scripts should work just fine again. You will have fixed the “Error connecting to database: SQLSTATE[HY000] [2054] The server requested authentication method unknown to the client”

Enjoy.

Posted in Uncategorized | Tagged , , , , , , , , , | Leave a comment

courier mta Virtual Users Deliver to mbox Instead of Maildir – Failure of maildrop – .mailfilter Doesn’t Work

In a recent server upgrade (and in fact in a prior server upgrade a few years back) I ran into a problem where my Courier Mail Server (courier-mta) virtual users (virtual domains) for use with IMAP or POP3 (and with ESMTP) no longer functioned correctly after the upgrade. I used courier on a fresh install of Ubuntu 20.04. I installed all the courier packages, including maildrop.

The problem was the mail was being delivered to a traditional mbox of the real system account I was using as the UID for virtual accounts. I was using an account called vmail and all the mail for the virtual users was being delivered into /var/mail/vmail in mbox format, rather than to the respecivet Maildir directories. Also, in my testing I found that the individual .mailfilter scripts were not being executed.

Perhaps you are even setting up a new courier server with virtual users (virtual domains), but you are just using the default apt-get install packages for courier. However, your virtual users are not working.

I was getting errors like these from courirerlocal:

/usr/bin/maildrop: Unable to create a dot-lock at /var/mail/3741.0.xxx.yyyyy.com.

OR

/usr/bin/maildrop: Unable to open mailbox.

I had copied the mysql database and the authmysqlrc and authdaemonrc from the old install on the old server. So surely that was all fine.

I recalled a similar issue during my last upgrade, but I failed to log what I had done. I knew it had something to do with /usr/bin/maildrop . So I checked the version of maildrop that install from apt on Ubuntu 20.04:

mybox> /usr/bin/maildrop -version
maildrop 2.9.3 Copyright 1998-2017 Double Precision, Inc.
GDBM/DB extensions enabled.
Courier Authentication Library extension enabled.
Maildir quota extension are now always enabled.
This program is distributed under the terms of the GNU General Public
License. See COPYING for additional information.

That sounded good. It’s from Double Precision, the maker of the courier mail server. It even says, “Courier Authentication Library extension enabled.” That sounded great! This has to be right maildrop.

Doesn’t it?

No. This is the generic maildrop from Double Precision. It lacks support for the courier style virtual users.

The maildrop you want is the one that comes with the courier sources (not with the courier package on Ubuntu apparently). But the confusion gets worse because the courier website has a maildrop package that you can download as a standalone. I tired this thinking I was getting the latest and great maildrop and all would be good. Nope.

mybox> /usr/local/bin/maildrop -version
maildrop 3.0.3 Copyright 1998-2018 Double Precision, Inc.
GDBM/DB extensions enabled.
Courier Authentication Library extension enabled.
Maildir quota extension are now always enabled.
This program is distributed under the terms of the GNU General Public License. See COPYING for additional information.

Again, this has the Courier Authentication Library and everything. This has got to be right!

No.

Finally, I downloaded the sources (be sure to get Package: Courier and NOT Package: maildrop) for the entire courier package and built those (required adding a few things… I may do a full howto someday, but for now you have to figure this bit out on your own). I built these new sources and then copied the maildrop binary from: /path/to/courier-1.1.4/courier/maildrop over to /usr/bin/maildrop and everything magically worked. (You might also have to make maildrop setuid root depending on your install).

The correct maildrop looks like this:

mybox> /usr/bin/maildrop -version
maildrop 3.0.3 Copyright 1998-2018 Double Precision, Inc.
Courier-specific maildrop build. This version of maildrop should only be used with Courier, and not any other mail server.
GDBM/DB extensions enabled.
Maildir quota extension are now always enabled.
This program is distributed under the terms of the GNU General Public License. See COPYING for additional information.

Note the critical line, “Courier-specific maildrop build.” If your maildrop doesn’t spit out this line when you check the version it most likely is not going to work right with courier virtual users (virtual domains) when you use the following option in your /etc/courier/courierd configuration file.

DEFAULTDELIVERY="| /usr/bin/maildrop"

You need this magic version of maildrop to get virtual users to work right with mysql or with userdb, etc. If you have the wrong version of maildrop then your .mailfilter files will not get executed, and you may get various errors like these from courierlocal:

/usr/bin/maildrop: Unable to create a dot-lock at /var/mail/193271.0.xxx.yyyy.com.

OR

/usr/bin/maildrop: Unable to open mailbox.

Or all the mail for the virtual users may instead end up going to a mbox formatted mailbox in the system account you setup as the real id of your virtual users (mail or vmail, etc.).

Posted in Uncategorized | Tagged , , , , , , , , , , , , , , , , , , , | Leave a comment