Debian 12 Update Broke My ASL Cloud Node

How to fix this if you are running AllStarLink on Debian 12

Last week, a new version of Debian 12 was released (see Debian 12.4 Released With Dozens Of Fixes and Updated Debian 12: 12.4 released). This update addresses several bugs and at least one vulnerability, so I chose to update my cloud node 57945 (see https://kj7t.net).

And the update broke my node. I reached out to Allan Nathanson WA3WCO, the creator of the Debian 12 install package I used to move up from Debian 10 to Debian 12:

While Allan had not yet encountered the problem I found, he had some good suggestions to figure it out.

The upgrade that broke ASL

I SSH’d into my cloud node and ran:

apt update

apt ugrade -y

I was surprised by how long the upgrade took. It ran longer than I expected and I saw it built a new kernel. (That makes sense since we were making a significant move up to Debian 12.4.)

When I rebooted the cloud server, Asterisk Manager wouldn’t run. The lower window in AllScan shows there was a problem:

Errors after upgrading Debian

I scratched my head and rebooted the server a couple more times, checked the error logs, and then emailed Allan for some advice.

Restored a working node from backup

To get the node back up and running while I made a plan on upgrading the machine, I restored a known good backup. That is very easy to do on Vultr with just a couple of mouse clicks. After the backup was restored, the AllStarLink node was working fine.

Upgrading to Debian 12.4

Armed with recommendations from Allan, I proceeded to do the update/upgrade dance again, as root:

apt update

apt upgrade -y

I saw the upgrade run longer than most of the upgrades I’ve installed. Sure enough, the AllStarLink node wasn’t working. Asterisk wasn’t connecting to the node and the node wasn’t reporting to the node list at AllStarLink.org.

Allan had postulated that the upgrade installed a new kernel.

You now need to rebuild/install the DAHDI code.  Should be simple enough with the ./asl-install script … and you should only need to run the (1) “setup”, (2) clean, and (3) DAHDI steps.

To get there, I SSH’d into the machine as root and made sure I was in root’s home directory:

cd ~

Then I changed to the directory that houses Allan’s installation script:

cd ASL-Install/

What’s the installation script? Let’s list the directory contents:

ls -al

And there’s the installation script:

The installation script is asl-install.sh

To proceed, run the installation script:

./asl-install.sh

I did exactly what Allan suggested: I ran steps 1 (Setup), 2 (Clean before build), and 3 (Build/install DAHDI):

Install screen after running first three steps

When you exit the install script menu, you are prompted to reboot the server, which I did.

After that, AllStarLink was running fine…except for a message I saw in the bottom window of AllScan that suggested I should run astdb.php. I changed into the supermon directory:

cd /var/www/html/supermon

Once there, I ran astdb.php from the command line:

Running astdb.php from the command line

And that fixed it. The node was running fine.

To confirm that Debian was upgraded, I rebooted the node and SSH’d in to look at the welcome message:

The resulting Debian version is the latest stable update

Debian 6.1.67-1 is identified as the version running. That is the latest stable update.

Stable update 6.1.67-1

Finally, I ran apt update again and found no packages needed to be upgraded.

Job done. Many thanks to Allan for the assistance. I hope this information helps someone in the AllStarLink community!

Postscript, December 20, 2023
More help from Allan. Until the next Debian release, you can mark you system to hold on updates. As root, run this in a terminal window (this is all one line):

dpkg --list "*$(uname -r)*" | grep linux | awk '{print $2}' | xargs sudo apt-mark hold

That set of commands produced this result for me:

And then run this command to show what is being held:

sudo apt-mark showhold

My result:

So now when I run apt update, apt upgrade, the kernel should not get upgraded, and that means I won’t have the problem caused by breaking DAHDI with a new kernel!