Michael Stanclift

Virtually Benevolent & Fully Federated

I moved the write.vmst.io instance over to a Docker container. If you can see this, it worked.

Should posts on here have a title? Or just be a longer version of what I might post on Mastodon.

This morning I removed just over 3000 dead instance connections from vmst.io. I know Mastodonizens don’t follow their follower accounts as closely as some folks on other networks, but I wanted to let you know you this operation might have caused some go missing in this purge.

These are instances that have quit responding for at least seven days, and in some cases they haven’t been around since November. A couple weeks ago I saw an uptick in the amount of SSL errors in Sidekiq communicating with various remote instances and noticed a trend of instances that were stood up around 90 days ago, with Let’s Encrypt certificates, that have since been abandoned.

It's likely that folks were trying to self host and then realized what a pain in the ass it is 😉 As @supernovae@universeodon.com just put it to me in the project Discord, “lots of failed dreams in there.”

I’d say that most of the domains listed didn’t have anyone following or being followed by vmst.io users, but some did. These accounts would not have been in communication with us for some time, as Mastodon stops actively trying to communicate with systems after they have failed for seven days.

While some of these were removed manually if they were larger instances, this process was largely accomplished by doing a dump of the unavailable_domains table in the Mastodon Postgres database, and exporting the list to a giant string of domains.

RAILS_ENV=production ./bin/tootctl domains purge away.we.go not.here.anymore bye.bye.birdie

In my spot checking of these instances, most of them had no direct follow/follower relationship with our users, but some of them did.

Going forward this process will be run more frequently.

I've doing some blue/green (maybe it's just A/B) testing on vmst.io this week with server updates and Mastodon dependency testing. Specifically focused on some upgraded versions of Nginx and Puma (which are now fully live for everyone) and some backend libraries for the streaming API to things like Redis and Postgres.

I've been rolling out more updates this way, since things are pretty happy and functional with multiple systems running each component. I haven't decided what the right period of time to let things bake-in is, but I suppose that depends on what is changed.

The streaming updates are only on half the things right now. So if something either works better or fails spectacularly 50% of the time, that’s why.

I hate running old code. I've been running Debian 11 stable for a long time, but some of the packages that I end up using are past where they are on other distributions, so I recently flipped to the testing branch. So far I've not run into any issues other than needing to redownload/update some of the Ruby Gems used by Mastodon.

I'm also currently testing HAProxy as a replacement for Stunnel in our Mastodon Redis configuration. So far so good. I've noticed some activity in the Mastodon GitHub focused on a replacement for the current Redis libraries that don't work with TLS connections. Being able to completely remove Stunnel/HAProxy from the setup would be a fantastic simplification for me.

I exported the contents of my previous Ghost blog at vmstan.com, converted the contents to pure Markdown files, and then told WriteFreely to import those files … and it blasted them all straight to public posts and flooded the @vmstan@write.vmst.io timeline.

So for the 4 of you that follow it directly. Sorry. 😆

I’ve processed the post dated and titles, and honestly it was easier once I logged in and edited the MySQL database directly. I still need to cleanup formatting on some things, like code blocks that became block quotes, and upload images in places where it makes sense to continue using them.

In the coming days I will decommission vmstan.com and setup a redirect in Nginx To many of the more popular links.

I think I now have write.vmst.io running behind our load balancer between two nodes. If this post works ... 🤞

elk.vmst.io and semaphore.vmst.io are also on the same setup.

I spent some time this morning moving some of the ancillary services of vmst.io behind our Digital Ocean managed load balancer, and the pair of Nginx hosts that front end Mastodon.

I'd previously been fronting all these things with a separate instance of Caddy, which was pretty flexible and easy to setup, but having a single place where things flow in makes it simpler to monitor and secure. Having them load balanced for the frontend even if it's a single app behind it also makes me feel a little better, and gives some flexibility for making things like Elk or WriteFreely more redundant in the future.

It also means fewer places where I need to have TLS certificates configured, replaced, etc.

Due to the generous nature of the folks that are contributors to vmst.io, I'm now in the process of adding additional services to the vmst.io offering. Both of these are in “beta” for the time being.

While I don't anticipate any changes that would impact their long term consistency (URLs, usernames, etc) things may need some adjustments based on load or may need to be shuffled around until it's full production.


The first is what you're reading right now. This is from an instance of WriteFreely that we host. WriteFreely is free and open source software for easily publishing writing that may be longer than the 500 character limit on Mastodon.

Users can follow and boost your posts on write.vmst.io just like your regular Mastodon account, because it's uses ActivityPub. The format of your WriteFreely account will be @username@write.vmst.io.

They can also follow your blog using good ole' RSS.

You can boost your own long form writing from your Mastodon account. Best of all you login to write.vmst.io using your primary vmst.io Mastodon account, so there's no second set of passwords to manage. When you set two factor authentication on your Mastodon account, it carries over to write.vmst.io.

When you create your write.vmst.io Blog, it'll ask you to set your display name, this is like the name of your blog. It can be your real name, or whatever you'd like to call your blog. Your username will automatically populate, and should stay the same as your Mastodon username for consistency.

That's it. WriteFreely uses Markdown. For more information on how to use the platform, check out their User Guide.


Matrix is an open network for secure, decentralized communication. Matrix lets you chat in real time in 1:1 sessions or in groups, with end to end encryption.

While it doesn't use ActivityPub it does use WebFinger to communicate server to server, just like Mastodon. You can use a variety of different clients to login to Matrix. I'm in the process of getting documentation on docs.vmst.io that outlines the process, but if you're familiar with Matrix you can grab your client of choice (I suggest starting with Element) and login.

Your “homeserver” will be matrix.vmst.io, and when you login you'll use your Mastodon credentials, so there's no second set of passwords to manage. When you set two factor authentication on your Mastodon account, it carries over to Matrix. (Notice a theme here?)

The first time you login you'll need to establish your account, your username may automatically populate, but needs to stay the same as your Mastodon username for consistency.

You can now talk to anyone else on the Matrix platform, not just other vmst.io users who have activated their accounts.

Your Matrix username will be as @username:vmst.io, very similar to a Mastodon username.

You can also join the #general:vmst.io and chat with other vmst.io users.


Our rules for these new tools are the same as our rules for our core Mastodon offering.

This is a test of the emergency broadcast system. This is only a test.

I've been doing things with computers, since last October.

In addition to launching a Mastodon instance at vmst.io I've since launched an integrated WriteFreely platform at write.vmst.io. As a result I will be blogging there regularly from now on.