deleted by creator

Yes, I’ll get this done

UPDATE: here you go https://oldsh.itjust.works/

There was a little hiccup this morning unrelated to the migration tonight. The Lemmy services like to be restarted every once in a while. Once we move to the new hardware I’ll be able to look into implementing a better logging system and hopefully be more proactive when situations like these happen.

The biggest consumer of storage on this instance is related to the image hosting which we use an external object storage provider for. The second is the database which is no were near the 2TB capacity. 1TB SSDs are cheaper than 2TB SSDs and I also didn’t want to spend more than I needed. As other mentioned if we need more space or IOPs in the future, I could accomplish this by adding more drives as a quick fix. This server does not support NVME unless I leverage its PCIe ports but I don’t plan on doing that. By the time this instance gets to the point where 10 SSD drives just isn’t cutting it anymore I’ll probably have come across another opportunities on getting a new server with better NVME support.

I’ve started the preparation steps to minimize the maintenance window needed for this upgrade. These operations are running some background tasks and once they are done I’ll make an announcement to give everyone a few days heads up of the anticipated upgrade time and date. Should be soon.

Hey!

The best way is to @TheDude on our matrix channel. This action triggers notifications across all my devices, creating a barrage of alerts that almost always grab my attention, a.k.a a ddos to my sensory system.

@kersploosh@sh.itjust.works explained it well. It was related to a small bug on the pictrs service that I’ve already reported which they quickly fixed in a newer version. I will tackle this upgrade at the same time as I get this instance up to v19

Hi there

Sent you a message on the matrix chat. This should now be fixed.

Thank you

@Buddahriffic@lemmy.world Please send me a DM with the username and email address used to sign up and I’ll manually verify your account.

Could you provide some additional information on what is not functioning well? Screenshots with errors are also very valuable.

Hey all,

I have made some changes to how this instance sends email that should help with the email issues people have been experiencing. If anyone continues to have issues receiving emails please let me know

Thank you

Hey,

I did some research on this.

I will not share the list of accounts as I feel its everyone right to vote and to have their own opinions whether they align with yours or not.

I will however say, Its mostly the same accounts that are down voting you. The majority of the accounts are over 2 months old with content generated by them (they do not seem like bot account). There are a few that are just 10 days old. 99% of the down votes are from other instances with feddit.uk and feddit.nl being the primary source of them.

Please DM me your username and email address used to register and i’ll get it sorted out.

Thank you

The big email email providers are not liking the amount of email coming from the server ip. The permanent fix is to use a mail sending service. I do plan on making that adjustment soon.

In the meanwhile send me a dm with your username and email address used and I’ll verify the account for you.

Thanks

You should not be blocked from editing communities. If that’s the case we’ll have to open an issue with lemmy

Hey there,

Send me a PM with what you want to create and i’ll create and transfer it over to you. Disabled it for now until i add additional functionality to the mod bot.

Hey all, I’m always excited to upgrade to the latest versions that become available. Unless the upgrade is related to security I will tend to wait a few days to give other instances a chance to share their experiences. Before upgrading I usually go over the issues that have been submitted to help understand the risks and solutions to mitigate them.

That being said the new version does bring in a whole bunch of performance gains (mostly related to the database) that i’m looking forward to.

Lemmy.world is due to upgrade later today and I plan on upgrading shortly after them (either Monday or Tuesday).

I will put up a notification up prior to the upgrade.

Thanks

Hi there,

Some providers are rejecting mail from the IP. In the meanwhile send me a PM with your account name and email and I’ll get you sorted.

Thank you

Hi all,

I somewhat explain some of the recent downtime here https://sh.itjust.works/comment/1341394

Postmortems are really great when they are done well and you have a team of sysadmins to collaborate with. As you probably know most instance operators and moderators are volunteering their time (and sometimes money) to keep these instances online. As the lemmyverse matures in age and development things will get better. In the meanwhile there will be some hiccups along the way.

When I have some extra time I plan on sharing more details about all of this. In the meanwhile check in on our chat. I’m pretty active there and can always provide additional insights on whats going on during an outage.

This is an issue with Lemmy. You cannot delete private messages, it needs to be done from the database. Luckily this instance and lemmy.world have implemented some measures to have these links removed automagically. If other instance admins are reading this and would like to know more about the measures we put into place feel free to reach out to me via pm or over on our matrix channel.

Hey all,

There has been a few issues over the last couple of weeks. Here’s a rundown of some of the issues we’ve had.

• Broken Images: A few weeks ago we had issues with broken images. This was due to me migrating our local image storage to object storage (like AWS S3). When I did the switch it broke all old images as they needed to be migrated. Pictrs at that time did not support concurrent uploads which means the migration would have taken days or weeks to complete during which time the image service would be offline. Instead I waiting for a newer version to be released that supported concurrent connections and did the migration in about 30-40 minutes one evening.

• 5xx server errors: Some of you may have experience a lemmy page with an error code on it. This was due to me trying to implement an additional proxy to shield and mitigate future risks. While rolling this out I hit a few blockers that caused downtime as we worked to rectify it. I’m glad to say that as of Today this has since been implemented.

In the addition to the above, the lemmyverse and especially this instance has been under bot attack almost daily. These bot attacks are eating resources and causing a query floods.

Lemmy is still very young and in its early phases. In time these issues will slowly go away.

P.S you don’t have to worry about me leaving you guys hanging.