Hi all, new to Lemmy but this seems to be the best community for this that is decently active. Apologies if not!

I got into home servers in my first house a couple years ago, but our stay in that house was unexpectedly brief and everything got put back into boxes. It’s time to setup at the new place, and I have many improvements in mind from the first implementation - so while I wait for server parts to arrive, I decided to update the diagram for planning.

In no order, here’s a list of lessons I learned from V1:

  • The blade form factor doesn’t work for me. I enjoyed getting one and learning about them, but my use cases are small (&quiet) enough that a tower and a small network rack works better.
  • In the quest for automatic home lighting, I shouldn’t have gone all-in on smart bulbs rather than switches. There get to be too many in the house, and when a couple start inevitably failing, expensive bulbs and misplaced warranty info are a gigantic pain. So now the bulbs are just for special things like ceiling fans and floor lamps.
  • I need to put more attention on storage. That’s what gets used the most, by multiple users, so I will use TrueNAS Scale as my host instead of ESXi. I was not enough of a power user for that to be important to me. The rest of it is mostly for play and doesn’t need to be perfect.
  • My media streaming needs are very simple, so I think I may like Jellyfin better than Plex.
  • I need to be ‘a little’ more lax about security. I don’t think my server is realistically likely to be heavily attacked, and when I tried to go all out on best practices, more often than not I just broke things and upset my family users. My server will not have an outside access except via VPN, and my IOT devices will not speak unless spoken to - I think that will be enough.

In particular, I tried so hard last time to have a tagged management VLAN in UniFi and always just broke connectivity between something that required a hard reset. I’m planning to skip that this time but if someone has a pointer to a good setup guide, I could try that again.

Thanks for reading/looking, all comments or suggestions are welcome! I also still need to find more applications I can selfhost so I will be keeping an eye on the community for ideas.

    • mauns@lemmy.worldOP
      link
      fedilink
      English
      arrow-up
      10
      ·
      1 year ago

      unga bunga

      I enjoy making diagrams, and I spent as much time on this as I did because my internet has been out for 2 days and counting and it was an offline tool I already had. Nevertheless, thanks for the link to a new tool to look at

  • roofuskit@kbin.social
    link
    fedilink
    arrow-up
    12
    ·
    edit-2
    1 year ago

    My Home Automation and Home Assistant brain is asking “Why two concurrent Zigbee networks?”

    Phillips Hue is based on Zigbee and should work with your Conbee setup. Running them separately weakens both of them. Especially since the line of sight that light bulbs have makes them some of the best Zigbee routers in your household.

    • mauns@lemmy.worldOP
      link
      fedilink
      arrow-up
      7
      ·
      1 year ago

      You’re definitely right, I’m going to remove the Hue hub and connect everything to the ConBee. They were separated before because Hue came first, and I didn’t want to rebuild my lighting setup at the time. HA automations were set up to use Hue-side groups, rather than HA-side groups. The thought hadn’t occurred to me yet that I can unify them now - but I definitely will.

      I did think I remembered that Hue bulbs don’t work properly as ZigBee routers, but could be wrong there

      • roofuskit@kbin.social
        link
        fedilink
        arrow-up
        3
        ·
        1 year ago

        The only bulbs I know of that don’t are SengLED bulbs because it was a deliberate (and misguided) choice by the manufacturer. I have some Hue bulbs mixed in with my network and they work the same as everything else.

  • Octavius@lemmy.world
    link
    fedilink
    English
    arrow-up
    11
    ·
    1 year ago

    Hey. That looks great. How did you mange to make it that pretty? Just one Question: is your Modem really part of VLAN 1? I am running a pretty similar setup and my modem just runs into my USG and terminates my internet connection there (so my public IP is at my USG).

    • mauns@lemmy.worldOP
      link
      fedilink
      English
      arrow-up
      1
      ·
      edit-2
      1 year ago

      Thanks! Too much spare time on the weekend, haha

      Good point, mine might be the same. I was linking things up from memory since it’s been >1year since I last had the server interface up, and I can’t remember how the modem was represented now that you mention it - I’ll have to fix that up

  • mr47@kbin.social
    link
    fedilink
    arrow-up
    5
    ·
    1 year ago

    That’s a really nice setup! I run most of my things on a docker swarm (the docker hosts are VMs running on Proxmox hosts), though that was an overkill in retrospect, and causes more problems with no practical advantages.

    The range of services I run is similar to yours, but I also have a bunch of services for personal finance (beancont/fava, as well as automatic importers and such), a more extensive media setup (with qBitTorrent and *arr apps), a gitea server, and a vaultwarden instance.

    • karlthemailman@sh.itjust.works
      link
      fedilink
      arrow-up
      3
      ·
      1 year ago

      I’m curious which part you think is overkill and how you would redo this? I have a proxmox cluster and run docker amongst other things, but haven’t set up any sort of high availability.

      I don’t need live migrations, but something that could help with load balancing and reducing any potential downtime if a host fails would be great.

    • mauns@lemmy.worldOP
      link
      fedilink
      arrow-up
      2
      ·
      1 year ago

      Thanks! My last setup was quite similar to you, just with ESXi - definitely wanted to cut out as many problem points as possible this time.

      Personal finance has come up twice now, I hadn’t thought of it but will go check out the services now, I bet I could find a use as well.

  • KrokanteBamischijf@feddit.nl
    link
    fedilink
    English
    arrow-up
    5
    ·
    1 year ago

    Wow that’s a cool setup, I’ll definitely steal some ideas.

    I’m used to slinging lots of data around and one of the more helpful tools for general purpose automation has been n8n. Though it might have limited use if you’re not trying to glue all kinds of services together. I also host actualbudget to keep track of finances. Both are running comfortably in their own little docker containers.

    I’m currently looking into setting up Nextcloud and experimenting some more with presence detection for Home Assistant. I’m considering CO2 sensors, which will either tell me my home is ventilated properly, or which rooms are occupied.

    • mauns@lemmy.worldOP
      link
      fedilink
      English
      arrow-up
      3
      ·
      1 year ago

      Thanks for the ideas! I hadn’t thought about budget apps but that could be a good one.

      Not sure if I have use cases for n8n or not, but I’ll go learn about it and find out!

    • mr47@kbin.social
      link
      fedilink
      arrow-up
      3
      ·
      1 year ago

      That awesomebudget looks nice! I’m more of a beancont/fava guy, and too invested in my setup to try something vastly different - but it sure looks like a cool option for people starting out.

  • digger@latte.isnot.coffee
    link
    fedilink
    English
    arrow-up
    5
    ·
    1 year ago

    For media streaming, what mattered most for me was client software. When I first started with Jellyfin, the Roku client wasn’t as feature rich as the Android and web. I stayed with Plex until the Roku Jellyfin clients had what I needed.

    If you’re just on Android and the web, you should be fine with Jellyfin. If you’ve got a smart TV with it’s own app store, you may look at what’s there and what features are available.

    • mauns@lemmy.worldOP
      link
      fedilink
      English
      arrow-up
      5
      ·
      1 year ago

      Thanks for the hint, it’s a good point. I need to check that my TV app store has it, I could see that being the catch.

      That being said, I have been learning that my TV’s software is complete garbage and I may just go back to Android casting for TV control in general…

  • SleepyBear@lemmy.myspamtrap.com
    link
    fedilink
    English
    arrow-up
    5
    ·
    1 year ago

    Moving to Caseta for lighting from the random mix of bulbs which never quite work was amazing. It’s also much cheaper to put in one controllable switch than replace the 6 bulbs in the light fittings connected to the wall switch. Those bulbs always fail in weird and non-debuggable ways.

    I use Crafty Controller (https://craftycontrol.com/) to manage the minecraft servers. It runs in a docker instance and gives you a nice web UI to manage each minecraft server. I use it to delegate control to my kids to create and manage servers as necessary.

    Finally, if you’re not using a config mgmt tool, I’d start looking, so you can make everything easily re-doable. Personally I’m using Ansible, but puppet, chef, salt, etc all work too. Ansible is easiest given it does need it’s own infra. I like it so if something dies I can redeploy everything onto a different server.

    • LittleLily@shinobu.cloud
      link
      fedilink
      English
      arrow-up
      3
      ·
      1 year ago

      I personally use Pterodactyl for my Minecraft servers because it’s versatile enough that it can host any game server, not just Minecraft. It’s pretty much guaranteed that any game you’ll want to host will already have an install script someone has made for it.

      • SleepyBear@lemmy.myspamtrap.com
        link
        fedilink
        English
        arrow-up
        3
        ·
        1 year ago

        That’s cool. I’ve used crafty for years, but mainly because I outgrew my scripts to manage each server instance and my kids need instance responses to restarting servers. So I went looking for something Minecraft specific to give them restart and reset commands.

    • mauns@lemmy.worldOP
      link
      fedilink
      English
      arrow-up
      3
      ·
      1 year ago

      This is awesome, I didn’t know there were game server managers. I definitely see the point as I did some ugly file hacking a few times. I’ll check out both Crafty Controller and Pterodactyl!

      I’ve used Ansible for other stuff but never thought about it for server config, definitely going to figure that out too, been bitten there before as well.

      Thank you!

    • mauns@lemmy.worldOP
      link
      fedilink
      English
      arrow-up
      3
      ·
      1 year ago

      I do! I am hosting HAOS as one of my VMs, I love it. I’m planning an upgrade to have a wall mounted tablet with the dashboard this time.

      That’s a good point - I had Hue first, then got a ZigBee stick to add later devices, but left Hue alone because ‘not broken/don’t fix’ - but this time, I would like to ditch the Hue hub and set up everything on the proper ZigBee network. Thanks for reminding me!

  • Jonsk@lemmy.halfhosted.com
    link
    fedilink
    English
    arrow-up
    4
    ·
    edit-2
    1 year ago

    You don’t have a logging stack? You should set one up, it helps greatly to see your servers long term snd short term status at a glance, especially if you hook it up with grafana to visualize. You can even use grafana with Home Assistant, and do some pretty crazy things with notifications, but i must say, it is a rabbit hole, especially grafana, and quite a large one at that.

    I’m currently using grafana with loki and looking to set up notifications with ntfy but if you have the resources you could set up some other logging stacks like elk, or greylog but they’re a bit resource hungry and you dont seem to have too many services set up to warrant a larger stack.

    Anyway you have a really nice setup, good job!

    Edit: typo

    • mauns@lemmy.worldOP
      link
      fedilink
      English
      arrow-up
      2
      ·
      1 year ago

      Thank you, and good suggestion! I installed grafana once last time and failed to get anything useful as I didn’t invest enough time. Notifications sound great so I may look into the same combo you mentioned and see if I can wrangle it this time. Did you follow any particular guide when you started out with that? Otherwise I’ll just dive into the google

      • Jonsk@lemmy.halfhosted.com
        link
        fedilink
        English
        arrow-up
        2
        ·
        edit-2
        1 year ago

        Sorry for the late reply. I followed a great tutorial on youtube by Techno Tim, it explains everything pretty well. It’s a bit long, but thorough.

        I used this docker-compose file as a base because it connects loki automatically, but you have to add the volumes manually and its not too hard to connect it manually. You can just use the one that Techno Tim uses in the video if you want less complexity in your compose files.

        My docker compose file for reference:

        version: "3"
        services:
          loki:
            container_name: 'loki'
            image: grafana/loki:2.8.0
            ports:
              - "20110:3100"
            command: -config.file=/etc/loki/loki-config.yaml
            volumes: 
              - ./loki:/etc/loki
            networks:
              - loki
        
          promtail:
            image: grafana/promtail:2.8.0
            volumes:
              - /var/log:/var/log
              - ./promtail:/etc/promtail
            command: -config.file=/etc/promtail/promtail-config.yaml
            networks:
              - loki
        
          grafana:
            container_name: 'grafana'
            image: grafana/grafana-oss:latest
            environment:
              GF_PATHS_PROVISIONING: /etc/grafana/provisioning
        
              GF_AUTH_ANONYMOUS_ENABLED: false
        #      GF_LOG_MODE: "console file" #optional, used if you want a log file
         #     GF_SERVER_ROOT_URL: "(Full url here eg. https://grafana.example.com") #optional, used for redirects
            entrypoint:
              - sh
              - -euc
              - |
                mkdir -p /etc/grafana/provisioning/datasources
                cat <<EOF > /etc/grafana/provisioning/datasources/ds.yaml
                apiVersion: 1
                datasources:
                - name: Loki
                  type: loki
                  access: proxy 
                  orgId: 1
                  url: http://loki:3100
                  basicAuth: false
                  isDefault: true
                  version: 1
                  editable: false
                EOF
                /run.sh
            volumes:
              - ./grafana/data:/var/lib/grafana
        #      - ./grafana/logs:/var/log/grafana #log file
            ports:
              - "20100:3000"
            networks:
              - loki
        
        networks:
          loki:
            name: loki
          frontend:
            external: true
        

        Sorry if the formatting looks bad, since I’m on mobile. I use frontend as a network that includes containers that connect to nginx proxy manager.

        If you have ARM then you might have problems, more info in the comment below

        Edit: Better wording Edit2:typo, edited refrence for clarity and added arm warning

          • Jonsk@lemmy.halfhosted.com
            link
            fedilink
            English
            arrow-up
            2
            ·
            1 year ago

            Oh and one last thing (i promise) is that if you are using or will use ARM for loki, you have to build the docker driver for loki from source.

            A quick tutorial is here for anyone that needs it: Install go with sudo apt install go Then clone the github repo with git clone https://github.com/grafana/loki.git Then cd into it with cd loki Once you’re inside the directory, do GOOS=linux/windows/whatever GOARCH=arm-version(eg. armv7,arm64) go build ./clients/cmd/docker-driver and wait for it to finish. The resulting file should (if i remember correclty) be called either loki or docker-driver. It’s reccomended to do this on another machine then import it into your arm machine.

    • mauns@lemmy.worldOP
      link
      fedilink
      English
      arrow-up
      2
      ·
      1 year ago

      Good point, possibly - I haven’t set one up before (new house has a wider range to cover) so I threw it in the diagram and didn’t think further. I’ll fix it up :)

  • AlternateHuman02@kbin.social
    link
    fedilink
    arrow-up
    3
    ·
    1 year ago

    I see you have a container for a Space Engineers server, would you mind telling me what you are using? I would love to have a container for it instead of running torch on the desktop.

  • KevinKraft@lemmy.world
    link
    fedilink
    English
    arrow-up
    2
    ·
    1 year ago

    Looks awesome. Would you ever consider replacing the server with a cloud hosted solution? I know it kinda defeats the point of hosting it yourself, but wouldn’t it be easier/cheaper/more flexible? Of course it could also be slower to transfer data.

    • mauns@lemmy.worldOP
      link
      fedilink
      English
      arrow-up
      2
      ·
      1 year ago

      Fast data is priority 1, and learning how to host and administrate is priority 2, so I wouldn’t consider it - but it’s a fair question!