Weeknotes the Third
- 2 minutes read - 424 words - Page SourceI had intended to write this weeknotes on the amusing rabbit-hole of yak-shaving I’d fallen down:
- I found that a likely cause for Keycloak stopping working on my system is that it recently changed to only support traffic over HTTPS.
- So I want to finally get around to using LetsEncrypt on my system, as per this guide
- But I should really take that opportunity to convert my cert-manager definition to my fancy new libsonnet-based approach.
- But before doing that, I want to install the GitHub Vault Plugin to provide automated scoped authentication for Gitea Actions, so that I don’t need to keep refreshing the credentials for my Commit Report Sync tool.
…but while attempting to start writing the blogpost on that rabbit hole, I found that the recent restart of my NAS (due to the manufacturer’s sending me a replacement PSU1) had caused k8s dynamic PVC provision to get into a weird state, necessitating some force-deletion of PVCs and pods, meaning that my Gitea install’s Redis cluster got into a broken state2, so it’s taken the better part of my entire Sunday evening to even be able to publish this post3.
Nobody ever said that self-hosting was easy 😝
-
which, irritatingly, turned out to have the wrong number of connectors, so I’ll still need to keep using the current holdover one (which has the wrong form factor, so can’t actually be installed in the case but is sitting loose outside it in the rack) until they can send a proper replacement. But I didn’t notice that until I’d already powered down the system. More fool me for assuming and not checking that the manufacturers would send the correct part! ↩︎
-
shout out to this SO answer for providing the command to force-reset a Redis cluster with unreachable masters:
redis-cli --cluster fix ONE_OF_HEALTHY_NODE_IP:PORT --cluster-fix-with-unreachable-masters; and to this discussion for pointing out that settingRUN_MODE=devin Giteaapp.iniwill print actual error messages on a500page. ↩︎ -
and, at the time of writing (which is naturally pre-publication), I’m wary that Gitea Actions will probably need some gentle resetting and reauthentication before it can execute the publish, too…EDIT: heh, yep - Jack-from-a-half-hour-later can confirm that the automated job to create a registration token for the runners was bugged out, meaning that the k8s secret containing the token contained incorrect data, and irritatingly that token was cached on the runner at
/data/.runner(thanks to this post for identifying that!) so even after I populated the secret with a correct token it was still picking up the incorrect cached one. ↩︎