services should be back up - power outage

Chirana Gheorghita Eugeniu Theodor office at adaptcom.ro
Thu Oct 1 11:23:12 PDT 2020


Or we could sponsor a F5 bigip virtual appliance and that should solve the
load balancing. Correct me if I am wrong.

On Thu, Oct 1, 2020, 20:11 Jeroen Massar <jeroen at massar.ch> wrote:

>
> On 20201001, at 18:23, Dirk Hohndel <dirk at hohndel.org> wrote:
>
> > [..]
> >> Just rsync/git-pull the stuff over, so that if the primary dies, that
> you can switch over manually.
> >
> > I actually use some simple git hooks to completely automate that. The
> system is actively keeping its standby in sync, and the standby does the
> same with the live system if it were to become the live system. So yes, one
> could switch between them.
> > BUT - you can't switch in the middle of a transaction, and there's of
> course a short delay, so a simple web round-robin style handoff creates a
> mess.
> > Additionally, I have not thought through the solution to the
> authentication problem. That mysql database also runs on a VM in that same
> room - and making that be redundant and consistent... that's usually when I
> decide that this isn't a life-and-death critical system and go back to
> doing something else :-)
>
> Git-hooks are indeed great for that purpose.
>
> Store the auth info in git, really, just have a passwd file or a file per
> user (htpasswd is a great format).
> Of course, if you are facebook that does not scale, as the Ops/s on git
> are not very high.
>
> Master-slave MySQL can be done, but can give you big headaches which
> typically means re-doing the initial sync. A network disruption and you
> typically see the slave log stop following the master; I have only found it
> finicky.
> Master-Master MySQL.... just don't bother.
>
> >> Split-brain is the biggest issue in these kind of setups.
> >
> > With the way git works, as long as you don't switch in the middle of a
> transaction it's actually fairly resilient. But still, I'm sure I'm missing
> subtle issues that could go wrong.
>
> You could do it with a multi-stage git commit:
>  - commit happens
>  - in git-hook:
>    - flock(LOCK_EX)
>    - rsync or git-pull/push the tree to a different directory (git-hook
> still has a lock on the first thus it won't change)
>    - rsync or git-pull/push the secondary directory to remote 'syncdir'
>    - trigger remote to swap directories from 'syncdir' to 'active'
>    - flock(LOCK_UN)
>
> That way, you only are syncing 'clean' repos, never half ones and
> 'syncdir' could be dirty but 'active' is not.
>
> Thus might be a bit too convoluted, I am sure others have another idea
> which might be better.
>
> >> That said though, I use private colo'd servers (because of home
> connectivity not always being superb and could not get more than slow cable
> speeds and upload at home still is slow), if you need a secondary/third VM
> somewhere on a static IP (colo'd), don't hesitate to yell, more than happy
> to donate one for Subsurface (and I am sure others here can do the same; it
> is always funny the names you see on this list ;) ).
> >
> > Symmetrical Gb/s plus backup 200Mb/s on a different provider. Great UPS
> setup (cough, cough), soon a solar setup plus Tesla PowerWall to take the
> UPS setup to the ridiculous level :-)
>
> Well, many 1U boxes have dual-power feeds, thus just use two UPSs and two
> power feeds.
>
> >> On Oct 1, 2020, at 5:57 AM, Attilla de Groot <attilla at attilla.nl>
> wrote:
> >>
> >>> On 1 Oct 2020, at 07:00, Dirk Hohndel via subsurface <
> subsurface at subsurface-divelog.org> wrote:
> >>>
> >>> The beauty of the current situation is that outside the (grumble,
> grumble) hours that I spend on this
> >>
> >> How / where can we do an anti-grumble, pro-infrastructure donation?
> >
> >
> > Thank you. I got a lot of generous offers of donations and support. I
> really appreciate them. But I have a simple rule. I never take money for
> the work on Subsurface that I do. That makes things so much simpler. And my
> day job does cover my basic and not so basic needs.
> >
> >
> > All of you: thank you. What a nice thread to wake up to. This community
> is what really makes it worth investing in Subsurface. Be it time or money.
>
> I think many of us here on the list have provided 'community service' in
> one way or another, hence, knowing that a 'thank you' is something that
> goes a long way.
>
> That said, if other types of resources are needed, I bet there is a pizza
> place that can take orders ;)
> And otherwise we'll have a drink of one or another one day!
>
> Greets,
>  Jeroen
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.subsurface-divelog.org/pipermail/subsurface/attachments/20201001/cbdd8eeb/attachment-0001.html>


More information about the subsurface mailing list