making remote git storage work [was Re: Subsurface ans Dropbox]

[Dirk Hohndel]

On Mon, Jun 01, 2015 at 10:15:08AM +0100, Long, Martin wrote:
> >
> >
> > Doing this via ssh seemed easier and straight forward as it integrated
> > into an existing backend infrastructure, but the more time I spent
> > thinking about the details of how this would be implemented, the more I
> > questioned my earlier assumptions.
>
> I always just felt icky about this. Using various git and ssh tools on
> Windows (for work) is depressing, and they all make a real mess of ssh,
> trying to push you towards using putty/pageant as opposed to
> linux/cygwin/msys .ssh dir. It's a nightmare trying to support other
> developers unfamiliar with git/linux when you're trying to be an
> evangelist. I realise those same problems wouldn't exist with libgit as it
> would all be hidden, but it just seemed like a lot of hoop jumping.

Based on my test this would indeed all have been hidden from the user.
They wouldn't even know that there is ssh involved. And the client support
is still there, so it's something that advanced users definitely can play
with.

> > I spent a good chunk of today stubbing out what I think Martin suggested.
> > I pushed this to master so people can play with it.
>
> Great. I'll certainly have a look.

Thanks

> > To avoid people messing with things I am encoding email so that everything
> > that isn't a letter, number, '.', '-', '_' is dropped and the '@' is
> > changed to "_at_". So subsurface at hohndel.org turns into
> > subsurface_at_hohndel.org. I think this should still keep them uniqu but
> > prevent bad people from trying to insert shell escapes, functions,
> > processes and what not into things...
> 
> That seems like a sane choice. Dropping chars, though, could lead to
> different email addresses mapping to the same account. I know that is
> unlikely to happen, but would it be just as simple to use an SHA hash of
> the email for the username? Please feel free to shoot me down on that one.

Are there other characters that are legal? A quick search for patterns for
legal email addresses was a bit inconclusive, but it seems that most of
the "dangerous" characters are illegal in email addresses, anyway. It felt
safer to pick the safe characters than to try to remove the unsafe ones.
And since email addresses are already unique...

I'll look into using the SHA here, but I'm not convinced.

> > So this is all done via https (and yes, I ended up buying a wildcard
> > certificate for Subsurface). Once I figured out (and cleaned up) the UI,
> > we can even allow additional branches so people easily can track the dives
> > for multiple divers without having to change their email/password in the
> > settings).
> 
> Slightly off-topic and perhaps a sensitive subject. Might it be time to
> consider accepting donations to cover hosting, certs, bandwidth, etc?

So I talked to a few people running open source projects that have a
"donate" button and in general the money received is not worth the hassle.

Additionally, there is a psychological shift that goes on once someone
thinks they have "paid" for something. At least some people assume that
with paying come entitlements.

While I love to complain about the money I spend on this (I am German
after all - at least I was born and raised there), in reality this is not
causing me considerable harm. We also have a little money left from last
year's GSoC (I need to talk to the LF who's holding the money for us about
this - I haven't tracked closely what happened... much of it was supposed
to be used to cover the travel to the GSoC reunion for a couple of our
mentors) - I guess I could legally use some of that money for these
expenses. But it's not really something I lose a lot of sleep over.

> > Right now I realize that there are no checks on what users store on the
> > server - I need to figure out a way to do some sanity checks so that this
> > doesn't become an abused free git server for people. But no one on this
> > list would do something like that to me, right?
>
> You could probably use a server-side hook which simply rejects commits
> containing any files other than those matching the pattern used for the
> dive format.

That's what I was thinking of doing. Just haven't gotten around to it.
Right now this infrastructure is intentionally NOT running on my server
but instead on an EC2 host, so if this gets hacked or abused, the damage
should be fairly minimal.

> > Anyway, I'm sure I'm missing two thirds of the explanation, but please
> > poke at it and let me know what you think, what I need to change, etc.
> >
> > I simply ran out of steam trying to implement the automated repo setup
> > (the authentication / authorization infrastructures already is in place).
>
> This is really exciting stuff, and is something I've been looking for since
> I started diving. It'll take subsurface to a new level.

I hope so.

And of course I have been thinking about a completely logical extension of
this... we have Gehad's HTML export. It should be pretty straight forward
to also have a git hook that uses a headless app built from the Subsurface
backend libraries to create an HTML export for each user's dive data. And
to allow them to use the same email/password to log into the server on the
web to visualize their data...

Food for thought :-)

/D