Any brave dive computer download testers out there?

[Jef Driesen]

On 2018-04-23 19:07, Linus Torvalds wrote:
> On Mon, Apr 23, 2018 at 12:56 AM, Sébastien Dugué
> <sebastien.dugue.subsurface at gmail.com> wrote:
>> On Sun, Apr 22, 2018 at 10:09 PM, Linus Torvalds
>> <torvalds at linux-foundation.org> wrote:
>>> 
>>> It still has the exact same problem it always had: it's random binary 
>>> data
>>> that we'd have to encode some way.
>> 
>>   Well, it's not so random. It's in fact a timestamp in the DC format 
>> which
>> specifies from which dive we wish to download.
> 
> I wish that was the case. It's not.
> 
> It's a timestamp on _some_ dive computers. On others, it's other random 
> data.
> 
> On most dive computers, it's 4 bytes, on others it's 5-6, and on some
> it's 16 bytes of random binary data.
> 
> On many dive computers it's entirely pointless (yes, yes, you may be
> able to avoid downloading that one last dive), but they still have a
> fingerprint, and there's no way to know "don't bother".
> 
> So to really get it right and download the least amount of dives you 
> have to:
> 
>  - encode this insane random binary data without knowing what it is
> 
>  - also encode the insane garbage "serial number" (that isn't actually
> a valid serial number at all, and hasn't even been stable across
> libdivecomputer versions)
> 
>  - then before downloading, match the garbage serial number and
> vendor/device string to find the last dive with that dive computer, in
> order to find the other garbage random binary data
> 
> honestly, it's nasty nasty nasty.
> 
> What we *could* do is to do it knowingly wrong, the way dctool does
> it: don't save the fingerprint data with the real dive data at all,
> just have a random "last download" cache indexed by device family and
> serial number.
> 
> That's how the feature is designed to be used, and it would avoid the
> nasty "pollute our logs with meaningless and badly defined crazy data
> that will never be useful in the future".
> 
> "So why don't you do that, then?" I hear you say.
> 
> It's probably the best approach, and works fine if you just always
> download from the same device (because the cache would be specific to
> the installation, not be part of our dive data). And when it doesn't
> work, arguably we wouldn't be in any worse condition that we already
> are in (ie "download everything" from the devices that can't download
> dives one by one), so why not?
> 
> The "why not" is because it makes re-downloading dives much nastier.
> If you actually want to re-download dives (maybe because you didn't
> save the dives after the last download due to some other problem?) you
> now need a way to clear the cache data. And the "download all dives"
> thing could do that, but then  you wouldn't have the existing "don't
> duplicate dives" that we *currently* have.
> 
> But it also means that when you download from the same dive computer
> on a different device (ie your phone vs your laptop), you'll get get
> potentiall ydifferent results. It can be very confusing.
> 
> So it's a hack, but it might be acceptable. As long as I don't have to
> save that stupid fingerprint - and that broken serial number that
> isn't - in the logs, I'm certainly happier with it.

The fingerprint usually contains the raw date/time data, but it depends 
on the dive computer. Why should the application need to know what's 
inside? It only needs to store the data somewhere until the next 
download.

Using something less arbitrary for the fingerprint data, like a hash 
over the dive data or even the date/time of the dive, doesn't work. 
There are several reasons for that:

1. As explained before, the Uwatec/Scubapro protocol requires sending 
the timestamp to the dive computer. That already rules out the use of a 
hash, because we can't retrieve the timestamp back from the hash. Even 
using the date/time is tricky here, because this timestamp should be 
relative to the device clock, not the host clock. Since some of those 
devices rely on the host clock for parsing the date/time, that means you 
would have to store those values as well. Using the raw dive timestamp 
as the fingerprint doesn't have those problems.

2. Many dive computer protocols support downloading some kind of index 
with a summary of the available dives. That allows libdivecomputer to 
check the fingerprint and decide which dives need to be downloaded 
before actually downloading the dive. That's not only a bit more 
efficient, but it also allows to deliver much more accurate progress 
events. That improves the user experience. So this is not only about 
being a little bit faster. But this is only possible if libdivecomputer 
can use whatever information is available in the index that can uniquely 
identify a single dive, and that's of course different for each 
protocol. Hence the need to use arbitrary data.

So even if you don't really care about this last argument, then how 
would you handle the first one?


The only alternative solution that I can think of is to replace the 
fingerprint with the raw dive data itself. That way, the feature would 
still work exactly the same as before, but with the entire dive as the 
fingerprint. Internally libdivecomputer can still extract and use 
whatever piece of info it needs for its implementation. The disadvantage 
is that you now need to store the entire dive (which is a lot larger 
than just the fingerprint), but at least it's some more meaningful data.


Note that some of your other remarks are indeed valid. But the 
fingerprint feature was designed only to download the new dives. Nothing 
more, nothing less. Additional logic to detect duplicates and other more 
advanced stuff, belongs in the application. This kind of logic is 
certainly not mutually exclusive with the fingerprint feature. 
Libdivecomputer can't even do that kind of stuff because it has no 
access to your previously downloaded dives. The fingerprint of the most 
recent dive is all it needs.


The serial number is encoded as a number for historic reasons. I'm well 
aware of that, and it's already on my todo list to take care of that. 
There is need to bring that up every time.


Jef