[PATCH] Visualisation of individual oxygen sensor data for CCR dives

[Jef Driesen]

On 2017-04-12 10:47, Anton Lundin wrote:
> On 12 April, 2017 - Davide DB wrote:
> 
>> On 11 April 2017 at 23:44, Jef Driesen <jef at libdivecomputer.org> 
>> wrote:
>> 
>> > I'll need to look again at the old discussion, but if I remember
>> > correctly, your patch produced bogus results for some of the data I tested
>> > against. And that's why I wanted to double check with Shearwater. But so
>> > far I wasn't able to get any info from Shearwater. So yes, let's have
>> > another look
> 
> Directly after that revision of the patch, I sent you another which
> always handled the calibration values the same, no matter what they
> were, which produced sane values even in that case.
> 
> I didn't have that test data you referenced when developing the code,
> but after seeing that data, it clear that its the right thing to do.

I don't entirely agree with that. If you check the available memory 
dumps, 5 of them contain CCR dives. I analyzed all of them, and this is 
what I found:

  * petrel.marklee.bin

The three mV values in the samples are always zero. But since all three 
O2 sensors have their calibrated bit in the header disabled, no ppO2 
samples are reported. So that looks good.

Sensor 0: 0 0 2100 -1
Sensor 1: 0 0 2100 -1
Sensor 2: 0 0 2100 -1

(These numbers are respectively the voted bit, the calibrated bit, the 
calibration value and the ADC offset.)

  * petrel.larrybainbridge.bin

The three mV values in the samples are always zero. But the O2 sensors 
are calibrated, and thus we report ppO2 samples with all zeros. That's 
not good. This might indicate that we need to take into account the 
voted bit too, because that is disabled for all dives:

Sensor 0: 0 1 2045 44
Sensor 1: 0 1 2025 -9
Sensor 2: 0 1 2045 51

petrel.rainer.bin

Here we have non-zero mV values in the samples. The O2 sensors are 
calibrated. In most samples, manually averaging the resulting ppO2 gives 
results that are very close to the average ppO2 value reported by the 
device. In roughly 90% of the samples where the result is different, the 
difference in average ppO2 is only 0.01. That might be some rounding 
error. In only a few percent of the cases there is a significant 
difference, and that are probably cases where a value is voted out. So 
everything looks reasonable here.

Sensor 0: X 1 YYYY 0   (with YYYY ranging from 1671 to 2112)
Sensor 1: X 1 YYYY 39  (with YYYY ranging from 1810 to 2326)
Sensor 2: 1 1 YYYY 2   (with YYYY ranging from 1621 to 2108)

Note that sometimes sensor 0 or 1 have the voted bit disabled in the 
header! So if we would take that bit into account (see above), then I 
think we're disabling a sensor when it shouldn't. So I'm not really sure 
what to do with the voted bit.

  * predator.janschubert.bin

All sensors are calibrated and voted in:

Sensor 0: 1 1 YYYY -16   (with YYYY ranging from 859 to 895)
Sensor 1: 1 1 YYYY -24   (with YYYY ranging from 812 to 843)
Sensor 2: 1 1 YYYY -11   (with YYYY ranging from 827 to 856)

Note that compared to the petrel data, the calibration values are now in 
an entirely different range. So the resulting ppO2 are way off. That's 
what you tried to address in the second revision of you patch with this 
code:

if (calibration < 1000)
     calibration += 1024;

With this patch, the results look good again. When comparing the 
calculated with the stored average ppO2, the largest difference is only 
0.021.

  * predator.marklee.bin

All sensors are calibrated and voted in:

Sensor 0: 1 1 YYYY -41   (with YYYY ranging from  885 to 1069)
Sensor 1: 1 1 YYYY -39   (with YYYY ranging from  989 to 1016)
Sensor 2: 1 1 YYYY -5    (with YYYY ranging from 1179 to 1376)

Very similar, except that some calibration values are above and other 
below 1000. And that breaks the above patch. If we always add the value 
1024 (or just 1000) unconditionally, things are getting better. I guess 
that's an indication that this is something specific for the predator, 
but not the petrel. But it's not perfect yet. The difference in average 
ppO2 is now between 0.043 and 0.212. This is much larger as before, and 
if you look at the values, it's not caused by an outlier that is getting 
voted out.

A random example:

Sensor 0: 0.639 = 33 mV * (885 + 1024) / 100000.0
Sensor 1: 0.583 = 29 mV * (989 + 1024) / 100000.0
Sensor 2: 0.572 = 26 mv * (1179 + 1024) / 100000.0

But the average ppO2 reported by the device is 0.65. That's more than 
our highest value, so it can't be due to sensor voting. Maybe the 
calibration is done different for the predator?

>> Maybe I'm really messing up things but reading Dirk email regarding 
>> his
>> contact at Shearwater "Perdix / Perdix AI info from Shearwater" it 
>> seems
>> that Perdix and Petrel devices don't carry O2 info via their protocol:
> 
> </snip>
> 
>> They carry only a global pO2 (I guess it's the result PO2 from so 
>> called
>> voting logic hence used in all further calculation) and mV values for 
>> each
>> sensor. BTW individual mV 02 sensor graph was added in version 2.3.5:
>> https://www.shearwater.com/release-notes-firmware/shearwater-desktop-2-3-5-released/
>> <https://www.shearwater.com/release-notes-firmware/shearwater-desktop-2-3-5-released/>
>> Hence would be impossible to display single individual pO2.
> 
> No, its not impossible. The calibration values are in the Dive header,
> and with those, the mV values can be converted into pO2 values.
> 
> 
> The issue with current libdivecomputer DC_SAMPLE_PPO2 is that you cant
> distinguish between ha "real" "voted" pO2 and the raw sensor value.
> 
> I would like to see a option to export both, and be able to handle them
> differently

We could introduce a new structure that carries a sensor index (similar 
to DC_SAMPLE_PRESSURE):

struct ppo2 {
     unsigned int sensor;
     double value;
};

That way we can easily report multiple DC_SAMPLE_PPO2 values, each with 
a
different sensor id. And for the voted/calculated ppO2, we can use some 
magic value, like:

#define DC_SENSOR_NONE 0xFFFFFFFF

(Just like we already have DC_GASMIX_UNKNOWN).

Jef