m241-bin incorrect track creation

classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

m241-bin incorrect track creation

Stefan Bauer-12
Hi,


I'm currently investigating an erroneous behavior in the m241-bin input
format. The device I'm using is the Holux GPSport 245.

Command line:
gpsbabel -D 5 -t -i m241-bin -f testcase.bin -o gpx -F testcase.gpx

What I understand is that when reading the bin file, the stream of track
points is split when certain events happen. For my device, it looks like
the wrong events are selected to trigger such a split. The result is an
"off-by-one" like scenario: The split always happens one track point too
late.

Please find attached a test case which demonstrates the issue.

The following change fixes this for my device:

https://github.com/stefan-a-bauer/gpsbabel/commit/f5c4589fb60fe57af796a0c082f587403843fafe

What I'm unsure about is how this affects the other devices handled by
the m241-bin format.

Is anyone else with MTK-based devices seeing the same issues as I?
If so, does my patch fix this for you, too?


Thanks,
Stefan


------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code

new.gpx (10K) Download Attachment
old.gpx (10K) Download Attachment
testcase.bin (64K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: m241-bin incorrect track creation

tsteven4-2
Your change seems to break our regression, have your tried "testo mtk"?  I suppose it is possible that the expected output of our test case is wrong.


On 1/10/2016 11:56 AM, Stefan Bauer wrote:
Hi,


I'm currently investigating an erroneous behavior in the m241-bin input format. The device I'm using is the Holux GPSport 245.

Command line:
gpsbabel -D 5 -t -i m241-bin -f testcase.bin -o gpx -F testcase.gpx

What I understand is that when reading the bin file, the stream of track points is split when certain events happen. For my device, it looks like the wrong events are selected to trigger such a split. The result is an "off-by-one" like scenario: The split always happens one track point too late.

Please find attached a test case which demonstrates the issue.

The following change fixes this for my device:

https://github.com/stefan-a-bauer/gpsbabel/commit/f5c4589fb60fe57af796a0c082f587403843fafe

What I'm unsure about is how this affects the other devices handled by the m241-bin format.

Is anyone else with MTK-based devices seeing the same issues as I?
If so, does my patch fix this for you, too?


Thanks,
Stefan



------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140


_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code


------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: m241-bin incorrect track creation

Stefan Bauer-12
Am 10.01.2016 um 20:32 schrieb tsteven4:
> Your change seems to break our regression, have your tried "testo mtk"?

Thanks for pointing me to the testo script, I wasn't aware of it.

> I suppose it is possible that the expected output of our test case is wrong.

The reference file looks correct, given the large time gaps between the
existing tracks.

So I'll adapt my changes to only apply to the GR245 and add a test case.

Nevertheless, I would appreciate if any of the Holux GR245 (GPSport) or
M-1200 owners could check if they're seeing the issue as well.

Stefan

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: m241-bin incorrect track creation

Stefan Bauer-12
Am 10.01.2016 um 21:00 schrieb Stefan Bauer:

> Am 10.01.2016 um 20:32 schrieb tsteven4:
>> Your change seems to break our regression, have your tried "testo mtk"?
>
> Thanks for pointing me to the testo script, I wasn't aware of it.
>
>> I suppose it is possible that the expected output of our test case is wrong.
>
> The reference file looks correct, given the large time gaps between the
> existing tracks.
>
> So I'll adapt my changes to only apply to the GR245 and add a test case.

I just noticed that there is already a GP245 test file. As this file
only features a single track, I'd opt for replacing it with my test
case. Or would an additional test case be preferable?

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: m241-bin incorrect track creation

Stefan Bauer-12
Am 10.01.2016 um 21:18 schrieb Stefan Bauer:

> Am 10.01.2016 um 21:00 schrieb Stefan Bauer:
>> Am 10.01.2016 um 20:32 schrieb tsteven4:
>>> Your change seems to break our regression, have your tried "testo mtk"?
>>
>> Thanks for pointing me to the testo script, I wasn't aware of it.
>>
>>> I suppose it is possible that the expected output of our test case is wrong.
>>
>> The reference file looks correct, given the large time gaps between the
>> existing tracks.
>>
>> So I'll adapt my changes to only apply to the GR245 and add a test case.
>
> I just noticed that there is already a GP245 test file. As this file
> only features a single track, I'd opt for replacing it with my test
> case. Or would an additional test case be preferable?

OK, it turns out that the firmware version also plays a role in this
game. The existing GP245 test looks like it is recorded with an older
v1.x firmware. For this firmware, correctly reading the track split
events didn't work until now; only a single large track was created.

My device has firmware v2.00. This caused the tracks to get split, but
at the wrong position.

This change fixes both GP245 FW versions without breaking M241:

https://github.com/stefan-a-bauer/gpsbabel/tree/holuxgr245-fix-trackstart

As always, further feedback, especially from other device owners, is
appreciated.


Stefan

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: m241-bin incorrect track creation

Erik Krause
In reply to this post by Stefan Bauer-12
Am 10.01.2016 um 21:00 schrieb Stefan Bauer:
> Nevertheless, I would appreciate if any of the Holux GR245 (GPSport) or
> M-1200 owners could check if they're seeing the issue as well.

The issue is also present for the original Holux M-241 logger. Compare
http://sourceforge.net/p/gpsbabel/mailman/message/24280896/

--
Erik Krause
Herchersgarten 1
79249 Merzhausen

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: m241-bin incorrect track creation

Robert Lipe-4
I'm not an expert on this family of devices. I'm with tsteven4; it's entirely probable that our captured reference file was from an older device with buggy(ier) firmware. Rather than enshrining that behaviour, we should replace it.

Erik and Stefan, if you could capture your learned lessons for improvements to http://www.gpsbabel.org/htmldoc-development/fmt_m241.html or http://www.gpsbabel.org/htmldoc-development/fmt_m241-bin.html, that would be helpful to the next user of these devices.  We don't really talk much about firmware versions, how to check, wehere to upgrade and such. Perhaps we should.  The source is in xmldoc/formats - if you can provide the words, I can do the markup.

That csv option in the m241bin reader to write a file is weird and should be a format of its own...


On Mon, Jan 11, 2016 at 2:07 PM, Erik Krause <[hidden email]> wrote:
Am 10.01.2016 um 21:00 schrieb Stefan Bauer:
> Nevertheless, I would appreciate if any of the Holux GR245 (GPSport) or
> M-1200 owners could check if they're seeing the issue as well.

The issue is also present for the original Holux M-241 logger. Compare
http://sourceforge.net/p/gpsbabel/mailman/message/24280896/

--
Erik Krause
Herchersgarten 1
79249 Merzhausen

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code


------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: m241-bin incorrect track creation

Stefan Bauer-12
In reply to this post by Erik Krause
Am 11.01.2016 um 21:07 schrieb Erik Krause:
> Am 10.01.2016 um 21:00 schrieb Stefan Bauer:
>> Nevertheless, I would appreciate if any of the Holux GR245 (GPSport) or
>> M-1200 owners could check if they're seeing the issue as well.
>
> The issue is also present for the original Holux M-241 logger. Compare
> http://sourceforge.net/p/gpsbabel/mailman/message/24280896/

Very interesting. On my GR245, I see exactly what P. Borgentun described
in this message:

http://sourceforge.net/p/gpsbabel/mailman/message/24340952/

I'd like to further investigate this issue. Erik, if you could send me a
short binary file that shows the issue I'll try to adapt my fix to the
M241 as well. If it's ok for you I'd also add a test case to the
regression test suite from your sample.

Stefan

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: m241-bin incorrect track creation

Thorben Thuermer
In reply to this post by Stefan Bauer-12

i'll just throw in my comments as an m241 (holux m241 logger) user:

for what i can remember, after some firmware update i found that the
code in gpsbabel at the time failed to parse anything from the bin
files. (iirc the format was changed with an new field added.)
back then i ended up hacking some custom code that just brute-force
scans the bin file for valid trackpoint entries, outputting anything
that has a header and valid crc.
with that i found (iirc) that i had been missing at least some data when
processing files previously.
(sometimes the data is corrupt, and even the windows tool fails to
 process it, but i can do a full dump and extract some data with the
 brute-force method.)
personally i don't really care about the formatting of tracks,
as long as i can get (ts,lat,lon) tuples.

overall i found that the firmware on the m241 is really horrible.
it sometimes ends up corrupting it's internal state, causing it to stop
logging anything (besides manually recorded waypoints).
(with no way to reset it, other than repeatedly re-flashing the
 firmware...!)
also some of by devices continuously report that they are in
"overwrite" mode (acting as a ring buffer because memory is full),
even after erasing them.

- T.

On Sun, 10 Jan 2016 19:56:05 +0100
Stefan Bauer <[hidden email]> wrote:
> Hi,
> I'm currently investigating an erroneous behavior in the m241-bin
> input format. The device I'm using is the Holux GPSport 245.
[...]
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code

m241_parse.c.gz (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: m241-bin incorrect track creation

tsteven4-2
In reply to this post by Robert Lipe-4
Robert,

I think the capture reference input file (${REFERENCE}/track/mtk_logger_m241.bin) isn't what is wrong, it is our expected output (${REFERENCE}/track/mtk_logger_m241.gpx) from this captured reference file.  I think Stefan has already got a change for mtk_logger_m241.gpx up on github.   I think we should keep both so we can test against both versions of device firmware.

Steve

On 1/11/2016 1:59 PM, Robert Lipe wrote:
I'm not an expert on this family of devices. I'm with tsteven4; it's entirely probable that our captured reference file was from an older device with buggy(ier) firmware. Rather than enshrining that behaviour, we should replace it.

Erik and Stefan, if you could capture your learned lessons for improvements to http://www.gpsbabel.org/htmldoc-development/fmt_m241.html or http://www.gpsbabel.org/htmldoc-development/fmt_m241-bin.html, that would be helpful to the next user of these devices.  We don't really talk much about firmware versions, how to check, wehere to upgrade and such. Perhaps we should.  The source is in xmldoc/formats - if you can provide the words, I can do the markup.

That csv option in the m241bin reader to write a file is weird and should be a format of its own...


On Mon, Jan 11, 2016 at 2:07 PM, Erik Krause <[hidden email]> wrote:
Am 10.01.2016 um 21:00 schrieb Stefan Bauer:
> Nevertheless, I would appreciate if any of the Holux GR245 (GPSport) or
> M-1200 owners could check if they're seeing the issue as well.

The issue is also present for the original Holux M-241 logger. Compare
http://sourceforge.net/p/gpsbabel/mailman/message/24280896/

--
Erik Krause
Herchersgarten 1
79249 Merzhausen

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code



------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140


_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code


------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: m241-bin incorrect track creation

Stefan Bauer-12
Am 12.01.2016 um 00:18 schrieb tsteven4:
> Robert,
>
> I think the capture reference input file
> (${REFERENCE}/track/mtk_logger_m241.bin) isn't what is wrong,

ACK

> it is our
> expected output (${REFERENCE}/track/mtk_logger_m241.gpx) from this
> captured reference file.

Rather not, the mtk_logger_m241.gpx is totally fine. The tracks are
split at the right positions. My first proposal broke that.

> I think Stefan has already got a change for
> mtk_logger_m241.gpx up on github.

My current/second proposal

https://github.com/stefan-a-bauer/gpsbabel/commit/9555d19409b4c0f532429482cbdec8dc04e1b8da

doesn't break the m241 test anymore. It does, OTOH, update the
mtk_logger_gp245.gpx (notice: 245, not 241) reference file. This one was
wrong because it did only contain a single track instead of four.

> I think we should keep both so we
> can test against both versions of device firmware.

Given these FW differences we're seeing, I absolutely agree. This is why
I just added my test case next to the others in the above commit.


Thanks,
Stefan

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: m241-bin incorrect track creation

Robert Lipe-4
Thanx for the explanation. What a mess.

This change looks good to me. Are you hip with it, Steven?

On Tue, Jan 12, 2016 at 1:00 AM, Stefan Bauer <[hidden email]> wrote:
Am 12.01.2016 um 00:18 schrieb tsteven4:
> Robert,
>
> I think the capture reference input file
> (${REFERENCE}/track/mtk_logger_m241.bin) isn't what is wrong,

ACK

> it is our
> expected output (${REFERENCE}/track/mtk_logger_m241.gpx) from this
> captured reference file.

Rather not, the mtk_logger_m241.gpx is totally fine. The tracks are
split at the right positions. My first proposal broke that.

> I think Stefan has already got a change for
> mtk_logger_m241.gpx up on github.

My current/second proposal

https://github.com/stefan-a-bauer/gpsbabel/commit/9555d19409b4c0f532429482cbdec8dc04e1b8da

doesn't break the m241 test anymore. It does, OTOH, update the
mtk_logger_gp245.gpx (notice: 245, not 241) reference file. This one was
wrong because it did only contain a single track instead of four.

> I think we should keep both so we
> can test against both versions of device firmware.

Given these FW differences we're seeing, I absolutely agree. This is why
I just added my test case next to the others in the above commit.


Thanks,
Stefan

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code


------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: m241-bin incorrect track creation

tsteven4-2
I would be hip with it, but not until Stefan feels confident enough to submit a pull request.  We are at the mercy of users of these devices to figure out the format variations.

There is at least one dissenting opinion about the application to a 241 (Erik Krause), but the latest proposal (https://github.com/stefan-a-bauer/gpsbabel/commit/9555d19409b4c0f532429482cbdec8dc04e1b8da) will not change the 241.

On 1/12/2016 10:39 AM, Robert Lipe wrote:
Thanx for the explanation. What a mess.

This change looks good to me. Are you hip with it, Steven?

On Tue, Jan 12, 2016 at 1:00 AM, Stefan Bauer <[hidden email]> wrote:
Am 12.01.2016 um 00:18 schrieb tsteven4:
> Robert,
>
> I think the capture reference input file
> (${REFERENCE}/track/mtk_logger_m241.bin) isn't what is wrong,

ACK

> it is our
> expected output (${REFERENCE}/track/mtk_logger_m241.gpx) from this
> captured reference file.

Rather not, the mtk_logger_m241.gpx is totally fine. The tracks are
split at the right positions. My first proposal broke that.

> I think Stefan has already got a change for
> mtk_logger_m241.gpx up on github.

My current/second proposal

https://github.com/stefan-a-bauer/gpsbabel/commit/9555d19409b4c0f532429482cbdec8dc04e1b8da

doesn't break the m241 test anymore. It does, OTOH, update the
mtk_logger_gp245.gpx (notice: 245, not 241) reference file. This one was
wrong because it did only contain a single track instead of four.

> I think we should keep both so we
> can test against both versions of device firmware.

Given these FW differences we're seeing, I absolutely agree. This is why
I just added my test case next to the others in the above commit.


Thanks,
Stefan

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code



------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140


_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code


------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: m241-bin incorrect track creation

Stefan Bauer-12
In reply to this post by Stefan Bauer-12
Am 10.01.2016 um 19:56 schrieb Stefan Bauer:
> What I understand is that when reading the bin file, the stream of track
> points is split when certain events happen. For my device, it looks like
> the wrong events are selected to trigger such a split. The result is an
> "off-by-one" like scenario: The split always happens one track point too
> late.

As it seems that some more people see this, I'd like to share this
little XSLT which fixes the issue for existing GPX files:

https://gist.githubusercontent.com/stefan-a-bauer/7df2545ccc1ce96b5ba7/raw/07c33cca7fbaf4c04d670065702f76d6ceb4e809/1-fixtracks.xslt

Use it like this:

xsltproc 1-fixtracks.xslt broken.gpx > fixed.gpx

Or, if you're brave and have a recent backup:

find . -name "*.gpx" -exec xsltproc -o {} 1-fixtracks.xslt {} \;

Note: Please verify that you're actually affected by the issue. If this
is not the case, the transformation will break your tracks.


Stefan

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: m241-bin incorrect track creation

Robert Lipe-4
In reply to this post by tsteven4-2
If Stefan and Erik can reach a happy place soon, I'd consider it in a release bump (with version number bump, sigh) this weekend.

On Tue, Jan 12, 2016 at 8:45 PM, tsteven4 <[hidden email]> wrote:
I would be hip with it, but not until Stefan feels confident enough to submit a pull request.  We are at the mercy of users of these devices to figure out the format variations.

There is at least one dissenting opinion about the application to a 241 (Erik Krause), but the latest proposal (https://github.com/stefan-a-bauer/gpsbabel/commit/9555d19409b4c0f532429482cbdec8dc04e1b8da) will not change the 241.


On 1/12/2016 10:39 AM, Robert Lipe wrote:
Thanx for the explanation. What a mess.

This change looks good to me. Are you hip with it, Steven?

On Tue, Jan 12, 2016 at 1:00 AM, Stefan Bauer <[hidden email]> wrote:
Am 12.01.2016 um 00:18 schrieb tsteven4:
> Robert,
>
> I think the capture reference input file
> (${REFERENCE}/track/mtk_logger_m241.bin) isn't what is wrong,

ACK

> it is our
> expected output (${REFERENCE}/track/mtk_logger_m241.gpx) from this
> captured reference file.

Rather not, the mtk_logger_m241.gpx is totally fine. The tracks are
split at the right positions. My first proposal broke that.

> I think Stefan has already got a change for
> mtk_logger_m241.gpx up on github.

My current/second proposal

https://github.com/stefan-a-bauer/gpsbabel/commit/9555d19409b4c0f532429482cbdec8dc04e1b8da

doesn't break the m241 test anymore. It does, OTOH, update the
mtk_logger_gp245.gpx (notice: 245, not 241) reference file. This one was
wrong because it did only contain a single track instead of four.

> I think we should keep both so we
> can test against both versions of device firmware.

Given these FW differences we're seeing, I absolutely agree. This is why
I just added my test case next to the others in the above commit.


Thanks,
Stefan

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code



------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140


_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code



------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: m241-bin incorrect track creation

Stefan Bauer-12
Am 14.01.2016 um 19:12 schrieb Robert Lipe:
> If Stefan and Erik can reach a happy place soon, I'd consider it in a
> release bump (with version number bump, sigh) this weekend.

Thanks for offering such a fast publication of the fixes.

I'm in contact with Erik regarding his data. I'll post back here if I
gained any further insights. Unfortunately, I cannot give you any
commitments about when something will be ready. I'll try my best :)


Thanks, Stefan

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: m241-bin incorrect track creation

Robert Lipe-4
No problem and no pressure. I just wanted to be sure we weren't deadlocked.

Besides, based on the comments in this thread and others, I get the impression that Holux users are used to some degree of suffering. :-)

On Thu, Jan 14, 2016 at 12:27 PM, Stefan Bauer <[hidden email]> wrote:
Am 14.01.2016 um 19:12 schrieb Robert Lipe:
If Stefan and Erik can reach a happy place soon, I'd consider it in a
release bump (with version number bump, sigh) this weekend.

Thanks for offering such a fast publication of the fixes.

I'm in contact with Erik regarding his data. I'll post back here if I gained any further insights. Unfortunately, I cannot give you any commitments about when something will be ready. I'll try my best :)


Thanks, Stefan


------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: m241-bin incorrect track creation

Stefan Bauer-12
In reply to this post by Robert Lipe-4
Am 14.01.2016 um 19:12 schrieb Robert Lipe:
> If Stefan and Erik can reach a happy place soon, I'd consider it in a
> release bump (with version number bump, sigh) this weekend.

It turned out that the new 'split mode' that I introduced in my second
proposal is not only valid for GR245, but for the M241 as well. The
previously existing mode is only relevant for the MTK_LOGGER.

So here's proposal number 3:
https://github.com/gpsbabel/gpsbabel/pull/27

Erik, can you please verify that the GPX output that is now generated
matches your expectations?
https://raw.githubusercontent.com/stefan-a-bauer/gpsbabel/holux-fix-trackstart/reference/track/mtk_logger_m241_multiple_tracks.gpx

Stefan

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: m241-bin incorrect track creation

Stefan Bauer-12
In reply to this post by Stefan Bauer-12
Am 13.01.2016 um 21:48 schrieb Stefan Bauer:

> Am 10.01.2016 um 19:56 schrieb Stefan Bauer:
>> What I understand is that when reading the bin file, the stream of track
>> points is split when certain events happen. For my device, it looks like
>> the wrong events are selected to trigger such a split. The result is an
>> "off-by-one" like scenario: The split always happens one track point too
>> late.
>
> As it seems that some more people see this, I'd like to share this
> little XSLT which fixes the issue for existing GPX files:
>
> https://gist.githubusercontent.com/stefan-a-bauer/7df2545ccc1ce96b5ba7/raw/07c33cca7fbaf4c04d670065702f76d6ceb4e809/1-fixtracks.xslt
>
> Use it like this:
>
> xsltproc 1-fixtracks.xslt broken.gpx > fixed.gpx
>
> Or, if you're brave and have a recent backup:
>
> find . -name "*.gpx" -exec xsltproc -o {} 1-fixtracks.xslt {} \;
>
> Note: Please verify that you're actually affected by the issue. If this
> is not the case, the transformation will break your tracks.

It turned out that this was a bad idea. The track split is off-by-one in
most of the cases, but not always. I guess it relates to GPS cold start
/ start of logging before a fix has been acquired.

Bottom line: The above stylesheet is not always fixing things up
correctly and sometimes makes things worse.

If someone's interested in fixing up his old tracks, I'd propose to use
GPSBabels track filter with a time-based split. I used

gpsbabel -t -w -i gpx -f broken.gpx -x track,merge,split=1m -o gpx -F
less-broken.gpx

This also removes duplicate timestamps along the way.

Stefan

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: m241-bin incorrect track creation

Erik Krause
In reply to this post by Stefan Bauer-12
Am 14.01.2016 um 23:52 schrieb Stefan Bauer:

> Erik, can you please verify that the GPX output that is now generated
> matches your expectations?

Yes, that's what I expected. No more orphan points and tracks jumping
all across Europe :-) Many thanks.

--
Erik Krause
Herchersgarten 1
79249 Merzhausen

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
12