All EXIF Lat/Lon writes fail with "Aborted"

classic Classic list List threaded Threaded
22 messages Options
12
Reply | Threaded
Open this post in threaded view
|

All EXIF Lat/Lon writes fail with "Aborted"

Alan Burlison
I'm attempting to write Lat/Lon values from a waypoint file into the
EXIF section of a JPEG file. All attempts fail with "Aborted" being
printed on stdout, no other error is reported. Strace of the gpsbabel
executable reveals that it is calling the libc abort() function which is
causing the process to be terminated with SIGABRT.

The offending line seems to be this:
https://github.com/gpsbabel/gpsbabel/blame/master/exif.cc#L1393

 From looking at the odd indentation and the other changes made in the
same commit
(https://github.com/gpsbabel/gpsbabel/commit/847718da4591488afbd3c2f7270cc3b28df4576d)
I'm guessing this was added for debugging and not removed before commit.

This needs fixing as it looks like it prevents any write operations to
JPEG files.

--
Alan Burlison
--

------------------------------------------------------------------------------
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: All EXIF Lat/Lon writes fail with "Aborted"

Robert Lipe-2

Hi, and welcome.

Unfortunately, that means you're the first to execute that code path in about two years, which also means we have no automated test coverage there.

If you remove the abort(), does it work?

On Jan 2, 2016 9:35 AM, "Alan Burlison" <[hidden email]> wrote:
I'm attempting to write Lat/Lon values from a waypoint file into the
EXIF section of a JPEG file. All attempts fail with "Aborted" being
printed on stdout, no other error is reported. Strace of the gpsbabel
executable reveals that it is calling the libc abort() function which is
causing the process to be terminated with SIGABRT.

The offending line seems to be this:
https://github.com/gpsbabel/gpsbabel/blame/master/exif.cc#L1393

 From looking at the odd indentation and the other changes made in the
same commit
(https://github.com/gpsbabel/gpsbabel/commit/847718da4591488afbd3c2f7270cc3b28df4576d)
I'm guessing this was added for debugging and not removed before commit.

This needs fixing as it looks like it prevents any write operations to
JPEG files.

--
Alan Burlison
--

------------------------------------------------------------------------------
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code

------------------------------------------------------------------------------

_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: All EXIF Lat/Lon writes fail with "Aborted"

Alan Burlison

> If you remove the abort(), does it work?

Don't know, I had a quick go at building but it blew up because of a missing qt dependency. Are there instructions on how to build somewhere? Apologies if I missed them, I only had time today for a quick poke at it.


------------------------------------------------------------------------------

_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: All EXIF Lat/Lon writes fail with "Aborted"

Robert Lipe-2
http://www.gpsbabel.org/htmldoc-development/Source.html should reflect the Git era now.   A fresh set of eyes on it is welcome.

RJL

On Sat, Jan 2, 2016 at 3:01 PM, Alan Burlison <[hidden email]> wrote:

> If you remove the abort(), does it work?

Don't know, I had a quick go at building but it blew up because of a missing qt dependency. Are there instructions on how to build somewhere? Apologies if I missed them, I only had time today for a quick poke at it.



------------------------------------------------------------------------------

_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: All EXIF Lat/Lon writes fail with "Aborted"

tsteven4-2
In reply to this post by Robert Lipe-2
Robert,

It certainly works better.  However, we rewrite a bunch of tags.  We appear to be using gpsversionid 2.0. The image in our references is uses gpsversionid 2.2, and we try to rewrite it as version 2.0 which sounds shaky to me.  The exif specs I have found (2.1, 2.2, 2.3) seem to require a relationship between the exif version and the gpsversion.
exif version 2.1 => gpsversion id 2.0.0.0
exif version 2.2 => gpsversion id 2.2.0.0
exif version 2.3 => gpsversion id 2.3.0.0

Steve

On 1/2/2016 1:56 PM, Robert Lipe wrote:

Hi, and welcome.

Unfortunately, that means you're the first to execute that code path in about two years, which also means we have no automated test coverage there.

If you remove the abort(), does it work?

On Jan 2, 2016 9:35 AM, "Alan Burlison" <[hidden email]> wrote:
I'm attempting to write Lat/Lon values from a waypoint file into the
EXIF section of a JPEG file. All attempts fail with "Aborted" being
printed on stdout, no other error is reported. Strace of the gpsbabel
executable reveals that it is calling the libc abort() function which is
causing the process to be terminated with SIGABRT.

The offending line seems to be this:
https://github.com/gpsbabel/gpsbabel/blame/master/exif.cc#L1393

 From looking at the odd indentation and the other changes made in the
same commit
(https://github.com/gpsbabel/gpsbabel/commit/847718da4591488afbd3c2f7270cc3b28df4576d)
I'm guessing this was added for debugging and not removed before commit.

This needs fixing as it looks like it prevents any write operations to
JPEG files.

--
Alan Burlison
--

------------------------------------------------------------------------------
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code


------------------------------------------------------------------------------


_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code


------------------------------------------------------------------------------

_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: All EXIF Lat/Lon writes fail with "Aborted"

Alan Burlison
In reply to this post by Robert Lipe-2
> http://www.gpsbabel.org/htmldoc-development/Source.html should reflect the
> Git era now.   A fresh set of eyes on it is welcome.

I'm using Linux Mint Debian Edition and it appears that the Qt
development bits aren't installed as default. I've installed what I
believe are the required Qt5 packages. It's also necessary to set the
QT_SELECT envvar to qt5-x86_64-linux-gnu to get a working build, but
with that the normal "configure; make" chant works.

Which is a better choice for gpsbabel, Qt 4 or 5?

------------------------------------------------------------------------------
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: All EXIF Lat/Lon writes fail with "Aborted"

Alan Burlison
In reply to this post by tsteven4-2
> It certainly works better.

Confirmed, after removing the abort() it doesn't fail any longer and
generates a new JPEG file.

>  However, we rewrite a bunch of tags.  We appear
> to be using gpsversionid 2.0. The image in our references is uses
> gpsversionid 2.2, and we try to rewrite it as version 2.0 which sounds shaky
> to me.  The exif specs I have found (2.1, 2.2, 2.3) seem to require a
> relationship between the exif version and the gpsversion.
> exif version 2.1 => gpsversion id 2.0.0.0
> exif version 2.2 => gpsversion id 2.2.0.0
> exif version 2.3 => gpsversion id 2.3.0.0

Confirmed again. Here's the exiv2 dump of the GPS information in the
original image:

1024_L_SK0533895053.JPG  Exif.Image.GPSTag
Long        1  4892
1024_L_SK0533895053.JPG  Exif.GPSInfo.GPSVersionID
Byte        4  2.3.0.0
1024_L_SK0533895053.JPG  Exif.GPSInfo.GPSLatitudeRef
Ascii       2  North
1024_L_SK0533895053.JPG  Exif.GPSInfo.GPSLatitude
Rational    3  53deg 27' 7.932"
1024_L_SK0533895053.JPG  Exif.GPSInfo.GPSLongitudeRef
Ascii       2  West
1024_L_SK0533895053.JPG  Exif.GPSInfo.GPSLongitude
Rational    3  1deg 55' 15.948"
1024_L_SK0533895053.JPG  Exif.GPSInfo.GPSAltitudeRef
Byte        1  Above sea level
1024_L_SK0533895053.JPG  Exif.GPSInfo.GPSAltitude
Rational    1  307.5 m
1024_L_SK0533895053.JPG  Exif.GPSInfo.GPSTimeStamp
Rational    3  15:21:18
1024_L_SK0533895053.JPG  Exif.GPSInfo.GPSStatus
Ascii       2  Measurement in progress
1024_L_SK0533895053.JPG  Exif.GPSInfo.GPSMapDatum
Ascii       7  WGS-84
1024_L_SK0533895053.JPG  Exif.GPSInfo.GPSDateStamp
Ascii      11  2015:12:29

And here it is in the output image. The change in Lat/Lon is expected,
I'm using gpsbabel to correct incorrect Lat/Lon values in a bunch of
photos.

1024_L_SK0456594824.jpg  Exif.Image.GPSTag
Long        1  4932
1024_L_SK0456594824.jpg  Exif.GPSInfo.GPSVersionID
Byte        4  2.0.0.0
1024_L_SK0456594824.jpg  Exif.GPSInfo.GPSLatitudeRef
Ascii       2  North
1024_L_SK0456594824.jpg  Exif.GPSInfo.GPSLatitude
Rational    3  53deg 27' 0.526"
1024_L_SK0456594824.jpg  Exif.GPSInfo.GPSLongitudeRef
Ascii       2  West
1024_L_SK0456594824.jpg  Exif.GPSInfo.GPSLongitude
Rational    3  1deg 55' 57.864"
1024_L_SK0456594824.jpg  Exif.GPSInfo.GPSAltitudeRef
Byte        1  Above sea level
1024_L_SK0456594824.jpg  Exif.GPSInfo.GPSAltitude
Rational    1  307.5 m
1024_L_SK0456594824.jpg  Exif.GPSInfo.GPSTimeStamp
Rational    3  15:21:18
1024_L_SK0456594824.jpg  Exif.GPSInfo.GPSStatus
Ascii       2  Measurement in progress
1024_L_SK0456594824.jpg  Exif.GPSInfo.GPSMapDatum
Ascii       7  WGS-84
1024_L_SK0456594824.jpg  Exif.GPSInfo.GPSDateStamp
Ascii      11  2015:12:29

--
Alan Burlison
--

------------------------------------------------------------------------------
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: All EXIF Lat/Lon writes fail with "Aborted"

Alan Burlison
Actually it looks like there are widespread modifications to the EXIF
information in the images written out by gpsbabel, not just the GPS
section. I haven't dug down into the code enough to find out why...

Here's the test case:

$ cp ORIG.JPG NEW.JPG
$ gpsbabel -i ozi -f test.wpt -o exif,overwrite,name="PIC001" -F NEW.JPG
$ exiv2 -pa ORIG.JPG > ORIG.exif
$ exiv2 -pa NEW.JPG > NEW.exif
$ diff ORIG.exif NEW.exif > /tmp/exif.diff

diff attached.

--
Alan Burlison
--

------------------------------------------------------------------------------

_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code

exif.diff (11K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: All EXIF Lat/Lon writes fail with "Aborted"

tsteven4-2
Weird.  exiftool (libimage-exiftool-perl 9.46-1) has a different opinion
that exiv2 (eviv2 0.23-1ubuntu2) about the Canon related information.  
exiftool shows we don't corrupt it, while exiv2 shows we do.


On 1/3/2016 9:22 AM, Alan Burlison wrote:

> Actually it looks like there are widespread modifications to the EXIF
> information in the images written out by gpsbabel, not just the GPS
> section. I haven't dug down into the code enough to find out why...
>
> Here's the test case:
>
> $ cp ORIG.JPG NEW.JPG
> $ gpsbabel -i ozi -f test.wpt -o exif,overwrite,name="PIC001" -F NEW.JPG
> $ exiv2 -pa ORIG.JPG > ORIG.exif
> $ exiv2 -pa NEW.JPG > NEW.exif
> $ diff ORIG.exif NEW.exif > /tmp/exif.diff
>
> diff attached.
>


------------------------------------------------------------------------------
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: All EXIF Lat/Lon writes fail with "Aborted"

tsteven4-2
In reply to this post by Alan Burlison
I would use Qt5 if it was available, but we currently support later
levels of Qt 4.  Our continuous integration builds are using Qt 4.8.1 at
the moment.

On 1/3/2016 7:47 AM, Alan Burlison wrote:

>> http://www.gpsbabel.org/htmldoc-development/Source.html should reflect the
>> Git era now.   A fresh set of eyes on it is welcome.
> I'm using Linux Mint Debian Edition and it appears that the Qt
> development bits aren't installed as default. I've installed what I
> believe are the required Qt5 packages. It's also necessary to set the
> QT_SELECT envvar to qt5-x86_64-linux-gnu to get a working build, but
> with that the normal "configure; make" chant works.
>
> Which is a better choice for gpsbabel, Qt 4 or 5?
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Gpsbabel-code mailing list  http://www.gpsbabel.org
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gpsbabel-code


------------------------------------------------------------------------------
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: All EXIF Lat/Lon writes fail with "Aborted"

Alan Burlison
> I would use Qt5 if it was available, but we currently support later levels
> of Qt 4.  Our continuous integration builds are using Qt 4.8.1 at the
> moment.

OK, will do - ta :-)

------------------------------------------------------------------------------
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: All EXIF Lat/Lon writes fail with "Aborted"

Alan Burlison
In reply to this post by tsteven4-2
> Weird.  exiftool (libimage-exiftool-perl 9.46-1) has a different opinion
> that exiv2 (eviv2 0.23-1ubuntu2) about the Canon related information.
> exiftool shows we don't corrupt it, while exiv2 shows we do.

As far as I can tell from the limited subset it displays, Win7 also
believes the info is unchanged. I wonder if this is related to the
EXIF version changing when gpsbabel writes out the file, and some of
the fields being interpreted differently as a result? I'll see if I
can update the EXIF version by hand on the gpsbabel-processed file and
see if it makes a difference.

--
Alan Burlison
--

------------------------------------------------------------------------------
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: All EXIF Lat/Lon writes fail with "Aborted"

Alan Burlison
On 03/01/16 18:49, Alan Burlison wrote:

> I'll see if I can update the EXIF version by hand on the
> gpsbabel-processed file and see if it makes a difference.

Nope, that makes no difference. I see this in the exiv2 documentation
(http://exiv2.org/makernote.html)

> Most vendors write the makernote in TIFF format, i.e., in the same
> format as the rest of the Exif information is encoded. This appears
> to be a sensible thing at first glance. Unfortunately, in general it
> means that any change of an Exif tag, which moves the makernote
> field, will corrupt it. It is an inherent problem of the TIFF format
> that a writer must know the format and all extensions used, in order
> to be able to write changes correctly; unknown tags are potentially
> corrupted when they are moved (rearranged). But since makernotes are
> usually proprietary, Exif writers often don't know these details. The
> reason to write to the Exif data could be as simple as to add
> copyright information, an Exif comment, etc. Some camera
> manufacturers seem to have recognized this problem and now use a
> modified TIFF format with offsets relative to somewhere at the
> beginning of the makernote field for the makernote IFD to address the
> issue.

Could that be the problem?

--
Alan Burlison
--

------------------------------------------------------------------------------
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: All EXIF Lat/Lon writes fail with "Aborted"

tsteven4-2
Alan,

I was just going to write you.  In the case I am looking at, which is
reference/IMG_2065.JPG, the problem with MakerNote described below is
what is happening.  In this case the MakerNote tag points to a series of
IFD entries, and the value offsets of those entries are relative to the
beginning of the APP1 (exif) attribute information, i.e. the image file
header.  We are not sophisticated enough to know the format of the
MakerNote as it is "up to the manufacturer" , so we end up corrupting it
when it gets moved as described below.  It is interesting that exiftool
can overcome this corruption and get the correct information despite our
relocation of it.

I will probably clean up some of the issues with the conditional macro
EXIF_DBG.  Working on the MakerNote issue is unattractive since it
depends on each manufacturer.

Steve

On 1/3/2016 4:30 PM, Alan Burlison wrote:

> On 03/01/16 18:49, Alan Burlison wrote:
>
>> I'll see if I can update the EXIF version by hand on the
>> gpsbabel-processed file and see if it makes a difference.
>
> Nope, that makes no difference. I see this in the exiv2 documentation
> (http://exiv2.org/makernote.html)
>
>> Most vendors write the makernote in TIFF format, i.e., in the same
>> format as the rest of the Exif information is encoded. This appears
>> to be a sensible thing at first glance. Unfortunately, in general it
>> means that any change of an Exif tag, which moves the makernote
>> field, will corrupt it. It is an inherent problem of the TIFF format
>> that a writer must know the format and all extensions used, in order
>> to be able to write changes correctly; unknown tags are potentially
>> corrupted when they are moved (rearranged). But since makernotes are
>> usually proprietary, Exif writers often don't know these details. The
>> reason to write to the Exif data could be as simple as to add
>> copyright information, an Exif comment, etc. Some camera
>> manufacturers seem to have recognized this problem and now use a
>> modified TIFF format with offsets relative to somewhere at the
>> beginning of the makernote field for the makernote IFD to address the
>> issue.
>
> Could that be the problem?
>
> --
> Alan Burlison
> --


------------------------------------------------------------------------------
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: All EXIF Lat/Lon writes fail with "Aborted"

Robert Lipe-2
(I'm following this only casually while working on release-y things.)

Lightly researching this, that sounds amazingly dumb, but it's plagued users of EXIF for a Very Long Time.  The prescription at http://www.sno.phy.queensu.ca/~phil/exiftool/faq.html#Q8 seems to be throw out the makernote. Of course, since you don't know what's in it, that may have bad side effects.  Since the Makernote may contain byte offsets (internal pointers) to other things in the file itself, pretty much any change that adds a tag is going to break those even if we pass them along opaquely.

Don Lind's answer in https://productforums.google.com/d/msg/picasa/32jkT3GQSOw/D6Iaa7qdcl8J seems to say that Picasa (which, unlike us, has a valid reason to contain code for a few dozen camera quirks)  also tosses them, or at least did in 2009.  It looks like many other apps do the same, but it also looks like this makes camera aficionados unhappy.





On Sun, Jan 3, 2016 at 5:55 PM, tsteven4 <[hidden email]> wrote:
Alan,

I was just going to write you.  In the case I am looking at, which is reference/IMG_2065.JPG, the problem with MakerNote described below is what is happening.  In this case the MakerNote tag points to a series of IFD entries, and the value offsets of those entries are relative to the beginning of the APP1 (exif) attribute information, i.e. the image file header.  We are not sophisticated enough to know the format of the MakerNote as it is "up to the manufacturer" , so we end up corrupting it when it gets moved as described below.  It is interesting that exiftool can overcome this corruption and get the correct information despite our relocation of it.

I will probably clean up some of the issues with the conditional macro EXIF_DBG.  Working on the MakerNote issue is unattractive since it depends on each manufacturer.

Steve


On 1/3/2016 4:30 PM, Alan Burlison wrote:
On 03/01/16 18:49, Alan Burlison wrote:

I'll see if I can update the EXIF version by hand on the
gpsbabel-processed file and see if it makes a difference.

Nope, that makes no difference. I see this in the exiv2 documentation (http://exiv2.org/makernote.html)

Most vendors write the makernote in TIFF format, i.e., in the same
format as the rest of the Exif information is encoded. This appears
to be a sensible thing at first glance. Unfortunately, in general it
means that any change of an Exif tag, which moves the makernote
field, will corrupt it. It is an inherent problem of the TIFF format
that a writer must know the format and all extensions used, in order
to be able to write changes correctly; unknown tags are potentially
corrupted when they are moved (rearranged). But since makernotes are
usually proprietary, Exif writers often don't know these details. The
reason to write to the Exif data could be as simple as to add
copyright information, an Exif comment, etc. Some camera
manufacturers seem to have recognized this problem and now use a
modified TIFF format with offsets relative to somewhere at the
beginning of the makernote field for the makernote IFD to address the
issue.

Could that be the problem?

--
Alan Burlison
--



------------------------------------------------------------------------------

_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: All EXIF Lat/Lon writes fail with "Aborted"

tsteven4-2
In reply to this post by tsteven4-2
I one test case I can ADD the gps data without corrupting the
MakerNote.  Further analysis of the code is necessary to verify if this
generally will be true in the case that the original image does not
contain a GPS IFD.   Note that this should be the normal use case, i.e.
we want to add GPS information to an image that has none as opposed to
what Alan and I were both testing - MODIFYING existing GPS information
in an image.

On 1/3/2016 4:55 PM, tsteven4 wrote:

> Alan,
>
> I was just going to write you.  In the case I am looking at, which is
> reference/IMG_2065.JPG, the problem with MakerNote described below is
> what is happening.  In this case the MakerNote tag points to a series
> of IFD entries, and the value offsets of those entries are relative to
> the beginning of the APP1 (exif) attribute information, i.e. the image
> file header.  We are not sophisticated enough to know the format of
> the MakerNote as it is "up to the manufacturer" , so we end up
> corrupting it when it gets moved as described below.  It is
> interesting that exiftool can overcome this corruption and get the
> correct information despite our relocation of it.
>
> I will probably clean up some of the issues with the conditional macro
> EXIF_DBG.  Working on the MakerNote issue is unattractive since it
> depends on each manufacturer.
>
> Steve
>
> On 1/3/2016 4:30 PM, Alan Burlison wrote:
>> On 03/01/16 18:49, Alan Burlison wrote:
>>
>>> I'll see if I can update the EXIF version by hand on the
>>> gpsbabel-processed file and see if it makes a difference.
>>
>> Nope, that makes no difference. I see this in the exiv2 documentation
>> (http://exiv2.org/makernote.html)
>>
>>> Most vendors write the makernote in TIFF format, i.e., in the same
>>> format as the rest of the Exif information is encoded. This appears
>>> to be a sensible thing at first glance. Unfortunately, in general it
>>> means that any change of an Exif tag, which moves the makernote
>>> field, will corrupt it. It is an inherent problem of the TIFF format
>>> that a writer must know the format and all extensions used, in order
>>> to be able to write changes correctly; unknown tags are potentially
>>> corrupted when they are moved (rearranged). But since makernotes are
>>> usually proprietary, Exif writers often don't know these details. The
>>> reason to write to the Exif data could be as simple as to add
>>> copyright information, an Exif comment, etc. Some camera
>>> manufacturers seem to have recognized this problem and now use a
>>> modified TIFF format with offsets relative to somewhere at the
>>> beginning of the makernote field for the makernote IFD to address the
>>> issue.
>>
>> Could that be the problem?
>>
>> --
>> Alan Burlison
>> --
>


------------------------------------------------------------------------------
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: All EXIF Lat/Lon writes fail with "Aborted"

Alan Burlison
In reply to this post by tsteven4-2
> I was just going to write you.  In the case I am looking at, which is
> reference/IMG_2065.JPG, the problem with MakerNote described below is what
> is happening.  In this case the MakerNote tag points to a series of IFD
> entries, and the value offsets of those entries are relative to the
> beginning of the APP1 (exif) attribute information, i.e. the image file
> header.  We are not sophisticated enough to know the format of the MakerNote
> as it is "up to the manufacturer" , so we end up corrupting it when it gets
> moved as described below.  It is interesting that exiftool can overcome this
> corruption and get the correct information despite our relocation of it.

Right, I agree that seems to be the problem - moving the MakerNote
section invalidates the offsets and that causes the observed symptoms.
At least we've figured out the cause, if not a solution :-)

--
Alan Burlison
--

------------------------------------------------------------------------------
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: All EXIF Lat/Lon writes fail with "Aborted"

tsteven4-2
In reply to this post by tsteven4-2
Drat, false alarm.  my new test case has corrupted MakerNote data as well.

On 1/4/2016 7:38 AM, tsteven4 wrote:

> I one test case I can ADD the gps data without corrupting the
> MakerNote.  Further analysis of the code is necessary to verify if
> this generally will be true in the case that the original image does
> not contain a GPS IFD.   Note that this should be the normal use case,
> i.e. we want to add GPS information to an image that has none as
> opposed to what Alan and I were both testing - MODIFYING existing GPS
> information in an image.
>
> On 1/3/2016 4:55 PM, tsteven4 wrote:
>> Alan,
>>
>> I was just going to write you.  In the case I am looking at, which is
>> reference/IMG_2065.JPG, the problem with MakerNote described below is
>> what is happening.  In this case the MakerNote tag points to a series
>> of IFD entries, and the value offsets of those entries are relative
>> to the beginning of the APP1 (exif) attribute information, i.e. the
>> image file header. We are not sophisticated enough to know the format
>> of the MakerNote as it is "up to the manufacturer" , so we end up
>> corrupting it when it gets moved as described below.  It is
>> interesting that exiftool can overcome this corruption and get the
>> correct information despite our relocation of it.
>>
>> I will probably clean up some of the issues with the conditional
>> macro EXIF_DBG.  Working on the MakerNote issue is unattractive since
>> it depends on each manufacturer.
>>
>> Steve
>>
>> On 1/3/2016 4:30 PM, Alan Burlison wrote:
>>> On 03/01/16 18:49, Alan Burlison wrote:
>>>
>>>> I'll see if I can update the EXIF version by hand on the
>>>> gpsbabel-processed file and see if it makes a difference.
>>>
>>> Nope, that makes no difference. I see this in the exiv2
>>> documentation (http://exiv2.org/makernote.html)
>>>
>>>> Most vendors write the makernote in TIFF format, i.e., in the same
>>>> format as the rest of the Exif information is encoded. This appears
>>>> to be a sensible thing at first glance. Unfortunately, in general it
>>>> means that any change of an Exif tag, which moves the makernote
>>>> field, will corrupt it. It is an inherent problem of the TIFF format
>>>> that a writer must know the format and all extensions used, in order
>>>> to be able to write changes correctly; unknown tags are potentially
>>>> corrupted when they are moved (rearranged). But since makernotes are
>>>> usually proprietary, Exif writers often don't know these details. The
>>>> reason to write to the Exif data could be as simple as to add
>>>> copyright information, an Exif comment, etc. Some camera
>>>> manufacturers seem to have recognized this problem and now use a
>>>> modified TIFF format with offsets relative to somewhere at the
>>>> beginning of the makernote field for the makernote IFD to address the
>>>> issue.
>>>
>>> Could that be the problem?
>>>
>>> --
>>> Alan Burlison
>>> --
>>
>


------------------------------------------------------------------------------
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: All EXIF Lat/Lon writes fail with "Aborted"

Alan Burlison
In reply to this post by Robert Lipe-2
> Lightly researching this, that sounds amazingly dumb, but it's plagued users
> of EXIF for a Very Long Time.  The prescription at
> http://www.sno.phy.queensu.ca/~phil/exiftool/faq.html#Q8 seems to be throw
> out the makernote. Of course, since you don't know what's in it, that may
> have bad side effects.  Since the Makernote may contain byte offsets
> (internal pointers) to other things in the file itself, pretty much any
> change that adds a tag is going to break those even if we pass them along
> opaquely.

There are some interesting hints in the ExifTool documentation about
this problem, and how they get around it:

----------
3) The maker note information is copied as a block, so it isn't
affected like other information by subsequent tag assignments on the
command line. Also, since the PreviewImage referenced from the maker
notes may be rather large, it is not copied, and must be transferred
separately if desired.
----------

So it appears they don't try to parse it for each different camera
vendor, it's just copied verbatim.

----------
-F[OFFSET] (-fixBase)
Fix the base for maker notes offsets. A common problem with some image
editors is that offsets in the maker notes are not adjusted properly
when the file is modified. This may cause the wrong values to be
extracted for some maker note entries when reading the edited file.
This option allows an integer OFFSET to be specified for adjusting the
maker notes base offset. If no OFFSET is given, ExifTool takes its
best guess at the correct base. Note that exiftool will automatically
fix the offsets for images which store original offset information
(eg. newer Canon models). Offsets are fixed permanently if -F is used
when writing EXIF to an image.
----------

I'm operating at the limits of my sketchy EXIF knowledge here, but
that sounds like it is possible to fix up the maker Notes section
without having to parse it on a per-manufacturer basis, although I
freely admit I don't know how that's achieved by ExifTool.

--
Alan Burlison
--

------------------------------------------------------------------------------
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
Reply | Threaded
Open this post in threaded view
|

Re: All EXIF Lat/Lon writes fail with "Aborted"

Alan Burlison
In reply to this post by tsteven4-2
> Drat, false alarm.  my new test case has corrupted MakerNote data as well.

exiv2 also has a 'write' option, I've used it to modify the GPS
location of a file and it has done so without corrupting the Maker
Note block. Dumping out the EXIF before and after the update reveals
the following differences:

27c27
< Exif.Photo.MakerNote                         Undefined 3792  (Binary
value suppressed)
---
> Exif.Photo.MakerNote                         Undefined 3784  (Binary value suppressed)
108c108
< Exif.GPSInfo.GPSLatitude                     Rational    3  53deg 27' 7.932"
---
> Exif.GPSInfo.GPSLatitude                     Rational    3  0deg
110c110
< Exif.GPSInfo.GPSLongitude                    Rational    3  1deg 55' 15.948"
---
> Exif.GPSInfo.GPSLongitude                    Rational    3  0deg

Assuming Exif.Photo.MakerNote is the offset, it looks like it has been
updated by exiv2 without corrupting the Maker Note block.

--
Alan Burlison
--

------------------------------------------------------------------------------
_______________________________________________
Gpsbabel-code mailing list  http://www.gpsbabel.org
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gpsbabel-code
12