Merging lots of GPX files, while removing waypoints?

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Merging lots of GPX files, while removing waypoints?

Gilles
Hello

I need to merge a lot of GPX files in a single file, and also ignoring waypoints so as to only keep tracks.

The following doesn't work in Windows:
[CODE]
c:\>setlocal enabledelayedexpansion
c:\>set f=
c:\>for %%f in (*.gpx) do set f=!f! -f "%%f"
%%f was unexpected at this time.
[/CODE]

In the GUI, is there the option to ignore waypoints?

Thank you.
Reply | Threaded
Open this post in threaded view
|

Re: Merging lots of GPX files, while removing waypoints?

Robert Lipe-4
I defer your question about Windows batch file programming to a Windows user, but on "ignore waypoint", "-x nuketypes,waypoints" will remove them from the stream.  See 

...and notice that order matters. We process left to right:

-i type -f file1 -f file2 -f file3 -x nuketypes,waypoints -o outputtype -F outputfile.

On Thu, Mar 9, 2017 at 4:46 AM, Gilles <[hidden email]> wrote:
Hello

I need to merge a lot of GPX files in a single file, and also ignoring
waypoints so as to only keep tracks.

The following doesn't work in Windows:
[CODE]
c:\>setlocal enabledelayedexpansion
c:\>set f=
c:\>for %%f in (*.gpx) do set f=!f! -f "%%f"
%%f was unexpected at this time.
[/CODE]

In the GUI, is there the option to ignore waypoints?

Thank you.



--
View this message in context: http://gpsbabel.2324879.n4.nabble.com/Merging-lots-of-GPX-files-while-removing-waypoints-tp12229.html
Sent from the GPSBabel - General mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Announcing the Oxford Dictionaries API! The API offers world-renowned
dictionary content that is easy and intuitive to access. Sign up for an
account today to start using our lexical data to power your apps and
projects. Get started today and enter our developer competition.
http://sdm.link/oxford
_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc


------------------------------------------------------------------------------
Announcing the Oxford Dictionaries API! The API offers world-renowned
dictionary content that is easy and intuitive to access. Sign up for an
account today to start using our lexical data to power your apps and
projects. Get started today and enter our developer competition.
http://sdm.link/oxford
_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc
Reply | Threaded
Open this post in threaded view
|

Re: Merging lots of GPX files, while removing waypoints?

Gilles
Thank you.

Turns out that the example I found was specifically for use in a batch file, and the "!f!" part had to be changed to "%f%" .

This runs without error:
=======
setlocal enabledelayedexpansion
set f=
for %%f in (*.gpx) do set f=!f! -f "%%f"
gpsbabel -i gpx %f% -o gpx -F merge.gpx
=======

However, there's no trace of merge.gpx, although the commands looks like the example:
c:\>gpsbabel -i gpx  -f file1.gpx -f file2.gpx -o gpx -F merge.gpx

https://www.gpsbabel.org/htmldoc-development/Advanced_Usage.html
Reply | Threaded
Open this post in threaded view
|

Re: Merging lots of GPX files, while removing waypoints?

Gilles
FWIW, I only copied gpsbabel.exe from the installation directory:

Directory of c:\Program Files\GPSBabel

31/12/2016  16:24            18ÿ003 COPYING.txt
01/01/2017  18:18            24ÿ945 gmapbase.html
08/03/2017  22:12         1ÿ903ÿ616 gpsbabel.exe
08/03/2017  22:12           902ÿ656 gpsbabelfe.exe
08/03/2017  22:12        22ÿ378ÿ434 icudt51.dll
08/03/2017  22:12         3ÿ369ÿ922 icuin51.dll
08/03/2017  22:12         1ÿ978ÿ690 icuuc51.dll
08/03/2017  22:12           544ÿ817 libgcc_s_dw2-1.dll
08/03/2017  22:12           989ÿ805 libstdc++-6.dll
08/03/2017  22:12            73ÿ901 libwinpthread-1.dll
17/10/2014  16:59    <DIR>          plugins
31/12/2016  16:24               102 qt.conf
08/03/2017  22:12         4ÿ607ÿ488 Qt5Core.dll
08/03/2017  22:12         4ÿ624ÿ896 Qt5Gui.dll
08/03/2017  22:12           824ÿ320 Qt5Multimedia.dll
08/03/2017  22:12           126ÿ464 Qt5MultimediaWidgets.dll
08/03/2017  22:12         1ÿ393ÿ152 Qt5Network.dll
08/03/2017  22:12           402ÿ432 Qt5OpenGL.dll
08/03/2017  22:12           230ÿ400 Qt5Positioning.dll
08/03/2017  22:12           314ÿ368 Qt5PrintSupport.dll
08/03/2017  22:12         4ÿ108ÿ288 Qt5Qml.dll
08/03/2017  22:12         3ÿ403ÿ264 Qt5Quick.dll
08/03/2017  22:12           216ÿ064 Qt5Sensors.dll
08/03/2017  22:12           262ÿ656 Qt5Sql.dll
08/03/2017  22:12        31ÿ421ÿ952 Qt5WebKit.dll
08/03/2017  22:12           255ÿ488 Qt5WebKitWidgets.dll
08/03/2017  22:12         6ÿ282ÿ752 Qt5Widgets.dll
08/03/2017  22:12           254ÿ464 Qt5Xml.dll
08/03/2017  22:12    <DIR>          translations
08/03/2017  22:12            18ÿ006 unins000.dat
08/03/2017  22:12           777ÿ377 unins000.exe

Although it triggers no error, could it be needing some other files?
Reply | Threaded
Open this post in threaded view
|

Re: Merging lots of GPX files, while removing waypoints?

Gilles
Further experiment shows that…
1. The GUI works. The "nuke options" are in the Translation Options > Filters > Miscellaneous

2. Even when using the exact same syntax as shown in the output window of the GUI, the CLI application still generates no file, silently.
Originally, I thought it could be due to relative/absolute pathnames, the use of "/" vs. "\", etc. but it makes not difference.
Maybe it has dependencies so simply copying gpsbabel.exe elsewhere won't do, although I would expect the app to complain if it were missing some dependency.

Anyhow… after importing the big GPX file into a layer in Google My Maps, I notice that GMM still displays waypoints: I assume it does this when a GPX file contains multiple tracks.

https://s10.postimg.org/y5uiok8gp/Google_My_Maps_GPX_multiple_tracks_waypoints.png

Is there a way to have Gpsbabel merge all the tracks into a single track?

Thank you.
Reply | Threaded
Open this post in threaded view
|

Re: Merging lots of GPX files, while removing waypoints?

Robert Lipe-4
In reply to this post by Gilles
Do you have write permission into whatever directdory you're placing merge.gpx? If you had it to create file1, file2, probably so, but it doesn't hurt to check.

In general copying files out of the installed location is a poor strategy. Use the installed version where you told the installer to put it. The exe absolutely needs those .dll's for example, and if it's linking against some other version of libstdc++ or QtCore, bad things will certainly happen.

On Thu, Mar 9, 2017 at 5:42 AM, Gilles <[hidden email]> wrote:
FWIW, I only copied gpsbabel.exe from the installation directory:

Directory of c:\Program Files\GPSBabel

31/12/2016  16:24            18ÿ003 COPYING.txt
01/01/2017  18:18            24ÿ945 gmapbase.html
08/03/2017  22:12         1ÿ903ÿ616 gpsbabel.exe
08/03/2017  22:12           902ÿ656 gpsbabelfe.exe
08/03/2017  22:12        22ÿ378ÿ434 icudt51.dll
08/03/2017  22:12         3ÿ369ÿ922 icuin51.dll
08/03/2017  22:12         1ÿ978ÿ690 icuuc51.dll
08/03/2017  22:12           544ÿ817 libgcc_s_dw2-1.dll
08/03/2017  22:12           989ÿ805 libstdc++-6.dll
08/03/2017  22:12            73ÿ901 libwinpthread-1.dll
17/10/2014  16:59    <DIR>          plugins
31/12/2016  16:24               102 qt.conf
08/03/2017  22:12         4ÿ607ÿ488 Qt5Core.dll
08/03/2017  22:12         4ÿ624ÿ896 Qt5Gui.dll
08/03/2017  22:12           824ÿ320 Qt5Multimedia.dll
08/03/2017  22:12           126ÿ464 Qt5MultimediaWidgets.dll
08/03/2017  22:12         1ÿ393ÿ152 Qt5Network.dll
08/03/2017  22:12           402ÿ432 Qt5OpenGL.dll
08/03/2017  22:12           230ÿ400 Qt5Positioning.dll
08/03/2017  22:12           314ÿ368 Qt5PrintSupport.dll
08/03/2017  22:12         4ÿ108ÿ288 Qt5Qml.dll
08/03/2017  22:12         3ÿ403ÿ264 Qt5Quick.dll
08/03/2017  22:12           216ÿ064 Qt5Sensors.dll
08/03/2017  22:12           262ÿ656 Qt5Sql.dll
08/03/2017  22:12        31ÿ421ÿ952 Qt5WebKit.dll
08/03/2017  22:12           255ÿ488 Qt5WebKitWidgets.dll
08/03/2017  22:12         6ÿ282ÿ752 Qt5Widgets.dll
08/03/2017  22:12           254ÿ464 Qt5Xml.dll
08/03/2017  22:12    <DIR>          translations
08/03/2017  22:12            18ÿ006 unins000.dat
08/03/2017  22:12           777ÿ377 unins000.exe

Although it triggers no error, could it be needing some other files?



--
View this message in context: http://gpsbabel.2324879.n4.nabble.com/Merging-lots-of-GPX-files-while-removing-waypoints-tp12229p12232.html
Sent from the GPSBabel - General mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Announcing the Oxford Dictionaries API! The API offers world-renowned
dictionary content that is easy and intuitive to access. Sign up for an
account today to start using our lexical data to power your apps and
projects. Get started today and enter our developer competition.
http://sdm.link/oxford
_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc


------------------------------------------------------------------------------
Announcing the Oxford Dictionaries API! The API offers world-renowned
dictionary content that is easy and intuitive to access. Sign up for an
account today to start using our lexical data to power your apps and
projects. Get started today and enter our developer competition.
http://sdm.link/oxford
_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc
Reply | Threaded
Open this post in threaded view
|

Re: Merging lots of GPX files, while removing waypoints?

Robert Lipe-4
In reply to this post by Gilles


On Thu, Mar 9, 2017 at 7:34 AM, Gilles <[hidden email]> wrote:
Further experiment shows that…
1. The GUI works. The "nuke options" are in the Translation Options >
Filters > Miscellaneous

2. Even when using the exact same syntax as shown in the output window of
the GUI, the CLI application still generates no file, silently.
Originally, I thought it could be due to relative/absolute pathnames, the
use of "/" vs. "\", etc. but it makes not difference.
Maybe it has dependencies so simply copying gpsbabel.exe elsewhere won't do,

If you want the app installed someplace else, let the installer put it someplace else.
 
although I would expect the app to complain if it were missing some
dependency.

I would, too, but Windows a/v tend to impair apps from working in sane ways if they detect things have been "damaged" by modifications after installation.

 
Anyhow… after importing the big GPX file into a layer in Google My Maps, I
notice that GMM still displays waypoints: I assume it does this when a GPX
file contains multiple tracks.

https://s10.postimg.org/y5uiok8gp/Google_My_Maps_GPX_multiple_tracks_waypoints.png

You say you see <wpt>, but your screenshot shows <trk>. They're not the same. I see no <wpt> tags in that shot. We have no control over how GMM displays things. It quite possibly produces a paddle for the beginning and end points of a <trk>; that's certainly a valid representation of a track.
 
Is there a way to have Gpsbabel merge all the tracks into a single track?
 

------------------------------------------------------------------------------
Announcing the Oxford Dictionaries API! The API offers world-renowned
dictionary content that is easy and intuitive to access. Sign up for an
account today to start using our lexical data to power your apps and
projects. Get started today and enter our developer competition.
http://sdm.link/oxford
_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc
Reply | Threaded
Open this post in threaded view
|

Re: Merging lots of GPX files, while removing waypoints?

Gilles
In reply to this post by Robert Lipe-4
Robert Lipe-4 wrote
Do you have write permission into whatever directdory you're placing
merge.gpx? If you had it to create file1, file2, probably so, but it doesn't hurt to check.
The install directory is admin-only, so instead of updating the system PATH variable, and possibly having to reboot, I simply copied gpsbabel.exe elsewhere, expecting the app to complain if it needed some extra files.

Turns out it was indeed the problem.

This works:
======= nuke.bat
setlocal enabledelayedexpansion
set f=

for %%f in (*.gpx) do set f=!f! -f "%%f"

"c:\Program Files\GPSBabel\gpsbabel.exe" -w -r -t -i gpx %f% -x nuketypes,waypoints -o gpx -F merge.nuked.gpx
=======
Reply | Threaded
Open this post in threaded view
|

Re: Merging lots of GPX files, while removing waypoints?

Robert Lipe-4
You can change path without a reboot (it's just an environmental variable) and you can execute code not in path. z:/path/to/gpsbabel  ...

We definitely do check for an error when opening a file for write in the gpx writer:

 mkdir /tmp/cc
 chmod 000 /tmp/cc
 ./gpsbabel -i geo -f geocaching.loc -o gpx -F /tmp/cc/blah.gpx
Cannot open '/tmp/cc/blah.gpx' for write.  Error was 'Permission denied'.


Windows antivirus and security settings make it hard to say what's going on. We've seen cases where Windows (A/V?) doesn't return an error on system calls but silently processes them, so there's not much we can do about that. You're so far off the beaten path that you'd have to investigate with a debugger and see what's going on if you really want to find out. From your name, it sounds like that's an option.



On Thu, Mar 9, 2017 at 7:45 AM, Gilles <[hidden email]> wrote:
Robert Lipe-4 wrote
> Do you have write permission into whatever directdory you're placing
> merge.gpx? If you had it to create file1, file2, probably so, but it
> doesn't hurt to check.

The install directory is admin-only, so instead of updating the system PATH
variable, and possibly having to reboot, I simply copied gpsbabel.exe
elsewhere, expecting the app to complain if it needed some extra files.

Turns out it was indeed the problem.

This works:
======= nuke.bat
setlocal enabledelayedexpansion
set f=

for %%f in (*.gpx) do set f=!f! -f "%%f"

"c:\Program Files\GPSBabel\gpsbabel.exe" -w -r -t -i gpx %f% -x
nuketypes,waypoints -o gpx -F merge.nuked.gpx
=======



--
View this message in context: http://gpsbabel.2324879.n4.nabble.com/Merging-lots-of-GPX-files-while-removing-waypoints-tp12229p12236.html
Sent from the GPSBabel - General mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Announcing the Oxford Dictionaries API! The API offers world-renowned
dictionary content that is easy and intuitive to access. Sign up for an
account today to start using our lexical data to power your apps and
projects. Get started today and enter our developer competition.
http://sdm.link/oxford
_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc


------------------------------------------------------------------------------
Announcing the Oxford Dictionaries API! The API offers world-renowned
dictionary content that is easy and intuitive to access. Sign up for an
account today to start using our lexical data to power your apps and
projects. Get started today and enter our developer competition.
http://sdm.link/oxford
_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc
Reply | Threaded
Open this post in threaded view
|

Re: Merging lots of GPX files, while removing waypoints?

Gilles
In reply to this post by Robert Lipe-4
Robert Lipe-4 wrote
Is there a way to tell the track filter to ignore timestamps?

=======
"c:\Program Files\GPSBabel\gpsbabel.exe" -i gpx %f% -x nuketypes,waypoints -x track,pack -o gpx -F merge.nuked.gpx

trackfilter-init: Track points badly ordered (timestamp 2010-01-01T00:50:07Z > 2010-01-01T00:00:00Z)!
=======
"c:\Program Files\GPSBabel\gpsbabel.exe" -i gpx %f% -x nuketypes,waypoints -x track,merge -o gpx -F merge.nuked.gpx

trackfilter-init: Found track point at 47.888885,7.536324 without time!
=======
Reply | Threaded
Open this post in threaded view
|

Re: Merging lots of GPX files, while removing waypoints?

Robert Lipe-4
Man, this is the morning for people with GPS data sets running backward...

Oh, wait, you're merging a "lot" of GPX files, probably in a random-ish order, and thus hoping that file2 wasn't recorded before file1. That's going to be tricky.

If this is a case where you don't actually care about the time (perhaps you're just collecting geometry for an OSM-like collection) use the faketime option.

https://www.gpsbabel.org/htmldoc-development/filter_track.html

You may want to look at the merge option of the track filter, too. 


On Thu, Mar 9, 2017 at 8:16 AM, Gilles <[hidden email]> wrote:
Robert Lipe-4 wrote
> See the pack option
> https://www.gpsbabel.org/htmldoc-development/filter_track.html

Is there a way to tell the track filter to ignore timestamps?

=======
"c:\Program Files\GPSBabel\gpsbabel.exe" -i gpx %f% -x nuketypes,waypoints
-x track,pack -o gpx -F merge.nuked.gpx

trackfilter-init: Track points badly ordered (timestamp 2010-01-01T00:50:07Z
> 2010-01-01T00:00:00Z)!
=======
"c:\Program Files\GPSBabel\gpsbabel.exe" -i gpx %f% -x nuketypes,waypoints
-x track,merge -o gpx -F merge.nuked.gpx

trackfilter-init: Found track point at 47.888885,7.536324 without time!
=======



--
View this message in context: http://gpsbabel.2324879.n4.nabble.com/Merging-lots-of-GPX-files-while-removing-waypoints-tp12229p12238.html
Sent from the GPSBabel - General mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Announcing the Oxford Dictionaries API! The API offers world-renowned
dictionary content that is easy and intuitive to access. Sign up for an
account today to start using our lexical data to power your apps and
projects. Get started today and enter our developer competition.
http://sdm.link/oxford
_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc


------------------------------------------------------------------------------
Announcing the Oxford Dictionaries API! The API offers world-renowned
dictionary content that is easy and intuitive to access. Sign up for an
account today to start using our lexical data to power your apps and
projects. Get started today and enter our developer competition.
http://sdm.link/oxford
_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc
Reply | Threaded
Open this post in threaded view
|

Re: Merging lots of GPX files, while removing waypoints?

Gilles
Indeed, those are routes I downloaded from a cycling site, where each route consists in multiple tracks (one track = one GPX file.)

I ran the following command, but it doesn't return, like it's stuck in a loop:

==========
%APP% -i gpx %f% -x nuketypes,waypoints -x track,faketime=f20100705200000 -x track,merge -o gpx -F merge.nuked.gpx
==========

I had to kill the command (DOS) box.
Reply | Threaded
Open this post in threaded view
|

Re: Merging lots of GPX files, while removing waypoints?

Gilles
The issue was that the filenames weren't sequential, so that the route made no sense. I wrongly assumed GpsBabel would just load all the tracks in memory, and build a route from there.

The solution was to start by renaming the files by appending digits in the right order, eg. 1.foo.gpx, 2.bar.gpx, etc., before running the command.

=============
SET APP="c:\Program Files\GPSBabel\gpsbabel.exe"
setlocal enabledelayedexpansion
set f=

for %%f in (*.gpx) do set f=!f! -f "%%f"

%APP% -t -i gpx %f% -x nuketypes,waypoints -o gpx -F merge.nuked.gpx
=============

Thank you very much for your help.
Reply | Threaded
Open this post in threaded view
|

Re: Merging lots of GPX files, while removing waypoints?

Robert Lipe-4


On Mon, Mar 13, 2017 at 8:48 AM, Gilles <[hidden email]> wrote:
The issue was that the filenames weren't sequential, so that the route made
no sense. I wrongly assumed GpsBabel would just load all the tracks in
memory, and build a route from there.


As I said on March 9.

... order matters. We process left to right:


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc
Reply | Threaded
Open this post in threaded view
|

Re: Merging lots of GPX files, while removing waypoints?

Gilles
Yes, but I thought you were talking about options (-x).