Error when using swedish letters in filenames and category

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Error when using swedish letters in filenames and category

Marcus
Hi!

GPSBabel is not working with Swedish letters (å/ä/ö/Å/Ä/Ö), Iv'e been testing converting universal csv to Garmin gpi

* In Input: GRPSBabel won't find the file
* In output: The resulting filename is scrambled eg. _�_�_�_�_�_�.gpi (_å_ä_ö_Å_Ä_Ö.gpi)
* In category: Similar to output file (above mentioned letters will take up 2 bytes per character)

The filenames is not a big problem, I could just rename them, however the category is a bit of a pain in the... editing manually with an hex editor is exhausting.

Following hex values is the correct ones:
Å = C5
Ä = C4
Ö = D6
å = E5
ä = E4
ö = F6



------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc
Reply | Threaded
Open this post in threaded view
|

Re: Error when using swedish letters in filenames and category

tsteven4-2
Can you supply a copy of your input file (raw.csv)?
What version of gpsbabel are you running?

On 1/21/2016 2:30 PM, Marcus wrote:
Hi!

GPSBabel is not working with Swedish letters (å/ä/ö/Å/Ä/Ö), Iv'e been testing converting universal csv to Garmin gpi

* In Input: GRPSBabel won't find the file
* In output: The resulting filename is scrambled eg. _�_�_�_�_�_�.gpi (_å_ä_ö_Å_Ä_Ö.gpi)
* In category: Similar to output file (above mentioned letters will take up 2 bytes per character)

The filenames is not a big problem, I could just rename them, however the category is a bit of a pain in the... editing manually with an hex editor is exhausting.

Following hex values is the correct ones:
Å = C5
Ä = C4
Ö = D6
å = E5
ä = E4
ö = F6




------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140


_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc


------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc
Reply | Threaded
Open this post in threaded view
|

Re: Error when using swedish letters in filenames and category

Marcus
I'm running version 1.5.3

Swedish characters in name and description is working just fine, the problem is how GPSBabel handles Swedish characters from the commandline.

raw.csv:

Name, Latitude, Longitude, Description
"Vxl xx", "60,4731307388851", "15,4165038553414", "Bandel: 000<BR>Bis: 201<BR>Spår: j<BR>Ägare: -”
"Vxl xx", "60,4728171768849", "15,416281572783", "Bandel: 000<BR>Bis: 203<BR>Spår: j<BR>Ägare: -”
On 22 januari 2016 00:52:21 +01:00, tsteven4 wrote:
Can you supply a copy of your input file (raw.csv)?
What version of gpsbabel are you running?


On 1/21/2016 2:30 PM, Marcus wrote:
Hi!

GPSBabel is not working with Swedish letters (å/ä/ö/Å/Ä/Ö), Iv'e been testing converting universal csv to Garmin gpi

* In Input: GRPSBabel won't find the file
* In output: The resulting filename is scrambled eg. _�_�_�_�_�_�.gpi (_å_ä_ö_Å_Ä_Ö.gpi)
* In category: Similar to output file (above mentioned letters will take up 2 bytes per character)

The filenames is not a big problem, I could just rename them, however the category is a bit of a pain in the... editing manually with an hex editor is exhausting.

Following hex values is the correct ones:
Å = C5
Ä = C4
Ö = D6
å = E5
ä = E4
ö = F6




------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140


_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc


------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc
Reply | Threaded
Open this post in threaded view
|

Re: Error when using swedish letters in filenames and category

Robert Lipe-4
So we're handling the CP1252 internally correctly in the GPI file, it's "just" the command line built by the GUI that's using UTF-8, right?

On Thu, Jan 21, 2016 at 9:34 PM, Marcus <[hidden email]> wrote:
I'm running version 1.5.3

Swedish characters in name and description is working just fine, the problem is how GPSBabel handles Swedish characters from the commandline.

raw.csv:

Name, Latitude, Longitude, Description
"Vxl xx", "60,4731307388851", "15,4165038553414", "Bandel: 000<BR>Bis: 201<BR>Spår: j<BR>Ägare: -”
"Vxl xx", "60,4728171768849", "15,416281572783", "Bandel: 000<BR>Bis: 203<BR>Spår: j<BR>Ägare: -”
On 22 januari 2016 00:52:21 +01:00, tsteven4 wrote:
Can you supply a copy of your input file (raw.csv)?
What version of gpsbabel are you running?


On 1/21/2016 2:30 PM, Marcus wrote:
Hi!

GPSBabel is not working with Swedish letters (å/ä/ö/Å/Ä/Ö), Iv'e been testing converting universal csv to Garmin gpi

* In Input: GRPSBabel won't find the file
* In output: The resulting filename is scrambled eg. _�_�_�_�_�_�.gpi (_å_ä_ö_Å_Ä_Ö.gpi)
* In category: Similar to output file (above mentioned letters will take up 2 bytes per character)

The filenames is not a big problem, I could just rename them, however the category is a bit of a pain in the... editing manually with an hex editor is exhausting.

Following hex values is the correct ones:
Å = C5
Ä = C4
Ö = D6
å = E5
ä = E4
ö = F6




------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140


_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc


------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc



------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc
Reply | Threaded
Open this post in threaded view
|

Re: Error when using swedish letters in filenames and category

tsteven4-2

I have a handle on a least part of this.  there is an implicit conversion to QString in gbfopen.  We should probably be using fromlocal8bit for this conversion.  This is not exclusively a GUI problem.

On 1/21/2016 9:05 PM, Robert Lipe wrote:
So we're handling the CP1252 internally correctly in the GPI file, it's "just" the command line built by the GUI that's using UTF-8, right?

On Thu, Jan 21, 2016 at 9:34 PM, Marcus <[hidden email]> wrote:
I'm running version 1.5.3

Swedish characters in name and description is working just fine, the problem is how GPSBabel handles Swedish characters from the commandline.

raw.csv:

Name, Latitude, Longitude, Description
"Vxl xx", "60,4731307388851", "15,4165038553414", "Bandel: 000<BR>Bis: 201<BR>Spår: j<BR>Ägare: -”
"Vxl xx", "60,4728171768849", "15,416281572783", "Bandel: 000<BR>Bis: 203<BR>Spår: j<BR>Ägare: -”
On 22 januari 2016 00:52:21 +01:00, tsteven4 wrote:
Can you supply a copy of your input file (raw.csv)?
What version of gpsbabel are you running?


On 1/21/2016 2:30 PM, Marcus wrote:
Hi!

GPSBabel is not working with Swedish letters (å/ä/ö/Å/Ä/Ö), Iv'e been testing converting universal csv to Garmin gpi

* In Input: GRPSBabel won't find the file
* In output: The resulting filename is scrambled eg. _�_�_�_�_�_�.gpi (_å_ä_ö_Å_Ä_Ö.gpi)
* In category: Similar to output file (above mentioned letters will take up 2 bytes per character)

The filenames is not a big problem, I could just rename them, however the category is a bit of a pain in the... editing manually with an hex editor is exhausting.

Following hex values is the correct ones:
Å = C5
Ä = C4
Ö = D6
å = E5
ä = E4
ö = F6




------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140


_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc


------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc




------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc
Reply | Threaded
Open this post in threaded view
|

Re: Error when using swedish letters in filenames and category

tsteven4-2
ugg.  QTextCodec::codecForLocale seems to think my locale is UTF-8 despite having it set to en_US.iso88591.  If I force conversions to/from Latin1 I can get it to work, but in general this seems to be a poor assumption.  we have had QTextCodec issues before, see main.cc.  Also there is more back and forth between QStrings and char* after gbfopen.



On 1/22/2016 6:17 PM, tsteven4 wrote:

I have a handle on a least part of this.  there is an implicit conversion to QString in gbfopen.  We should probably be using fromlocal8bit for this conversion.  This is not exclusively a GUI problem.

On 1/21/2016 9:05 PM, Robert Lipe wrote:
So we're handling the CP1252 internally correctly in the GPI file, it's "just" the command line built by the GUI that's using UTF-8, right?

On Thu, Jan 21, 2016 at 9:34 PM, Marcus <[hidden email]> wrote:
I'm running version 1.5.3

Swedish characters in name and description is working just fine, the problem is how GPSBabel handles Swedish characters from the commandline.

raw.csv:

Name, Latitude, Longitude, Description
"Vxl xx", "60,4731307388851", "15,4165038553414", "Bandel: 000<BR>Bis: 201<BR>Spår: j<BR>Ägare: -”
"Vxl xx", "60,4728171768849", "15,416281572783", "Bandel: 000<BR>Bis: 203<BR>Spår: j<BR>Ägare: -”
On 22 januari 2016 00:52:21 +01:00, tsteven4 wrote:
Can you supply a copy of your input file (raw.csv)?
What version of gpsbabel are you running?


On 1/21/2016 2:30 PM, Marcus wrote:
Hi!

GPSBabel is not working with Swedish letters (å/ä/ö/Å/Ä/Ö), Iv'e been testing converting universal csv to Garmin gpi

* In Input: GRPSBabel won't find the file
* In output: The resulting filename is scrambled eg. _�_�_�_�_�_�.gpi (_å_ä_ö_Å_Ä_Ö.gpi)
* In category: Similar to output file (above mentioned letters will take up 2 bytes per character)

The filenames is not a big problem, I could just rename them, however the category is a bit of a pain in the... editing manually with an hex editor is exhausting.

Following hex values is the correct ones:
Å = C5
Ä = C4
Ö = D6
å = E5
ä = E4
ö = F6




------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140


_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc


------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc





------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc
Reply | Threaded
Open this post in threaded view
|

Re: Error when using swedish letters in filenames and category

tsteven4-2
This seems to be related to our resistance to using QCoreApplication in gpsbabel.  I think this is the second time we have gotten caught with something like this.

This works:
#include <QtCore/QCoreApplication>
#include <QtCore/QDebug>
#include <QtCore/QTextCodec>

int main(int argc, char *argv[])
{
    QCoreApplication a(argc, argv);
  QStringList args = a.arguments();
qDebug() << args.at(0);
qDebug() << args.at(1);
qDebug() << QTextCodec::codecForLocale()->name();

    a.exit(0); // 4.
}
:~$ ./qta work/tsteven4/radius*csv
"./qta"
"work/tsteven4/radiusÅ.csv"
"ISO-8859-1"

but this doesn't (excerpt from main.cc)
main(int argc, char* argv[])
{
  int c;
  int argn;
  ff_vecs_t* ivecs = NULL;
  ff_vecs_t* ovecs = NULL;
  filter_vecs_t* fvecs = NULL;
  char* fname = NULL;
  char* ofname = NULL;
  const char* ivec_opts = NULL;
  const char* ovec_opts = NULL;
  char* fvec_opts = NULL;
  int opt_version = 0;
  int did_something = 0;
  const char* prog_name = argv[0]; /* argv is modified during processing */
  queue* wpt_head_bak, *rte_head_bak, *trk_head_bak;    /* #ifdef UTF8_SUPPORT */
  signed int wpt_ct_bak, rte_ct_bak, trk_ct_bak;    /* #ifdef UTF8_SUPPORT */
  arg_stack_t* arg_stack = NULL;
qDebug() << QTextCodec::codecForLocale()->name();
:~/work/tsteven4$ ./gpsbabel
"UTF-8"



On 1/22/2016 6:55 PM, tsteven4 wrote:
ugg.  QTextCodec::codecForLocale seems to think my locale is UTF-8 despite having it set to en_US.iso88591.  If I force conversions to/from Latin1 I can get it to work, but in general this seems to be a poor assumption.  we have had QTextCodec issues before, see main.cc.  Also there is more back and forth between QStrings and char* after gbfopen.



On 1/22/2016 6:17 PM, tsteven4 wrote:

I have a handle on a least part of this.  there is an implicit conversion to QString in gbfopen.  We should probably be using fromlocal8bit for this conversion.  This is not exclusively a GUI problem.

On 1/21/2016 9:05 PM, Robert Lipe wrote:
So we're handling the CP1252 internally correctly in the GPI file, it's "just" the command line built by the GUI that's using UTF-8, right?

On Thu, Jan 21, 2016 at 9:34 PM, Marcus <[hidden email]> wrote:
I'm running version 1.5.3

Swedish characters in name and description is working just fine, the problem is how GPSBabel handles Swedish characters from the commandline.

raw.csv:

Name, Latitude, Longitude, Description
"Vxl xx", "60,4731307388851", "15,4165038553414", "Bandel: 000<BR>Bis: 201<BR>Spår: j<BR>Ägare: -”
"Vxl xx", "60,4728171768849", "15,416281572783", "Bandel: 000<BR>Bis: 203<BR>Spår: j<BR>Ägare: -”
On 22 januari 2016 00:52:21 +01:00, tsteven4 wrote:
Can you supply a copy of your input file (raw.csv)?
What version of gpsbabel are you running?


On 1/21/2016 2:30 PM, Marcus wrote:
Hi!

GPSBabel is not working with Swedish letters (å/ä/ö/Å/Ä/Ö), Iv'e been testing converting universal csv to Garmin gpi

* In Input: GRPSBabel won't find the file
* In output: The resulting filename is scrambled eg. _�_�_�_�_�_�.gpi (_å_ä_ö_Å_Ä_Ö.gpi)
* In category: Similar to output file (above mentioned letters will take up 2 bytes per character)

The filenames is not a big problem, I could just rename them, however the category is a bit of a pain in the... editing manually with an hex editor is exhausting.

Following hex values is the correct ones:
Å = C5
Ä = C4
Ö = D6
å = E5
ä = E4
ö = F6




------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140


_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc


------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc






------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc
Reply | Threaded
Open this post in threaded view
|

Re: Error when using swedish letters in filenames and category

Robert Lipe-4
Gross. Sorry that you bore the brunt of that again, Steve.

For command line apps, QCoreApplication is (or at least was) a heavy constructor and introduces (or at least did) weird races in teardown - that are solvable - looks like it's time to exit denial and make it happen so we (you) can stop rediscovering this. Having event loops for things like threads and network calls will be pretty handy...

I'll put this second on my list, behind fixing Mac dmg creation after the 5.5 upgrade broke it. :-/  I'll check in with you before I start laying bits in case you beat me to it.

RJL

On Sat, Jan 23, 2016 at 2:24 PM, tsteven4 <[hidden email]> wrote:
This seems to be related to our resistance to using QCoreApplication in gpsbabel.  I think this is the second time we have gotten caught with something like this.

This works:
#include <QtCore/QCoreApplication>
#include <QtCore/QDebug>
#include <QtCore/QTextCodec>

int main(int argc, char *argv[])
{
    QCoreApplication a(argc, argv);
  QStringList args = a.arguments();
qDebug() << args.at(0);
qDebug() << args.at(1);
qDebug() << QTextCodec::codecForLocale()->name();

    a.exit(0); // 4.
}
:~$ ./qta work/tsteven4/radius*csv
"./qta"
"work/tsteven4/radiusÅ.csv"
"ISO-8859-1"

but this doesn't (excerpt from main.cc)
main(int argc, char* argv[])
{
  int c;
  int argn;
  ff_vecs_t* ivecs = NULL;
  ff_vecs_t* ovecs = NULL;
  filter_vecs_t* fvecs = NULL;
  char* fname = NULL;
  char* ofname = NULL;
  const char* ivec_opts = NULL;
  const char* ovec_opts = NULL;
  char* fvec_opts = NULL;
  int opt_version = 0;
  int did_something = 0;
  const char* prog_name = argv[0]; /* argv is modified during processing */
  queue* wpt_head_bak, *rte_head_bak, *trk_head_bak;    /* #ifdef UTF8_SUPPORT */
  signed int wpt_ct_bak, rte_ct_bak, trk_ct_bak;    /* #ifdef UTF8_SUPPORT */
  arg_stack_t* arg_stack = NULL;
qDebug() << QTextCodec::codecForLocale()->name();
:~/work/tsteven4$ ./gpsbabel
"UTF-8"



On 1/22/2016 6:55 PM, tsteven4 wrote:
ugg.  QTextCodec::codecForLocale seems to think my locale is UTF-8 despite having it set to en_US.iso88591.  If I force conversions to/from Latin1 I can get it to work, but in general this seems to be a poor assumption.  we have had QTextCodec issues before, see main.cc.  Also there is more back and forth between QStrings and char* after gbfopen.



On 1/22/2016 6:17 PM, tsteven4 wrote:

I have a handle on a least part of this.  there is an implicit conversion to QString in gbfopen.  We should probably be using fromlocal8bit for this conversion.  This is not exclusively a GUI problem.

On 1/21/2016 9:05 PM, Robert Lipe wrote:
So we're handling the CP1252 internally correctly in the GPI file, it's "just" the command line built by the GUI that's using UTF-8, right?

On Thu, Jan 21, 2016 at 9:34 PM, Marcus <[hidden email]> wrote:
I'm running version 1.5.3

Swedish characters in name and description is working just fine, the problem is how GPSBabel handles Swedish characters from the commandline.

raw.csv:

Name, Latitude, Longitude, Description
"Vxl xx", "60,4731307388851", "15,4165038553414", "Bandel: 000<BR>Bis: 201<BR>Spår: j<BR>Ägare: -”
"Vxl xx", "60,4728171768849", "15,416281572783", "Bandel: 000<BR>Bis: 203<BR>Spår: j<BR>Ägare: -”
On 22 januari 2016 00:52:21 +01:00, tsteven4 wrote:
Can you supply a copy of your input file (raw.csv)?
What version of gpsbabel are you running?


On 1/21/2016 2:30 PM, Marcus wrote:
Hi!

GPSBabel is not working with Swedish letters (å/ä/ö/Å/Ä/Ö), Iv'e been testing converting universal csv to Garmin gpi

* In Input: GRPSBabel won't find the file
* In output: The resulting filename is scrambled eg. _�_�_�_�_�_�.gpi (_å_ä_ö_Å_Ä_Ö.gpi)
* In category: Similar to output file (above mentioned letters will take up 2 bytes per character)

The filenames is not a big problem, I could just rename them, however the category is a bit of a pain in the... editing manually with an hex editor is exhausting.

Following hex values is the correct ones:
Å = C5
Ä = C4
Ö = D6
å = E5
ä = E4
ö = F6




------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140


_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc


------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc







------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc
Reply | Threaded
Open this post in threaded view
|

Re: Error when using swedish letters in filenames and category

tsteven4-2
I see two parts:
  1. getting codecForLocale set correctly
    1. using QCoreApplication OR
    2. fool around to get the codecForLocale set correctly w/o QCoreApplication
  2. argument forwarding as QStrings

In addition to argument processing problems like Marcus pointed out not having codecForLocale set correctly will result in other issues, e.g. I suspect qPrintable isn't doing what it should.

From looking at the qt source qcoreapplication.cpp I see how to get codecForLocale set correctly w/o QCoreApplication, but I think doing so is not preferable.  I would rather trust Qt to process command line args, which is must be able to do, than keep in sync with how Qt works on all the platforms and versions.

changing our argument forwarding to QStrings is going to hit every format.  e.g. it would be good to change
typedef void (*ff_init)(char const*);
to use QStrings for rd_init, wr_init.
And as Marcus demonstrated there are other arguments besides file names that need to be taken care of as well.



On 1/23/2016 11:41 PM, Robert Lipe wrote:
Gross. Sorry that you bore the brunt of that again, Steve.

For command line apps, QCoreApplication is (or at least was) a heavy constructor and introduces (or at least did) weird races in teardown - that are solvable - looks like it's time to exit denial and make it happen so we (you) can stop rediscovering this. Having event loops for things like threads and network calls will be pretty handy...

I'll put this second on my list, behind fixing Mac dmg creation after the 5.5 upgrade broke it. :-/  I'll check in with you before I start laying bits in case you beat me to it.

RJL

On Sat, Jan 23, 2016 at 2:24 PM, tsteven4 <[hidden email]> wrote:
This seems to be related to our resistance to using QCoreApplication in gpsbabel.  I think this is the second time we have gotten caught with something like this.

This works:
#include <QtCore/QCoreApplication>
#include <QtCore/QDebug>
#include <QtCore/QTextCodec>

int main(int argc, char *argv[])
{
    QCoreApplication a(argc, argv);
  QStringList args = a.arguments();
qDebug() << args.at(0);
qDebug() << args.at(1);
qDebug() << QTextCodec::codecForLocale()->name();

    a.exit(0); // 4.
}
:~$ ./qta work/tsteven4/radius*csv
"./qta"
"work/tsteven4/radiusÅ.csv"
"ISO-8859-1"

but this doesn't (excerpt from main.cc)
main(int argc, char* argv[])
{
  int c;
  int argn;
  ff_vecs_t* ivecs = NULL;
  ff_vecs_t* ovecs = NULL;
  filter_vecs_t* fvecs = NULL;
  char* fname = NULL;
  char* ofname = NULL;
  const char* ivec_opts = NULL;
  const char* ovec_opts = NULL;
  char* fvec_opts = NULL;
  int opt_version = 0;
  int did_something = 0;
  const char* prog_name = argv[0]; /* argv is modified during processing */
  queue* wpt_head_bak, *rte_head_bak, *trk_head_bak;    /* #ifdef UTF8_SUPPORT */
  signed int wpt_ct_bak, rte_ct_bak, trk_ct_bak;    /* #ifdef UTF8_SUPPORT */
  arg_stack_t* arg_stack = NULL;
qDebug() << QTextCodec::codecForLocale()->name();
:~/work/tsteven4$ ./gpsbabel
"UTF-8"



On 1/22/2016 6:55 PM, tsteven4 wrote:
ugg.  QTextCodec::codecForLocale seems to think my locale is UTF-8 despite having it set to en_US.iso88591.  If I force conversions to/from Latin1 I can get it to work, but in general this seems to be a poor assumption.  we have had QTextCodec issues before, see main.cc.  Also there is more back and forth between QStrings and char* after gbfopen.



On 1/22/2016 6:17 PM, tsteven4 wrote:

I have a handle on a least part of this.  there is an implicit conversion to QString in gbfopen.  We should probably be using fromlocal8bit for this conversion.  This is not exclusively a GUI problem.

On 1/21/2016 9:05 PM, Robert Lipe wrote:
So we're handling the CP1252 internally correctly in the GPI file, it's "just" the command line built by the GUI that's using UTF-8, right?

On Thu, Jan 21, 2016 at 9:34 PM, Marcus <[hidden email]> wrote:
I'm running version 1.5.3

Swedish characters in name and description is working just fine, the problem is how GPSBabel handles Swedish characters from the commandline.

raw.csv:

Name, Latitude, Longitude, Description
"Vxl xx", "60,4731307388851", "15,4165038553414", "Bandel: 000<BR>Bis: 201<BR>Spår: j<BR>Ägare: -”
"Vxl xx", "60,4728171768849", "15,416281572783", "Bandel: 000<BR>Bis: 203<BR>Spår: j<BR>Ägare: -”
On 22 januari 2016 00:52:21 +01:00, tsteven4 wrote:
Can you supply a copy of your input file (raw.csv)?
What version of gpsbabel are you running?


On 1/21/2016 2:30 PM, Marcus wrote:
Hi!

GPSBabel is not working with Swedish letters (å/ä/ö/Å/Ä/Ö), Iv'e been testing converting universal csv to Garmin gpi

* In Input: GRPSBabel won't find the file
* In output: The resulting filename is scrambled eg. _�_�_�_�_�_�.gpi (_å_ä_ö_Å_Ä_Ö.gpi)
* In category: Similar to output file (above mentioned letters will take up 2 bytes per character)

The filenames is not a big problem, I could just rename them, however the category is a bit of a pain in the... editing manually with an hex editor is exhausting.

Following hex values is the correct ones:
Å = C5
Ä = C4
Ö = D6
å = E5
ä = E4
ö = F6




------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140


_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc


------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc








------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc
Reply | Threaded
Open this post in threaded view
|

Re: Error when using swedish letters in filenames and category

tsteven4-2
With regards to # 2 I have switched ff_init to const QString& for 99/113 formats.
no test yet.

On 1/24/2016 8:34 AM, tsteven4 wrote:
I see two parts:
  1. getting codecForLocale set correctly
    1. using QCoreApplication OR
    2. fool around to get the codecForLocale set correctly w/o QCoreApplication
  2. argument forwarding as QStrings

In addition to argument processing problems like Marcus pointed out not having codecForLocale set correctly will result in other issues, e.g. I suspect qPrintable isn't doing what it should.

From looking at the qt source qcoreapplication.cpp I see how to get codecForLocale set correctly w/o QCoreApplication, but I think doing so is not preferable.  I would rather trust Qt to process command line args, which is must be able to do, than keep in sync with how Qt works on all the platforms and versions.

changing our argument forwarding to QStrings is going to hit every format.  e.g. it would be good to change
typedef void (*ff_init)(char const*);
to use QStrings for rd_init, wr_init.
And as Marcus demonstrated there are other arguments besides file names that need to be taken care of as well.



On 1/23/2016 11:41 PM, Robert Lipe wrote:
Gross. Sorry that you bore the brunt of that again, Steve.

For command line apps, QCoreApplication is (or at least was) a heavy constructor and introduces (or at least did) weird races in teardown - that are solvable - looks like it's time to exit denial and make it happen so we (you) can stop rediscovering this. Having event loops for things like threads and network calls will be pretty handy...

I'll put this second on my list, behind fixing Mac dmg creation after the 5.5 upgrade broke it. :-/  I'll check in with you before I start laying bits in case you beat me to it.

RJL

On Sat, Jan 23, 2016 at 2:24 PM, tsteven4 <[hidden email]> wrote:
This seems to be related to our resistance to using QCoreApplication in gpsbabel.  I think this is the second time we have gotten caught with something like this.

This works:
#include <QtCore/QCoreApplication>
#include <QtCore/QDebug>
#include <QtCore/QTextCodec>

int main(int argc, char *argv[])
{
    QCoreApplication a(argc, argv);
  QStringList args = a.arguments();
qDebug() << args.at(0);
qDebug() << args.at(1);
qDebug() << QTextCodec::codecForLocale()->name();

    a.exit(0); // 4.
}
:~$ ./qta work/tsteven4/radius*csv
"./qta"
"work/tsteven4/radiusÅ.csv"
"ISO-8859-1"

but this doesn't (excerpt from main.cc)
main(int argc, char* argv[])
{
  int c;
  int argn;
  ff_vecs_t* ivecs = NULL;
  ff_vecs_t* ovecs = NULL;
  filter_vecs_t* fvecs = NULL;
  char* fname = NULL;
  char* ofname = NULL;
  const char* ivec_opts = NULL;
  const char* ovec_opts = NULL;
  char* fvec_opts = NULL;
  int opt_version = 0;
  int did_something = 0;
  const char* prog_name = argv[0]; /* argv is modified during processing */
  queue* wpt_head_bak, *rte_head_bak, *trk_head_bak;    /* #ifdef UTF8_SUPPORT */
  signed int wpt_ct_bak, rte_ct_bak, trk_ct_bak;    /* #ifdef UTF8_SUPPORT */
  arg_stack_t* arg_stack = NULL;
qDebug() << QTextCodec::codecForLocale()->name();
:~/work/tsteven4$ ./gpsbabel
"UTF-8"



On 1/22/2016 6:55 PM, tsteven4 wrote:
ugg.  QTextCodec::codecForLocale seems to think my locale is UTF-8 despite having it set to en_US.iso88591.  If I force conversions to/from Latin1 I can get it to work, but in general this seems to be a poor assumption.  we have had QTextCodec issues before, see main.cc.  Also there is more back and forth between QStrings and char* after gbfopen.



On 1/22/2016 6:17 PM, tsteven4 wrote:

I have a handle on a least part of this.  there is an implicit conversion to QString in gbfopen.  We should probably be using fromlocal8bit for this conversion.  This is not exclusively a GUI problem.

On 1/21/2016 9:05 PM, Robert Lipe wrote:
So we're handling the CP1252 internally correctly in the GPI file, it's "just" the command line built by the GUI that's using UTF-8, right?

On Thu, Jan 21, 2016 at 9:34 PM, Marcus <[hidden email]> wrote:
I'm running version 1.5.3

Swedish characters in name and description is working just fine, the problem is how GPSBabel handles Swedish characters from the commandline.

raw.csv:

Name, Latitude, Longitude, Description
"Vxl xx", "60,4731307388851", "15,4165038553414", "Bandel: 000<BR>Bis: 201<BR>Spår: j<BR>Ägare: -”
"Vxl xx", "60,4728171768849", "15,416281572783", "Bandel: 000<BR>Bis: 203<BR>Spår: j<BR>Ägare: -”
On 22 januari 2016 00:52:21 +01:00, tsteven4 wrote:
Can you supply a copy of your input file (raw.csv)?
What version of gpsbabel are you running?


On 1/21/2016 2:30 PM, Marcus wrote:
Hi!

GPSBabel is not working with Swedish letters (å/ä/ö/Å/Ä/Ö), Iv'e been testing converting universal csv to Garmin gpi

* In Input: GRPSBabel won't find the file
* In output: The resulting filename is scrambled eg. _�_�_�_�_�_�.gpi (_å_ä_ö_Å_Ä_Ö.gpi)
* In category: Similar to output file (above mentioned letters will take up 2 bytes per character)

The filenames is not a big problem, I could just rename them, however the category is a bit of a pain in the... editing manually with an hex editor is exhausting.

Following hex values is the correct ones:
Å = C5
Ä = C4
Ö = D6
å = E5
ä = E4
ö = F6




------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140


_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc


------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc









------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
[hidden email]
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc