Libc++ locale tests

Hi Howard and others,

I am struggling to get the locale related tests in libc++ working like they should. In attached patch I added a header in the main test directory, defining the locale names for Windows and the rest. I replaced all occurrences of the name in a test’s source code with an include to this file and the macro I defined. For every platform != windows, this should come down to the same thing. for Windows, I could not use UTF-8 locales, as they are unsupported and always return 0 when trying to setlocale or newlocale (_create_locale). The patch also fixes a bad #endif placement (if I’m not mistaken) in localization/locale.categories/category.time/locale.time.get.byname/get_one_wide.pass.cpp

The patch fixed exceptions being thrown because of bad locale names, but not the test’s failures. To compare, I used this test code (freely adapted to resemble the true test):
http://stackoverflow.com/questions/7627445/stdget-time-and-other-locale-functionality-not-working-correctly-on-windows

and tried to run the test on MSVC10 and its STL. It compiles, but the asserts fail, as all values are 0. GCC 4.6.1+stdlibc++ isn’t capable of compiling this code: it’s missing the function std::get_time for some reason. These kinds of unexplained failures are present across the board. Their causes seem either unrelated or hidden in the same place in locale-code not yet fixed for Windows. The fact remains that MSVC10, which has the above function, fails the test, which makes me think the input character buffer needs to be in different format.

I plan on adding floating point number macros (which create a string with the expected format from a base and exponent) to this header as well:
FLOAT_MACRO(1234.1234, -56)
would become
“1234.1234” “e” “-” “56”
or
“1234.1234” “e” “-” “0” “56”
(implicitely letting the compiler concatenate the string literals) or whatever depending on what it is on that platform. I think this will solve the locale related numerics failures.

The attached patch got rid of only two test failures (438 to 436), but as I said, at least the locales are being constructed properly, and the remaining failures are due to something else now.

Ruben

localenames.zip (8.43 KB)

platform_support.h appears to be missing.

Howard

2011/10/3 Howard Hinnant <hhinnant@apple.com>

Hi Howard and others,

I am struggling to get the locale related tests in libc++ working like they should. In attached patch I added a header in the main test directory, defining the locale names for Windows and the rest. I replaced all occurrences of the name in a test’s source code with an include to this file and the macro I defined. For every platform != windows, this should come down to the same thing. for Windows, I could not use UTF-8 locales, as they are unsupported and always return 0 when trying to setlocale or newlocale (_create_locale). The patch also fixes a bad #endif placement (if I’m not mistaken) in localization/locale.categories/category.time/locale.time.get.byname/get_one_wide.pass.cpp

The patch fixed exceptions being thrown because of bad locale names, but not the test’s failures. To compare, I used this test code (freely adapted to resemble the true test):
http://stackoverflow.com/questions/7627445/stdget-time-and-other-locale-functionality-not-working-correctly-on-windows

and tried to run the test on MSVC10 and its STL. It compiles, but the asserts fail, as all values are 0. GCC 4.6.1+stdlibc++ isn’t capable of compiling this code: it’s missing the function std::get_time for some reason. These kinds of unexplained failures are present across the board. Their causes seem either unrelated or hidden in the same place in locale-code not yet fixed for Windows. The fact remains that MSVC10, which has the above function, fails the test, which makes me think the input character buffer needs to be in different format.

I plan on adding floating point number macros (which create a string with the expected format from a base and exponent) to this header as well:
FLOAT_MACRO(1234.1234, -56)
would become
“1234.1234” “e” “-” “56”
or
“1234.1234” “e” “-” “0” “56”
(implicitely letting the compiler concatenate the string literals) or whatever depending on what it is on that platform. I think this will solve the locale related numerics failures.

The attached patch got rid of only two test failures (438 to 436), but as I said, at least the locales are being constructed properly, and the remaining failures are due to something else now.

Ruben

<localenames.zip>

platform_support.h appears to be missing.

Fixed patch attached.

There is another point about <[c]stdint[.h]> which is not quite clear to me. The macros __STDC_FORMAT_MACROS and __STDC_LIMIT_MACROS control macro definitions in <stdint.h> for C++. C++11 FDIS says that these macros have no influence for C++, but that’s with regards to , not <stdint.h>.

So either <stdint.h> needs to ignore these macros and define everything if in C++11 mode, or the <stdint.h> test needs to define these before including <stdint.h>, and libc++ needs to define these macros before it includes <stdint.h>. Which is it? I doubt C99 <stdint.h> would need adaptation for C++11… (although I submitted a patch to mingw-w64 earlier to this effect)

Ruben

localenames.zip (9 KB)

> Hi Howard and others,
>
> I am struggling to get the locale related tests in libc++ working like they should. In attached patch I added a header in the main test directory, defining the locale names for Windows and the rest. I replaced all occurrences of the name in a test's source code with an include to this file and the macro I defined. For every platform != windows, this should come down to the same thing. for Windows, I could not use UTF-8 locales, as they are unsupported and always return 0 when trying to setlocale or newlocale (_create_locale). The patch also fixes a bad #endif placement (if I'm not mistaken) in localization/locale.categories/category.time/locale.time.get.byname/get_one_wide.pass.cpp
>
> The patch fixed exceptions being thrown because of bad locale names, but not the test's failures. To compare, I used this test code (freely adapted to resemble the true test):
> http://stackoverflow.com/questions/7627445/stdget-time-and-other-locale-functionality-not-working-correctly-on-windows
>
> and tried to run the test on MSVC10 and its STL. It compiles, but the asserts fail, as all values are 0. GCC 4.6.1+stdlibc++ isn't capable of compiling this code: it's missing the function std::get_time for some reason. These kinds of unexplained failures are present across the board. Their causes seem either unrelated or hidden in the same place in locale-code not yet fixed for Windows. The fact remains that MSVC10, which has the above function, fails the test, which makes me think the input character buffer needs to be in different format.
>
> I plan on adding floating point number macros (which create a string with the expected format from a base and exponent) to this header as well:
> FLOAT_MACRO(1234.1234, -56)
> would become
> "1234.1234" "e" "-" "56"
> or
> "1234.1234" "e" "-" "0" "56"
> (implicitely letting the compiler concatenate the string literals) or whatever depending on what it is on that platform. I think this will solve the locale related numerics failures.
>
> The attached patch got rid of only two test failures (438 to 436), but as I said, at least the locales are being constructed properly, and the remaining failures are due to something else now.
>
> Ruben
> <localenames.zip>

platform_support.h appears to be missing.

Fixed patch attached.

Thanks, committed revision 141003. There are a few minor changes from what you proposed. Please make sure they are ok with you.

There is another point about <[c]stdint[.h]> which is not quite clear to me. The macros __STDC_FORMAT_MACROS and __STDC_LIMIT_MACROS control macro definitions in <stdint.h> for C++. C++11 FDIS says that these macros have no influence for C++, but that's with regards to <cstdint>, not <stdint.h>.

So either <stdint.h> needs to ignore these macros and define everything if in C++11 mode, or the <stdint.h> test needs to define these before including <stdint.h>, and libc++ <cstdint> needs to define these macros before it includes <stdint.h>. Which is it? I doubt C99 <stdint.h> would need adaptation for C++11... (although I submitted a patch to mingw-w64 earlier to this effect)

The intent is that these macros play no role whatsoever (in any header) in C++11. My impression is that that the standard is sufficiently clear on this. However if you disagree, a defect report is probably in order. Instructions for opening an issue are at:

http://www.open-std.org/jtc1/sc22/wg21/docs/lwg-active.html#submit_issue

Howard