UTS #35: Locale Data Markup Language

The deprecated items can be used to indicate elements, attributes, and attribute values
that are deprecated. This means that the items are valid, but
that their usage is strongly discouraged. When the same deprecatedItems element contains
combinations of elements, attributes, and values, then the "least significant" items are only
deprecated if they occur with the "more significant" items. For example:
Deprecated Items
elements="A B">
A and B are deprecated
attributes="C D">
C and D are deprecated on all
elements
elements="A B" attributes="C D">
C and D are deprecated, but only
if they occur on elements A or B.
elements="A B" attributes="C D" values="E">
E is deprecated, but only if it
is a value of C in an element A or B
In each case, multiple items are space-delimited.
C.2
The characters element provides a way for non-Unicode systems, or systems that only
support a subset of Unicode characters, to transform CLDR data. It gives a list of characters with
alternative values that can be used if the main value is not available. For example:



ss


Ö
O


Pts


Fr.



The ordering of the substitute elements indicates the preference among them.
C.3 Calendar Data




...
The common values provide a list of the calendars that are in common
use, and thus should be shown in UIs that provide choice of calendars. (An 'Other...' button could
give access to the other available calendars.)





...
These values provide information
on how a calendar is used in a particular territory. It may also be used in computing week
boundaries for other purposes. The default is provided by the element with
territories="001".
The minDays indicates the minimum
number of days to count as the first week (of a month or year). The first day of the week is typically used for calendar
presentation.
What is meant by the weekend varies from country to country. It
is typically when most non-retail businesses are closed. The time should not be specified unless
it is a well-recognized part of the day.
The weekendStart day defaults to "sat", and weekendEnd
day defaults to "sun".
The weekendStart time defaults to "00:00:00" (midnight
at the start of the day). The weekendEnd time defaults to "24:00:00" (midnight at the end of the
day). (That is, Friday at 24:00:00 is the same time as Saturday at 00:00:00.) Thus the
following are equivalent:


00:00
"/>

24:00
"/>

The week information was formerly in the main LDML file.
C.4 Measurement System






The measurement system is the normal measurement system in common
everyday use (except for date/time). The values are "metric" (= ISO 1000), "US", or "UK"; others
may be added over time. The "US" value indicates the customary system of measurement with
feet, inches, pints, quarts, etc. as used in the United States. The "UK" value indicates the
customary system of measurement with feet, inches, pints, quarts, etc. as used in the United
Kingdom. It is also called the Imperial system: the pint, quart, etc. are different sizes than in
"US".
The paperSize attribute gives the height and width of paper
used for normal business letters. The values are A4 and US.
The measurement information was formerly in the main LDML file,
and had a somewhat different format.
Appendix D:
Language and Locale IDs
People have very slippery notions of what distinguishes a language code vs. a locale code. The
problem is that both are somewhat nebulous concepts.
In practice, many people use [
RFC3066
] codes to mean locale codes
instead of strictly language codes. It is easy to see why this came about; because [
RFC3066
includes an explicit region (territory) code, for most people it was sufficient for use as a
locale code as well. For example, when typical web software receives an [
RFC3066
code, it will use it as a locale code. Other typical software will do the same: in practice,
language codes and locale codes are treated interchangeably. Some people recommend distinguishing
on the basis of "-" vs "_" (e.g.
zh-TW
for language code,
zh_TW
for locale code),
but in practice that does not work because of the free variation out in the world in the use of
these separators. Notice that Windows, for example, uses "-" as a separator in its locale codes.
So pragmatically one is forced to treat "-" and "_" as equivalent when interpreting either one on
input.
Another reason for the conflation of these codes is that
very
little data in most
systems is distinguished by region alone; currency codes and measurement systems being some of the
few. Sometimes date or number formats are mentioned as regional, but that really doesn't make much
sense. If people see the sentence "You will have to adjust the value to १,२३४.५६७ from ૭૧,૨૩૪.૫૬"
(using Indic digits), they would say that sentence is simply not English. Number format is far
more closely associated with language than it is with region. The same is true for date formats:
people would never expect to see intermixed a date in the format "2003年4月1日" (using Kanji) in text
purporting to be purely English. There are regional differences in date and number format —
differences which can be important — but those are different in kind than other language
differences between regions.
As far as we are concerned —
as a completely practical matter
— two languages are
different if they require substantially different localized resources. Distinctions according to
spoken form are important in some contexts, but the written form is by far and away the most
important issue for data interchange. Unfortunately, this is not the principle used in [
ISO639
],
which has the fairly unproductive notion (for data interchange) that only spoken language matters
(it is also not completely consistent about this, however).
RFC3066
can
express a difference if the use of written
languages happens to correspond to region boundaries expressed as [
ISO3166
region codes, and has recently added codes that allow it to express some important cases that are
not distinguished by [
ISO3166
] codes. These written
languages include simplified and
traditional Chinese (both used in Hong Kong S.A.R.); Serbian in Latin script; Azerbaijani
in Arab script, and so on.
Notice also that
currency codes
are different than
currency localizations
. The
currency localizations should largely be in the language-based resource bundles, not in the
territory-based resource bundles. Thus, the resource bundle
en
contains the localized
mappings in English for a range of different currency codes: USD → US$, RUR → Rub, AUD → $A etc. Of course, some currency symbols are used for more than one currency, and in such cases specializations appear in the territory-based bundles. Continuing the example,
en_US
would have USD → $, while
en_AU
would have AUD → $. (In
protocols, the currency codes should always accompany any currency amounts; otherwise the data is
ambiguous, and software is forced to use the user's territory to guess at the currency. For some
informal discussion of this, see
JIT
Localization
.)
D.1
Written Language
Criteria for what makes a written language should be purely pragmatic;
what would
copy-editors say?
If one gave them text like the following, they would respond that is far
from acceptable English for publication, and ask for it to be redone:
"Theatre Center News: The date of the last version of this document was 2003年3月20日.
A copy can be obtained for $50,0 or 1.234,57 грн. We would like to acknowledge contributions by
the following authors (in alphabetical order): Alaa Ghoneim, Behdad Esfahbod, Ahmed Talaat, Eric
Mader, Asmus Freytag, Avery Bishop, and Doug Felt."
So one would change it to either B or C below, depending on which orthographic variant of
English was the target for the publication:
"Theater Center News: The date of the last version of this document was 3/20/2003. A copy
can be obtained for $50.00 or 1,234.57 Ukrainian Hryvni. We would like to acknowledge
contributions by the following authors (in alphabetical order): Alaa Ghoneim, Ahmed Talaat,
Asmus Freytag, Avery Bishop, Behdad Esfahbod, Doug Felt, Eric Mader."
"Theatre Centre News: The date of the last version of this document was 20/3/2003. A copy
can be obtained for $50.00 or 1,234.57 Ukrainian Hryvni. We would like to acknowledge
contributions by the following authors (in alphabetical order): Alaa Ghoneim, Ahmed Talaat,
Asmus Freytag, Avery Bishop, Behdad Esfahbod, Doug Felt, Eric Mader."
Clearly there are many acceptable variations on this text. For example, copy editors might
still quibble with the use of first vs. last name sorting in the list, but clearly the first list
was
not
acceptable English alphabetical order. And in quoting a name, like "Theatre Centre
News", one may leave it in the source orthography even if it differs from the publication target
orthography. And so on. However, just as clearly, there limits on what is acceptable English, and
"2003年3月20日", for example, is
not
Note that the language of locale data may differ from the language of localized software or web sites, when those latter are not localized into the user's preferred language. In such cases, the kind of incongruous juxtapositions described above may well appear, but this situation is usually preferable to forcing unfamiliar date or number formats on the user as well.
Appendix E:
Unicode Sets
A UnicodeSet is a set of Unicode characters (and possibly strings) determined by a pattern,
following
UTS #18: Unicode Regular Expressions
URegex
],
Level 1 and RL2.5, including the syntax where given. For an
example of a concrete implementation of this, see [
ICUUnicodeSet
].
Patterns are a series of characters bounded by square brackets that contain lists of characters
and Unicode property sets. Lists are a sequence of characters that may have ranges indicated by a
'-' between two characters, as in "a-z". The sequence specifies the range of all characters from
the left to the right, in Unicode order. For example,
[a c d-f m]
is equivalent to
[a c
d e f m]
. Whitespace can be freely used for clarity, as
[a c d-f m]
means the same as
[acd-fm]
Unicode property sets are specified by any Unicode property and a value of
that property, such as
[:General_Category=Letter:]
The property names are defined by the PropertyAliases.txt
file and the property values by the PropertyValueAliases.txt file. For more
information, see [
UCD
]. The syntax for specifying the property
sets
is an extension of either POSIX or Perl syntax, by the addition of "=". For example, you can
match letters by using the POSIX-style syntax:
[:General_Category=Letter:]
or by using the Perl-style syntax
\p{General_Category=Letter}
Property names and values are case-insensitive, and
whitespace, "-", and "_" are ignored. The property name can be omitted for the Category and Script properties, but is required for other
properties. If the property value is omitted, it is
assumed to represent a boolean property with the value "true". Thus
[:Letter:]
is equivalent to
[:General_Category=Letter:]
, and
[:Wh-ite-s pa_ce:]
is equivalent to
[:Whitespace=true:]
The table below shows the two kinds of syntax: POSIX and Perl style. Also, the table shows the
"Negative", which is a property that excludes all characters of a given kind. For example,
[:^Letter:]
matches all characters that are not
[:Letter:]
Positive
Negative
POSIX-style Syntax
[:type=value:]
[:^type=value:]
Perl-style Syntax
\p{type=value}
\P{type=value}
These following low-level lists or properties then can be freely combined with the normal set
operations (union, inverse, difference, and intersection):
To union two sets, simply concatenate them. For example,
[[:letter:] [:number:]]
To intersect two sets, use the '&' operator. For example,
[[:letter:] & [a-z]]
To take the set-difference of two sets, use the '-' operator. For example,
[[:letter:] -
[a-z]]
To invert a set, place a '^' immediately after the opening '['. For example,
[^a-z]
In any other location, the '^' does not have a special meaning.
The binary operators '&', '-',
and the implicit union have equal precedence and bind left-to-right. Thus
[[:letter:]-[a-z]-[\u0100-\u01FF]]
is equal to
[[[:letter:]-[a-z]]-[\u0100-\u01FF]]
Another example is the set
[[ace][bdf] - [abc][def]]
, which is not the empty set,
but
instead equal to
[[[[ace] [bdf]] - [abc]] [def]]
, which equals
[[[abcdef] - [abc]] [def]]
which equals
[[def] [def]]
, which equals
[def]
One caution: the '&' and '-' operators operate between sets. That is, they
must be immediately preceded and immediately followed by a set. For example, the pattern
[[:Lu:]-A]
is illegal, since it is interpreted as the set
[:Lu:]
followed by the
incomplete range
-A
. To specify the set of uppercase letters except for 'A', enclose the
'A' in a set:
[[:Lu:]-[A]]
A multi-character string can be in a Unicode set, to represent a
tailored grapheme cluster for a particular language. The syntax uses curly braces for that case.
In Unicode Sets, there are two ways to quote syntax characters and whitespace:
E.1 Single Quote
Two single quotes represents a single quote, either inside or outside single quotes. Text
within single quotes is not interpreted in any way (except for two adjacent single quotes). It is
taken as literal text (special characters become non-special).
E.2 Backslash Escapes
Outside of single quotes, certain backslashed characters have special meaning:
\uhhhh
Exactly 4 hex digits; h in [0-9A-Fa-f]
\Uhhhhhhhh
Exactly 8 hex digits
\xhh
1-2 hex digits
\ooo
1-3 octal digits; o in [0-7]
\a
U+0007 (BELL)
\b
U+0008 (BACKSPACE)
\t
U+0009 (HORIZONTAL TAB)
\n
U+000A (LINE FEED)
\v
U+000B (VERTICAL TAB)
\f
U+000C (FORM FEED)
\r
U+000D (CARRIAGE RETURN)
\\
U+005C (BACKSLASH)
\N{name}
The Unicode character named "name".
Anything else following a backslash is mapped to itself, except in an environment where it is
defined to have some special meaning. For example,
\p{uppercase}
is the set of uppercase
letters in Unicode.
Any character formed as the result of a backslash escape loses any special meaning and is
treated as a literal. In particular, note that \u and \U escapes create literal characters. (In
contrast, Java treats Unicode escapes as just a way to represent arbitrary characters in an ASCII
source file, and any resulting characters are
not
tagged as literals.)
The following table summarizes the syntax that can
be used.
Example
Description
[a]
The set containing 'a' alone
[a-z]
The set containing 'a' through 'z' and all letters in between, in
Unicode order.
Thus it is the same as [\u0061-\u007A].
[^a-z]
The set containing all characters but 'a' through 'z'.
Thus it is the same as [\u0000-\u0061
\u007B..\U0010FFFF].
[[pat1][pat2]]
The union of sets specified by pat1 and pat2
[[pat1]&[pat2]]
The intersection of sets specified by pat1 and pat2
[[pat1]-[pat2]]
The asymmetric difference of sets specified by pat1 and pat2
[a {ab} {ac}]
The character 'a' and the multi-character strings "ab" and "ac"
[:Lu:]
The set of characters with a given property value, as defined by
PropertyValueAliases.txt. In this case,
these are the Unicode uppercase letters. The long form for this is
[:General_Category=Uppercase_Letter:]
[:L:]
The set of characters belonging to all Unicode categories starting with 'L', that is,
[[:Lu:][:Ll:][:Lt:][:Lm:][:Lo:]]
. The long form for this is
[:General_Category=Letter:]
Appendix F:
Date Format Patterns
A date pattern is a string of characters, where specific strings of characters are
replaced with date and time data from a calendar when formatting or used to generate data for a
calendar when parsing. The following are the characters used in patterns to show the appropriate
formats for a given locale. The following are examples:
Pattern
Result (in a particular locale)
yyyy.MM.dd G 'at' HH:mm:ss zzz
1996.07.10 AD at 15:08:56 PDT
EEE, MMM d, ''yy
Wed, July 10, '96
h:mm a
12:08 PM
hh 'o''clock' a, zzzz
12 o'clock PM, Pacific Daylight Time
K:mm a, z
0:00 PM, PST
yyyyy.MMMM.dd GGG hh:mm aaa
01996.July.10 AD 12:08 PM
Characters may be used multiple times. For example, if y is used for the year, 'yy'
might produce '99', whereas 'yyyy' produces '1999'. For most numerical
fields, the number of characters specifies the field width. For example, if h is the hour, 'h'
might produce '5', but 'hh' produces '05'. For some characters, the
count specifies whether an abbreviated or full form should be used, but may have other choices, as
given below.
Two single quotes represents a literal single quote,
either inside or outside single quotes. Text within single quotes is not
interpreted in any way (except for two adjacent single quotes). Otherwise
all ASCII letter from a to z and A to Z are reserved as syntax characters,
and require quoting if they are to represent literal characters. In
addition, certain ASCII punctuation characters may become variable in the
future (eg ":" being interpreted as the time separator and '/' as a date
separator, and replaced by respective locale-sensitive characters in
display).
Note: the counter-intuitive use of 5 letters for the narrow form of weekdays and months
is forced by backwards compatibility.
Date Field Symbol Table
Field
Sym.
No.
Example
Description
era
1..3
AD
Era - Replaced with the
Era string for the current date. One to three letters for the
abbreviated form, four letters for the long form, five for the narrow form.
Anno Domini
year
1..n
1996
Year. Normally the length specifies the padding, but for two letters it also
specifies the maximum length. Example:
Year
yy
yyy
yyyy
yyyyy
AD 1
01
001
0001
00001
AD 12
12
12
012
0012
00012
AD 123
123
23
123
0123
00123
AD 1234
1234
34
1234
1234
01234
AD 12345
12345
45
12345
12345
12345
1..n
1997
Year (of "Week of Year"), used in ISO year-week calendar. May differ from calendar
year.
1..n
4601
Extended year. This is a single number designating the year of this calendar system,
encompassing all supra-year fields. For example, for the Julian calendar system, year numbers
are positive, with an era of BCE or CE. An extended year value for the Julian calendar system
assigns positive values to CE years and negative values to BCE years, with 1 BCE being year 0.
quarter
1..2
02
Quarter - Use one or two for the numerical quarter, three for the abbreviation, or four for
the full name.
Q2
2nd quarter
1..2
02
Stand-Alone
Quarter - Use one or two for the numerical quarter, three for the
abbreviation, or four for the full name.
Q2
2nd quarter
month
1..2
09
Month - Use one or two for
the numerical month, three for the abbreviation, or four for the full name, or five for the
narrow name.
Sept
September
1..2
09
Stand-Alone
Month - Use one or two for the numerical month, three for the abbreviation, or
four for the full name, or 5 for the narrow name.
Sept
September
week
1..2
27
Week of Year.
Week of Month
day
1..2
Date - Day of the month
1..3
345
Day of year
Day of Week in Month. The example is for the 2nd Wed in July
1..n
2451334
Modified Julian day. This is different from the conventional Julian day number in two
regards. First, it demarcates days at local zone midnight, rather than noon GMT. Second, it is
a local number; that is, it depends on the local time zone. It can be thought of as a single
number that encompasses all the date-related fields.
week
day
1..3
Tues
Day of week - Use one
through three letters for the short day, or four for the full name, or five for the narrow
name.
Tuesday
1..2
Local day of week. Same as E
except adds a numeric value that will depend on the local
starting day of the week, using one or two letters. For this example, Monday is the first day
of the week.
Tues
Tuesday
Stand-Alone
local day of week - Use one letter for the local numeric value (same as 'e'),
three for the short day, or four for the full name, or five for the narrow name.
Tues
Tuesday
period
AM
AM or PM
hour
1..2
11
Hour [1-12].
1..2
13
Hour [0-23].
1..2
Hour [0-11].
1..2
24
Hour [1-24].
minute
1..2
59
Minute. Use one or two for zero padding.
second
1..2
12
Second. Use one or two for zero padding.
1..n
3457
Fractional Second - rounds to the count of letters. (example is for 12.34567)
1..n
69540000
Milliseconds in day. This field behaves
exactly
like a composite of all
time-related fields, not including the zone fields. As such, it also reflects discontinuities
of those fields on DST transition days. On a day of DST onset, it will jump forward. On a day
of DST cessation, it will jump backward. This reflects the fact that is must be combined with
the offset field to obtain a unique local time value.
zone
1..3
PDT
Timezone - Use one to three letters for the short timezone or four for the full name. For more
information, see
Appendix J: Time Zone Display
Names
Pacific Daylight Time
1..3
-0800
Use
one to three letters for RFC 822, four letters for GMT
format.
GMT-08:00
PT
Use
one letter for short wall (generic) time, four for long wall time. For more information, see
Appendix J: Time Zone Display Names
Pacific Time
All non-letter character represent themselves in a pattern, except for the single quote.
It is used to 'escape' letters. Two single quotes in a row, whether inside or outside a quoted
sequence, represent a 'real' single quote.
F.1
Localized Pattern Characters
(deprecated)
These are characters that can be used when displaying a date pattern to an end user. This
can occur, for example, when a spreadsheet allows users to specify date patterns. Whatever is in
the string is substituted one-for-one with the characters "
GyMdkHmsSEDFwWahKzYe",
with the above meanings
. Thus, for example, if "J" is to be used instead of
"Y" to mean Year, then the string would be: "
GyMdkHmsSEDFwWahKz
e".
This element is deprecated. It is recommended instead that a more sophisticated UI be
used for localization, such as using icons to represent the different formats (and lengths) in the
Date Field Symbol Table
F.2 AM / PM
Even for countries where the customary date format only has a 24 hour format, both the am
and pm localized strings must be present and must be distinct from one another. Note that as long
as the 24 hour format is used, these strings will normally never be used, but for testing and
unusual circumstances they must be present.
F.3 Eras
There are only two values for an era in a Gregorian calendar, "BC" and "AD". These values
can be translated into other languages, like "a.C." and and "d.C."
for Spanish, but there are no other eras in the Gregorian calendar. Other calendars have a
different numbers of eras. Care should be taken when translating the era names for a specific
calendar.
F.4 Week of Year
Values calculated for the Week of Year field range from 1 to 53
for the Gregorian calendar (they may have different ranges for other
calendars). Week 1 for a year is the
first week that contains at least the specified minimum number of days from that year. Weeks
between week 1 of one year and week 1 of the following year are numbered sequentially from 2 to 52
or 53 (if needed). For example, January 1, 1998 was a Thursday. If the first day of the week is
MONDAY and the minimum days in a week is 4 (these are the values reflecting ISO 8601 and many
national standards), then week 1 of 1998 starts on December 29, 1997, and ends on January 4, 1998.
However, if the first day of the week is SUNDAY, then week 1 of 1998 starts on January 4, 1998,
and ends on January 10, 1998. The first three days of 1998 are then part of week 53 of 1997.
Values are similarly calculated for the Week of Month.
F.5 Week Elements
firstDay
A number indicating which day of the week is considered the 'first' day, for calendar
purposes. Because the ordering of days may vary between calendar, keywords are used for this
value, such as sun, mon,... These values will be replaced by the
localized name when they are actually used.
minDays (Minimal Days in First Week)
Minimal days required in the first week of a month or year. For example, if the first
week is defined as one that contains at least one day, this value will be 1. If it must contain
a full seven days before it counts as the first week, then the value would be 7.
weekendStart, weekendEnd
Indicates the day and time that the weekend starts or ends. As with
firstDay, keywords are used instead of numbers.
Appendix G:
Number Format Patterns
G.1 Number Patterns
The
NumberElements
resource
affects how these patterns are interpreted in a localized context. Here are some examples, based
on the French locale. The "." shows where the decimal point should go. The "," shows where the
thousands separator should go. A "0" indicates zero-padding: if the number is too short, a zero
(in the locale's numeric set) will go there. A "#" indicates no padding: if the number is too
short, nothing goes there. A "¤" shows where the currency sign will go. The following illustrates
the effects of different patterns for the French locale, with the number "1234.567". Notice how
the pattern characters ',' and '.' are replaced by the characters appropriate for the locale.
Pattern
Currency
Text
#,##0.##
n/a
1 234,57
#,##0.###
n/a
1 234,567
###0.#####
n/a
1234,567
###0.0000#
n/a
1234,5670
00000.0000
n/a
01234,5670
# ##0.00 ¤
EUR
1 234,57 €
JPY
1 235 ¥
The number of # placeholder characters before the decimal do not matter, since no limit is
placed on the maximum number of digits. There should, however, be at least one zero someplace in
the pattern. In currency formats, the number of digits after the decimal also do not matter, since
the information in the supplemental data (see
Appendix C: Supplemental Data
is used to
override the number of decimal places — and the rounding — according to the currency that is being
formatted. That can be seen in the above chart, with the difference between Yen and Euro
formatting.
G.2 Special Pattern Characters
Many characters in a pattern are taken literally; they are matched during parsing and
output unchanged during formatting. Special characters, on the other hand, stand for other
characters, strings, or classes of characters. For example, the '#' character is replaced by a
localized digit. Often the replacement character is the same as the pattern character; in the U.S.
locale, the ',' grouping character is replaced by ','. However, the replacement is still
happening, and if the symbols are modified, the grouping character changes. Some special
characters affect the behavior of the formatter by their presence; for example, if the percent
character is seen, then the value is multiplied by 100 before being displayed.
To insert a special character in a pattern as a literal, that is, without any special
meaning, the character must be quoted. There are some exceptions to this which are noted below.
Symbol
Location
Localized?
Meaning
Number
Yes
Digit
1-9
Number
Yes
'1' through '9' indicate rounding.
Number
No
Significant digit
Number
Yes
Digit, zero shows as absent
Number
Yes
Decimal separator or monetary decimal separator
Number
Yes
Minus sign
Number
Yes
Grouping separator
Number
Yes
Separates mantissa and exponent in scientific notation.
Need not
be quoted in prefix or suffix.
Exponent
Yes
Prefix positive exponents with localized plus sign.
Need not be
quoted in prefix or suffix.
Subpattern boundary
Yes
Separates positive and negative subpatterns
Prefix or suffix
Yes
Multiply by 100 and show as percentage
(\u2030)
Prefix or suffix
Yes
Multiply by 1000 and show as per mille
¤ (\u00A4)
Prefix or suffix
No
Currency sign, replaced by currency symbol. If doubled, replaced by international
currency symbol. If tripled, uses the long form of the decimal symbol. If
present in a pattern, the monetary decimal separator and grouping separators (if
available) are used instead of the numeric ones.
Prefix or suffix
No
Used to quote special characters in a prefix or suffix, for example,
"'#'#"
formats 123 to
"#123"
To create a single quote itself, use two in a row:
"# o''clock"
Prefix or suffix boundary
Yes
Pad escape, precedes pad character
A pattern contains a postive and may contain a
negative subpattern, for example, "#,##0.00;(#,##0.00)". Each subpattern has a prefix, a numeric part, and a suffix. If there is no explicit
negative subpattern, the negative subpattern
is the localized minus sign prefixed to the positive subpattern. That
is, "0.00" alone is equivalent to "0.00;-0.00". If there is an explicit negative
subpattern, it serves only to specify the negative prefix and suffix; the number of
digits, minimal digits, and other characteristics are ignored in the negative
subpattern. That means that "#,##0.0#;(#)" has precisely the same result as
"#,##0.0#;(#,##0.0#)".
Note:
The thousands separator and decimal separator in this pattern are always
',' and '.'. They are substituted by the code with the correct local values according to other
fields in CLDR.
The prefixes, suffixes, and various symbols used for infinity, digits, thousands
separators, decimal separators, etc. may be set to arbitrary values, and they will appear properly
during formatting.
However, care must be taken that the symbols and strings do not
conflict, or parsing will be unreliable.
For example, either the positive and
negative prefixes or the suffixes must be distinct for any parser using this data to be able to
distinguish positive from negative values. Another example is that the decimal separator and
thousands separator should be distinct characters, or parsing will be impossible.
The
grouping separator
is a character that separates
clusters of integer digits to make large numbers more legible. It commonly used for thousands, but
in some locales it separates ten-thousands. The
grouping size
is the number of digits between the grouping separators, such as 3 for "100,000,000" or 4 for "1
0000 0000". There are actually two different grouping sizes: One used for the least significant
integer digits, the
primary grouping size
, and one used for all
others, the
secondary grouping size
. In most locales these are
the same, but sometimes they are different. For example, if the primary grouping interval is 3,
and the secondary is 2, then this corresponds to the pattern "#,##,##0", and the number 123456789
is formatted as "12,34,56,789". If a pattern contains multiple grouping separators, the interval
between the last one and the end of the integer defines the primary grouping size, and the
interval between the last two defines the secondary grouping size. All others are ignored, so
"#,##,###,####" == "###,###,####" == "##,#,###,####".
When parsing using a number format, a more lenient parse should be used where possible.
In particular, it should implement at least the following rules.
If the separator is a NO BREAK SPACE, all input in general category Zs
is treated as
matching.
If the separator is an apostrophe (curly or straight), all input apostrophes
are treated as matching. The supplemental data
can be used to implement this.
If the separator has a fullwidth or halfwidth equivalent, that is treated as
matching.
For more on parsing, see
Lenient Parsing
For consistency in the CLDR data, the following conventions should be observed so as to
have a canonical representation:
All number patterns should be minimal: there should be no leading # marks except to
specify the position of the grouping separators (e.g. avoid  ##,##0.###).
All formats should have one 0 before the decimal point (e.g. avoid #,###.##)
Decimal formats should have three hash marks in the fractional position (e.g.
#,##0.###).
Currency formats should have two zeros in the fractional position (e.g. ¤ #,##0.00).
The exact number of decimals is overridden with the decimal count in
supplementary data.
The only time two thousands separators needs to be used is when the number of
digits varies, such as for Hindi: #,##,##0.
G.3 Formatting
Formatting is guided by several parameters, all of which can be specified either using a
pattern or using the API. The following description applies to formats that do not use
scientific notation
or
significant digits
If the number of actual integer digits exceeds the
maximum integer
digits
, then only the least significant digits are shown. For example, 1997 is
formatted as "97" if the maximum integer digits is set to 2.
If the number of actual integer digits is less than the
minimum
integer digits
, then leading zeros are added. For example, 1997 is formatted
as "01997" if the minimum integer digits is set to 5.
If the number of actual fraction digits exceeds the
maximum fraction
digits
, then half-even rounding it performed to the maximum fraction digits.
For example, 0.125 is formatted as "0.12" if the maximum fraction digits is 2. This behavior can
be changed by specifying a rounding increment and a rounding mode.
If the number of actual fraction digits is less than the
minimum
fraction digits
, then trailing zeros are added. For example, 0.125 is
formatted as "0.1250" if the mimimum fraction digits is set to 4.
Trailing fractional zeros are not displayed if they occur
positions after the decimal, where
is less than the maximum
fraction digits. For example, 0.10004 is formatted as "0.1" if the maximum fraction digits is
four or less.
Special Values
NaN
is represented as a single character, typically
(\uFFFD)
. This character is
determined by the localized number symbols. This is the only value for which the prefixes and
suffixes are not used.
Infinity is represented as a single character, typically
(\u221E)
, with the positive or negative prefixes and
suffixes applied. The infinity character is determined by the localized number symbols.
G.4
Scientific Notation
Numbers in scientific notation are expressed as the product of a mantissa and a power of
ten, for example, 1234 can be expressed as 1.234 x 10
. The
mantissa is typically in the half-open interval [1.0, 10.0) or sometimes [0.0, 1.0), but it need
not be. In a pattern, the exponent character immediately followed by one or more digit characters
indicates scientific notation. Example: "0.###E0" formats the number 1234 as "1.234E3".
The number of digit characters after the exponent character gives the minimum exponent
digit count. There is no maximum. Negative exponents are formatted using the localized minus
sign,
not
the prefix and suffix from the pattern. This allows
patterns such as "0.###E0 m/s". To prefix positive exponents with a localized plus sign, specify
'+' between the exponent and the digits: "0.###E+0" will produce formats "1E+1", "1E+0", "1E-1",
etc. (In localized patterns, use the localized plus sign rather than '+'.)
The minimum number of integer digits is achieved by adjusting the exponent. Example:
0.00123 formatted with "00.###E0" yields "12.3E-4". This only happens if there is no maximum
number of integer digits. If there is a maximum, then the minimum number of integer digits is
fixed at one.
The maximum number of integer digits, if present, specifies the exponent grouping. The
most common use of this is to generate
engineering notation
in which the exponent is a multiple of three, e.g., "##0.###E0". The number 12345 is formatted
using "##0.####E0" as "12.345E3".
When using scientific notation, the formatter controls the digit counts using
significant digits logic. The maximum number of significant digits limits the total number of
integer and fraction digits that will be shown in the mantissa; it does not affect parsing. For
example, 12345 formatted with "##0.##E0" is "12.3E3". See the section on significant digits for
more details.
Exponential patterns may not contain grouping separators.
G.5
Significant Digits
There are two ways of controlling how many digits are shows: (a) significant digits
counts, or (b) integer and fraction digit counts. Integer and fraction digit counts are described
above. When a formatter is using significant digits counts, the number of integer and fraction
digits is not specified directly, and the formatter settings for these counts are ignored.
Instead, the formatter uses however many integer and fraction digits are required to display the
specified number of significant digits. Examples:
Pattern
Minimum significant digits
Maximum significant digits
Number
Output
@@@
12345
12300
@@@
0.12345
0.123
@@##
3.14159
3.142
@@##
1.23004
1.23
In order to enable significant digits formatting, use a pattern containing the
'@'
pattern character. In order to disable significant digits
formatting, use a pattern that does not contain the
'@'
pattern character.
Significant digit counts may be expressed using patterns that specify a minimum and
maximum number of significant digits. These are indicated by the
'@'
and
'#'
characters. The minimum number of significant
digits is the number of
'@'
characters. The maximum
number of significant digits is the number of
'@'
characters plus the number of
'#'
characters following on
the right. For example, the pattern
"@@@"
indicates
exactly 3 significant digits. The pattern
"@##"
indicates
from 1 to 3 significant digits. Trailing zero digits to the right of the decimal separator are
suppressed after the minimum number of significant digits have been shown. For example, the
pattern
"@##"
formats the number 0.1203 as
"0.12"
If a pattern uses significant digits, it may not contain a decimal separator, nor the
'0'
pattern character. Patterns such as
"@00"
or
"@.###"
are
disallowed.
Any number of
'#'
characters may be prepended to the left of the leftmost
'@'
character. These have no effect on the minimum and maximum significant digits counts, but may be
used to position grouping separators. For example,
"#,#@#"
indicates a minimum of one significant digits, a maximum of two significant digits, and a
grouping size of three.
The number of significant digits has no effect on parsing.
Significant digits may be used together with exponential notation. Such patterns are
equivalent to a normal exponential pattern with a minimum and maximum integer digit count of
one, a minimum fraction digit count of
Minimum Significant Digits - 1
and a maximum fraction digit count of
Maximum Significant Digits - 1
For example, the pattern
"@@###E0"
is equivalent to
"0.0###E0"
G.6 Padding
Patterns support padding the result to a specific width. In a pattern the pad escape
character, followed by a single pad character, causes padding to be parsed and formatted. The pad
escape character is '*'. For example,
"$*x#,##0.00"
formats
123 to
"$xx123.00"
, and 1234 to
"$1,234.00"
When padding is in effect, the width of the positive subpattern,
including prefix and suffix, determines the format width. For example, in the pattern
"* #0 o''clock"
, the format width is 10.
Some parameters which usually do not matter have meaning when padding is used, because
the pattern width is significant with padding. In the pattern "*
##,##,#,##0.##",
the format width is 14. The initial characters "##,##," do not affect the grouping size or
maximum integer digits, but they do affect the format width.
Padding may be inserted at one of four locations: before the prefix, after the prefix,
before the suffix, or after the suffix. No padding can be specified in any other location. If
there is no prefix, before the prefix and after the prefix are equivalent, likewise for the
suffix.
When specified in a pattern, the code point immediately following the pad escape is
the pad character. This may be any character, including a special pattern character. That is,
the pad escape
escapes
the following character. If there is
no character after the pad escape, then the pattern is illegal.
Rounding
Patterns support rounding to a specific increment. For example, 1230 rounded to the
nearest 50 is 1250. Mathematically, rounding to specific increments is performed by multiplying by
the increment, rounding to an integer, then dividing by the increment. To take a more bizarre
example, 1.234 rounded to the nearest 0.65 is 1.3, as follows:
Original:
1.234
Divide by increment (0.65):
1.89846...
Round:
Multiply by increment (0.65):
1.3
To specify a rounding increment in a pattern, include the increment in the pattern
itself. "#,#50" specifies a rounding increment of 50. "#,##0.05" specifies a rounding increment of
0.05.
Rounding only affects the string produced by formatting. It does not affect parsing or
change any numerical values.
An implementation may allow the specification of a
rounding mode
to determine how values are rounded. In the absence of such choices, the default is to round
"half-even", as described in IEEE arithmetic. That is, it rounds towards the "nearest neighbor"
unless both neighbors are equidistant, in which case, it rounds towards the even neighbor.
Behaves as for round "half-up" if the digit to the left of the discarded fraction is odd;
behaves as for round "half-down" if it's even. Note that this is the rounding mode that
minimizes cumulative error when applied repeatedly over a sequence of calculations.
Some locales use rounding in their currency formats to reflect the smallest currency
denomination.
In a pattern, digits '1' through '9' specify rounding, but otherwise behave
identically to digit '0'.
decimalFormats
The normal locale specific way to write a base 10 number.
currencyFormats
Use \u00A4 where the local currency symbol should be. Doubling the currency symbol
(\u00A4\u00A4) will output the international currency symbol (a 3-letter code).
percentFormats
Pattern for use with percentage formatting
scientificFormats
Pattern for use with scientific (exponent) formatting.
G.7 Quoting rules
Single quotes, (
), enclose bits of the pattern that
should be treated literally. Inside a quoted string, two single quotes ('') are replaced with a
single one ('). For example:
'X '
' Q '
->
X 1939 Q
(Literal strings
underlined
.)
G.8
Number Elements
Localized symbols used in number formatting and parsing.
decimal
- separates the integer and fractional part of the number.
group
- groups (for example) units of thousands: 10
= 1,000,000. The grouping
separator is commonly used for thousands, but in some countries for ten-thousands. The interval
is a constant number of digits between the grouping characters, such as 100,000,000 or
1,0000,0000. If you supply a pattern with multiple grouping characters, the interval between the
last one and the end of the integer is the one that is used. So "#,##,###,####" == "######,####"
== "##,####,####".
list
- separates lists of numbers
percentSign
- symbol used to indicate a percentage (1/100th) amount. (If present, the value is
also multiplied by 100 before formatting. That way 1.23 → 123%)
nativeZeroDigit
- Symbol used to indicate a digit in the pattern, or zero if that place would
otherwise be empty. For example, with the digit of '0', the pattern "000" would format "34" as
"034", but the pattern "0" would format "34" as just "34". As well, the digits 1-9 are expected
to follow the code point of this specified 0 value.
patternDigit
- Symbol used to indicate any digit value, typically #. When that digit is zero, then
it is not shown.
minusSign
- Symbol used to denote negative value.
plusSign
- Symbol used to denote negative value.
exponential
- Symbol separating the mantissa and exponent values.
perMille
- symbol used to indicate a per-mille (1/1000th) amount. (If present, the value is
also multiplied by 1000 before formatting. That way 1.23 → 1230 [1/000])
infinity
- The infinity sign. Corresponds to the IEEE infinity bit pattern.
nan - Not a number
- The NaN sign. Corresponds to the IEEE NaN
bit pattern.
currencySeparator
This is used as the decimal separator in currency formatting/parsing, instead of the
DecimalSeparator from the
NumberElements
list. This item is optional in the CLDR.
currencyGroup
This is used as the grouping separator in currency formatting/parsing, instead of the
DecimalSeparator from the
NumberElements
list. This item is optional in the CLDR.
Appendix H:
Choice Patterns
A choice pattern is a string that chooses among a number of strings, based on numeric
value. It has the following form:
= ( '|' )*
=
= ('+' | '-')? (
'∞' | [0-9]+ ('.' [0-9]+)?)
= '<' | '
≤'
The interpretation of a choice pattern is that given a number N, the pattern is scanned
from right to left, for each choice evaluating N. The first choice that
matches results in the corresponding string. If no match is found, then the first string is used.
For example:
Pattern
Result
0≤Rf|1≤Ru|1∞,
-3, -1, -0.000001
Rf (defaulted to first string)
0, 0.01, 0.9999
Rf
Ru
1.00001, 5, 99,
Re
Quoting is done using ' characters, as in date or number formats.
Appendix I:
Inheritance and Validity
The following describes in more detail how to determine the exact inheritance of
elements, and the validity of a given element in LDML.
I.1 Definitions
Attributes that serve to distinguish multiple elements at the same level are called
distinguishing
attributes. These currently consist of the following:
key
request
id
registry
alt
iso4217
iso3166
type
deprecatedItems
Note:
the
type
attribute on
the following elements was not distinguishing:
abbreviationFallback
default
mapping
measurementSystem
preferenceOrdering
That usage of the
type
attribute has
been deprecated in favor of the
choice
attribute.
Blocking
elements are those whose subelements do not inherit from parent locales.
For example, a element is a blocking element: everything in a element is
treated as a single lump of data, as far as inheritance is concerned.
Certain elements are called
attribute-information
elements. They do not have
element content; their information is carried in their attribute values. This is unlike the other
elements, whose attributes are used to distinguish different types of data.
A list of blocking and
attribute-information
elements is found in
Appendix K: Valid Attribute Values
For any element in an XML file,
an element chain
is a
resolved XPath leading from the root to an element, with attributes on
each element in alphabetical order. So in, say,
we may have:








Αραβικά
...
Which gives the following element chains (among others):
//ldml[@version="1.1"]/identity/version[@number="1.1"]
//ldml[@version="1.1"]/localeDisplayNames/languages/language[@type="ar"]
An element chain A is an
extension
of an element chain B
if B is equivalent to an initial portion of A. For example, #2 below is an extension of #1.
(Equivalent, depending on the tree, may not be "identical to". See below for an example.)
//ldml[@version="1.1"]/localeDisplayNames
//ldml[@version="1.1"]/localeDisplayNames/languages/language[@type="ar"]
An LDML file can be thought of as an ordered list of
element pairs
: , where the element chains are all the
chains for the end-nodes. (This works because of restrictions on the structure of
LDML, including that it doesn't allow mixed content.) The ordering is the ordering
that the element chains are found in the file, and thus determined by the DTD.
For example, some of those pairs would be the following. Notice that the first has the
null string as element contents.
//ldml[@version="1.1"]/identity/version[@number="1.1"]
""
//ldml[@version="1.1"]/localeDisplayNames/languages/language[@type="ar"]
"Αραβικά"
Note:
There are two exceptions to this:
Blocking nodes and their contents are treated as a single end note.
For attribute-information elements, in terms of computing inheritance, the element
pair consists of the element chain minus the attributes in the final element and the value is
the list of attributes for that final element.
Thus instead of the element pair being (a) below, it is (b):
//ldml[@version="1.1"]/dates/calendars/calendar[@type='gregorian']/week/weekendStart[@day='sun'][@time='00:00']
"">
//ldml[@version="1.1"]/dates/calendars/calendar[@type='gregorian']/week/weekendStart
[@day='sun'][@time='00:00']
Two LDML element chains are
equivalent
when they would be identical if all attributes and their values were removed
except for distinguishing attributes. Thus the following are equivalent:
//ldml[@version="1.1"]/localeDisplayNames/languages/language[@type="ar"]
//ldml[@version="1.1"]/localeDisplayNames/languages/language[@type="ar"][@draft="unconfirmed"]
For any locale ID, an
locale chain
is an ordered list
starting with the root and leading down to the ID. For example:

I.2 Resolved Data File
To produce fully resolved locale data file from CLDR for a locale ID L, you start with
L, and successively add unique items from the parent locales until you
get up to root. More formally, this can be expressed as the following procedure.
Let Result be initially empty.
For each Li in the locale chain for L, starting at L and going up to root:
Let Temp be a copy of the pairs in the LDML file for Li
Replace each alias in Temp by the list of pairs it points to.
That alias now blocks any inheritance from the parent. (See
Section 5.1 Common Elements
for an
example.)
For each element pair P in Temp:
If P does not contain a blocking element, and Result
does not have an element pair Q with an equivalent element chain, add P
to Result.
Note:
when adding an element pair to a result, it has to go in the right order for it to
be valid according to the DTD.
I.3
Valid Data
The attribute
draft="unconfirmed"
or
draft="provisional"
in LDML
means that the data has not been approved for release. However, some data that is not explicitly
marked as
unconfirmed
or
provisional
may be implicitly
unconfirmed
or
provisional
, either because it inherits it from a parent, or from an
enclosing element.
Example 2.
Suppose that new locale data is added for af
(Afrikans). To indicate that all of the data is
unconfirmed
, the attribute
can be added to the top level.






...
...

Any data can be added to that file, and the status will all be draft="unconfirmed". Once an item is
vetted --
whether it is inherited or explicitly in the file
--
then its status can be changed to
approved
. This can be done either by leaving draft="unconfirmed" on
the enclosing element and marking the child with draft="approved", such as:






...
...




However, normally the draft status should be canonicalized, which
means it is pushed down to leaf nodes: see
Appendix L: Canonical Form
Note:
A missing draft attribute is
not
the same as either a true or false value. A missing attribute means instead:
inherit
the draft status from enclosing elements and parent locales.
The attribute
validSubLocales
allows sublocales in a
given tree to be treated as though a file for them were present when there isn't one. It can be
applied to any element. It only has an effect for locales that inherit from the current file where
a file is missing, and the elements wouldn't otherwise be draft.
Example 1.
Suppose that in a particular LDML
tree, there are no region locales for German, e.g. there is a de.xml
file, but no files for de_AT.xml, de_CH.xml, or de_DE.xml.
Then no elements are valid for any of those region locales. If we want to mark one of those files
as having valid elements, then we introduce an empty file, such as the following.








With the
validSubLocales
attribute, instead of adding
the empty files for de_AT.xml, de_CH.xml, and de_DE.xml, in
the de file we can add to the parent locale a list of the child locales that should behave as if
files were present.






...

More formally, here is how to determine whether data for an element chain E is implicitly
or explicitly draft, given a locale L. Sections 1, 2, and 4 are simply formalizations of what is
in LDML already. Item 3 adds the new element.
I.4 Checking for Draft Status:
Parent Locale Inheritance
Walk through the locale chain until you find a locale ID L' with a data file D. (L'
may equal L).
Produce the fully resolved data file D' for D.
In D', find the first element pair whose element chain E' is either equivalent to or
an extension of E.
If there is no such E', return
true
If E' is not equivalent to E, truncate E' to the length of E.
Enclosing Element Inheritance
Walk through the elements in E', from back to front.
If you ever encounter draft=
, return
If L' = L, return
false
Missing File Inheritance
Otherwise, walk again through the elements in E', from back to front.
If you encounter a validSubLocales attribute:
If L is in the attribute value, return
false
Otherwise return
true
Otherwise
Return
true
The validSubLocales in the most specific (farthest from root
file) locale file "wins" through the full resolution step (data from more specific files replacing
data from less specific ones).
I.5 Keyword and Default Resolution
When accessing data based on keywords, the following process is used. Consider the
following example:
The locale 'de' has collation types A, B, C, and no element
The locale 'de_CH' has
Here are the searches for various combinations.
1.
de_CH
not found
de
not found
root
not found: so get the default type in de_CH
de@collation=B
found
2.
de
not found
root
not found: so get the default type in de, which itself falls back to root
de@collation=standard
not found
root@collation=standard
found
3.
de@collation=A
found
4.
de@collation=standard
not found
root@collation=standard
found
Note:
It is an invariant that the default in root for a given element must
always be a value that exists in root. So you can't have the following in root:


...
...


For identifiers, such as language codes, script codes, region codes, variant codes,
types, keywords, currency symbols or currency display names, the default value is the identifier
itself whenever if no value is found in the root. Thus if there is no display name for the region
code 'QA' in root, then the display name is simply 'QA'.
Appendix J:
Time Zone Display
Names
There are three types of formats for zone identifiers: GMT, generic
(wall time), and standard/daylight. Standard and daylight are equivalent to a particular offset
from GMT, and can be represented by a GMT offset as a fallback. In general, this is not true for
the generic format, which is used for picking
timezones or for conveying a timezone for specifying a recurring time (such as a meeting in a
calendar). For either purpose, a GMT offset would lose information.
When a timezone is to be displayed, the following process is used. It
uses explicit display names where they are available, and otherwise uses a fallback to GMT for
non-wall time (standard and daylight). For generic, it falls back to the exemplar city if
available, otherwise the country if possible, and otherwise the last field of the zone ID. Only the generic time (or its fallback) should be used in menus, in order to avoid possible collisions in the display names of standard and daylight time.
Each step is followed until a "return" is reached. Some of the
examples are drawn from real data, while for illustration the region format is "Tampo de {0}". The
fallback format is "{0} ({1})", which is what is in root.
Canonicalize the
TZ
ID according to the table in supplemental data. Use that canonical TZID in each
of the following steps.
America/Atka → America/Adak
Australia/ACT → Australia/Sydney
For RFC 822 format ("Z") return the results according to the RFC.
America/Los_Angeles → "-0800"
Note:
The digits in this case are always from the western digits,
0..9.
If there is an explicit translation for the TZID according to type
(generic, standard, or daylight) in the resolved locale, return it.
America/Los_Angeles → "Heure du Pacifique (ÉUA)" // generic
America/Los_Angeles → 太平洋標準時 // standard
America/Los_Angeles → Yhdysvaltain Tyynenmeren kesäaika //
daylight
Note:
This translation may not at all be literal: it would
be what is most recognizable for people using the target language.
For non-wall-time (ie, GMT, daylight, or standard) or where there
is no country for the TZID (eg, Etc/GMT+3), use the localized GMT format.
America/Los_Angeles → "GMT-08:00" //  standard time
America/Los_Angeles → "HMG-07:00" //  daylight time
Etc/GMT+3 → "GMT-03.00" // note that
TZ
tzids have inverse polarity!
Note:
The digits should be whatever are appropriate for the locale
used to format the time zone, not necessarily from the western digits, 0..9.
For example, they might be from ०..९.
Thus the remaining steps are only applicable to the generic
format. In these steps, use as the country name the an explicitly localized country if
available, otherwise the raw country code. If the localized exemplar city is not available, use
as the exemplar city the last field of the raw TZID, stripping off the prefix and turning _ into
space.
CU → "CU" // no localized country name for Cuba
America/Los_Angeles → "Los Angeles" // no localized exemplar
city
From get the country code for the zone, and
determine whether there is only one timezone in the country. If there is only one timezone or
the zone id is in the singleCountries list, format the country name with the region format, and
return it.
Africa/Monrovia → LR → "Tampo de Liberja"
America/Havana → CU → "Tampo de CU" // if CU is not localized
Note:
If a language does require grammatical changes when
composing strings, then it should either use a neutral format such as what is in root, or put
all exceptional cases in explicitly translated strings.
Get the exemplar city and country name, and format them with the
fallback format (as parameters 0 and 1, respectively).
America/Buenos_Aires → "Буэнос-Айрес (Аргентина)"
America/Buenos_Aires → "Буэнос-Айрес (AR)" // if Argentina isn't
translated
America/Buenos_Aires → "Buenos Aires (Аргентина)" // if Buenos
Aires isn't
America/Buenos_Aires → "Buenos Aires (AR)" // if both aren't
Note:
As with the region format, exceptional cases need to
be explicitly translated.
In parsing, an implementation will be able to either determine the
zone id, or a simple offset from GMT for anything formatting according to the above process. The
following process should be used, stopping in the first step that matches.
Check for explicitly localized strings.
"Tampo de Pacifica" → America/Los_Angeles
2. Check for RFC 822 and localized GMT formats
"-0800" → Etc/GMT+8
"GMT-03:00" → Etc/GMT+3
Check for using the fallback format. Remember to
check for fallback localizations (last field of zone id and the raw country code).
“Sydney (Australia)” → Australia/Sydney
Check for localized using the region format. Remember to
check for fallback localizations (raw country code).
"Tampo de CU" → America/Havana
Using this process, a correct parse will roundtrip the generic format
(v and vvvv) back to the canonical zoneid.
Australia/ACT → Australia/Sydney → “Sydney (Australia)” →
Australia/Sydney
The GMT formats (Z and ZZZZ) will return back an offset, and thus
lose the original canonical zone id.
Australia/ACT → Australia/Sydney → "GMT+11:00" → GMT+11
The daylight and standard time formats (z and zzzz) may either
roundtrip back to the original canonical zone id, or to just an offset, depending on the available
translation data. Thus:
Australia/ACT → Australia/Sydney → "GMT+11:00" → GMT+11
PST8PDT → America/Los_Angeles → “PST” → America/Los_Angeles
Parsing can be more lenient than the above, allowing for different
spacing, punctuation, or other variation.
Many time zone IDs only represent differences that are important historically, but do not make
any difference in modern times. The preferenceOrdering element can be used to select the preferred
modern IDs when desired, either in presenting a list of localized
timezone names in a user interface, or in formatting. (The choice of the period to use
as "modern" when determining when two time zone IDs are
equivalent is left to the implementation.)
Whenever two timezone IDs are equivalent in effect and are in the same country, the preference
ordering list is examined according to the following process. When used
in formatting, this process is used to add additional canonicalization in Step 1 above.
If x, y are in the list, then the earlier one in the list is preferred.
Else if x is in the list and y isn't, then x is preferred
Else if not in root, repeat #1 and #2 using the parent locale's list
If all else fails, use a case-insensitive comparison of the timezone IDs.
For example, the following table lists the modern equivalents for Mexico on separate rows. If
the preference ordering has one element: "America/Mexico_City", then the bolded items would be
chosen as the preferred timezone IDs.
America/Merida,
America/Mexico_City,
America/Monterrey, America/Cancun
America/Chihuahua,
America/Mazatlan
America/Hermosillo
America/Tijuana
Note:
The hoursFormat and abbreviationFallback used in earlier
versions of this appendix are deprecated.
Appendix K: Valid Attribute Values
The valid attribute values, as well as other validity information is
contained in the metadata.xml file. (Some, but not all, of this information could have been
represented in XML Schema or a DTD.)
The following specify the ordering of elements / attributes in the file
ldml identity alias localeDisplayNames layout ...
type key registry alt source path day date...
The suppress elements are those that are
suppressed in canonicalization.
The serialElements are those that do not inherit,
and may have ordering
variable comment
tRule reset p pc s sc t tc q qc i ic x extend first_variable last_variable
first_tertiary_ignorable last_tertiary_ignorable first_secondary_ignorable
last_secondary_ignorable first_primary_ignorable last_primary_ignorable
first_non_ignorable last_non_ignorable first_trailing last_trailing
The validity elements give the possible attribute values.
They are in the format of a series of variables, followed by attributeValues.

buddhist coptic ethiopic chinese gregorian hebrew islamic islamic-civil
japanese arabic civil-arabic thai-buddhist persian

The types indicate the style of match:
choice: for a list of possible values
regex: for a regular expression match
notDoneYet: for items without matching
criteria
locale: for locale IDs
list: for a space-delimited list of values
path: for a valid XPath
If the attribute order="given" is supplied, it
indicates the order of elements when canonicalizing (see below).
The element lists elements,
attributes, and attribute values that are deprecated. If any deprecatedItems
element contains more than one attribute, then only the listed combinations
are deprecated. Thus the following means not that the draft attribute is
deprecated, but that the true and false values for that attribute are:

Similarly, the following means that the
type
attribute is deprecated, but only for the listed elements:

Appendix L:
Canonical Form
The following are restrictions on the format of LDML files to allow for easier parsing
and comparison of files.
Peer elements have consistent order. That is, if the DTD or this specification requires the
following order in an element foo:




It can never require the reverse order in a different element bar.




Note that there was one case that had to be corrected in order to make this true. For
that reason, pattern occurs twice under currency:
decimal?, group?, special*)) >
XML
files can have a wide variation in
textual form, while representing precisely the same data. By putting the LDML files in the
repository into a canonical form, this allows us to use the simple diff tools used widely (and in
CVS) to detect differences when vetting changes, without those tools being confused. This is not a
requirement on other uses of LDML; just simply a way to manage repository data more easily.
L.1 Content
All start elements are on their own line, indented by
depth
tabs.
All end elements (except for leaf nodes) are on their own line, indented by
depth
tabs.
Any leaf node with empty content is in the form .
There are no blank lines except within comments or content.
Spaces are used within a start element. There are no extra spaces within elements.

, not


, not

All attribute values use double quote ("), not single (').
There are no CDATA sections, and no escapes except those absolutely required.
no ' since it is not necessary
no 'a', it would be just 'a'
All attributes with defaulted values are suppressed. See the
Defaulted Attributes Table
XXX
The draft and alt="proposed.*" attributes are only on leaf elements.
The tzid are canonicalized in the following way:
All tzids as of as CLDR 1.1 (2004.06.08) in zone.tab are canonical.
After that point, the first time a tzid is introduced, that is the canonical form.
That is, new IDs are added, but existing ones keep the original form. The
TZ
timezone database keeps a set of equivalences in
the "backward" file. These are used to map other tzids to the canonical form. For example, when
America/Argentina/Catamarca
was introduced as the new name for the previous
America/Catamarca
, a link was added in the backward file.
Link America/Argentina/Catamarca America/Catamarca
Example:











¤#,##0.00;(¤#,##0.00)





L.2 Ordering
Element names are ordered by the
Element Order Table
Attribute names are ordered by the
Attribute Order Table
Attribute value comparison is a bit more complicated, and may depend on the attribute
and type. Compare two values by using the following steps:
If two values are in the
Value Order Table
, compare according to the order in the table. Otherwise if just one is,
it goes first.
If two values are numeric [0-9], compare numerically (2 < 12). Otherwise if just one
is numeric, it goes first.
Otherwise values are ordered alphabetically
An attribute-value pair is ordered first by attribute name, and then if the attribute
names are identical, by the value.
An element is ordered first by the element name, and then if the element names are
identical, by the sorted set of attribute-value pairs (sorted by #4). For the latter, compare
the first pair in each (in sorted order by attribute pair). If not identical, go to the second
pair, etc.
Any future additions to the DTD must be structured so as to allow compatibility with
this ordering.
See also Appendix K:
Valid Attribute Values
L.3 Comments
Comments are of the form .
They are logically attached to a node. There are 4 kinds:
Inline always appear after a leaf node, on the same line at the end. These are a
single line.
Preblock comments always precede the attachment node, and are indented on the same
level.
Postblock comments always follow the attachment node, and are indented on the same
level.
Final comment, after
Multiline comments (except the final comment) have each line after the first indented
to one deeper level.
Examples:

BC
...



...


L.4 Canonicalization
The process of canonicalization is fairly straightforward, except for comments. Inline
comments will have any linebreaks replaced by a space. There may be cases where the attachment
node is not permitted, such as the following.




In those cases, the comment will be made into a block comment on the last previous leaf
node, if it is at that level or deeper. (If there is one already, it will be appended, with a
line-break between.) If there is no place to attach the node (for example, as a result of
processing that removes the attachment node), the comment and its node's xpath will be appended to
the final comment in the document.
Multiline comments will have leading tabs stripped, so any indentation should be done
with spaces.
L.5
Element Order Table
The order of attributes is given by the elementOrder
table in the supplemental metadata.
L.6
Attribute Order Table
The order of attributes is given by the
attributeOrder table in the supplemental metadata.
L.7
Value Order Table
The order of attribute values is given by the order
of the values in the attributeValues elements that have the attibute
order="given". Numeric values are sorted in numeric order, while tzids are ordered by country, then
longitude, then latitude.
L.8
Defaulted Values Table
The defaulted attributes are given by the
suppress
table in the supplemental metadata. There is one special value
_q; that is used on serial elements internally to preserve ordering.
Appendix M: Coverage Levels
The following defines the coverage levels:
Level
Description
level 100
comprehensive
Has complete localizations (or valid inheritance) for every
possible field
level 80
modern
Localizations (or valid inheritance) as given below
level 60
moderate
level 40
basic
level 20
posix
Only what is required for POSIX generation; example, only one
country name, only one currency symbol, etc.
level 0
rudimentary
Doesn't meet any of the above levels. (default, if nothing
specified)
Levels 40 and 60 are based on the following definitions and
specifications.
M.1 Definitions
Target-Language
is the language under consideration.
Target-Territories
is the list of territories found by
looking up
Target-Language
in the elements in
supplementalData.xml
Language-List
is
Target-Language
, plus
basic:
Chinese, English, French, German, Italian,
Japanese, Portuguese, Russian, Spanish (de, en, es, fr, it, ja, pt, ru, zh)
moderate:
basic + Arabic, Hindi, Korean, Indonesian,
Dutch, Bengali, Turkish, Thai, Polish (ar, hi, ko, in, nl, bn, tr, th, pl). If an EU language,
add the remaining official EU languages, currently: Danish, Greek, Finnish, Swedish, Czech,
Estonian, Latvian, Lithuanian, Hungarian, Maltese, Slovak, Slovene (da, el, fi, sv, cs, et, lv,
lt, hu, mt, sk, sl)
modern:
all languages that are official or major
commercial languages of modern territories
Target-Scripts
is the list of scripts in which
Target-Language
can be customarily written (found by looking up
Target-Language
in
the elements in
supplementalData.xml
Script-List
is the
Target-Scripts
plus the major
scripts used for multiple languages
Latin, Simplified Chinese, Traditional Chinese, Cyrillic, Arabic
(Latn, Hans, Hant, Cyrl, Arab)
Territory-List
is the list of territories formed by taking
the
Target-Territories
and adding:
basic:
Brazil, China, France, Germany, India, Italy,
Japan, Russia, United Kingdom, United States (BR, CN, DE, GB, FR, IN, IT, JP, RU, US)
moderate:
basic + Spain, Canada, Korea, Mexico,
Australia, Netherlands, Switzerland, Belgium, Sweden, Turkey, Austria, Indonesia, Saudi
Arabia, Norway, Denmark, Poland, South Africa, Greece, Finland, Ireland, Portugal, Thailand,
Hong Kong SAR China, Taiwan (ES, BE, SE, TR, AT, ID, SA, NO, DK, PL, ZA, GR, FI, IE, PT, TH,
HK, TW). If an EU language, add the remaining member EU
countries: Luxembourg, Czech Republic, Hungary, Estonia, Lithuania, Latvia, Slovenia,
Slovakia, Malta (LU, CZ, HU, ES, LT, LV, SI, SK, MT).
modern:
all current ISO 3166 territories, plus the UN M.49 regions in
supplementalData.xml
Currency-List
is the list of current official currencies
used in any of the territories in
Territory-List
, found by looking at the region elements
in
supplementalData.xml
Calendar-List
is the set of calendars in customary use in
any of
Target-Territories
, plus Gregorian.
Timezone-List
is the set of all timezones for multi-zone
territories in
Target-Territories
, plus each of following timezones whose territories is
in
Territory-List
Argentina:
America/Buenos_Aires
Brazil:
America/Rio_Branco,
America/Campo_Grande, America/Sao_Paulo
Australia:
Australia/Perth, Australia/Darwin,
Australia/Brisbane, Australia/Adelaide, Australia/Sydney, Australia/Hobart
Canada:
America/Vancouver, America/Edmonton,
America/Regina, America/Winnipeg, America/Toronto, America/Halifax, America/St_Johns
Mexico:
America/Tijuana, America/Hermosillo,
America/Chihuahua, America/Mexico_City
US:
Pacific/Honolulu, America/Anchorage, America/Los_Angeles,
America/Phoenix, America/Denver, America/Chicago, America/Indianapolis, America/New_York
Machine-readable
information for this is contained in the element in the
supplemental metadata file.
M.2 Data Requirements
The required data to qualify for the level is then the following.
identity
localeDisplayNames
languages:
localized names for all languages in
Language-List.
scripts:
localized names for all scripts in
Script-List
territories:
localized names for all territories in
Territory-List
variants, keys, types:
localized names for any in use in
Target-Territories
; e.g. a translation for PHONEBOOK in a German locale.
layout, orientation
exemplarCharacters
measurementSystem, paperSize
dates: all of the following for each calendar in
Calendar-List
calendars: localized names
monthNames & dayNames
context=format and width=narrow, wide, & abbreviated
plus context=standAlone and width=narrow, wide, & abbreviated,
if the grammatical forms of these are different than for context=format.
week: minDays, firstDay, weekendStart, weekendEnd
if some of these vary in territories in
Territory-List
include territory locales for those that do.
am, pm, eraNames, eraAbbr
dateFormat, timeFormat: full, long, medium, short
timeZoneNames:
exemplar cities the timezones in
Timezone-List
hourFormat, hoursFormat, gmtFormat, regionFormat, fallbackFormat
numbers: symbols, decimalFormats, scientificFormats,
percentFormats, currencyFormats
currencies: displayName and symbol for all currencies in
Currency-List
collation sequence
yesstr, nostr
transforms:
basic:
none
moderate
transliteration between Latin and each other script in
Target-Scripts.
M.3 Default Values
Items should
only
be included if they are not the same as the
default, which is:
what is in root, if there is something defined there.
for timezone IDs: the name computed according to
Appendix J: Time Zone Display Names
for collation sequence, the UCA DUCET (Default Unicode Collation
Element Table)
however, in that case the locale must be added to the
validSubLocale list in
collation/root.xml
for currency symbol, language, territory, script names, variants,
keys, types, the internal code identifiers, eg,
currencies: EUR, USD, JPY, ...
languages: en, ja, ru, ...
territories: GB, JP, FR, ...
scripts: Latn, Thai, ...
variants: PHONEBOOK,...
Appendix N:
Transform Rules
The transform rules are similar to regular-expression substitutions,
but adapted to the specific domain of text transformations. The rules and
comments in this discussion will be intermixed, with # marking the comments.
In the xml format these in separate elements:
comment
and
tRule
. The simplest rule is a conversion
rule, which replaces one string of characters with another. The conversion rule takes the
following form:
xy → z ;
This converts any substring "xy" into "z". Rules are executed in
order; consider the following rules:
sch → sh ;
ss → z ;
This conversion rule transforms "bass school" into "baz shool". The
transform walks through the string from start to finish. Thus given the rules above "bassch" will
convert to "bazch", because the "ss" rule is found before the "sch" rule in the string (later,
we'll see a way to override this behavior). If two rules can both apply at a given point in the
string, then the transform applies the first rule in the list.
All of the ASCII characters except numbers and letters are reserved
for use in the rule syntax, as are the characters →, ←, ↔. Normally, these characters do not need
to be converted. However, to convert them use either a pair of single quotes or a slash. The pair
of single quotes can be used to surround a whole string of text. The slash affects only the
character immediately after it. For example, to convert from an arrow signs to the word "arrow",
use one of the following rules:
\←   →  arrow\ sign ;
'←'   →   'arrow sign' ;
'←'   →   arrow' 'sign ;
Spaces may be inserted anywhere without any effect on the rules. Use
extra space to separate items out for clarity without worrying about the effects. This feature is
particularly useful with combining marks; it is handy to put some spaces around it to separate it
from the surrounding text. The following is an example:
→ i ; # an iota-subscript diacritic turns into an i.
For a real space in the rules, place quotes around it. For a real
backslash, either double it \\, or quote it '\'. For a real single quote, double it '', or place a
backslash before it \'.
Any text that starts with a hash mark and concludes a line is a
comment. Comments help document how the rules work. The following shows a comment in a rule:
x → ks ; # change every x into ks
The "\u" notation can be used instead of any letter. For instance,
instead of using the Greek π, one could write:
\u03C0 → p ;
One can also define and use variables, such as:
$pi = \u03C0 ;
$pi → p ;
N.1 Dual Rules
Rules can also specify what happens when an inverse transform is
formed. To do this, we reverse the direction of the "←" sign. Thus the above example becomes:
$pi ← p ;
With the inverse transform, "p" will convert to the Greek p. These
two directions can be combined together into a dual conversion rule by using the "↔" operator,
yielding:
$pi ↔ p ;
N.2 Context
Context can be used to have the results of a transformation be
different depending on the characters before or after. The following means "Remove hyphens, but
only when they follow lowercase letters":
[:lowercase letter:] } '-' → '' ;
The context itself ([:lowercase letter:]) is unaffected by the
replacement; only the text between the curly braces is changed.
N.3 Revisiting
If the resulting text contains a vertical bar "|", then that means
that processing will proceed from that point and that the transform will revisit part of the
resulting text. Thus the | marks a "cursor" position. For example, if we have the following, then
the string "xa" will convert to "w".
x → y | z ;
z a → w;
First, "xa" is converted to "yza". Then the processing will continue
from after the character "y", pick up the "za", and convert it. Had we not had the "|", the result
would have been simply "yza". The '@' character can be used as filler character to place the
revisiting point off the start or end of the string. Thus the following causes x to be replaced,
and the cursor to be backed up by two characters.
x → |@@y;
N.4 Example
The following shows how these features are combined together in the
Transliterator "Any-Publishing". This transform converts the ASCII typewriter conventions into
text more suitable for desktop publishing (in English). It turns straight quotation marks or UNIX
style quotation marks into curly quotation marks, fixes multiple spaces, and converts
double-hyphens into a dash.
# Variables
$single = \' ;
$space = ' ' ;
$double = \" ;
$back = \` ;
$tab = '\u0008' ;
# the following is for spaces, line ends, (, [, {, ...
$makeRight = [[:separator:][:start punctuation:][:initial punctuation:]] ;
# fix UNIX quotes
$back $back → “ ; # generate right d.q.m. (double quotation mark)
$back → ‘ ;
# fix typewriter quotes, by context
$makeRight { $double ↔ “ ; # convert a double to right d.q.m. after certain chars
^ { $double → “ ; # convert a double at the start of the line.
$double ↔ ” ; # otherwise convert to a left q.m.
$makeRight {$single} ↔ ‘ ; # do the same for s.q.m.s
^ {$single} → ‘ ;
$single ↔ ’;
# fix multiple spaces and hyphens
$space {$space} → ; # collapse multiple spaces
'--' ↔ — ; # convert fake dash into real one
N.5 Rule Syntax
The following describes the full format of the list of rules used to
create a transform. Each rule in the list is terminated by a semicolon. The list consists of the
following:
an optional filter rule
zero or more transform rules
zero or more variable-definition rules
zer or more conversion rules
an optional inverse filter rule
The filter rule, if present, must appear at the beginning of the
list, before any of the other rules.  The inverse filter rule, if present, must appear at the end
of the list, after all of the other rules.  The other rules may occur in any order and be freely
intermixed.
The rule list can also generate the inverse of the transform. In that
case, the inverse of each of the rules is used, as described below.
N.6 Transform Rules
Each transform rule consists of two colons followed by a transform
name, which is of the form source-target. For example:
:: NFD ;
:: und_Latn-und_Greek ;
:: Latin-Greek; # alternate form
If either the source or target is 'und', it can be omitted, thus
'und_NFC' is equivalent to 'NFC'. For compatibility, the English names for scripts can be used
instead of the und_Latn locale name, and "Any" can be used instead of "und". Case is not
signficant.
The following transforms are defined not by rules, but by the
operations in the Unicode Standard, and may be used in building any other transform:
Any-NFC, Any-NFD, Any-NFKD, Any-NFKC
- the normalization
forms defined by [UAX15]
Any-Lower, Any-Upper, Any-Title
- full case transformations,
defined by [Unicode] Chapter 3.
In addition, the following special cases are defined:
Any-Null
- has no effect; that is, each character is left
alone.
Any-Remove
- maps each character to the empty string; this, removes each character.
The inverse of a transform rule uses parentheses to indicate what
should be done when the inverse transform is used. For example:
:: lower () ; # only executed for the normal
:: (lower) ; # only executed for the inverse
:: lower ; # executed for both the normal and the inverse
N.7 Variable Definition Rules
Each variable definition is of the following form:
$variableName = contents ;
The variable name can contain letters and digits, but must start with
a letter. More precisely, the variable names use Unicode identifiers as defined by [UAX31]. The
identifier properties allow for the use of foreign letters and numbers.
The contents of a variable definition is any sequence of Unicode sets
and characters or characters. For example:
$mac = M [aA] [cC] ;
Variables are only replaced within other variable definition rules
and within conversion rules. They have no effect on transliteration rules.
N.8 Filter Rules
A filter rule consists of two colons followed by a UnicodeSet. This
filter is global in that only the characters matching the filter will be affected by any transform
rules or conversion rules. The inverse filter rule consists of two colons followed by a UnicodeSet
in parentheses. This filter is also global for the inverse transform.
For example, the Hiragana-Latin transform can be implemented by
"pivoting" through the Katakana converter, as follows:
:: [:^Katakana:] ; # don't touch any katakana that was in the text!
:: Hiragana-Katakana;
:: Katakana-Latin;
:: ([:^Katakana:]) ; # don't touch any katakana that was in the text
# for the inverse either!
The filters keep the transform from mistakenly converting any of the
"pivot" characters. Note that this is a case where a rule list contains no conversion rules at
all, just transform rules and filters.
N.9 Conversion Rules
Conversion rules can be forward, backward, or double. The complete
conversion rule syntax is described below:
N.9.1
Forward
A forward conversion rule is of the following form:
before_context { text_to_replace } after_context → completed_result | result_to_revisit ;
If there is no before_context, then the "{" can be omitted. If there
is no after_context, then the "}" can be omitted. If there is no result_to_revisit, then the "|"
can be omitted. A forward conversion rule is only executed for the normal transform and is ignored
when generating the inverse transform.
N.9.2
Backward
A backward conversion rule is of the following form:
completed_result | result_to_revisit ← before_context { text_to_replace } after_context ;
The same omission rules apply as in the case of forward conversion
rules. A backward conversion rule is only executed for the inverse transform and is ignored when
generating the normal transform.
N.9.3
Dual
A dual conversion rule combines a forward conversion rule and a
backward conversion rule into one, as discussed above. It is of the form:
a { b | c } d ↔ e { f | g } h ;
When generating the normal transform and the inverse, the revisit
mark "|" and the before and after contexts are ignored on the sides where they don't belong. Thus,
the above is exactly equivalent to the sequence of the following two rules:
a { b c } d  →  f | g  ;
b | c  ←  e { f g } h ;
N.10 Intermixing Transform Rules and Conversion Rules
Transform rules and conversion rules may be freely intermixed.
Inserting a transform rule into the middle of a set of conversion rules has an important side
effect.
Normally, conversion rules are considered together as a group.  The
only time their order in the rule set is important is when more than one rule matches at the same
point in the string.  In that case, the one that occurs earlier in the rule set wins.  In all
other situations, when multiple rules match overlapping parts of the string, the one that matches
earlier wins.
Transform rules apply to the whole string.  If you have several
transform rules in a row, the first one is applied to the whole string, then the second one is
applied to the whole string, and so on.  To reconcile this behavior with the behavior of
conversion rules, transform rules have the side effect of breaking a surrounding set of conversion
rules into two groups: First all of the conversion rules before the transform rule are applied as
a group to the whole string in the usual way, then the transform rule is applied to the whole
string, and then the conversion rules after the transform rule are applied as a group to the whole
string.  For example, consider the following rules:
abc → xyz;
xyz → def;
::Upper;
If you apply these rules to “abcxyz”, you get “XYZDEF”.  If you move
the “::Upper;” to the middle of the rule set and change the cases accordingly, then applying this
to “abcxyz” produces “DEFDEF”.
abc → xyz;
::Upper;
XYZ → DEF;
This is because “::Upper;” causes the transliterator to reset to the
beginning of the string. The first rule turns the string into “xyzxyz”, the second rule uppercases
the whole thing to “XYZXYZ”, and the third rule turns this into “DEFDEF”.
This can be useful when a transform naturally occurs in multiple
“passes.”  Consider this rule set:
[:Separator:]* → ' ';
'high school' → 'H.S.';
'middle school' → 'M.S.';
'elementary school' → 'E.S.';
If you apply this rule to “high school”, you get “H.S.”, but if you
apply it to “high  school” (with two spaces), you just get “high school” (with one space).  To
have “high  school” (with two spaces) turn into “H.S.”, you'd either have to have the first rule
back up some arbitrary distance (far enough to see “elementary”, if you want all the rules to
work), or you have to include the whole left-hand side of the first rule in the other rules, which
can make them hard to read and maintain:
$space = [:Separator:]*;
high $space school → 'H.S.';
middle $space school → 'M.S.';
elementary $space school → 'E.S.';
Instead, you can simply insert “
::Null;
” in order to get things to
work right:
[:Separator:]* → ' ';
::Null;
'high school' → 'H.S.';
'middle school' → 'M.S.';
'elementary school' → 'E.S.';
The “::Null;” has no effect of its own (the null transform, by
definition, doesn't do anything), but it splits the other rules into two “passes”: The first rule
is applied to the whole string, normalizing all runs of white space into single spaces, and then
we start over at the beginning of the string to look for the phrases.  “high    school” (with four
spaces) gets correctly converted to “H.S.”.
This can also sometimes be useful with rules that have overlapping
domains.  Consider this rule set from before:
sch → sh ;
ss → z ;
Apply this rule to “bassch” results in “bazch” because “ss” matches
earlier in the string than “sch”.  If you really wanted “bassh”-- that is, if you wanted the first
rule to win even when the second rule matches earlier in the string, you'd either have to add
another rule for this special case...
sch → sh ;
ssch → ssh;
ss → z ;
...or you could use a transform rule to apply the conversions in two
passes:
sch → sh ;
::Null;
ss → z ;
N.11 Inverse Summary
The following table shows how the same rule list generates two
different transforms, where the inverse is restated in terms of forward rules (this is a contrived
example, simply to show the reordering):
Original Rules
Forward
Inverse
:: [:Uppercase Letter:] ;
:: latin-greek ;
:: greek-japanese ;
x ↔ y ;
z → w ;
r ← m ;
:: upper;
a → b ;
c ↔ d ;
:: any-publishing ;
:: ([:Number:]) ;
:: [:Uppercase Letter:] ;
:: latin-greek ;
:: greek-japanese ;
x → y ;
z → w ;
:: upper ;
a → b ;
c → d ;
:: any-publishing ;
:: [:Number:] ;
:: publishing-any ;
d → c ;
:: lower ;
y → x ;
m → r ;
:: japanese-greek ;
:: greek-latin ;
Note how the irrelevant rules (the inverse filter rule and the rules
containing ←) are omitted (ignored, actually) in the forward direction, and notice how things are
reversed: the transform rules are inverted and happen in the opposite order, and the groups of
conversion rules are also executed in the opposite relative order (although the rules within each
group are executed in the same order).
Appendix O: Lenient Parsing
O.1 Motivation
User input is frequently messy. Attempting to parse it by matching it exactly against a pattern is likely to be unsuccessful, even when the meaning of the input is clear to a human being. For example, for a date pattern of "MM/dd/yy", the input "June 1, 2006" will fail.
The goal of lenient parsing is to accept user input whenever it is possible to decipher what the user intended. Doing so requires using patterns as data to guide the parsing process, rather than an exact template that must be matched. This informative section suggests some heuristics that may be useful for lenient parsing of dates, times, and numbers.
O.2 Loose Matching
Loose matching ignores attributes of the strings being compared that are not important to matching. It involves the following steps:
Remove "." from currency symbols and other fields used for matching, and also from the input string unless:
"." is in the decimal set, and
its position in the input string is immediately before a decimal digit
Ignore all characters in [:Zs:] unless they occur between letters. (in the heuristics below, even those between letters are ignored except to delimit fields)
Map all characters in [:Dash:] to U+002D HYPHEN-MINUS
Use the data in to map equivalent characters (e.g. ” → ")
Apply mappings particular to the domain (i.e., for dates or for numbers, discussed in more detail below)
Apply case folding (possibly including language-specific mappings such as Turkish i)
Normalize to NFKC
Loose matching involves (logically) applying the above transform to both the input text and to each of the field elements used in matching, before applying the specific heuristics below. For example, if the input number text is " - NA f. 1,000.00", then it is mapped to "-naf1,000.00" before processing. The currency signs are also transformed, so "NA f." is converted to "naf" for purposes of matching. As with other Unicode algorithms, this is a logical statement of the process; actual implementations can optimize, such as by applying the transform incrementally during matching.
O.3 Parsing Numbers
The following elements are relevant to determining the value of a parsed number:
A possible prefix or suffix, indicating sign
A possible currency symbol or code
Decimal digits
A possible decimal separator
A possible exponent
A possible percent or per mille character
Other characters should either be ignored, or indicate the end of input, depending on the application. The key point is to disambiguate the sets of characters that might serve in more than one position, based on context. For example, a period might be either the decimal separator, or part of a currency symbol (e.g., "NA f."). Similarly, an "E" could be an exponent indicator, or a currency symbol (the Swaziland Lilangeni uses "E" in the "en" locale). An apostrophe might be the decimal separator, or might be the grouping separator.
Here is a set of heuristic rules that may be helpful:
Any character with the decimal digit property is unambiguous and should be accepted.
Note:
In some environments, applications may independently wish to restrict the decimal digit set to prevent security problems. See
[UTR36]
The exponent character can only be interpreted as such if it occurs after at least one digit, and if it is followed by at least one digit, with only an optional sign in between. A regular expression may be helpful here.
For the sign, decimal separator, percent, and per mille, use a set of all possible characters that can serve those functions. For example, the decimal separator set could include all of [.,']. (The actual set of characters can be derived from the number symbols in the By-Type charts
[ByType]
, which list all of the values in CLDR.) To disambiguate, the decimal separator for the locale must be removed from the "ignore" set, and the grouping separator for the locale must be removed from the decimal separator set. The same principle applies to all sets and symbols: any symbol must appear in at most one set.
Since there are a wide variety of currency symbols and codes, this should be tried before the less ambiguous elements. It may be helpful to develop a set of characters that can appear in a symbol or code, based on the currency symbols in the locale.
Otherwise, a character should be ignored unless it is in the "stop" set. This includes even characters that are meaningful for formatting, e.g., the grouping separator.
If more than one sign, currency symbol, exponent, or percent/per mille occurs in the input, the first found should be used.
A currency symbol in the input should be interpreted as the longest match found in the set of possible currency symbols.
Especially in cases of ambiguity, the user's input should be echoed back, properly formatted according to the locale, before it is actually used for anything.
O.4 Parsing Dates and Times
Lenient parsing of date and time strings is more complicated, due to the large number of possible fields and formats. The fields fall into two categories: numeric fields (hour, day of month, year, numeric month, etc.) and symbolic fields (era, quarter, month, weekday, AM/PM, time zone). In addition, the user may type in a date or time in a form that is significantly different from the normal format for the locale, and the application must use the locale information to figure out what the user meant. Input may well consist of nothing but a string of numbers with separators, e.g. "09/05/02 09:57:33".
The input can be separated into tokens: numbers, symbols, and literal strings. Some care must be taken due to ambiguity, e.g. in the Japanese locale the symbol for March is "3 月", which looks like a number followed by a literal. To avoid these problems, symbols should be checked first, and spaces should be ignored (except to delimit the tokens of the input string).
The meaning of symbol fields should be easy to determine; the problem is determining the meaning of the numeric fields. Disambiguation will likely be most successful if it is based on heuristics. Here are some rules that can help:
Always try the format string expected for the input text first. This is the only way to disambiguate 03/07 (March 2007, a credit card expiration date) from 03/07 (March 7, a birthday).
Attempt to match fields and literals against those in the format string, using loose matching of the tokens.
When matching symbols, try the narrow, abbreviated, and full-width forms, including standalone forms if they are unique. You may want to allow prefix matches too, or diacritic-insensitive, again, as long as they are unique. E.g., for a month, accept 9, 09, S, Se, Sep, Sept, Sept., etc.
When a field or literal is encountered that is not compatible with the pattern:
Synchronization is not necessary for symbolic fields, since they are self-identifying. Wait until a numeric field or literal is encountered before attempting to resynchronize.
Ignore whether the input token is symbolic or numeric, if it is compatible with the current field in the pattern.
Look forward or backward in the current format string for a literal that matches the one most recently encountered. See if you can resynchronize from that point. Use the value of the numeric field to resynchronize as well, if possible (e.g., a number larger than the largest month cannot be a month)
If that fails, use other format strings from the locale (including those in ) to try to match the previous or next symbol or literal (again, using a loose match).
References
Ancillary Information
To properly localize, parse, and format data requires
ancillary information, which is not expressed in Locale Data Markup Language. Some of the
formats for values used in Locale Data Markup Language are constructed according to external
specifications. The sources for this data and/or formats include the following:
Charts
The online code charts can be found at
An index to characters
names with links to the corresponding chart is found at
DUCET
The Default Unicode Collation Element Table (DUCET)
For the base-level collation, of which all the collation tables in this document are
tailorings.
FAQ
Unicode Frequently Asked Questions
For answers to common questions on technical issues.
FCD
As defined in UTN #5 Canonical Equivalences in Applications
Bugs
CLDR Bug Reporting form
Glossary
Unicode Glossary
For explanations of terminology used in this and other documents.
JavaChoice
Java ChoiceFormat
Olson
The
TZ
ID Database (aka
Olson timezone database)
Timezone and daylight savings information.
ftp://elsie.nci.nih.gov/pub/
For archived data,
see
ftp://munnari.oz.au/pub/oldtz/
For general information, see
Reports
Unicode Technical Reports
For information on the status and development process for technical reports, and for a
list of technical reports.
UCA
UTS #10: Unicode Collation Algorithm
UCD
The Unicode Character Database (UCD)
For character properties, casing behavior, default line-, word-, cluster-breaking behavior,
etc.
Unicode
The Unicode Consortium.
The Unicode Standard, Version 4.0
. Reading,
MA, Addison-Wesley, 2003. 0-321-18578-1.
Versions
Versions of the Unicode Standard
For information on version numbering, and citing and referencing the Unicode Standard,
the Unicode Character Database, and Unicode Technical Reports.
Other Standards
Various standards define codes that are used as keys or
values in Locale Data Markup Language. These include:
ISO639
ISO Language Codes
Actual List:
ISO3166
ISO Region Codes
Actual List
ISO4217
ISO Currency Codes
(Note that as of this point, there are significant problems with this list.
The supplemental data file contains the best compendium of currency
information available.)
ISO15924
ISO Script Codes
Older version with Actual List:
RFC3066
IETF Language Codes
Registered Exception List (those not of the form language + region)
RFC3066bis
While RFC 3066bis was approved in 2005, at time
of publication it has not yet been published or given a number. The
following is the latest editor's draft:
However, the registry is functioning, and
located at:
UNM49
UN M.49: UN Statistics Division
Country or area & region
codes
Composition of macro geographical (continental) regions, geographical sub-regions, and
selected economic and other groupings
XML Schema
W3C XML Schema
General
The following are general references from the text:
BIDI
UAX #9: The Bidirectional Algorithm
ByType
CLDR Comparison Charts
Calendars
Calendrical Calculations: The Millennium Edition by Edward M.
Reingold, Nachum Dershowitz; Cambridge University Press; Book and CD-ROM edition (July 1,
2001); ISBN: 0521777526. Note that the algorithms given in this book are copyrighted.
CharMapML
UTR #22: Character Mapping Tables
Comparisons
Comparisons between locale data from different sources
CurrencyInfo
UNECE Currency Data
DataFormats
CLDR Data Formats
Example
A sample in Locale Data Markup Language
ICUCollation
ICU rule syntax:
ICUTransforms
Transforms
Transforms Demo
ICUUnicodeSet
ICU UnicodeSet
API:
LocaleExplorer
ICU Locale Explorer
LocaleProject
Common Locale Data Repository Project
NamingGuideline
OpenI18N Locale Naming Guideline
RBNF
Rule-Based Number Format
RBBI
Rule-Based Break Iterator
Scripts
UAX #24: Script Names
UCAChart
Collation Chart
UAX14
UAX #14: Line Breaking Properties
UAX24
UAX #24: Script Names
UAX29
UAX #29: Text Boundaries
UAX31
UAX #31: Identifier and Pattern Syntax
URegex
UTR #18: Unicode Regular Expression Guidelines
UTR36
UTR #36: Unicode Security Considerations
UTCInfo
NIST Time and Frequency Division Home Page
U.S. Naval Observatory: What is Universal Time?
WindowsCulture
Windows Culture Info (with  mappings from [
RFC3066
]-style
codes to LCIDs)
Acknowledgments
Special thanks to the following people for their continuing overall contributions to the CLDR
project, and for their specific contributions in the following areas:
Mark Davis for creating the initial version of LDML, and adding to
and maintaining this specification, and for his work on the LDML code
and tests
John Emmons for the POSIX
conversion tool
Deborah Goldsmith for her contributions to LDML architecture and
this specification
Chris Hansten for
coordinating and managing data submissions and vetting
Erkki Kolehmainen and his team for their
work on Finnish
Steven Loomis for development of the survey tool and database
management
Peter Nugent for his
contributions to the POSIX tool and from Open Office, and for coordinating and managing data
submissions and vetting
George Rhoten for his work on currencies
Roozbeh Pournader (روزبه پورنادر) for his work
on South Asian countries
Ram Viswanadha (రఘురామ్ విశ్వనాధ) for all of his work on LDML code and data integration,
and for coordinating and managing data submissions and vetting.
Vladimir Weinstein (Владимир Вајнштајн) for his work on
collation.
Other contributors to CLDR are listed on the
CLDR Project Page
Modifications
The following summarizes modifications from the previous revision of this document.
Revision 7
Point at bug database
instead of Unicode reporting form.
Add "root" as valid
locale identifier, and clarify that "locale" in CLDR is really essentially
language.
Added the list of
private use language & script subtag codes that won't be used by CLDR.
Corrected the
dateTimeFormat assignments for {0} and {1}.
Revision 6
Incorporated Corrigendum 1 (see
into Appendix F:
Date Format Patterns
and Section
5.4

Revamped Appendix J:
Time Zone
Display Names
. Also changed "Fallback" to "Display Names" in the title of the
Appendix, and "Olson" to "TZ" in other places in the document.
Yesstr/nostr/yesexpr/noexpr changes in Section 5.11

Added Section 5.14

Moved week, measurement data to Appendix C:
Supplemental Data
Added coverage levels in Appendix M:
Coverage Levels
Added rule-based number
formats and transforms, in Section 5.15
Transforms
Appendix N:
Transform Rules
Added metadata, replacing
the contents of Appendix K:
Valid Attribute
Values
Added availableFormats, dateFormatItem, and appendItem in

to support more flexible date/time formatting
Added measurementSystemNames and measurementSystemName in

for localized names of
measurement systems
Added quarters, quarterContext, quarterWidth, and quarter to

for names of calendar quarters
Extended possible values for
alt tag
Added ethiopic calendar to
allowed calendar
values
Clarified usage of
quotation marks
and alternate marks
Corrected example
ISBN
Added eraNarrow to

for one-character version of era names
Added
Appendix O
, on lenient parsing
Added
3.1
Unknown or Invalid Identifiers
Other editing
Updated descriptions to final DTD and metadata.
Revision 5
The canonical form for variants is uppercase
Addition of UN M.49 codes
Addition of persian and coptic calendar IDs
Clarification of alias inheritance
New XML references section
Modified revision and generation field format
Use of language display names for whole initial segments of locale IDs names, such as nl-BE
Addition of the inList element
Clarification of 'narrow'
Additional dateTimeFormat description
Names of calendar fields, and relative times.
New element currencySpacing
Descriptions of POSIX yes/now
New supplemental data elements/attributes (end of Appendix C)
currency to/from
languageData
timezoneData
territoryContainment
mapTimezones
alias
deprecated
characters
in dateExtension of era to 1..3
Clarification of year padding
Deprecation of localizedPatternChars
Use of the singleCountries list
Appendix L: canonical form
Misc editing
Revision 1
(2005-06-30): added link to Corrigenda.
Copyright © 2001-2006 Unicode, Inc. All Rights Reserved. The Unicode
Consortium makes no expressed or implied warranty of any kind, and assumes no liability for errors
or omissions. No liability is assumed for incidental and consequential damages in connection with
or arising out of the use of the information or programs contained or accompanying this technical
report. The Unicode
apply.
Unicode and the Unicode logo are trademarks of Unicode, Inc., and are
registered in some jurisdictions.