Tibetan Unicode

ཡུ་ནི་ཀོཌ྄

Introduction

Before the Unicode Standard came along, there were hundreds of different standardized and non-standardized encoding systems for encoding the characters of different writing systems. No single character encoding had enough characters to encode all the characters used in all the different writing systems of the world. Even for Western European languages, which use an uncomplicated writing system, the 7-bit and 8-bit computer character sets, such as ASCII and ISO 8859-1, used for encoding the Roman script were inadequate for all the letters, punctuation, and technical symbols in common use.

The situation for Tibetan was particularly anarchical. There was no recognized standard for encoding Tibetan script characters. Word-processing applications and add-ins for Tibetan used non-standardized, proprietary font-based encodings - mapping the Tibetan glyphs in the fonts they used to character sets originally designed for encoding Roman or Chinese characters. Since each Tibetan system used its own encoding, one of the greatest obstacles to using electronic Tibetan data resulted from the fact that files could not be easily shared by different Tibetan word-processing programs or with other applications without converting files from one encoding scheme to another.

Background: Some 'Legacy' Tibetan Systems & Encodings

8-bit Tibetan word-processing systems

Tibetan texts may contain thousands of different character combinations (stacks displayed as a ligature or combination of glyphs) - but many of these systems, programmed to work with the 8-bit operating systems and applications in use in western countries, India, Bhutan and Nepal at the time they were designed, mapped the glyphs in their fonts to restricted 8-bit character sets supporting a maximum of 256 characters or less. This meant that these applications had to spread the glyph set required for a comprehensive coverage of Tibetan across a whole set or series of separate font files. The different glyphs in each font were necessarily mapped to the same underlying character set. A system using a set of five fonts (e.g. TCC system) to cover the Tibetan combinations it supports is in fact storing five different glyphs, each representing a different Tibetan character or different combination of characters, using a one common underlying character. A system using a set of thirty fonts (e.g. Nitartha’s Sambhota system with Dedris fonts) is using one and the same character for thirty different combinations of Tibetan letters.

With all of these systems, in order to specify which particular combination of Tibetan letters the underlying character represents, it is necessary to store information on which specific font within the set this underlying character should be displayed with. In other words, font formatting information has to be stored along with the text. Effectively these systems are switching between five or thirty different code pages for Tibetan - the code page being used specified by the font being used. This means that Tibetan text entered using one of these systems is tied to the system which produced it – or at least to the particular font set(s) it supported. These systems require Tibetan to be stored as rich-text (text plus formatting information) rather than plain-text and if the formatting information is lost or corrupted the Tibetan text data is corrupted – or becomes garbage.

Another consequence of representing and storing Tibetan text in this way is that it becomes practically impossible to reliably search, sort, index or spell-check Tibetan data. Since most searching, sorting, indexing and spell checking utilities are designed to work with plain text rather than rich text they either ignore - or choke on - any formatting information applied to the underlying characters. Transmission of text across the internet to a large number of users is also a problem as there is no guarantee that a system receiving the data has access to or even can support the fonts and application used to generate the original data. Long term archival storage of Tibetan data in these formats also relies on the highly risky assumption that the non-standardized applications and fonts used to generate them will be supported in future versions of operating systems for many years and decades to come.

Of course the systems referred to above were first developed at a time when most users were using them on stand alone computers not connected to the internet. They necessarily had to be designed to work within and leverage the technology and applications available at the time. Their primary purpose was to produce Tibetan documents which would be printed out rather than displayed on screen. Some of these Tibetan word-processing systems were developed and enhanced to effectively become sophisticated desktop publication systems for Tibetan pecha - and their fonts were evolved to a high standard of design. These systems indeed became very good for the purposes for which they were originally intended – but we should recognize their limitations.

Systems developed in India

Indian languages with their complex scripts faced similar problems to Tibetan. The Indian Government developed the ISCII standard (Indian Code for Information Interchange) which covered all major Indian scripts. An important feature of this encoding was that tried to separate characters from glyphs thus a large number of letter forms could be represented by a small number of characters which easily fitted into an 8 bit code page. Systems based on this encoding used a software layer at runtime to map from characters to glyphs.

Most of the software and fonts designed to work with this encoding was developed by the Centre for Advanced Computing (CDAC) in Pune. Since the Tibetan script is based on an Indian model, sharing many features with other Indic scripts, CDAC were able to develop a similar system for Tibetan & Dzongkha.

However, the CDAC system for Tibetan & Dzongkha was never widely adopted. (As far as I know, it was being used by a few organisations connected with the TGIE in Dharamsala; the Institute for Higher Tibetan Studies in Sarnath; Drepung Loseling monastic college in Mungod, Karnataka; and two or three departments of the Royal Government of Bhutan in Thimphu.) Reasons for this may have been that the software required a dongle or hardware key to run – and so couldn’t be freely copied; and the fonts, which built up stacks from parts, did not look as nice as those available for some other PC based systems.

However because ISCII based systems used a small number of characters and required no font formatting information embedded in data, this encoding could be successfully used in database applications – such as library catalogue applications – running on 8-bit operating systems. ISCII also seems to have had a major influence on the model chosen for encoding complex Indic, and Indic derived, scripts in the Unicode Standard.

Multi-byte Tibetan systems developed in China

The situation for Tibetan computing in China was very different. Computer systems in China were from the start designed to support the large character sets needed for the Chinese language – so didn’t suffer the limitations same limitations of a charcter set restricted to 256 charcters as in the west. Although in the early stages few Tibetan individuals could afford to purchase a computer, state owned publishing houses were set up to produce new editions of Tibetan texts and to publish newspapers, textbooks and magazines in the Tibetan language. Consequently two Chinese publishing systems ___ and ___ were adapted to work with Tibetan

How to install Tibetan Unicode software support

See Tibetan Unicode Installation for information on how to install the required software support for Tibetan Unicode on Windows, Linux and Mac OS-X.

See also

Tibetan Fonts

External links

• What is Unicode?
• Encoding model of the Tibetan script in the UCS - Explains how Tibetan characters are encoded in the ISO 10646 / Unicode Standard. by Christopher Fynn
• Tibetan Block of The Unicode Standard (code chart)
• Enabling Complex Script Text Support for Indic Scripts - on Wikipedia (includes Tibetan)