![[Show/Hide Right Column]](pics/icons/orightcol.png "[Show/Hide Right Column]"){kind=icon}
Understanding Encoding
Table of contents
Different languages require different characters. Classic character encodings were limited to 255 characters per character set to reduce the amount of disk space required to store text. The original character set, ASCII, was limited to 128 characters and was suitable for English.
The Unicode effort and the UTF-8 encoding were created to support all characters for all languages as efficiently as possible.
Tiki uses UTF-8 internally for all manipulations and has done so for a very long time.
Encoding scenarios
However, MySQL allows storage to use different encodings for more efficiency. In the past, Tiki poorly informed MySQL of the content it provided to it. Several outcomes could appear.The UTF-8 case
In this ideal scenario, the MySQL database server was configured to accept UTF-8 encoding and store the data as UTF-8. All features behaved correctly and data could freely be accessed from other applications. Some distributions provided settings leading to this configuration, but in most cases, it required intervention from the system administrator.The latin case
In many cases, MySQL was configured to receive and store content as latin1, and encoding suitable for most western Europe languages. Under this set-up, Tiki behaved correctly from the front-end. However, there were consequences:- Extracting the data stored by Tiki from other applications, such as phpMyAdmin, lead to strangely encoded characters.
- Moving the database to a different server could cause unexpected errors.
- Ordering in lists could appear to be random.
Double encoding
On most shared hosts, site administrators have no control over the default connection encoding on the server. The connection encoding would be used as latin1. However, when creating the database, the site administrator would create the database as UTF-8 to support all languages.This configuration had a pervert effect. When Tiki would send UTF-8 data, MySQL would think it was latin1. Because the database requested UTF-8, it would perform a conversion, leading to doubly-encoded UTF-8.
In reality, the consequences were very similar to the latin case.
Information loss
If the server was configured by default to accept UTF-8 connections and the database happened to be created as latin1, Tiki would correctly send UTF-8 data and MySQL would recognize it. However, when time arrived to store it, the data would be converted to latin1. Because UTF-8 has more possible characters than other encoding types, unknown characters got converted to ? signs or some other unknown character.Other than not supporting languages not supported by the database encoding, all features would work correctly in Tiki.
Changes
After a partially-failed attempt to resolve these issues in Tiki5, more options were added in Tiki51?.This section explains how the upgrade to 5.1 will affect your installation.
Upgrading from 2.x, 3.x or 4.x
For previous versions, Tiki will keep behaving as-is to avoid corrupting data.Correcting the encoding issue will require manual intervention. The Tiki installer comes with tools to assist in the conversion. However, you should have reliable backups before using them. Other techniques involving exporting data, modifying the dump and re-importing it can be used as well.
To avoid data loss, you should make sure your tables use UTF-8. This can be done from the install/upgrade page in the installer.
Removing the double-encoding can be done from Enter Your Tiki. It will require the client charset to be forced to UTF-8.
Follow the steps described here:
http://doc.tiki.org/Upgrade#Fix_the_encoding_issue
Upgrading from 5.0
In default 5.0 installations and upgrades, the connection character set was specified automatically in the configuration file.New 5.0 installations correctly specified the connection encoding and will lead to either the UTF-8 case or the information loss case.
Upgraded 5.0 installations will lead to different results. However, the upgrade procedure to 5.1 does not impact the changes that were made at that time. Some administrator judgment is advised if encoding issues appear. The client character set can be altered in the configuration file. Contact the developer mailing list if support is required.
New 5.1 installations
New 5.1 installations will specify the connection encoding to UTF-8 by default. If the database is not in UTF-8, this will lead to information loss. The installer will propose that you convert your database to UTF-8.If you have no control over the database encoding and still need to use non-latin characters (ex.: Arabic) , it is best not to force the connection encoding to UTF-8 or to set it as the same encoding as the database itself. It will lead to the latin case, which comes with certain downfalls, but still allow for multiple languages to be used on a single site.
If you can control your database encoding
Pre-Tiki6, make sure to set your MySQL (via PhpMyAdmin for example) so that new tables are also created in UTF-8. In general, for your databases, look for "MySQL connection collation:" and if it's at latin_1, change it to utf8_general_ci. (Generally, this is already the case in typical shared hosting)Then, in PhpMyAdmin, go to the database you'll be using your Tiki in Server: localhost -> Database: tiki -> Operations -> Collation, and also change from latin_1 to utf8_general_ci before running the installer. (Generally, this needs to be changed in typical shared hosting)
This is the equivalent of the following MySQL statement:
ALTER DATABASE DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
Tiki 5.2 or 5.3
If you have file upload issues (just upload an image and you will see if it works or not), you can solve by- applying revision 30522, which fixes it (and will be 5.4)
If you can't do that, you can try to
- add $api_tiki='adodb'; to your db/local.php,
and/or not forcing utf8
In db/local.php
$client_charset='utf8';
Now commented out
// $client_charset='utf8';
,
,
,
and
![[toggle]](javascript:icntoggle('mod-DocUserMenul1','module.png'); "[toggle]"){kind=small}
In Progress