Loading...
 

Search Index


The search engine in Tiki was rebuilt for Tiki7 this documentation applies only to versions 7 or later.


The search capabilities in tiki, such as those used by PluginList or the search at tiki-searchindex.php rely on a search index.



Tiki can support multiple search engines internally. Each of those will have different capabilities and limitations. The default engine should provide capabilities good enough for small and medium sites. Larger sites may need additional infrastructure to get the most performance.

Unified Index Comparison

Fields


Below is a matrix between the fields and the object types.

Legend:
  • Tokenized, as in decomposed in words for full text search
  • Sortable

TokenizedSortablewiki pageforum postblog postarticlefiletrackeritemsheetcommentuserAvailable in Tiki version
Generic
object_type XXXXXXXXXX7
object_id XXXXXXXXXX7
titleXXXXXXXXX?X7
language XX//X/?// 7
modification_date XXXXXXXXX 7
contributors XXXXXXXX 7
descriptionX X XX X 7
contentsX XXXXXXXXX7
Specific
wiki_content X X7
wiki_uptodateness XX 7
wiki_approval_state X 11
post_content X 7
post_snippet X 14
parent_thread_id (not to be confused with parent_object_id) X 8
root_thread_id X 14
parent_contributors X 14
blog_id X X 7
blog_excerpt X 7
blog_content X 7
topic_id X X 7
article_content X 7
article_topline X 7
article_subtitle X 7
article_author X 9
article_type X 9
published X 13
sitetitle X 13
siteurl X 13
gallery_id X X 7
filename_id X X 7
filetypeXX X 7
file_comment X 7
file_content X 7
tracker_id X X 7
tracker_status X X 7
tracker_field_IDX? X 7
sheet_content X 7
comment_contentX X 7
user_countryXX X10
Internal
view_permission X X X 7
parent_object_type X XXXXX X 7
parent_object_id X XXXXX X 7
parent_view_permission XXXXX X 7
global_view_permission X 7
hash X 7
url X X 7
Global
categories XXXXXXXX 7
deep_categories XXXXXXXX 7
allowed_groups XXXXXXXX 7
freetags XXXXXXXX 7
freetags_text XXXXXXXX 7
adv_rating_ID XXXXXXXXXX7
comment_count XXXXXXXXX 8
relations XXXXXXXXX8
attachments XXXXXXXX 7
attachment_contentsX XXXXXXXX 7
geo_located XXXXXXXXX9
geo_location XXXXXXXXX9
visits XX X 9.2


X Available
/ Static value
? Depends on the data


The indexing for tracker fields will vary depending on the field type. As a general rule, tracker_field_ID will be used as the field and will be sortable. However, there are a few exceptions:
  • Image and File fields are not indexed
  • TextArea is not sortable
  • Multilingual fields are indexed as multiple fields:
    • The normal one contains all languages
    • tracker_field_ID_lang contains one language only (tracker_field_12_fr for example)
  • Rating and related fields store as multiple fields
    • tracker_field_ID contains the average
    • tracker_field_ID_sum contains the vote totals
    • tracker_field_ID_count contains the number of votes
  • Items List and Item Link fields
    • tracker_field_ID_text contains the text instead of the IDs
  • If a language field is set for the tracker item, that language is indexed as the item language.

Some used in buildQuery/tiki-searchindex.php (need explanation on whether these are real fields or just helpers):
type: refers to object_type
deep: if this is set, categories will be considered deep-categories
autocomplete: Will search for items with title starting with this

Multivalue fields and the _array suffix

When using MySQL and Lucene engines which do not directly support indexing of arrays, multivalue fields are stored in the index as tokenized values in order to enable proper filtering. If you want to get the values in a real array, (supported from Tiki 14) please use fieldname_array instead. For example, contributors_array will give you the array of contributors, and categories_array will give you the array of categories. How this works is by additionally storing in the index a json_encoded string representation of the array.

Rebuild search index


The index is stored at temp/unified-index/ While the rebuild is occuring, a directory temp/unified-index-new/ will appear (This is to permit the existing index to be used until the new one is ready). If temp/unified-index-new/ doesn't disappear after the indexing, something must have gone wrong. You can delete it and try the re-indexing again. You may want to run sh setup.sh to make sure the permissions are OK.

From the Tiki interface

You can visit this url: tiki-admin.php?page=search&rebuild=now and search index will be rebuilt if the site is small. For medium to high load sites, you can do that from the command line.

From the command line

The search index can be rebuilt from the command line, and since Tiki9 can be run using a Cron job where the server runs the command automatically - see Cron Job to Rebuild Search Index.

Below are the commands that may be used to rebuild the index.

 Note

Lucene Engine: Important note These commands need to be run as the web server user when using the Lucene implementation, often www-data or nobody depending on your server setup, otherwise the web server cannot update and maintain the index. This means prepending the following commands with "sudo -u www-data ".

All commands below assume you are already in the Tiki root directory.

Tiki9 and Tiki10

Basic command
php lib/search/shell.php rebuild;
sh setup.sh -n

Create a log too
If you are troubleshooting, and would like a log (which will be created in temp/Search_Indexer.log:
php lib/search/shell.php rebuild log

From Tiki11

From Tiki11, you can also rebuild it using the unified console.php command, with the appropriate parameters. For example:
Basic command
php console.php index:rebuild

<-- or -->

php console.php i:r

 Note

The syntax "i:r" is equivalent to "index:rebuild".

Multitiki sites
For multitiki sites, you can rebuild with commands like:
php console.php index:rebuild --site=site1.example.com  
php console.php index:rebuild --site=site2.example.com  
...

Successful rebuild
If the rebuild is successful a message like the following will be produced (for cron jobs, this can usually be sent to you via email as part of automatically running the command):
Started rebuilding index...
Indexation
 wiki page: 150
 forum post: 67
 blog post: 412
 article: 61
 file: 1294
 trackeritem: 196
 comment: 0
Rebuilding index done
X-Powered-By: PHP/5.5.8
Content-type: text/html; charset=utf-8

Troubleshooting
If the rebuild is unsuccessful, instead of the above message you may get a message that indicates there has been an internal server error, or it may say "Rebuild in progress." This may be because the rebuild process uses more memory or takes more time than allowed by the server's php settings. Such settings can be changed on the fly as part of the rebuild command - examples of how to do this are shown below.

Increase memory limit
One way to increase memory is to change the memory_limit php setting as follows (this example changes the memory limit to 4 gigabytes while the rebuild process is running):
php -dmemory_limit=4G console.php i:r --log

You could also direct php to use a specific php.ini file, where there may be a higher memory limit setting or no limit. In this case you would use the -c parameter followed by the path to the php.ini file, as in the example below:
php -c /etc/php5/cli/php.ini console.php i:r --log


Increase maximum execution time
Getting an internal server error may indicate the rebuild process takes longer than the max_execution_time php setting. That can be increased as part of the command as shown below where the max execution time is set to 300 seconds, or 5 minutes. (This command is also increasing the memory limit as described above):
php -dmemory_limit=4G -dmax_execution_time=300 console.php i:r --log


Force rebuild
When the rebuild is unsuccessful with a "Rebuild in progress" message, this usually means that the rebuild failed previously in the middle of the process, leaving a temporary folder called temp/unified-index-new on the server. When a new rebuild is started and the program sees this folder, it thinks there is a rebuild already in progress and will stop. You can either delete this folder before rebuilding again or include the --force parameter in the rebuild command as follows:
php -dmemory_limit=4G -dmax_execution_time=300 console.php i:r --force --log

Related:


alias names for this page
Unified Search | UnifiedSearch | Enterprise search | Search Index | SearchIndex | UnifiedIndex | IndexRebuild | Index Rebuild

doc.tiki.org
[Toggle]


Keywords

Keywords serve as "hubs" for navigation within the Tiki documentation. They correspond to development keywords (bug reports and feature requests):

Accessibility (WAI and 508)
Accounting (7.x)
Articles and Submissions
Backlinks
Banners
Batch (6.x)
BigBlueButton audio/video/chat/screensharing (5.x)
Blog
Bookmark
Browser Compatibility
Link Cache
Calendar
Category
Chat
Clean URLs
Comments
Communication Center
Compression (gzip)
Contacts (Address Book)
Contact us
Content Templates
Contribution (2.x)
Cookie
Copyright
Credit (6.x)
Custom Home and Group Home Page
Date and Time
Debugger Console
Directory of hyperlinks
Documentation link from Tiki to doc.tiki.org (Help System)
Docs 8.x
Draw 7.x
Dynamic Content
Dynamic Variable
External Authentication
FAQ
Featured links
File Gallery
Forum
Friendship Network (Community)
Gmap Google maps
Groups
Hotword
HTML Page
i18n (Multilingual, l10n, Babelfish)
Image Gallery
Import-Export
Install
Integrator
Interoperability
Inter-User Messages
InterTiki
Kaltura video management (4.x)
Karma
Live Support
Login
Logs (system & action)
Look and Feel
Lost edit protection
Mail-in
Map with Mapserver
Menu
Meta Tags
Mobile Tiki and Voice Tiki
Mods
Module
MultiTiki
MyTiki
Newsletter
Notepad
Payment
Performance Speed / Load
Permissions
Platform independence (Linux-Apache, Windows/IIS, Mac, BSD)
Polls
Profiles
Profile Manager
Report
Toolbar
Quiz
Rating
Feeds
Score
Search engine optimization
Search
Search and Replace
Security
Semantic links (3.x)
Shadowbox
Shadow Layers
Share
Shopping cart
Shoutbox
Slideshow
Smiley
Social Networks
Spam protection (Anti-bot CATPCHA)
Spellcheck
Spreadsheet
Stats
Surveys
Tags (2.x)
Task
Tell a Friend, alert + Social Bookmarking
TikiTests (2.x)
Theme CSS & Smarty
Trackers
Transitions (5.x)
TRIM
User Administration including registration and banning
User Files
User Menu
Watch
WebHelp
WebDAV (5.x)
Webmail
Web Services
Wiki 3D
Wiki History, page rename, etc
Wiki Page Staging and Approval (2.x)
Wiki Plugin extends basic syntax
Wiki Syntax
Wiki structure (book and table of content)
Workspace
WSOD
WYSIWYCA
WYSIWYG (2.x)
XMLRPC


Tiki Newsletter

Delivered fresh to your email inbox!
Newsletter subscribe icon
Don't miss major announcements and other news!
Contribute to Tiki
Show php error messages