History: Tracker Import Export

Source of version: 21 (current)

Tracker Import-Export (Tracker Tabular) is an advanced import/export feature for Trackers, allowing for more flexibility and more predictable results. Additionally, it can be used to create custom listings for trackers using the same formatting rules. Applicable filters can be selected to create efficient task-specific interfaces.

{REMARKSBOX(type="warning" title="Feature renamed")}Feature renamed from Tracker Tabular to Tracker Import-Export in ((Tiki25)) (see: https://gitlab.com/tikiwiki/tiki/-/merge_requests/1880{REMARKSBOX}

! {{page}}

!! Usage
To use the feature, it must be enabled from the Tracker admin panel. It will then be accessible through the tracker listing along with the other import options. While basic import/export can be performed directly from the database, most usages require a functioning ((Unified Index)).

!!! Schema definition
With Tracker Import-Export (Tracker Tabular), the administrator must first define an import/export schema. This is essentially a list of columns and the format that will be used to present or manipulate the information. Each supported tracker field type may expose one or more supported formats. 

For instance, imagine that you have these data types as shown in the screenshot below:

{img src="display1313" link="display1313" width="400" rel="box[g]" imalign="center" desc="Click to expand" align="center" styleimage="border"}

You will need a schema like this one:

{img src="display1314" link="display1314" width="400" rel="box[g]" imalign="center" desc="Click to expand" align="center" styleimage="border"}

!!! Field Formats
Field format examples:
* The date fields that let you choose which date format that should be used.
* The text fields that normally allow to present a single value. However, if the field is multilingual, it will provide one format for each supported language, allowing to cleanly export all desired languages into a file, send the file to a translator and import it back.
* The category field provides a wide range of options, depending on if the field allows a single value or multiple. It can present the category ID, or the category name, make each category available as its own field to be marked with an X. This allows the administrator to provide the exported data in a human-readable format suitable to the required task.
* The group selector field can present the group ID or the group name.

!!! Field Filters
Along with the schema definition, a list of available filters can be selected. Just like formats, these filters are exposed per field type. The filters will be available for the UI listings and for partial exports. By default, filters will be available in a filter pop-up, but they can be configured to display above the listing or on the side.

Some filter examples:
* The text field can be filtered either as a full text search or as a prefix search.
* Multilingual text field filters can apply to a single translation or all of them.
* Dropdown fields can filter a single value at a time or allow multiple selection.

The available set of field formats and filters are expected to augment over time. The user interface allows to explore the available options.

!!!! Scope of Filters
Filters apply to the default listing view and partial export. The filters are defined per-field and can be configured in different locations:

* __Default__ filters will be available in a pop-up accessible through the filter icon.
* __Primary__ filters will be displayed above the listing and in the filter pop-up.
* __Side__ filters will be displayed to the left of the results and in the filter pop-up.

__Side__ filters are mostly suitable for dynamic filters, generated through ((Elasticsearch)) facets. The available options will vary depending on the other (non-dynamic) filters selected. If the filter does not provide a choice, it will be hidden. The dynamic filters are available for selection from the list of filters under the ''Dynamic Filters''. Some of the modes listed may never provide results as they come from a system-wide list rather than a tracker-specific list.

__Item Link__ fields that specify the __Index Remote__ option will also make the filters from those fields available for selection. For example, if you have  product linking to a supplier, you could filter products based on the city field defined in the supplier.

!!!! Presetting the Default Values of Filters
You can pass parameters on the partial export url to preset default values for the filters, like this to preset a username in a User Selector field:
{CODE()}tiki-tabular-filter?target=export&tabularId=1&tf_aTrackerUserField_dd=username{CODE}
If you're doing this in a custom smarty template you can do it like this:

{CODE()}{button href="tiki-tabular-filter?target=export&tabularId=1&tf_aTrackerUserField_dd=`$user`" _text="Export" _class='click-modal'}{CODE}

The basic format is -+ft_{$permName}_xx+- where xx changes depending on the type of field. Here are some examples:
||DropDown, User Selector single: | -+tf_permName_dd+-
DropDown, User Selector etc multi: | -+tf_permName_ms+-
Category (multi-select any) | -+tf_{$permName}_anyd+-
Category (checkboxes any) | -+tf_{$permName}_anyc+-
Text | -+tf_{$permName}_ft+-
Numeric (exact) | -+tf_{$permName}_em+-
ItemLink (object selector) | -+tf_{$permName}_os+-
Date Range | -+tf_{$permName}_range_fo=(unix timestamp)+-
           | -+tf_{$permName}_range_from=(unix timestamp)+-
Autoincrement, Freetags, Email and others | -+tf_{$permName}_lookup+-||

!! Import / Export
!!! Import

Import from CSV is supported for schemas containing a primary key. 

# From the schema listing, click on import for the desired schema
# Select the file to upload and submit
** If the file does not match the schema, it will be rejected.
** Items with a matching primary key will be updated
** Items without a matching primary key will be created

!!! Full Export

All schemas support exporting as long as they are valid.

# From the schema listing, click on click for the desired schema
# Save your file

!!! Partial Export

Partial export uses the ((Unified Index)) and the configured filters. It can be accessed from the primary schema listing or from the item listing. When used from the listing, using the export feature will default to the currently selected filters.

!!! Custom Export

Partial export relies on the ((Unified Index)) to export the results rather than directly obtaining the data from trackers. Depending on the configured search engine, performance may vary. On some engines, it may be faster than directly obtaining the data from trackers on a full export (no additional filters).

# From the schema listing, select search for the desired schema
** Alternative: Click the search icon from the tracker listing
** Alternative: Select the appropriate tracker from the search facets (when enabled)
# From the search interface you have been directed to, narrow down your resultset using any of the available filters
# Select the **Export** secondary action in the search form
# Conditional: If you selected the tracker rather than the schema, you will be prompted to select the schema
# Save your file

!!! Technicalities

Fields can be marked as __Read Only__. This will make sure that when importing, the field will be skipped, making the import process faster, but also making sure only the allowed fields get modified. It also allows including multiple formats that would otherwise conflict, such as the category ID and the category name, for example.

For the import to be possible, a single field can be marked as the __Primary Key__. The item ID can be used, but other fields can be used as well. For example, if you manage a product database and know your supplier-provided serial numbers are unique, you could create a schema that uses those as the reference and not include the auto-generated item ID at all.

A field can be marked as __Export Only__ to be excluded from the default web view. This is useful for non-meaningful primary keys such as the itemId. It is required to import back the data, but should otherwise not be presented to the users under normal circumstances.

See:
* http://sourceforge.net/p/tikiwiki/code/53165

!! Use from remote sources
You can alternatively use tracker data from remote sources through the __ODBC__ read/write support added in ((Tiki23)).

See ((ODBC))

Alias names of this page:
(alias(TrackerTabular)) | (alias(Tabular)) | (alias(Tracker Import-Export))
~tc~ (alias(Tracker Tabular)) ~/tc~