Skip to main content
Matrix42 Self-Service Help Center

Create an Import Definition

Overview

Adding an import definition requires that you fill out several consecutive dialogs to configure the import rules.

The chart below lists these dialogs and provides a brief overview of the actions to perform at each of the steps.

To create a new import definition:

  1. Start Matrix42 Workspace Management and click the Administration workspace.
  2. In the navigation area, click Import > Import Definitions.
  3. Click Create New Import Definition to open an input wizard and fill out the following:

Type Definition

  • Destination Type: Select the configuration item into which you want to import the data.
  • Show Detailed Information: Select this checkbox to display data definitions that constitute the selected configuration item.

2017-11-08_9-49-13.jpg

Data Source Type

Select the type of the data source. This page is used for establishing the connection to the source of the data being imported. It has different sets of fields depending on the data source type that has been selected at the previous step. The sample of the dialog for .csv data files is given below.

2017-11-08_9-50-23.jpg

Source Table Definition

At this step you should define which table(s) should be imported. As a rule, it is one simple table, but you can define more specific settings if necessary. You can also specify a list of parameters that should be considered during the import.

2017-11-09_14-48-56.jpg

Central Table

 

By default, the system will automatically select the first table from your data source. The central table of a data source can belong to one of two types.

Simple table: Select this option to import a simple table through the import definition. It means that the table contains one row per destination object. Example: If we are importing computers, then one row contains information about one computer. 
2017-11-08_14-22-19.jpg

Specify child tables (if required): If you select the Simple table option, then an additional grid becomes available. Use this grid to specify any number of child tables. When you click the Add button, the source object details dialog for a child table opens. Set the foreign key for the child table and make sure that the primary key is specified for the central table. These fields link the central and child tables together.

Example: The structure that is described above is represented by two separate files or we connect directly to a relational database. The import definition allows you to specify several source tables.

It requires that you should have one "central" table that contains one row per object and any number of "child" tables that are related to the primary key of the central table.
2017-11-08_14-33-05.jpg

Selecting the Complex Table Type

The central table contains several lines per target object: Select this option to import this type of data. Example: You have two tables, Computers and Hard Disks, and each computer can have several hard disks. In relational databases, such data is usually stored in different tables, but when it comes to importing files, the data is combined in one file. In this case, the file contains more than one row per computer. That is what we call a complex table. To import such data, we should know which fields uniquely identify a computer. For example, it can be Computer-Name.

2017-11-08_14-38-42.jpg

There is no need to specify explicitly which fields identify an object: the objects are identified by matching criteria which are specified at the mapping rule definition stage.

In practice, complex data sources are possible in two cases:

  • When we import objects together with their multifragments (Computer + Hard disks).
  • When we import objects together with their 1->N, N->M relations (User + Accounts).

GDIE supports both scenarios.

When objects are imported with their relations to other objects, the import processes only relations to those objects that already exist in Matrix42 Workspace Management. The import adds only records of the configuration item that is specified as the destination object.

Specify parameters

At this step you can introduce certain parameters for this import definition. Parameters allow you to modify values of fields (columns) of the imported objects.

When parameters are added, you will be able to use them for establishing mapping rules for this import definition. If an import definition contains parameters, the system will prompt you to enter the actual values for these parameters each time when you run the Execute Import Definition action. If an import definition is run within a workflow by the Execute GDI Sequence workflow activity, the import definition should receive the required parameters from this workflow. When an import is executed, it uses the entered values of parameters to transform the initial values that have been imported.

Use the Specify parameters grid to add a new parameter.

  • Name: Enter the arbitrary name without spaces.
  • Type: Select the type of value this parameter can take.

It is required that parameter values are entered each time when the import definition is run.

Mapping Rule Definition

Mapping rules define transformations of source columns into destination attributes. They also contain matching criteria for identifying the source data against the existing data.

  • Source Table: Name or alias of the source table as previously defined.
  • Source Column: Column name of the source table.
  • Match: The selected checkbox signifies that a source column is used for matching. By comparing values in this column with the values of the specified destination attribute, the system detects whether the data that is being imported already exists or whether new configuration items are being created. You can use more than one column for matching. As a rule, all source columns with the selected Match checkbox are used for the resulting matching query.
  • Destination Data Definition: Name of the destination data definition, to which the source column should be mapped.
  • Destination Attribute: Name of the destination attribute to which the source column should be mapped.
  • Relation Match: If the destination attribute is actually a reference to an attribute of a different data definition (the data definition that is not included in the selected configuration item), this column specifies the related attribute.

Add a new mapping rule or double-click the existing one to open the mapping rule details dialog for this rule. Those rules that have been selected as matching criteria are automatically added to the resulting matching query by using the AND operator. However, you can also open the query builder and edit the conditions that have been added automatically.

Mapping Rules for Table Columns

  • Define source column
    • Source Table: Select the source table from the drop-down list. The list of tables was defined at the previous step. If Alias was specified, it will be shown.
    • Column Type: Select the Table Column option.
    • Source Column: Select the source column from the drop-down list.
  • Define destination attribute
    • Destination Data Definition: Select the data definition from the drop-down list. It contains all data definitions of the specified configuration item. The [Multi] postfix indicates a multifragment data definition.
    • Destination Attribute: Select the attribute from the drop-down list. It contains all attributes of the selected data definition. If an attribute is a reference to another data definition, you will see the [Relation] postfix and cardinality of the corresponding relation.

If you select the Relation attribute, you should specify additional settings. A relation is only a reference to the row of the related table. You cannot populate it directly because its value is Guid. However, you can match your source value against the value of some simple attribute (typically the String data type) in the related data definition.

  • Related Data Definition: The field is non-editable and displays a name of the related data definition.
  • Match Type: Select the required option.
  • Related Attribute This field is available if the Attribute Value value is selected in the Match Type field. You can choose from the simple attributes (not relations) of the related data definition.
  • Match expression: This field is available if the corresponding value is selected in the Match Type field. When entering an ASQL expression, just use the Production keyword to refer to the related data definition.
  • GDIE requires the data type of this expression to be String (implementation feature). If it is not, then GDIE will internally add the CAST operator to convert it to String. You only need to ensure that the data type of your expression is not one of the following: Binary, Image, Text, or Virtual relation.
  • Define matching options

Mapping Rules for Static Values

Define source column

  • Source Table: Select the source table from the drop-down list. The list of tables was defined at the previous step. If Alias was specified, it will be shown.
  • Column Type: Select the Static Value option.
  • Column Name: Enter the column name that will be displayed in the mapping rule definition dialog.
  • Column Value: Enter the static value that will be passed to the destination attribute. This value will be the same for all imported objects.

Mapping Rules for Source

This option of a mapping rule is used in conjunction with the transformation option. Matrix42 Workspace Management cannot transform values directly in a source table. Therefore, it creates a staging table and imports all source data into this table. By adding a mapping rule for the Source column type, you will only add a corresponding column to the staging table. The column will not be mapped to any destination attribute, but it can be further used in a transformation mapping rule.

  • Define source column
    • Source Table: Select the source table from the drop-down list. The list of tables was defined at the previous step. If Alias was specified, it will be shown.
    • Column Type: Select the Source option.
    • Source Column: Select the source column from the drop-down list.
  • Define destination attribute
    • Target Column Name: Enter a name for the target column.
    • Column Type: Select the column type.
    • Column Length: The field is editable for certain column types. Enter the integer number to set the column length.
    • Column Scale: If it is a decimal number column, you can set the quantity of digits after decimal point.
    • Column Precision: If it is a decimal number column, you can set the total quantity of digits.

Mapping Rules for Transformations

The transformation mapping rule allows you to modify values that are being imported into the system and extract data from an attribute that houses numerous properties.

Define source column

  • Source Table: Select the source table from the drop-down list. The list of tables was defined at the previous step. If Alias was specified, it will be shown.
  • Column Type: Select the Transformation option.

Define destination attribute

  • Transform Expression: Specify an ASQL expression. It can use those source columns for which a mapping rule of the Source type has been specified. You can also reference parameters if the import definition has any.

Example: In the figure above the transformation expression uses the user Account Control column. Initially this column is an attribute from Active Directory. It has been previously mapped to a column in the staging table by using the Source mapping rule. The ASQL expression also contains parameters that have been added for the import definition. When a user executes the import definition, they enter values for the parameters. The expression then uses these values and the value of the user Account Control attribute to insert a certain resulting value into the destination attribute in Matrix42 Workspace Management.

Define matching options

  • Limitations of Mapping Rules
    • One source table can be mapped to several destination data definitions, but you cannot map several source tables to one destination data definition.
    • One source column can be mapped to several destination attributes, but you cannot map several source columns to one destination attribute.

Query Builder

In the mapping rule definition dialog, when all source columns have been mapped to destination attributes, you can customize the resulting query that will be used to match source objects with destination objects. Click Query Builder to open the query builder dialog.

As usual, when the dialog opens, it already contains some matching criteria. For a newly created import definition, they are combined by the AND operator. These are mapping rules that have either Use this column to match criteria to the destination type or Use this column to match criteria to the multifragment data definition checkbox selected. However, you can remove all criteria from the query builder and specify a more complex and precise query.

  • AND/OR: Select the logical operator if you have two or more matching criteria.
  • Add Expression: Click the button to add a new field where you can specify an ASQL expression as matching criteria.
  • Add Condition: Click the button to add a drop-down field where you will be able to choose from the added mapping rules. The field will contain only those mapping rules that have the matching criteria checkbox selected.
  • Add Group: Click the button to add a new group of conditions and a logical operator that will connect them. This group will be added into the upper-level group of conditions.
  • Remove Group: Click the button to delete the specific group of conditions and expressions that are connected by the operator.
  • OK/Cancel: Click OK to save the matching criteria or Cancel to leave the criteria unchanged.

Import Action Definition

Review validation details

When you leave the Mapping Rule Definition page, all mapping rules are validated. If any warnings appear, they will be shown in the Review validation details text box.

Validation includes the following checks:

  • If the class has a mandatory attribute without a default value and this attribute is not mapped, then such class cannot be inserted.
  • If the mandatory class cannot be inserted, then the entire object cannot be inserted.

Such warnings do not prevent the definition from execution, but the available actions become limited.

Select destination data definition/relation, which corresponds to one row in the source table

The drop-down list is shown only if you have selected the Multifragment Import option when choosing the source table type.

The complex table contains more than one row per destination object. It means that the information about multifragment class or the 1->N (N->M) relation exists in the table. Thus we can say that each row in this table corresponds to one row in the appropriate class/relation. This class/relation should be selected from the drop-down list.

Example: A table has more than one row per computer because it also contains the hard disk information. Thus each row in the file corresponds to one hard disk, and we select the Hard Disk multifragment.

When an import definition processes a complex table, it identifies records by using the resulting matching query. If it finds another row with the same criteria, it considers it to be the same object. In this case, the import definition ignores all data in such row, except fields that are mapped to the selected multifragment class/relation. In our example, it will be two fields, Model and Total Capacity, that are mapped to the Hard Disk multifragment.

There is also the Any Mapped Multifragment Class option in this drop-down list. If you choose it, then all fields that are mapped to a multifragment class will be processed in a duplicate row.

If you have not mapped any multifgrament classes, but use a complex table, then the duplicate rows will be skipped.

Specify import actions, which should occur for the entire CI and all mapped multifragment data definitions

Here you can customize the exact actions that the import definition can perform. The list contains records for the main configuration item and all mapped multifragment classes. You can specify the required actions separately for each multifragment class (with some limitations).

  • Update: Specify whether you want to update the matched records.
  • Insert: Specify whether you want to insert the non-matched records or skip them.
    • You cannot check this option for the main CI if the Object insert is impossible warning appears in validation details.
    • You cannot check this option for the multifragment class if the Class cannot be inserted warning appears in validation details.
    • If you select this option for the main CI, you should select this option for all mandatory multifragment classes.
  • Delete: Specify whether you want to delete Matrix42 Workspace Management records that do not exist in the source data. It is prohibited for the main CI. For multifragment classes, only records within the matched objects will be deleted.

Specify further attributes that need special processing (optional)

You can also adjust actions on the attribute level. If you want special processing for some attributes, specify them in the appropriate grid. Only the mapped attributes will be shown.

For each attribute, you can select one of the following actions:

  • Insert Only: An attribute's value can be only inserted (when a row is inserted) and is never updated.
  • Update Only: An attribute's value can be only updated (the source value is ignored when a row is inserted).
  • Skip: An attribute's value is neither inserted nor updated (useful if you use it only for matching).
  • Any Action: The same as if you did not specify an attribute at all.

Import Definition Summary

Here you can see overview information about your definition.

  • Summary of the import definition: The source and destination details of the import definition.
  • Name: Enter or edit the name of the import definition. It is saved when you click Next.
  • Copy the import definition: The checkbox is available if you customize an existing definition. After you click Next, a copy of the definition with the specified name is created.
  • Description: Additional description of the import definition.
  • Next: Saves the import definition or creates a copy of the import definition.

Import Definition Execution

This dialog is the last step in creating and running an import definition. It is also shown when you click the Execute Import Definition action.

  • Parameters: If this import definition contains parameters, it is required that you enter parameter values for the definition to be executed. The type of value should match the parameter value type.
  • Select Option: Select the Execute import definition completely option to run the import definition without intermediate steps.
  • Run: Click the button to execute the import definition.
  • Result Preview: When the import is completed, you will see the result in the text box. At the bottom of the page you will see a progress bar, which displays the current import status and helps estimate the remaining time. The color of the progress bar and the result preview value are connected:
    • Complete Import (green): The import was completed without any warnings.
    • Partial Import (yellow): The import was completed with warnings (e.g., some records were skipped, some relations did not match, etc.).
    • Failure (red): The import failed. It means that none of the records were imported.
  • Execution Statistics: Detailed information about the import.
  • Abort: You can interrupt the execution at any time by clicking Abort.

Executing the Import Definition Step by Step

If you select the Execute import definition step by step option in the Select Option field, you will be able to execute the import gradually. This option is recommended because it displays the individual steps and any errors that occur.

  • Fill Staging: Populating the staging tables in the Matrix42 Workspace Management database with the raw data from the data source.
  • Match: Matching the data in the Matrix42 Workspace Management tables with the new data that is imported to the staging tables.
  • Import: Actual import of the new data into the productive tables of Matrix42 Workspace Management.

When you run the definition step by step, the progress is displayed separately for each step, not for the entire definition execution.

  • Was this article helpful?