Skip to main content
Matrix42 Self-Service Help Center

Workflow Activities Migration Guide

Adjusting the Workflow Activities for running on the new Matrix42 Worker Engine.

Goal

Starting from version 10.0.0 DWP introduces the new Workflow Engine based on Matrix42 Workers. The new concept includes the possibility to scale the execution of the Workflows on many computers located not only on the intranet but also on the internet (including Cloud).  The new architecture of the Workflow Engine puts additional restrictions on the Workflow Activities implementation to guarantee it runs successfully regardless of the Workflow that is running on the Application Server or remotely.

Principals

Matrix42 Worker is a dedicated Windows Service process hosted either on the Application Server or any other computer in an intranet or even the Internet. On Workflow execution the Worker resolves dependent assemblies and automatically downloads them from Server. This mechanism can handle the automatic deployment of the Workflow Activities and their dependent assemblies to Worker, but it does not deliver Business Components to Worker which keeps running only on the Application Server. Therefore you need to review the Custom Activities for referencing Business Components, and if present, replace these references with the Web Service call.

The Matrix42 Workers is an application that runs on an independent process, either on the Application Server or on any other computer. This, compared to the classic AppFabric implementation, brings extra tasks for migration, as previously all Workflow Activities have been executed in-process of the Application and had direct access to all Business Components.

Default Workflow Activities

All standard Workflow Activities delivered with the Product out-of-the-box are already migrated and can be executed on the Matrix42 Worker.

Particularities

Due to the fact, the Workflows can be executed on another process (or even another computer) some standard Workflow Activities changed a little the default behavior, which in some cases need to be taken into consideration when moving execution of the Workflow to the Worker.

Invoke PowerShell Activity

By default, the Activity executes the specified PS Script on the Worker.

For cases when the Matrix42 Worker is installed on the remote computer (or in Cloud), the Activity execution can fail as:

  • the environment does not have required PS Libraries installed
  • it is trying to reach resources that are not accessible on the Worker computer.

Each such case needs to be analyzed and fixed to let the script keep being executed on Worker. If the issue cannot be solved and the script anyway has to be executed on the Application Server, then you need to rework the Workflow and set the property "Execute on Server" for the appropriate Invoke PowerShell activity.

Execute SQL Activities

The System delivers two Workflow Activities "ExecuteSqlNonQuery" and "ExecuteSqlQuery" which run native-SQL queries on Databases specified with the ConnectionString. For cases when the Production database is used and the connection string is referenced by the name "m42store". For security reasons, the Connection String is not distributed to the remote Matrix42 Worker is the SQL Authorization is used (username and password defined in the connection string).  Connection Strings with Integrated security can be used on Matrix42 Workers installed on the same intranet with the Application Server. 

Custom Workflow Activities

If you created your own custom Workflow Activities, all of them need to be reviewed for compatibility with the Matrix42 Worker. For more details please follow the steps described in the Workflow Activities Migration section below.

Workflow Activities Migration

The entire Worker Engine migration process including custom Workflow Activities migration examples are shown in details on the video below: 

Workflow Activity Compatibility

All incompatible custom Workflow Activities must be reviewed because the Workflows that use such custom Workflow Activities cannot be migrated to the Worker technology. See also Workflow Engine Migration Guide.

Workflow Studio

To be able to be executed on the Matrix42 Worker, the Activity has to be compatible with it, which means the Activity either has all the required modules for the execution deployed on the Matrix42 Worker process, or the Activity is able to delegate the execution back to the Application Server via a Web Service method call. To signal the System uses the Workflow Activity property PLSLBinaryComponentClassBase.WorkerCompatible to signal that the Workflow Activity can be executed on the Matrix42 Worker. 

In the Workflow Studio, the Activities that are compatible with Matrix42 Worker are highlighted as follows:

WFCompatible.png

It provides better transparency to the Author and shows why the Workflow cannot be executed by the Matrix42 Worker.

Administration application

Incompatible Workflow Activities in the Administration application are marked as follows:

WF_activity_incompatible1.png

Workflow Activity implementation

To change the custom Workflow Activity implementation you need to edit and adjust the code. Manually copy the example from the article and adjust the code according to the Workflow Activity particularities:

  1. Remove reference to the Data Layer in the Workflow Activity project and replace it with Data Layer operation;
  2. Change the conditions and data sources;
  3. Adjust the Workflow Activity to call Web API methods;  
  4. Adjust the access to files that are not on the Application Server.  See also File Exists Workflow Activity example;
  5. Recompile the WF activity;
  6. Upload new assembly;
  7. Mark the Workflow Activity as Matrix42 Worker compatible;
  8. Run Workflow with Custom Activity on Matrix42 Worker.

Using Data Layer operation in Activity

The Workflow Framework provides the approach for working with the Production database fro the Workflow Activities.  Depends on Matrix42 Worker installation case the module transforms the database request directly over the Data Layer (Matrix42 Workers installed in intranet) or triggers REST web service for data operation on the Application Server when the Worker installed on the internet (Cloud or Data Gateway ). Using this practice guarantees the best option is automatically picked depending on the deployment model.

The Workflow Engine infrastructure provides a few components that are dealing with the data and schema operation.  

Interface Namespace Description
IDataReaderProvider Matrix42.Contracts.Platform.Data Provides the basic functions for reading data from the Production database
IDbProvider Matrix42.ServiceRepository.Contracts.Components Provides the functionality for CRUD operations
ISchemaReaderProvider Matrix42.Contracts.Platform.Data Keeps the methods for retrieving schema metadata information

The instances of these components are initialized on Workflow Engine start and could be received over the Dependency Injection mechanism

protected override void Execute(NativeActivityContext context)
{
    var executor = context.GetExtension<IExtensionExecutor>();
    var schemaReader = executor.Get<ISchemaReaderProvider>();
    var dataProvider = executor.Get<IDataReaderProvider>();
    var objectsData = dataProvider.GetDataList("SPSCommonClassBase", "[Expression-ObjectID] AS ObjectID, TypeID AS TypeID",
        string.Format("[Expression-ObjectID] IN ({0})", ids)
    );
}

Calling Web API methods from the Workflow Activity

If your Workflow Activity uses resources that are present only on the Application Server, then it needs to be reworked to use Web Services instead to guarantee the Activity is correctly executed regardless of Matrix42 Worker location it is running on. 

Create a Custom Web Service

If the required functionality is not exposed over the standard Web Service, you need to create a custom Web Service. For more details, see Register Custom Web Service.

Call a Web Service

The Web Service which already registered in the Solution Builder Web Service Repository can be triggered using the special component for running Web Methods which already properly initialized on Workflow Engine start, and can be retrieved from the container of the component over the Dependency Injection mechanism addressing the interface Matrix42.Http.Client.Contracts.IWebApiClient. The component exposes a few methods which help to generate and send Web Request to Web Server, and then handle the response.

T Run<T>(Guid operationId, Argument[] arguments = null, object data = null, int? lcid = null);
JToken Run(Guid operationId, Argument[] arguments = null, object data = null, int? lcid = null);
void Call(Guid operationId, Argument[] arguments = null, object data = null);

Input arguments:

  • operationId (Guid) - Id of the Web Service operation in the Services Repository (PLSLWebServiceOperation.ID);
  • arguments (Matrix42.Http.Client.Contracts.Argument[]) - list of the operation incoming argument values;
  • data (object) - for POST and PUT requests defines the request body;
  • lcid (Integer) -  Optional. Sets the Locale of the Client;

Response:

  • <T> - The Web  Response is casted to specified .NET Type ;
  • JToken - the raw response  in a form of JSON;

The following code snippet demonstrates how the WebApiClient component can be used for triggering GDIE job on Server

using Matrix42.Http.Client.Contracts;

protected override void Execute(NativeActivityContext context)
{
    var executor = context.GetExtension<IExtensionExecutor>();
    var proxy = executor.Get<IProxyContainer>();
    IWebApiClient client = proxy.GetWebApiClient();
    client.Run(new Guid("c04a51a2-a720-c94c-e965-08d77a3ca264"), null, new
                    {
                        SequenceId = sequenceId,
                        InputFile = @"C:\Temp\data.xml"
                    });
}

Upload new assembly

Open Administration application → Services & ProcessesWorkflowsManage AssembliesActionsUpload Assembly. Select the Assembly from the development machine and upload it to the Application Server, click to select the checkbox “Directly Publish Assemblies":

WF_upload_assembly1.png

Set Workflow Activity compatible with Matrix42 Workers

Execute the SQL script on the SQL Server using the Technical Name of the Workflow Activity:

WF_activity_technical_name.png

You need implicitly set in the Production database a flag ([PLSLBinaryComponentClassBase].[WorkerCompatible]) for each Workflow Activity which indicates the Activity is "compatible" with the Matrix42 Workers Engine. 

update PLSLBinaryComponentClassBase
SET WorkerCompatible = 1
From PLSLBinaryComponentClassBase B
    inner join PLSLComponentClassBase C On C.[Expression-ObjectID] = B.[Expression-ObjectID] AND
    C.Name = '<Name of Custom Activity>'

Now the Workflow Activity will be marked as Worker compatible. To check it, open the Workflow Activity preview page.

Run Workflow with Custom Activity on Matrix42 Worker

To validate the Workflow Activity is working correctly on Matrix42 Worker, you need to "Set Execution Engine" to Worker for one of the Workflows which uses the Activity.

Make also sure the System is configured in Administration Settings (tab Workflows) to use the Matrix42 Worker engine for Worker compatible workflows. 

Start Workflow execution and assure it finished successfully and the targeted Activity was executed. 

If Workflow is part of your Custom Package and could be installed on multiple environments then the package needs to be adjusted to include the information that the Workflow could be executed on Matrix42 Worker. For that the flag [PLSLXamlComponentClassBase].UseWorkflowWorker needs to be set.

Key learnings

  • Use DataReader component instead of direct access to the database
  • Call web services via IWebApiClient instead of using Service Locator and Web API Service Layer
  • Move all business logic from Workflow Activities to REST Services
  • Remember that Worker may run not only on App Server, so all resources used should be available on all Worker machines (relevant for PowerShell Activities that require some cmdlets, make sure these cmdlets are installed on every machine that has Worker installed OR mark the PowerShell Activity as to be executed on the Application Server only)

Example: "File Exists" Workflow Activity

The activity checks the presence of the file of the Application Server. At the original, it has a very simple implementation as Workflow on AppFabric always running on Application Server:

public sealed class FileExists : NativeActivity
{
    public InArgument<string> Path { get; set; }
    public OutArgument<bool> Result { get; set; }

    protected override void Execute(NativeActivityContext context)
    {
        string path = context.GetValue(Path);
        context.SetValue(Result, File.Exists(path));
    }
}

 As far as the Matrix42 Worker could be installed remotely the original implementation could not be valid as it will check the File on the Matrix42 Worker computer, not on the Application Server. To make this Activity compatible with the Matrix42 Worker includes the following steps:

  1. Try to find a Web Service that implements the logic. See Web Services: REST API integration.
  2. If the Web Service is not present in the Product, then introduce a custom Web Service. For more details, see Register Custom Web Service.
  3. Rework the Workflow Activity code and recompile the Workflow Activity. See the code example below.
  4. Mark "File Exists" Workflow Activity as Compatible with Matrix42 Worker. See, Set Workflow Activity compatible with Worker
  5. Ensure the "File Exists" works, as expected on a Matrix42 Worker, installed remotely. See, Run Workflow with Custom Activity on Worker
protected override void Execute(NativeActivityContext context)
{
      string path = context.GetValue(Path);

      bool result;
             
      // Get the current execution engine
      var workerContext = context.GetExtension<IWorkflowContextExtention>()?.CurrentContext as IWorkflowWorkerContext;
      WorkflowEngine engine = workerContext?.Engine ?? WorkflowEngine.AppFabric;

      //Retrieve the WebAPI Client
      var executor = context.GetExtension<IExtensionExecutor>();
      var proxy = executor.Get<IProxyContainer>();
      IWebApiClient client = proxy.GetWebApiClient();
            
      if (engine != WorkflowEngine.AppFabric)
      {
          //Custom Web Service 
           result = client.Run<bool>(new Guid("3E73BB20-4399-40D5-979B-43177FD11713"));
      }
      else
      {
           result = File.Exists(path);
      }

      context.SetValue(Result, result);
} 

AppFabric vs Matrix42 Workers

Also, you need to take into consideration that meanwhile the AppFabric Workflow Engine is not fully discontinued (does not happen in 2020), depending on the Product Environment configuration the Workflow Activity can be simultaneously executed as on Matrix42 Worker as on AppFabric. Therefore, at least for some period of time, the Activity implementation should consider both cases.

  AppFabric Matrix42 Workers
Hosted Always on Application Server Anywhere (App Server, Intranet, Internet, Cloud)
Using  Business Components or Server Resources
  • Direct Reference
  • Over the WCF Service Layer (generation 2)

Web Service call over Web API Service Layer (generation 3)

Using Datalayer
  • Direct Datalayer Referencing
  • Over IDbProvider, IDataReaderProvider interfaces
  • Over IDbProvider, IDataReaderProvider interfaces
  • Direct Web Service call to Generic Data Service

The Workflow Execution context provides a simple approach to figure out the run-time Workflow Execution engine. The following code snippet demonstrates how to obtain this information from the Activity context. 

using Matrix42.Workflows.Contracts;
using Matrix42.Workflows.Contracts.Process;

protected override void Execute(NativeActivityContext context)
{
    var workerContext = context.GetExtension<IWorkflowContextExtention>()?.CurrentContext as IWorkflowWorkerContext;
    WorkflowEngine engine = workerContext?.Engine ?? WorkflowEngine.AppFabric;
}

The enumeration Matrix42.Workflows.Contracts.Process.WorkflowEngine  keeps the following values:

  1. AppFabric (1) - the Workflow is running on AppFabric Workflow Engine. 
  2. Matrix42 Workers (2)  - the Workflow is running the Matrix42 Worker engine.
  3. Cloud Workers (3) - the Workflow is executing on the Matrix42 Worker hosted in the Cloud (e.g. Microsoft Azure). In general, this option is identical to "Matrix42 Workers", but for some special cases, it could be considered.
  • Was this article helpful?