Skip to main content
Matrix42 Self-Service Help Center

Test Management System (TMS)

How to Create and Configure Test Environment

TMS requires a Test Environment as a prerequisite for all activities.

Let’s describe how to create a Test Environment.

1. Register the desired build:

·         Open TMS Console → Administration → Development à Test Environment Management à Build Definitions and click New.

·         Enter Name and Type.

·         (IMPORTANT) Add Download path (win share path where the desired build resides).                        Example: \\192.168.1.30\alpha\ServiceStore\Builds\BPM Main Continuous

·         (IMPORTANT) Add Source control path. TFS specific branch to desired build.

Example: $/Matrix42/ServiceStore/Main

clipboard_e9626407d489026804ca0b7720fd17b35.png

2. Register a virtual machine:

·         Open TMS Console → Administration → Development à Test Environment Management → Virtual Machines → New.

·         Enter all required data about your virtual machine:

Host – Actual DNS name of machine;

Virtual Host – Virtual machine server name (ESX Server, for example).

clipboard_e9eec7aad288c51f5b59a305aebf72b04.png

3.1. Register a Test Environment:

 

* Administration → Development à Test Environment Management → Test Environments → New.

* In property dialog define all required data regarding environment: environment name, installation path (where service store is or should be installed), application server, DB server, build definition (e.g., BPM Main Continuous), etc. Use data that has been prepared in previous steps

clipboard_e40d18cfaa542a76ee95e4a5abadf183b.png

3.2. Add additional Update configuration (needed only for Update):

·         Go to the Setup Configuration tab. Select checkboxes if you want to skip some steps during the update.

clipboard_e7f1e46539d287a0bad6a2f192b850e52.png

Specify the Domain Controllers settings:

·        For LDAP tests (or for custom tests if required) you will need to specify LDAP settings.

·        (IMPORTANT) ServiceAccount AND ServiceAccountPassword must be set if any test cases will be run.

clipboard_eadff68a60dc87af0645afc41796f7159.png

 

 

 

How to Run Tests and check results

Test Management System (TMS) provides a possibility of automatic Test Environment update both manually (by action) and automatically by scheduled engine activation.

Let’s describe basic required configurations.

           

Running Tests

 

1.      To run now (from Test Environment):

 

·         Open TMS Console → Administration → Test Environment Management → Test Environments.

·         Select a Test Environment or open its property dialog and click the Run Test Plan action.

clipboard_ec6d017e75810e4c93c9040f63451b90f.png

·         In the opened wizard, select the needed Test Plan and click the Finish button to run the test.

clipboard_e092bdb55d980bd392e8a123bf858c861.png

1.      To run now (From Engine Activations):

 

·         Open TMS Console → Administration → Services and Processes → Engine Activations.

·         Create a new Activation.

·          Select the “RunTestActivation” engine to be activated and the “EngineWF_RunTest” workflow. Then choose the relevant Test Environment and Test Plan and save the new activation.

 

clipboard_eadc7cb2df7b6a15c95714f0b923f0dff.png

 

·         After the new Engine Activation is created, you are able to run it just by clicking the Activate action.

clipboard_eb3bf833e3a17e365f7d56d9080da04ee.png

1.      To schedule a test run:

·         Execute steps from paragraph 2 and add a schedule for your Engine Activation.

clipboard_e5d2b49da5c7a3047bb0ececfc556d64d.png

1.      How to check results:

 

a.      Checking Results of Test Run

There are two ways to view Test Execution results:

·         Open TMS Console → Administration → Test Management → Test Executions.

·         Open the relevant Test Plan property dialog. These results are related to the selected plan.

 

a.      Checking Results of Individual Tests

 

To check results of tests:

·         Open TMS Console → Administration → Test Management → Test Execution Results.

·         Open the Results tab of the Test Executions property dialog.

Details of the test result provide additional information:

o   Test Name

o   Status

o   Start Date

o   End Date

o   Result (Out of PowerShell test).

 

clipboard_e16e960eef3e51e1552230c2c1f3aaf5c.png

 

How to Register Test Plan

TMS: How to register Test Plan

 

 

Test Management System (TMS) provides a possibility of automatic Test Environment update both manually (by action) and automatically by scheduled engine activation.

Let’s describe basic required configurations.

           

1. Registering Test Plan in TMS

·         Open TMS Console → Administration → Test Management →Test Plans.

clipboard_e3a5e112aa93c0ba0031a5fd7194b0bc3.png

 

 

 

 

·         Open a new Test Plan dialog by clicking “New” button.

clipboard_e2185ff54b64747e6e102d5f7fe174d54.png

 

·         Provide the following information:

-          Name. Recommended to be unique.

-          Description

-          Filter. Expression which will be used to select tests (by file name) that should be run with this test plan. It is used to group tests located at the same folder by different test plans.

Examples: *.ps1, WM*LDAP*.ps1, WM*LDAP*import*.ps1

 

-          File Path. Path to the test files in TFS under your branch (branch you will use).

Example:  /Test/PSAutomation/LDAPConnector, /Test/PSAutomation/Workflows

 

-          Disable TFS Sync (checkbox). In case TFS is down, select this checkbox and put your test files to TMS environment to folder path like C:\< File Path>\. This function will skip sync with TFS.

 

-          Capacity DataSource Service URL. In case you want to see analytics on your executed tests in relation to the expected coverage of your Test Plan, you need to create a Burndown chart that uses PushAPI data source. URL to a Push Service of this chart should be specified here.

 

clipboard_ee537d32d322d03b056e0717c72365674.png

 

-          Expected Coverage. Count of tests needed to cover this test plan functionality.

 

2.      Reviewing Test Results for a Test Plan

The Test Executions grid provides information of test runs (with result) for all test plans on all environments.

 

How to Update Test Environment 

 

TMS provides a possibility of automatic Test Environment update both manually (by action) and scheduled by engine activation.

Let’s describe basic required configurations.

1. Register the desired build:

·         Open TMS Console → Administration → Development à Test Environment Management à Build Definitions and click New.

·         Enter Name and Type.

·         (IMPORTANT) Add Download path (win share path where the desired build resides).           

Example: \\192.168.1.30\alpha\ServiceStore\Builds\BPM Main Continuous

·         (IMPORTANT) Add Source control path. TFS specific branch to desired build.

Example: $/Matrix42/ServiceStore/Main

clipboard_ec46626ba2f94420f7dbb2c8dad92ec11.png

 

2. Register a virtual machine:

·         Open TMS Console → Administration → Development à Test Environment Management → Virtual Machines → New.

·         Enter all required data about your virtual machine:

Host – Actual DNS name of machine;

Virtual Host – Virtual machine server name (ESX Server, for example).

clipboard_e52a4f7f4f7c1411e69b05223109f9056.png

 

 

                       

3.1. Register a Test Environment:

·         Administration → Development à Test Environment Management → Test Environments → New.

·         clipboard_eca14e5977310f0e728c05949d5e3139a.pngIn property dialog define all required data regarding environment: environment name, installation path (where service store is or should be installed), application server, DB server, build definition (e.g., BPM Main Continuous), etc. Use data that has been prepared in previous steps.

           

 

3.2. Add additional Update configuration (needed only for Update):

·         Go to the Setup Configuration tab. Select checkboxes if you want to skip some steps during the update.

           

3.3. Specify the Domain Controllers settings:

·        For LDAP tests (or for custom tests if required) you will need to specify LDAP settings.

·     clipboard_e319bf6b5d09998b2e2f050cce82bf551.png

 

   (IMPORTANT) ServiceAccount AND ServiceAccountPassword must be set if any test cases will be run.

clipboard_ef60902fc04ce0b0aed1b058b70533f48.png

4. Starting Update

4.1. Start an update using the action:

·        Select the required Test environment and run the “Start Setup” action.

clipboard_e608d1c77ec0174cd76a3bc9e05da016b.png

·        In the “Select Setup Type” drop-down field, select “Update” and click “Finish” to start the update.

 

clipboard_e8dd5be3d2de0467b0c1eb89a6c783661.png

 

 

4.2. To start an update via an Engine Activation:     

·        Go to Administration à Services & Processes à Engine Activations àNew.

·        Add new engine activation.

·        Select  the “EngineWF_UpdateBuild” Workflow to be run, the “UpdateContinousActivation” engine to be activated, and test environment.

clipboard_ea82f195785170acafb6c92ca0e258431.png

           Configure other engine activation parameters, such as scheduling, notes etc.

·        clipboard_eef93f1122398e41116d2d6a7c7ec8c5e.png

 

       Now you can start the update by running “Activate” Action, or just wait for the scheduled time.

 

                                   

 

 

 

 

How to prepare VM to work with TMS

How to prepare Host (Virtual Host) to work with TMS as Test Environment.

 

 

Test Management System (TMS) provides a possibility of automatic Test Environment update both manually (by action) and automatically by scheduled engine activation.

 

Using Virtual Host as Test Environment on TMS has the following prerequisites for Client OS (Virtual Host) and TMS Server.

 

Prerequisites for the Client Computer (Virtual Host)

1.      All WPM prerequisites are installed.

2.      The following commands should be executed to successfully run PowerShell scripts and open a PowerShell remote session:

-          Enable-PSRemoting –Force

-          Set-ExecutionPolicy Unrestricted

-          Enable-WSManCredSSP -Role server

Points 1 and 2 should be a part of snapshot of your Virtual Host. It will be used to revert your Test Environment for automatic installation of newest build.

3.      If you want to use Test Environment for an already installed WPM, you need to check SPS.config at <InstallationDirectory>/bin folder. The “serviceHost” parameter at the “Matrix42.ServiceLayer” section should contain FQDN of virtual host instead of “localhost”.

 

Example:
(<Matrix42.ServiceLayer    
    servicesHost="store10.neon.qa.matrix42.de"    
    discoveryHostName="store10.neon.qa.matrix42.de"   
     useHttps="False"    
     appFabricAdminGroup="store10\as_administrators"  
   useServiceAccountForAppFabricService="True" />
 )

 

Prerequisites for the TMS Server

1.      For the TMS Server to recognize your Virtual Host as a trusted host, you need to execute the following script on TMS server:

Enable-WSManCredSSP -Role Client -DelegateComputer <VIrtualHostFullDomainName>

How to run tests locally, without deploying to TMS

TMS: How to run tests locally, without deploying to TMS

 

 

Test Management System (TMS) provides a possibility of automatic Test Environment update both manually (by action) and automatically by scheduled engine activation.

 

To run your tests locally:

1.      Get scripts from Source Control  (Main branch location: $/Matrix42/ServiceStore/Main/Test/PSAutomation)

Required tests: AllTestExecutor.ps1, EnvironmentConfiguration.ps1, Helpers folder with all files, and folder with tests you want to run.

 

clipboard_ee9ff96e1ad00ebf22f5ec9570e232e39.png

 

2.      Place all these files and folders in some folder on local environment where you want to run them.

 

 

3.      Modify “AllTestExecutor.ps1” file by providing name of your folder with tests.

clipboard_ede4537c76a118995ba5a4daf3966ac5a.png 

4.      Open “EnvironmentConfiguration.ps1” file and provide all data you need for tests.

Required variables:  

$global:InstallationPath – path to the “Matrix42 Workplace Management” installation directory

(e.g., "C:\Program Files (x86)\Matrix42\Matrix42 Workplace Management")

$fileScriptFilter – filter by file name (e.g., "*User*.ps1" or "WM-*.ps1")

 

clipboard_efed24088b758ecd1cbd63633e366f7e6.png

 

5.      Run “AllTestExecutor.ps1” file.

How to debug Test Environment Update using Visual Tracking

TMS: How to debug Test Environment Update using Visual Tracking

 

 

Update environment Workflow consists of separate logical steps, which provides possibility of easy debugging using Visual Tracking in Workflow Studio.

Let`s simulate a situation when an update fails. For example, Matrix42 certificates are missing. If an update is run after deletion of certificates from local storage, the workflow will fail.

·        Run Update by activating the relevant engine activation or by clicking the Start Setup action.

·        Open TMS console à Administration à  Workflows à  Activity Monitor.

Here you can see “AutoDeploy_UpdateContinuousBuild” failed workflow.

·        Open the Workflow instance to see the concise exception:

clipboard_efcf16b4b2a9420d1d35a246f231e5271.png

·        Click the Visual Tracking action to open WFS debugging tool to see on which step workflow have failed.

clipboard_eb1f577a38f2864cf02ffd715459f50d5.png

As you can see, workflow failed on Checking Certificates Stage, which is correct in this case.

This way it is possible also possible to debug any Test Plan execution.

 

  • Was this article helpful?