Data Collection
A guide to data collection methods to ensure you are receiving the richest set of system events and data possible.
Overview
In order to ensure you are receiving the richest set of system events and data possible, FireScope offers a variety of data collection types to choose from. You can select data collection methods on a host-by-host basis. For instance, data on network equipment can be collected through SNMP, while servers communicate through rich agents. Not all data collection methods offer the same range of data types, and each has its own set of limitations and prerequisites. The table below will guide you through the available collection methods to help you strategize the best deployment for your unique environment.
| Method | Requirements | Additional Information |
| Agent Remote | A type of FireScope agent data access where agent sends data back to the Edge device periodically vs. utilizing the poller service from the Edge device. | |
| Attribute Incoming | This is also referred to as a Dynamic Data Attribute | |
| Attribute JSON Incoming | This is also referred to as a Dynamic Data Attribute | |
| Calculated Attribute | You can use this attribute to combine other attributes’ values. Configuring a Calculated Attribute | |
| Cisco UCS | Requires CI to have a link to Cisco UCS Credential | Collect metrics using Cisco UCS API |
| Derived Check | Internal metrics such as number of attributes or event definitions. | |
| FireScope Agent | Agent must be installed on device. Currently available for most operating systems, including Windows, Linux, Unix, BSD. | This method offers the largest scope of information collection. It includes log data from any log file on the host machine as well as system performance data and direct database connectivity.See Installing an Agent. |
| Grouped Check | Calculated values within a logical group of CI’s for a given an attribute’s operation Grouped Check Attribute | |
| Infoblox Attributes | Requires CI to have a link an Infoblox API credential | Collect network information defined in Infoblox Manager (ex. network ranges, etc) |
| LDAP Check | Requires CI to have a link to LDAP credential | Collect metrics using LDAP queries |
| NetApp Check | Requires CI to have a link to NetApp ONTAP credential | Collect metrics directly from a NetApp Filer using ONTAP API |
| NetApp DFM Check | Requires CI to have a link to NetApp DFM credential | Collect filer volume metrics from a NetApp DFM |
| Percentile Attribute | Value of other attributes at specified percentile for a specific time window in days. (information on percentile calculations) | |
| SNMP Trap | Collects messages forwarded to FireScope from SNMP devicesConfiguring an SNMP Trap Attribute | |
| SNMP v1 | Device must be SNMPv1 compatible or have an SNMP agent installed. This includes most networked assets. | The SNMPv1 SMI specifies the use of a number of SMI-specific data types, which are divided into two categories:Simple data typesApplication-wide data types |
| SNMP v2 | Device must be SNMPv2 compatible or have an SNMP agent installed. | The SNMPv2 SMI is described in RFC 2578. It makes certain additions and enhancements to the SNMPv1 SMI-specific data types, such as including bit strings, network addresses, and counters. Bit strings are defined only in SNMPv2 and comprise zero or more named bits that specify a value. Network addresses represent an address from a particular protocol family. SNMPv1 supports only 32-bit IP addresses, but SNMPv2 can support other types of addresses as well. Counters are non-negative integers that increase until they reach a maximum value and then return to zero. In SNMPv1, a 32-bit counter size is specified. In SNMPv2, 32-bit and 64-bit counters are defined.Additionally, SNMPv2 also specifies information modules, which specify a group of related definitions. Three types of SMI information modules exist: MIB modules, compliance statements, and capability statements.MIB modules contain definitions of interrelated managed objects.Compliance statements provide a systematic way to describe a group of managed objects that must be implemented for conformance to a standard.Capability statements are used to indicate the precise level of support that an agent claims with respect to a MIB group. An NMS can adjust its behavior toward agents according to the capabilities statements associated with each agent. |
| SNMP v3 | Device must be SNMPv3 compatible or have an SNMP agent installed. | Essentially offers the same information as SNMPv2, with the addition of 3 important security features:Message integrity to ensure that a packet has not been tampered with in transit.Authentication to verify that the message is from a valid source.Encryption of packets to prevent snooping by an unauthorized source. |
| Syslog Message | Device must have Syslog agent, which is common on most non-Microsoft operating systems. | Limited to data stored in log files, which does not include most system performance metrics. However, on most platforms the administrator can define what information to log. |
| TCP Check | Collects information by connecting via IP and Port. | |
| Simple Check | No requirements. Performed directly through the FireScope SPM appliance. | A simple check is ping testing to verify a host is reachable. Available/Unavailable and response times are the only types of information this method can collect. |
| VM Guest | Requires CI to have a link to VM credential | Collects metrics regarding VM Guests |
| VM Host | Requires CI to have a link to VM credential | Collects metrics regarding VM Hosts |
| VM Virtual Center | Requires CI to have a link to VM credential | Collects metrics regarding VM Virtual centers |
| Web Monitoring (User Experience Checks) | No requirements. Performed directly through the FireScope SPM installation. | Web monitoring simulates a user experience on a web-based application and measures response and download times.See User Experience |
Data Collection Settings
Data Retention
You can control how long historical data is stored in the SPM database. This affects how much storage space is required for saving the history. This will also affect how much storage space is required for creating backups.
Attribute Data Retention by Type* Numeric (float, 1.4, 235.2): (default: 7 Days)
- Numeric (integer 64bit): (default: 7 Days)
- Character: (default: 14 Days)
- Text: (default: 14 Days)
- Log: (default: 14 Days)
- JSON: (default: 14 Days)
- Trends older than: days (default: 90 Days)Trends are hourly and daily averages of numeric (float and integer) data stored separately from detailed data points. Each average entry also keeps track of maximum, minimum, average and number data points collected. This allows data to be stored for longer periods of time and still be usable for analysis.
Notifications and Events Retention* Notifications older than: days (default: 90 Days)
- Events older than: days (default: 30 Days)
- SLA older than: days (default: 90 Days)SLA trends are hourly and daily averages of availability percentage data calculated hourly and average daily. Each average entry also keeps track of number of Ok events, Failed events, Unkown events, up-time, down-time, business impact and any maintenance window results.
Log Retention by Type* Audit logs older than: days (default: 90 Days)
- Reports older than: days (default: 30 Days)
- ServiceNow CMDB auditlog older than: days (default: 30 Days)
- SNMP trap: (default: 90 Days)
- Syslogs older than: days (default: 90 Days)
- System messages older than: days (default: 90 Days)
- Windows event logs older than: days (default: 90 Days)
Value Translation
For UI displays of data, you can assign value translation to be associated with attributes. For example, ‘Ping’ values of 0 and 1 can be associated to display.
0 ⇒ Down 1 ⇒ Ok
Value translation are helpful to provide readable formats corresponding numeric status or codes. Often status values are dependent on individual vendor APIs and protocols. For example, for VMWare API, the following are some mappings:
| VM Guest State | reset ⇒ Reset suspended ⇒ Suspended poweredOn ⇒ Powered On poweredOff ⇒ Powered Off |
| VM Host State | connected ⇒ Connected disconnected ⇒ Disconnected notResponding ⇒ Not Responding |
| VM Tools Information | toolsNotInstalled ⇒ Not Installed toolsNotRunning ⇒ Not Running toolsOld ⇒ Out of Date toolsOk ⇒ Ok |
These values can be populated either manually by users, imported with CI definition XMLs, or results from discovery jobs.
Applying Translation to Attributes
When configuring an attribute, change the ‘Show value’ setting under Attribute Display Options.
Flexible Collection Schedule
Data collected by Attributes can be collected at flexible intervals.
To set the Flexible Collection Schedule:# Log in to FireScope SPM.
- Click Configuration > CIS > Attributes. The Attributes page displays the Attributes for the selected Configuration Item (in the top left corner).
- In Navigate or Search for a description, select the CI you want to create the Attribute for.
- Click Create (in the top right corner). The Create Attribute page will be displayed.
- Click on the Flexible Intervals, Flexible Intervals section will be displayed.
- Select Delay Period, Days of Week, Start Time, End Time or select Preset options
- Click Add to set the Flexible Intervals.
- Complete rest of the form.
- Click on the Save button.