CMIS Diagnostic Monitoring Overview in SONiC

1. Overview

The CMIS (Common Management Interface Specification) diagnostic monitoring feature is a standard for monitoring the performance of optical transceivers. It provides a way to monitor the performance of optical transceivers in real time. SONiC periodically reads the diagnostic monitoring data from the optical transceivers and stores the data in the database. The data can be retrieved using the SONiC CLI or by querying the database directly.

The current scope of the CMIS diagnostic monitoring feature in SONiC includes the following parameters:

DOM (Digital Optical Monitoring) data: Provides real-time monitoring of optical transceiver parameters such as temperature, voltage, and optical power.
VDM (Versatile Diagnostics Monitoring) data: Offers versatile diagnostic information for enhanced monitoring and troubleshooting.
PM (Performance Monitoring) data: Applicable only for C-CMIS transceivers, this includes performance metrics such as error counts and signal quality indicators.

2. Requirements, future enhancements Non-goals

Requirements

The CMIS diagnostic monitoring feature in SONiC requires the following:

Capturing the diagnostic monitoring data from the optical transceivers and storing it in the database at regular intervals.
Providing a way to retrieve the diagnostic monitoring data using the SONiC CLI as well as by querying the database directly.
Optimizing the periodic update time for the diagnostic monitoring data by storing diagnostic information for only 1 subport from a breakout port group.
Storing the flag change count, last flag set and clear time for each flag in the database during a link change event. This is necessary because the periodic update time for the diagnostic monitoring data may not be frequent enough to capture the flag change event.
Updating the last table update time in the database whenever the diagnostic monitoring data is updated.
Capturing the real diagnostic update interval in the database for each port since this time can vary with multiple factors such as the number of optics plugged in, I2C latency, host handling of diagnostic data and various other factors.

Future Enhancements

Creating a mechanism to store the flag change count and last flag set and clear time in the database so that this data is not lost during xcvrd warm restart or during device warm reboots. The current implementation deletes this data as part of the xcvrd shutdown process.

Non-goals (Exceptions and Scope Beyond the Current Implementation)

Handling Multiple Link Flaps In a Short Period to Update Flag Metadata:
- In the event of multiple link flaps occurring in a short period, the flag change count and the last flag set and clear time may not be accurate.
- The short period refers to a scenario where the periodic update time for the diagnostic monitoring data is not frequent enough to capture the link change events through the DomInfoUpdateTask thread.
- This behavior is due to the flags being clear-on-read latched values and the DomInfoUpdateTask thread being a single-threaded process that updates the flag metadata during link change events and periodically reads diagnostic data from the optics.
Example

The following table illustrates an example scenario where the last flag set time is not updated in line with link flap events:

Table 1: Example Scenario of Flag Metadata Update During Link Flap Events and Periodic Update Time

X refers to a specific type of flag on the module.DB means database.

Time Event Number	Timestamp of Event	DB Update Trigger (Periodic Polling or Link Change Event Handler)	Flag X Value on Module - Before SW Read	Flag X Value on Module - After SW Read	Flag X Value in DB After Reading from Module	Flag X Set Time in DB After Reading from Module	Flag X Clear Time in DB After Reading from Module	Flag X Change Count in DB After Reading from Module	Link Flap Count in APPL_DB
T1	Wed Oct 16 03:46:41 2024	Link DOWN event	Set	Set	Set	Wed Oct 16 03:46:41 2024	Never	1	1
T2	Wed Oct 16 03:46:42 2024	Link UP event	Set	Clear	Set	Wed Oct 16 03:46:41 2024	Never	1	2
T3	Wed Oct 16 03:46:42 2024	Link DOWN event	Set	Set	Set	Wed Oct 16 03:46:41 2024	Never	1	3
T4	Wed Oct 16 03:46:43 2024	Link UP event	Set	Clear	Set	Wed Oct 16 03:46:41 2024	Never	1	4
T5	Wed Oct 16 03:46:44 2024	Link DOWN event	Set	Set	Set	Wed Oct 16 03:46:41 2024	Never	1	5
T6	Wed Oct 16 03:46:55 2024	Link UP event	Set	Clear	Set	Wed Oct 16 03:46:41 2024	Never	1	6
T7	Wed Oct 16 03:47:01 2024	Periodic polling	Clear	Clear	Clear	Wed Oct 16 03:46:41 2024	Wed Oct 16 03:47:01 2024	2	6

Explanation:
- In the Table 1 scenario, the flag X value on the module is set during the link down event at time T1.
- However, the flag set time is not updated even though, multiple link change events were detected.
- The flag clear time in the database is updated during the periodic polling event at time T7 and not during all the link up events at times T2, T4, and T6.
- The flag change count in the database is updated to 2 during the periodic polling event at time T7, even though the flag changed on the module 6 times.
Flag Clear Time Update:
- The clear time of the flag in the database will be updated after the second read of the flag register.
- This is because the flag registers are clear-on-read latched values.
Example:
- In Table 1, consider the time events T6 and T7.
- The flag X value on the module is cleared during the link up event at time T6.
- The flag clear time in the database is updated during the periodic polling event at time T7.

3. STATE_DB Schema for CMIS Diagnostic Monitoring

The CMIS diagnostic monitoring data is stored in the STATE_DB database. Each logical port on the switch has a corresponding entry in the STATE_DB schema unless the device is configured in breakout mode. For devices configured in breakout mode, only the first port of the breakout group will have the diagnostic monitoring data stored in the STATE_DB schema since the diagnostic monitoring data is the same for all ports in the breakout group.

The STATE_DB schema for the CMIS diagnostic monitoring feature includes the following tables:

TRANSCEIVER_DOM_SENSOR: Stores real-time DOM data for the optical transceivers.
TRANSCEIVER_DOM_THRESHOLD: Contains threshold values for DOM parameters.
TRANSCEIVER_DOM_FLAG: Stores flags indicating the status of various DOM parameters.
TRANSCEIVER_DOM_FLAG_CHANGE_COUNT: Keeps a count of how many times each DOM flag has changed.
TRANSCEIVER_DOM_FLAG_SET_TIME: Records the last timestamp when each DOM flag was set.
TRANSCEIVER_DOM_FLAG_CLEAR_TIME: Records the last timestamp when each DOM flag was cleared.
TRANSCEIVER_VDM_REAL_VALUE: Stores VDM sample data.
TRANSCEIVER_VDM_HALARM_THRESHOLD: Stores the high alarm threshold values for the VDM data.
TRANSCEIVER_VDM_LALARM_THRESHOLD: Stores the low alarm threshold values for the VDM data.
TRANSCEIVER_VDM_HWARN_THRESHOLD: Stores the high warning threshold values for the VDM data.
TRANSCEIVER_VDM_LWARN_THRESHOLD: Stores the low warning threshold values for the VDM data.
TRANSCEIVER_VDM_HALARM_FLAG: Stores the high alarm flag for the VDM data.
TRANSCEIVER_VDM_LALARM_FLAG: Stores the low alarm flag for the VDM data.
TRANSCEIVER_VDM_HWARN_FLAG: Stores the high warning flag for the VDM data.
TRANSCEIVER_VDM_LWARN_FLAG: Stores the low warning flag for the VDM data.
TRANSCEIVER_VDM_HALARM_FLAG_CHANGE_COUNT: Keeps a count of how many times each VDM high alarm flag has changed.
TRANSCEIVER_VDM_LALARM_FLAG_CHANGE_COUNT: Keeps a count of how many times each VDM low alarm flag has changed.
TRANSCEIVER_VDM_HWARN_FLAG_CHANGE_COUNT: Keeps a count of how many times each VDM high warning flag has changed.
TRANSCEIVER_VDM_LWARN_FLAG_CHANGE_COUNT: Keeps a count of how many times each VDM low warning flag has changed.
TRANSCEIVER_VDM_HALARM_FLAG_SET_TIME: Records the last timestamp when each VDM high alarm flag was set.
TRANSCEIVER_VDM_LALARM_FLAG_SET_TIME: Records the last timestamp when each VDM low alarm flag was set.
TRANSCEIVER_VDM_HWARN_FLAG_SET_TIME: Records the last timestamp when each VDM high warning flag was set.
TRANSCEIVER_VDM_LWARN_FLAG_SET_TIME: Records the last timestamp when each VDM low warning flag was set.
TRANSCEIVER_VDM_HALARM_FLAG_CLEAR_TIME: Records the last timestamp when each VDM high alarm flag was cleared.
TRANSCEIVER_VDM_LALARM_FLAG_CLEAR_TIME: Records the last timestamp when each VDM low alarm flag was cleared.
TRANSCEIVER_VDM_HWARN_FLAG_CLEAR_TIME: Records the last timestamp when each VDM high warning flag was cleared.
TRANSCEIVER_VDM_LWARN_FLAG_CLEAR_TIME: Records the last timestamp when each VDM low warning flag was cleared.
TRANSCEIVER_STATUS: Stores the module and datapath state data along with various flags related to it. Also stores various Tx and Rx related status values.
TRANSCEIVER_STATUS_FLAG: Stores the module and datapath status flags along with various Tx and Rx related status flags.
TRANSCEIVER_STATUS_FLAG_CHANGE_COUNT: Stores the count of times the transceiver status flag has changed.
TRANSCEIVER_STATUS_FLAG_SET_TIME: Records the timestamp when the transceiver status flag was set.
TRANSCEIVER_STATUS_FLAG_CLEAR_TIME: Records the timestamp when the transceiver status flag was cleared.
TRANSCEIVER_PM: Stores performance monitoring data.

3.1 Transceiver DOM

3.1.1 Transceiver DOM sensor data

The TRANSCEIVER_DOM_SENSOR table stores the real-time DOM data for the optical transceivers.

lane_num: Represents the lane number of the field. The lane number is an integer value that ranges from 1 to 8.

    ; Defines Transceiver DOM sensor information for a port
    key                          = TRANSCEIVER_DOM_SENSOR|ifname    ; information module DOM sensors on port
    ; field                      = value
    table_last_update_time       = STR                              ; last update time for diagnostic data
    temperature                  = FLOAT                            ; temperature value in Celsius
    voltage                      = FLOAT                            ; voltage value in V
    tx{lane_num}power            = FLOAT                            ; tx power in dBm for each lane
    rx{lane_num}power            = FLOAT                            ; rx power in dBm for each lane
    tx{lane_num}bias             = FLOAT                            ; tx bias in mA for each lane
    laser_temperature            = FLOAT                            ; laser temperature value in Celsius

    laser_config_freq            = FLOAT                            ; laser configured frequency in MHz
    laser_curr_freq              = FLOAT                            ; laser current frequency in MHz
    tx_config_power              = FLOAT                            ; configured tx output power in dbm

3.1.2 Transceiver DOM threshold data

The TRANSCEIVER_DOM_THRESHOLD table stores the threshold values for the DOM data.

    ; Defines Transceiver DOM threshold info for a port
    key                          = TRANSCEIVER_DOM_THRESHOLD|ifname ; DOM threshold information for module on port
    ; field                      = value
    table_last_update_time       = STR                              ; last update time for diagnostic data
    temphighalarm                = FLOAT                            ; temperature high alarm threshold in Celsius
    temphighwarning              = FLOAT                            ; temperature high warning threshold in Celsius
    templowalarm                 = FLOAT                            ; temperature low alarm threshold in Celsius
    templowwarning               = FLOAT                            ; temperature low warning threshold in Celsius
    vcchighalarm                 = FLOAT                            ; vcc high alarm threshold in V
    vcchighwarning               = FLOAT                            ; vcc high warning threshold in V
    vcclowalarm                  = FLOAT                            ; vcc low alarm threshold in V
    vcclowwarning                = FLOAT                            ; vcc low warning threshold in V
    txpowerhighalarm             = FLOAT                            ; tx power high alarm threshold in dBm
    txpowerlowalarm              = FLOAT                            ; tx power low alarm threshold in dBm
    txpowerhighwarning           = FLOAT                            ; tx power high warning threshold in dBm
    txpowerlowwarning            = FLOAT                            ; tx power low alarm threshold in dBm
    rxpowerhighalarm             = FLOAT                            ; rx power high alarm threshold in dBm
    rxpowerlowalarm              = FLOAT                            ; rx power low alarm threshold in dBm
    rxpowerhighwarning           = FLOAT                            ; rx power high warning threshold in dBm
    rxpowerlowwarning            = FLOAT                            ; rx power low warning threshold in dBm
    txbiashighalarm              = FLOAT                            ; tx bias high alarm threshold in mA
    txbiaslowalarm               = FLOAT                            ; tx bias low alarm threshold in mA
    txbiashighwarning            = FLOAT                            ; tx bias high warning threshold in mA
    txbiaslowwarning             = FLOAT                            ; tx bias low warning threshold in mA
    lasertemphighalarm           = FLOAT                            ; laser temperature high alarm threshold in Celsius
    lasertemplowalarm            = FLOAT                            ; laser temperature low alarm threshold in Celsius
    lasertemphighwarning         = FLOAT                            ; laser temperature high warning threshold in Celsius
    lasertemplowwarning          = FLOAT                            ; laser temperature low warning threshold in Celsius