[debug dump util] COPP Module Added #1670

vivekrnv · 2021-06-11T19:13:38Z

Signed-off-by: Vivek Reddy Karri vkarri@nvidia.com

What I did

HLD for Dump Utility: HLD.

Added the COPP Module to the Debug Dump Utility
Added the Corresponding UT's

How I did it

How to verify it

tests/dump_tests/module_tests/copp_test.py::TestCoppModule::test_all_args PASSED [ 85%]
tests/dump_tests/module_tests/copp_test.py::TestCoppModule::test_custom_trap_with_default_grp PASSED [ 85%]
tests/dump_tests/module_tests/copp_test.py::TestCoppModule::test_custom_trap_with_missing_group PASSED [ 85%]
tests/dump_tests/module_tests/copp_test.py::TestCoppModule::test_default_grp_and_custom_trap_with_diff PASSED [ 85%]
tests/dump_tests/module_tests/copp_test.py::TestCoppModule::test_default_trap_and_default_grp PASSED [ 85%]
tests/dump_tests/module_tests/copp_test.py::TestCoppModule::test_default_trap_and_default_grp_with_diff PASSED [ 85%]
tests/dump_tests/module_tests/copp_test.py::TestCoppModule::test_invalid_trap_id PASSED [ 85%]
tests/dump_tests/module_tests/copp_test.py::TestCoppModule::test_missing_appl PASSED [ 85%]
tests/dump_tests/module_tests/copp_test.py::TestCoppModule::test_missing_asic_dump PASSED [ 85%]
tests/dump_tests/module_tests/copp_test.py::TestCoppModule::test_missing_state PASSED [ 86%]

Previous command output (if the output of a command-line utility has changed)

New command output (if the output of a command-line utility has changed)

from dump.plugins.copp import Copp 
m_copp = Copp()
params = {Copp.ARG_NAME : "sample_packet", "sample_packet"  : "" }
dict = m_copp.execute(params)
print(dict)

{
    "CONFIG_DB": {
        "keys": [
            "COPP_GROUP|queue2_group1"
        ],
        "tables_not_found": []
    },
    "APPL_DB": {
        "keys": [
            "COPP_TABLE:queue2_group1"
        ],
        "tables_not_found": []
    },
    "ASIC_DB": {
        "keys": [
            "ASIC_STATE:SAI_OBJECT_TYPE_HOSTIF_TRAP:oid:0x220000000004de",
            "ASIC_STATE:SAI_OBJECT_TYPE_HOSTIF_TRAP_GROUP:oid:0x110000000004db",
            "ASIC_STATE:SAI_OBJECT_TYPE_POLICER:oid:0x120000000004dc",
            "ASIC_STATE:SAI_OBJECT_TYPE_QUEUE:oid:0x150000000002a1",
            "ASIC_STATE:SAI_OBJECT_TYPE_HOSTIF_TABLE_ENTRY:oid:0x230000000004d8",
            "ASIC_STATE:SAI_OBJECT_TYPE_HOSTIF:oid:0xd0000000004d6"
        ],
        "tables_not_found": []
    },
    "STATE_DB": {
        "keys": [
            "COPP_TRAP_TABLE|sflow",
            "COPP_GROUP_TABLE|queue2_group1"
        ],
        "tables_not_found": []
    },
    "CONFIG_FILE": {
        "keys": [
            "COPP_TRAP|sflow",
            "COPP_GROUP|queue2_group1"
        ],
        "tables_not_found": []
    }
}

admin@sonic:~$ dump state copp bgp
{
    "bgp": {
        "CONFIG_DB": {
            "keys": [],
            "tables_not_found": []
        },
        "APPL_DB": {
            "keys": [
                {
                    "COPP_TABLE:queue4_group1": {
                        "queue": "4",
                        "trap_action": "trap",
                        "trap_ids": "bgp,bgpv6,lacp",
                        "trap_priority": "4"
                    }
                }
            ],
            "tables_not_found": []
        },
        "ASIC_DB": {
            "keys": [
                {
                    "ASIC_STATE:SAI_OBJECT_TYPE_HOSTIF_TRAP:oid:0x22000000000c56": {
                        "SAI_HOSTIF_TRAP_ATTR_PACKET_ACTION": "SAI_PACKET_ACTION_TRAP",
                        "SAI_HOSTIF_TRAP_ATTR_TRAP_GROUP": "oid:0x11000000000c55",
                        "SAI_HOSTIF_TRAP_ATTR_TRAP_TYPE": "SAI_HOSTIF_TRAP_TYPE_BGP"
                    }
                },
                {
                    "ASIC_STATE:SAI_OBJECT_TYPE_HOSTIF_TRAP_GROUP:oid:0x11000000000c55": {
                        "SAI_HOSTIF_TRAP_GROUP_ATTR_QUEUE": "4"
                    }
                },
                {
                    "ASIC_STATE:SAI_OBJECT_TYPE_QUEUE:oid:0x15000000000626": {
                        "NULL": "NULL",
                        "SAI_QUEUE_ATTR_INDEX": "4",
                        "SAI_QUEUE_ATTR_TYPE": "SAI_QUEUE_TYPE_UNICAST"
                    }
                }
            ],
            "tables_not_found": [],
            "vidtorid": {
                "oid:0x22000000000c56": "oid:0x400300000022",
                "oid:0x11000000000c55": "oid:0x200000011",
                "oid:0x15000000000626": "oid:0x12e0000040015"
            }
        },
        "STATE_DB": {
            "keys": [
                {
                    "COPP_TRAP_TABLE|bgp": {
                        "state": "ok"
                    }
                },
                {
                    "COPP_GROUP_TABLE|queue4_group1": {
                        "state": "ok"
                    }
                }
            ],
            "tables_not_found": []
        },
        "CONFIG_FILE": {
            "keys": [
                {
                    "COPP_TRAP|bgp": {
                        "trap_ids": "bgp,bgpv6",
                        "trap_group": "queue4_group1"
                    }
                },
                {
                    "COPP_GROUP|queue4_group1": {
                        "trap_action": "trap",
                        "trap_priority": "4",
                        "queue": "4"
                    }
                }
            ],
            "tables_not_found": []
        }
    }
}


admin@sonic:~$ dump state copp arp_req --table --key-map
+-----------+-------------+-----------------------------------------------------------------------+
| trap_id   | DB_NAME     | DUMP                                                                  |
+===========+=============+=======================================================================+
| arp_req   | CONFIG_DB   | +------------------+                                                  |
|           |             | | Keys Collected   |                                                  |
|           |             | +==================+                                                  |
|           |             | +------------------+                                                  |
+-----------+-------------+-----------------------------------------------------------------------+
| arp_req   | APPL_DB     | +--------------------------+                                          |
|           |             | | Keys Collected           |                                          |
|           |             | +==========================+                                          |
|           |             | | COPP_TABLE:queue4_group2 |                                          |
|           |             | +--------------------------+                                          |
+-----------+-------------+-----------------------------------------------------------------------+
| arp_req   | ASIC_DB     | +-------------------------------------------------------------------+ |
|           |             | | Keys Collected                                                    | |
|           |             | +===================================================================+ |
|           |             | | ASIC_STATE:SAI_OBJECT_TYPE_HOSTIF_TRAP:oid:0x22000000000c5b       | |
|           |             | +-------------------------------------------------------------------+ |
|           |             | | ASIC_STATE:SAI_OBJECT_TYPE_HOSTIF_TRAP_GROUP:oid:0x11000000000c59 | |
|           |             | +-------------------------------------------------------------------+ |
|           |             | | ASIC_STATE:SAI_OBJECT_TYPE_POLICER:oid:0x12000000000c5a           | |
|           |             | +-------------------------------------------------------------------+ |
|           |             | | ASIC_STATE:SAI_OBJECT_TYPE_QUEUE:oid:0x15000000000626             | |
|           |             | +-------------------------------------------------------------------+ |
|           |             | +----------------------+---------------------+                        |
|           |             | | vid                  | rid                 |                        |
|           |             | +======================+=====================+                        |
|           |             | | oid:0x22000000000c5b | oid:0x200000000022  |                        |
|           |             | +----------------------+---------------------+                        |
|           |             | | oid:0x11000000000c59 | oid:0x300000011     |                        |
|           |             | +----------------------+---------------------+                        |
|           |             | | oid:0x12000000000c5a | oid:0x200000012     |                        |
|           |             | +----------------------+---------------------+                        |
|           |             | | oid:0x15000000000626 | oid:0x12e0000040015 |                        |
|           |             | +----------------------+---------------------+                        |
+-----------+-------------+-----------------------------------------------------------------------+
| arp_req   | STATE_DB    | +--------------------------------+                                    |
|           |             | | Keys Collected                 |                                    |
|           |             | +================================+                                    |
|           |             | | COPP_TRAP_TABLE|arp            |                                    |
|           |             | +--------------------------------+                                    |
|           |             | | COPP_GROUP_TABLE|queue4_group2 |                                    |
|           |             | +--------------------------------+                                    |
+-----------+-------------+-----------------------------------------------------------------------+
| arp_req   | CONFIG_FILE | +--------------------------+                                          |
|           |             | | Keys Collected           |                                          |
|           |             | +==========================+                                          |
|           |             | | COPP_TRAP|arp            |                                          |
|           |             | +--------------------------+                                          |
|           |             | | COPP_GROUP|queue4_group2 |                                          |
|           |             | +--------------------------+                                          |
+-----------+-------------+-----------------------------------------------------------------------+

Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com>

lgtm-com · 2021-06-11T19:27:52Z

This pull request introduces 2 alerts when merging 05edcad into a425ca2 - view on LGTM.com

new alerts:

2 for Unused import

prsunny · 2021-06-11T21:06:09Z

@dgsudharsan , can you review/sign-off?

Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com>

…o dump_util_copp

liat-grozovik · 2021-06-27T13:55:57Z

/azp run

azure-pipelines · 2021-06-27T13:56:07Z

Azure Pipelines successfully started running 1 pipeline(s).

vivekrnv · 2021-07-20T18:53:01Z

Request for 202106

Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com>

arlakshm

as comments

dump/plugins/copp.py

arlakshm · 2021-08-17T19:20:12Z

dump/plugins/copp.py

+            return sai_hostif_vid
+
+    def __get_asic_hostif_obj(self, sai_hostif_vid):
+         # Not adding tp tables_not_found because of the type of reason specified for policer obj


indentation is not correct

arlakshm · 2021-08-17T19:20:19Z

dump/plugins/copp.py

+        if not ret["error"] and len(ret["keys"]) > 0:
+            self.ret_temp["ASIC_DB"]["keys"].append(ret["keys"][0])
+
+     # When the user writes config to CONFIG_DB, that takes precedence over default config


indentation is not correct

arlakshm · 2021-08-17T19:35:29Z

dump/plugins/copp.py

+        elif trap_id_key_cf:
+            self.ret_temp["CONFIG_FILE"]["keys"].append(trap_id_key_cf)
+            self.copp_trap_cfg_key = trap_id_key_cf.split("|")[-1]
+            id_in_file, _ = self.__find_trap_id_in_db(self.copp_trap_cfg_key, False)


In line 247, the trap_id_key_cf is retrived only when there are no traps in the db. so in id_in_file will always be empty string and the key will not save to `ret_temp ?

Sry, i'll rename this to id_in_db.

And no, id_in_db can be trap_id_key_cf or NULL

Eg: Let's say user wanted a dump for trap_id: "ospfv6"

In Config File:

"COPP_TRAP":{ "bgp": { "trap_ids": "ospfv6,ospf" } }

In Config DB:

"COPP_TRAP": { "bgp": { "trap_ids": "bgp,bgpv6", "trap_group": "queue4_group1" } }

In this case, after running trap_id_key_db, trap_group_db = self.__find_trap_id_in_db(), trap_id_key_db will still be empty as there is no direct way to find out the COPP_TRAP|bgp as the config DB entry doesn't have ospfv6,

We then find "trap_id_key_cf" from the file and the use that to find any diff present in the Config DB.

The complexity comes from the fact that there are two sources of truth for COPP config and you can provide a diff of any particular table (COPP_TRAP & COPP_GROUP) in any of the sources.

arlakshm · 2021-08-17T19:35:31Z

dump/plugins/copp.py

+            self.ret_temp["CONFIG_DB"]["keys"].append(trap_id_key_db)
+            self.copp_trap_cfg_key = trap_id_key_db.split("|")[-1]
+            id_in_file, _ = self.__find_trap_id_in_conf_file(self.copp_trap_cfg_key, False)
+            if id_in_file == trap_id_key_db:  # If any diff


it seems we add the trap_id to return ret_temp only when they are same. if they are different they are ignored ?

id_in_file will either be trap_id_key_db or NULL/Empty String.

Eg: Let's say user wanted a dump for trap_id: "ospfv6"

In Config DB:

"COPP_TRAP":{ "bgp": { "trap_ids": "ospfv6,ospf" } }

In Config File:

"COPP_TRAP": { "bgp": { "trap_ids": "bgp,bgpv6", "trap_group": "queue4_group1" } }

So, here we found out trap_id_key_db is bgp given ospfv6 as the trap_id. id_in_file is just a check in config file for any potential diff present in the COPP_TRAP|bgp key and thus id_in_file will either be "bgp" or NULL.

If it's NULL, we won't add it to the "CONFIG_FILE" section in the return template

arlakshm · 2021-08-17T19:38:01Z

dump/plugins/copp.py

+from dump.helper import handle_error
+from .executor import Executor
+
+TRAP_ID_MAP = {


Does this handle all the hostif traps defined in the SAI

No, i've copied the table from copporch. LINK. Seems copporch doesn't handle all the hostif traps defined in the SAI

arlakshm · 2021-08-17T19:57:16Z

dump/plugins/copp.py

+            key_tg = ret["keys"][0]
+            self.ret_temp["CONFIG_FILE"]["keys"].append(key_tg)
+            return True
+        elif not_found_report:


do we need this elif ? seems like the not_found_report is always true

Yeah, but just to keep the implementation consistent with __fill_trap_group_in_conf_db, i've retained it.

tests/dump_tests/module_tests/copp_test.py

Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com>

liat-grozovik · 2021-08-26T07:25:56Z

/azp run

azure-pipelines · 2021-08-26T07:26:08Z

Azure Pipelines successfully started running 1 pipeline(s).

vivekrnv · 2021-08-30T22:27:24Z

/azpw run

mssonicbld · 2021-08-30T22:27:25Z

/AzurePipelines run

azure-pipelines · 2021-08-30T22:27:36Z

Azure Pipelines successfully started running 1 pipeline(s).

vivekrnv · 2021-09-07T21:38:16Z

@SuvarnaMeenakshi and @arlakshm Can you finish the review and sign-off?

dgsudharsan · 2021-09-11T00:23:48Z

@arlakshm @SuvarnaMeenakshi Can you please review and sign off?

arlakshm

lgtm

dgsudharsan · 2021-09-15T03:53:37Z

@SuvarnaMeenakshi Can you please review and sign off?

dgsudharsan · 2021-09-16T18:35:15Z

@SuvarnaMeenakshi Can you please help here?

What I did Implemented vlan and vlan_member modules for debug dump utility. How I did it Used infrastructure and followed examples in #1666 #1667 #1668 #1669 #1670 How to verify it On switch: dump state vlan <vlan_name> dump state vlan_member '<vlan_name|<member_name>' Unit test: pytest-3 dump_tests/module_tests/vlan_test.py (same test file covers both vlan and vlan_member)

vivekrnv added 9 commits May 29, 2021 05:04

[Debug Dump Util] COPP Module Added

ed7e8a8

Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com>

Refactoring Test names

09c326a

Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com>

Comments Addressed

2323728

Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com>

Minor fixes

a98a2b7

Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com>

Minor Fixes

aec2283

Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com>

Minor Fixes

c707944

Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com>

Moved mock files to a diff dir

47b214f

Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com>

HOSTIF_* entries added to copp

079f453

Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com>

Minor Comment Addressed

05edcad

Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com>

LGTM issue addressed

759c4d7

Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com>

vivekrnv mentioned this pull request Jun 14, 2021

Debug dump utility HLD sonic-net/SONiC#789

Merged

anshuv-mfst requested a review from SuvarnaMeenakshi June 22, 2021 15:50

vivekrnv added 2 commits June 23, 2021 01:10

Multi-Asic Fixes

1208768

Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com>

Merge branch 'master' of https://github.com/Azure/sonic-utilities int…

66b5873

…o dump_util_copp

dgsudharsan previously approved these changes Jul 12, 2021

View reviewed changes

liat-grozovik added the Request for 202106 Branch label Jul 25, 2021

pep-8 issues handled

5615eda

Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com>

vivekrnv dismissed dgsudharsan’s stale review via 5615eda August 17, 2021 06:14

arlakshm reviewed Aug 17, 2021

View reviewed changes

SuvarnaMeenakshi reviewed Aug 19, 2021

View reviewed changes

tests/dump_tests/module_tests/copp_test.py Outdated Show resolved Hide resolved

Fixed Long Strings & removed extra mock

2513f13

Signed-off-by: Vivek Reddy Karri <vkarri@nvidia.com>

dgsudharsan approved these changes Sep 7, 2021

View reviewed changes

arlakshm approved these changes Sep 13, 2021

View reviewed changes

SuvarnaMeenakshi approved these changes Sep 17, 2021

View reviewed changes

SuvarnaMeenakshi merged commit c0b9917 into sonic-net:master Sep 17, 2021

liat-grozovik removed the Request for 202106 Branch label Sep 19, 2021

raphaelt-nvidia mentioned this pull request Oct 13, 2021

[debug dump util] Vlan Module Added #1877

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[debug dump util] COPP Module Added #1670

[debug dump util] COPP Module Added #1670

vivekrnv commented Jun 11, 2021 •

edited

Loading

lgtm-com bot commented Jun 11, 2021

prsunny commented Jun 11, 2021

liat-grozovik commented Jun 27, 2021

azure-pipelines bot commented Jun 27, 2021

vivekrnv commented Jul 20, 2021

arlakshm left a comment

arlakshm Aug 17, 2021

arlakshm Aug 17, 2021

arlakshm Aug 17, 2021

vivekrnv Aug 20, 2021

arlakshm Aug 17, 2021

vivekrnv Aug 20, 2021 •

edited

Loading

arlakshm Aug 17, 2021

vivekrnv Aug 20, 2021

arlakshm Aug 17, 2021

vivekrnv Aug 20, 2021

liat-grozovik commented Aug 26, 2021

azure-pipelines bot commented Aug 26, 2021

vivekrnv commented Aug 30, 2021

mssonicbld commented Aug 30, 2021

azure-pipelines bot commented Aug 30, 2021

vivekrnv commented Sep 7, 2021

dgsudharsan commented Sep 11, 2021

arlakshm left a comment

dgsudharsan commented Sep 15, 2021

dgsudharsan commented Sep 16, 2021

[debug dump util] COPP Module Added #1670

[debug dump util] COPP Module Added #1670

Conversation

vivekrnv commented Jun 11, 2021 • edited Loading

What I did

How I did it

How to verify it

Previous command output (if the output of a command-line utility has changed)

New command output (if the output of a command-line utility has changed)

lgtm-com bot commented Jun 11, 2021

prsunny commented Jun 11, 2021

liat-grozovik commented Jun 27, 2021

azure-pipelines bot commented Jun 27, 2021

vivekrnv commented Jul 20, 2021

arlakshm left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vivekrnv Aug 20, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

liat-grozovik commented Aug 26, 2021

azure-pipelines bot commented Aug 26, 2021

vivekrnv commented Aug 30, 2021

mssonicbld commented Aug 30, 2021

azure-pipelines bot commented Aug 30, 2021

vivekrnv commented Sep 7, 2021

dgsudharsan commented Sep 11, 2021

arlakshm left a comment

Choose a reason for hiding this comment

dgsudharsan commented Sep 15, 2021

dgsudharsan commented Sep 16, 2021

vivekrnv commented Jun 11, 2021 •

edited

Loading

vivekrnv Aug 20, 2021 •

edited

Loading