Skip to content

Light clients

Carlos Rodriguez edited this page Oct 6, 2023 · 2 revisions

This document attempts to explore IBC's light client create and upddate flows in detail. For illustration purposes we will work with a CometBFT (Tendermint) light client, tracking the consensus state information of a counterparty chain running the CometBFT consensus algorithm. The goals of this tutorial are:

  • Explore the code of hermes and ibc-go that makes client creation and update possible.
  • Understand the data passed in the client creation and update messages and how/where it is obtained.

Some other useful resources:

Table of Contents

Setup

  • Two chains (chain1 and chain2) running ib-go's simd binary.
  • chain1's RPC endpoint runs on http://localhost:27000 and REST API runs on http://localhost:27001.
  • chain2's RPC endpoint runs on http://localhost:27010 and REST API runs on http://localhost:27011.
  • hermes relayer.

This document is updated for ibc-go v7.0.0 and hermes v1.3.0.

ICS-02 store paths

Client and consensus related information is stored in the IBC store:

key value
[]byte("clients/" + {clientID} + "/clientState") Connection end
[]byte("clients/" + {clientID} + "/consensusStates/" + {revision} + "-" + {height}) Consensus state at a given revision and height

Client creation

To create a light client on chain1 for chain2 (a light client that will track the counterparty's consensus state) we execute the following hermes command (see hermes documentation for more details):

> hermes --config config.toml create client \
--host-chain chain1 \
--reference-chain chain2
SUCCESS CreateClient(
  CreateClient(
    Attributes {
      client_id: ClientId(
        "07-tendermint-0",
      ),
      client_type: Tendermint,
      consensus_height: Height {
        revision: 0,
        height: 6,
      },
    },
  ),
)

Light clien creation succeeded. The identifier of the light client created is 07-tendermint-0. The consensus state stored during the creation corresponds to chain2's height 6. We will see now in details how this all worked out.

Building MsgCreateClient

To create the client the relayer submits MsgCreateClient on chain1 (see the message proto definition).

In hermes the MsgCreateClient is created and sent as followed:

The parameters needed to construct MsgCreateClient are:

client_state

Hermes queries the latest height of chain2 by querying CometBFT's /status RPC endpoint and reading the values for result.node_info.network (which is the revision number, chain2) and result.sync_info.latest_block_height (which is the revision height, 6).

Then hermes retrieves the Cosmos-specific client parameters from the chain configurations in its own config.toml) and calculates the settings to use when creating the client state.

The client state is then created by calling build_client_state. Inside build_client_state the client state is created.

consensus_state

The [consensus state is created by calling build_consensus_state](https://github.com/informalsystems/hermes/blob/master/crates/relayer/src/foreign_client.rs#L608). The consensus state is basically a block header from chain2` at the latest height (6 in this example).

signer

It is the address of the relayer that submits the message. Hermes retrieves it here.

Submitting MsgCreateClient

From the hermes log we can retrieve the hash of the transaction that executed MsgCreateClient on chain1:

2023-03-10T11:56:41.497282Z DEBUG ThreadId(25) send_messages_and_wait_commit{chain=chain1 tracking_id=create client}:send_tx_with_account_sequence_retry{chain=chain1 account.sequence=0}: tx was successfully broadcasted, increasing account sequence number response=Response { code: Ok, data: b"", log: "[]", hash: Hash::Sha256(95216348978324110B489F1FBD6CC8D9704C29323815E5C0947629794471AE99) } account.sequence.old=0 account.sequence.new=1

And we can use this hash to get the transaction information using CometBFT's /tx RPC endpoint:

http://localhost:27000/tx?hash=0x95216348978324110B489F1FBD6CC8D9704C29323815E5C0947629794471AE99

See sample JSON result.

The transaction was successfully executed and included in the block at height 175 of chain1. The value in the field result.tx is a base64-encoded string of the bytes of the messages that were executed as part of the transaction (e.g. hitting http://localhost:27000/block?height=175 on the browser, see the sample JSON result). This transaction contains only one message (MsgCreateClient) and if we decode these bytes we can retrieve back the message data that the relayer submitted:

> simd tx decode CsYDCooDCiMvaWJjLmNvcmUuY2xpZW50LnYxLk1zZ0NyZWF0ZUNsaWVudBLiAgqoAQorL2liYy5saWdodGNsaWVudHMudGVuZGVybWludC52MS5DbGllbnRTdGF0ZRJ5CgZjaGFpbjISBAgBEAMaBAiA6kkiBAiA324qAggoMgA6AhAGQhkKCQgBGAEgASoBABIMCgIAARAhGAQgDDABQhkKCQgBGAEgASoBABIMCgIAARAgGAEgATABSgd1cGdyYWRlShB1cGdyYWRlZElCQ1N0YXRlUAFYARKFAQouL2liYy5saWdodGNsaWVudHMudGVuZGVybWludC52MS5Db25zZW5zdXNTdGF0ZRJTCgsI87WsoAYQqNXnWRIiCiAQg/gjqhf01/yJyfbMDXUcF4DCMR9ptghSzD1MN3/aIhogRt7WE9jHiTQzsYgYzw/40ukY+aPOgkytdv3awfG6+vUaLWNvc21vczFtOHdwZ3g2cmw5eDR3Y3czdXRlbGxwZzMwOWpnZXR3OXFjemU3ZRI3aGVybWVzIDEuMy4wKzk5N2U1NDFiIChodHRwczovL2hlcm1lcy5pbmZvcm1hbC5zeXN0ZW1zKRJjCk4KRgofL2Nvc21vcy5jcnlwdG8uc2VjcDI1NmsxLlB1YktleRIjCiEDo+5d9XWGJhK5g0GguWC8NfxuKwX7Lg4tldRB44WkAb8SBAoCCAESEQoLCgVzdGFrZRICOTkQov8FGkBsny/UQJot4jFH6nCMPHmAYW04YITH4zihopmMQyGOM0+6cNkUWVQ0FmA8PuFoO+nFULUac0eOlJOaZrb32q+l
{
  "body":{
    "messages":[
      {
        "@type":"/ibc.core.client.v1.MsgCreateClient",
        "client_state":{
          "@type":"/ibc.lightclients.tendermint.v1.ClientState",
          "chain_id":"chain2",
          "trust_level":{
            "numerator":"1",
            "denominator":"3"
          },
          "trusting_period":"1209600s",
          "unbonding_period":"1814400s",
          "max_clock_drift":"40s",
          "frozen_height":{
            "revision_number":"0",
            "revision_height":"0"
          },
          "latest_height":{
            "revision_number":"0",
            "revision_height":"6"
          },
          "proof_specs":[
            {
              "leaf_spec":{
                "hash":"SHA256",
                "prehash_key":"NO_HASH",
                "prehash_value":"SHA256",
                "length":"VAR_PROTO",
                "prefix":"AA=="
              },
              "inner_spec":{
                "child_order":[
                  0,
                  1
                ],
                "child_size":33,
                "min_prefix_length":4,
                "max_prefix_length":12,
                "empty_child":null,
                "hash":"SHA256"
              },
              "max_depth":0,
              "min_depth":0
            },
            {
              "leaf_spec":{
                "hash":"SHA256",
                "prehash_key":"NO_HASH",
                "prehash_value":"SHA256",
                "length":"VAR_PROTO",
                "prefix":"AA=="
              },
              "inner_spec":{
                "child_order":[
                  0,
                  1
                ],
                "child_size":32,
                "min_prefix_length":1,
                "max_prefix_length":1,
                "empty_child":null,
                "hash":"SHA256"
              },
              "max_depth":0,
              "min_depth":0
            }
          ],
          "upgrade_path":[
            "upgrade",
            "upgradedIBCState"
          ],
          "allow_update_after_expiry":true,
          "allow_update_after_misbehaviour":true
        },
        "consensus_state":{
          "@type":"/ibc.lightclients.tendermint.v1.ConsensusState",
          "timestamp":"2023-03-10T11:56:35.188345Z",
          "root":{
            "hash":"EIP4I6oX9Nf8icn2zA11HBeAwjEfabYIUsw9TDd/2iI="
          },
          "next_validators_hash":"46DED613D8C7893433B18818CF0FF8D2E918F9A3CE824CAD76FDDAC1F1BAFAF5"
        },
        "signer":"cosmos1m8wpgx6rl9x4wcw3utellpg309jgetw9qcze7e"
      }
    ],
    "memo":"hermes 1.3.0+997e541b (https://hermes.informal.systems)",
    "timeout_height":"0",
    "extension_options":[
    ],
    "non_critical_extension_options":[
    ]
  },
  "auth_info":{
    "signer_infos":[
      {
        "public_key":{
          "@type":"/cosmos.crypto.secp256k1.PubKey",
          "key":"A6PuXfV1hiYSuYNBoLlgvDX8bisF+y4OLZXUQeOFpAG/"
        },
        "mode_info":{
          "single":{
            "mode":"SIGN_MODE_DIRECT"
          }
        },
        "sequence":"0"
      }
    ],
    "fee":{
      "amount":[
        {
          "denom":"stake",
          "amount":"99"
        }
      ],
      "gas_limit":"98210",
      "payer":"",
      "granter":""
    },
    "tip":null
  },
  "signatures":[
    "bJ8v1ECaLeIxR+pwjDx5gGFtOGCEx+M4oaKZjEMhjjNPunDZFFlUNBZgPD7haDvpxVC1GnNHjpSTmma299qvpQ=="
  ]
}

The field client_state.latest_height.revision_height is 6 (this is the height of chain2). The consensus_state object consists of the timestamp for the block header, the root.hash, which is the base64-encoded string of the byte representation of the app hash for block height 6 of chain2, and next_validators_hash, which is the hash of the next validator set. If we query chain2 locally for the block at height 6 by entering http://localhost:27010/block?height=6 on the browser, we can check that the value in the response for the field result.block.header.app_hash (1083F823AA17F4D7FC89C9F6CC0D751C1780C2311F69B60852CC3D4C377FDA22) matches the root hash from the consensus state (see the sample JSON response).

Executing MsgCreateClient

The message enters by the CreateClient RPC handler method on the message server, and after unpacking the client and consensus states, the execution continues in ClientKeeper's CreateClient function, to which the client and consensus states are passed in.

The CreateClient function generates a new client identifier (07-tendermint-0 in this example), then it creates an isolated prefix store for the provided client identifier, and finally it calls the client-specific (CometBFT/Tendermint in our example) implementation of Initialize to set any client-specific data in the store (this includes the client state, initial consensus state and any associated metadata in the case of CometBFT/Tendermint).

After MsgCreateClient successfully executes, there is a client state stored on chain1. We can use ibc-go's CLI or REST interface to check the existence of the client state end with client ID 07-tendermint-0. The REST interface works simply by entering http://localhost:27001/ibc/core/client/v1/client_states/07-tendermint-0 on the browser. But here we will use the CLI instead:

> simd q ibc client state 07-tendermint-0 --node http://localhost:27000
client_state:
  '@type': /ibc.lightclients.tendermint.v1.ClientState
  allow_update_after_expiry: true
  allow_update_after_misbehaviour: true
  chain_id: chain2
  frozen_height:
    revision_height: "0"
    revision_number: "0"
  latest_height:
    revision_height: "6"
    revision_number: "0"
  max_clock_drift: 40s
  proof_specs:
  - inner_spec:
      child_order:
      - 0
      - 1
      child_size: 33
      empty_child: null
      hash: SHA256
      max_prefix_length: 12
      min_prefix_length: 4
    leaf_spec:
      hash: SHA256
      length: VAR_PROTO
      prefix: AA==
      prehash_key: NO_HASH
      prehash_value: SHA256
    max_depth: 0
    min_depth: 0
  - inner_spec:
      child_order:
      - 0
      - 1
      child_size: 32
      empty_child: null
      hash: SHA256
      max_prefix_length: 1
      min_prefix_length: 1
    leaf_spec:
      hash: SHA256
      length: VAR_PROTO
      prefix: AA==
      prehash_key: NO_HASH
      prehash_value: SHA256
    max_depth: 0
    min_depth: 0
  trust_level:
    denominator: "3"
    numerator: "1"
  trusting_period: 1209600s
  unbonding_period: 1814400s
  upgrade_path:
  - upgrade
  - upgradedIBCState
proof: CusCCugCCiNjbGllbnRzLzA3LXRlbmRlcm1pbnQtMC9jbGllbnRTdGF0ZRKoAQorL2liYy5saWdodGNsaWVudHMudGVuZGVybWludC52MS5DbGllbnRTdGF0ZRJ5CgZjaGFpbjISBAgBEAMaBAiA6kkiBAiA324qAggoMgA6AhAGQhkKCQgBGAEgASoBABIMCgIAARAhGAQgDDABQhkKCQgBGAEgASoBABIMCgIAARAgGAEgATABSgd1cGdyYWRlShB1cGdyYWRlZElCQ1N0YXRlUAFYARoMCAEYASABKgQAAt4CIiwIARIFAgTeAiAaISBGQ/KS9slxYJKPUJeBAF/dBErbo2edgiwq3bBAW5vjxSIsCAESBQYK3gIgGiEgogptbQ9tW3ipDzD8kjrK74TlCDXjmRAtMDC49mnUWT8iLAgBEgUIEN4CIBohIB9Hw0IKXViNeX5STDfmLLcGcfBjckMSzgsTc7nBYB1lCv4BCvsBCgNpYmMSIKYdV8h+Trj64U09ppTtZKoxt1SFNZhCkUsdQN5CaXBXGgkIARgBIAEqAQAiJwgBEgEBGiDFBcD9SLHPK2VhnxKzFE4ZxgtOa2JSXuk2vpPQQWI+vyInCAESAQEaINorYzt2yR1EvM0mHhoFaSifFTwOJub275aEzBr42X0/IiUIARIhAUcOKnMQFKYk8mjce8mELcWxZQWeuMfgsVxzTh0bQlPkIiUIARIhAckND4egSkos9gOO4MhZw7gprsFN6fZr8VzCdXhAWcEuIicIARIBARogtRasVuJEM6mll0H3HmMoiJPp+1WP5ACfMrQWmvhyuEI=
proof_height:
  revision_height: "184"
  revision_number: "0"

We see that client_state.latest_height.revision_height is 6, as we expected. The client state also contains the proof_specs, which specifies the way to calculate a Merkle tree root hash for a Merkle proof.

We can also query the consensus state stored for height 6 of chain2 (using the REST interface we could enter http://localhost:27001/ibc/core/client/v1/consensus_states/07-tendermint-0/revision/0/height/6 on the browser):

./simd q ibc client consensus-state 07-tendermint-0 0-6 --node http://localhost:27000
consensus_state:
  '@type': /ibc.lightclients.tendermint.v1.ConsensusState
  next_validators_hash: 46DED613D8C7893433B18818CF0FF8D2E918F9A3CE824CAD76FDDAC1F1BAFAF5
  root:
    hash: EIP4I6oX9Nf8icn2zA11HBeAwjEfabYIUsw9TDd/2iI=
  timestamp: "2023-03-10T11:56:35.188345Z"
proof: Cs4CCssCCitjbGllbnRzLzA3LXRlbmRlcm1pbnQtMC9jb25zZW5zdXNTdGF0ZXMvMC02EoUBCi4vaWJjLmxpZ2h0Y2xpZW50cy50ZW5kZXJtaW50LnYxLkNvbnNlbnN1c1N0YXRlElMKCwjztaygBhCo1edZEiIKIBCD+COqF/TX/InJ9swNdRwXgMIxH2m2CFLMPUw3f9oiGiBG3tYT2MeJNDOxiBjPD/jS6Rj5o86CTK12/drB8br69RoMCAEYASABKgQAAt4CIioIARImAgTeAiACDtJnxD0zRuwPGYl+K8lpqi9/PUpri4QdIYU+qRNVJCAiLAgBEgUGCt4CIBohIKIKbW0PbVt4qQ8w/JI6yu+E5Qg145kQLTAwuPZp1Fk/IiwIARIFCBDeAiAaISAfR8NCCl1YjXl+Ukw35iy3BnHwY3JDEs4LE3O5wWAdZQr+AQr7AQoDaWJjEiCmHVfIfk64+uFNPaaU7WSqMbdUhTWYQpFLHUDeQmlwVxoJCAEYASABKgEAIicIARIBARogxQXA/UixzytlYZ8SsxROGcYLTmtiUl7pNr6T0EFiPr8iJwgBEgEBGiACJ2wEqdRBNkTz02gpx0EmjtORleV9fYZqridVT1gNDCIlCAESIQFHDipzEBSmJPJo3HvJhC3FsWUFnrjH4LFcc04dG0JT5CIlCAESIQHoPaEpKz/+N291AulrDIcZSXdzRvx1UggykQD7pCs7tyInCAESAQEaIB8IFijlIxoPO3Xr3oMLDFmU5AlM5ScJRIMFIuqferMH
proof_height:
  revision_height: "188"
  revision_number: "0"

The consensus_state.root.hash is the base64-encoded string of the byte representation of the app hash in the header for block at height 6 of chain2 (see the sample JSON response for block at height 6 of chain2).

We can also query all the consensus states for client with identifier 07-tendermint-0 (http://localhost:27001/ibc/core/client/v1/consensus_states/07-tendermint-0):

./simd q ibc client  consensus-states 07-tendermint-0 --node http://localhost:27000
consensus_states:
- consensus_state:
    '@type': /ibc.lightclients.tendermint.v1.ConsensusState
    next_validators_hash: 46DED613D8C7893433B18818CF0FF8D2E918F9A3CE824CAD76FDDAC1F1BAFAF5
    root:
      hash: EIP4I6oX9Nf8icn2zA11HBeAwjEfabYIUsw9TDd/2iI=
    timestamp: "2023-03-10T11:56:35.188345Z"
  height:
    revision_height: "6"
    revision_number: "0"
pagination:
  next_key: null
  total: "0"

As expected there is only one consensus state stored at the moment.

Out of curiosity we can perform an ABCI RPC query to the ibc store of chain1 to obtain a proof that can be used to verify that chain1 has stored a client state for client with identifier 07-tendermint-0 at a given height (we will use height 184 in this example). We can make the ABCI RPC query on the browser by simply using CometBFT's /abci_query REST endpoint:

path: "store/ibc/key"
data: "clients/07-tendermint-0/clientState" (in hex: 0x636c69656e74732f30372d74656e6465726d696e742d302f636c69656e745374617465)
prove: true (so that the response contains the merkle proof)
height: 184
URL: http://localhost:27000/abci_query?path="store/ibc/key"&data=0x636c69656e74732f30372d74656e6465726d696e742d302f636c69656e745374617465&prove=true&height=184

See sample JSON response.

The response to this ABCI RPC query contains both the value stored at the queried path (clients/07-tendermint-0/clientState) and the proof needed to verify it. If we take the base64-encoded byte string from the value for the field result.response.value and we decode it, then we can inspect the information for the client state stored in chain1:

chain_id:"chain2" 
trust_level:<numerator:1 denominator:3 > 
trusting_period:<seconds:1209600 > 
unbonding_period:<seconds:1814400 > 
max_clock_drift:<seconds:40 > 
frozen_height:<> 
latest_height:<revision_height:6 > 
proof_specs:<leaf_spec:<hash:SHA256 prehash_value:SHA256 length:VAR_PROTO prefix:"\000" > inner_spec:<child_order:0 child_order:1 child_size:33 min_prefix_length:4 max_prefix_length:12 hash:SHA256 > > 
proof_specs:<leaf_spec:<hash:SHA256 prehash_value:SHA256 length:VAR_PROTO prefix:"\000" > inner_spec:<child_order:0 child_order:1 child_size:32 min_prefix_length:1 max_prefix_length:1 hash:SHA256 > > 
upgrade_path:"upgrade"
upgrade_path:"upgradedIBCState"
allow_update_after_expiry:true
allow_update_after_misbehaviour:true 

We can as well perform an ABCI RPC query to the ibc store of chain1 to obtain a proof that can be used to verify that chain1 has stored a consensus state for client with identifier 07-tendermint-0 at a given height (we will use height 184 in this example) for a particular height of chain2 (6, in this example). We can make the ABCI RPC query on the browser by simply using CometBFT's /abci_query REST endpoint:

path: "store/ibc/key"
data: "clients/07-tendermint-0/consensusStates/0-6" (in hex: 0x636c69656e74732f30372d74656e6465726d696e742d302f636f6e73656e7375735374617465732f302d36)
prove: true (so that the response contains the merkle proof)
height: 184
URL: http://localhost:27000/abci_query?path="store/ibc/key"&data=0x636c69656e74732f30372d74656e6465726d696e742d302f636f6e73656e7375735374617465732f302d36&prove=true&height=184

See sample JSON response.

The response to this ABCI RPC query contains both the value stored at the queried path (clients/07-tendermint-0/consensusStates/0-6) and the proof needed to verify it. If we take the base64-encoded byte string from the value for the field result.response.value and we decode it, then we can inspect the information for the consensus state stored in chain1:

timestamp:<seconds:1664047549 nanos:509581000 >
root:<hash:"i\307\025\205\335n\363\2758\302\201}?\006\334H?\273\033V\023|]f\022\206\217j;F\004\334" > next_validators_hash:"=\231\271\250\261I\024\366W\353\031_\256yI1\005\355!-\311I\327\317\342G\277E\177/l\322" 

Client update

The hermes relayer submits a message to update the light client when it needs to submit another message that contains a proof extracted at a certain height of the counterparty chain. The way we are going to do this here is by initiating a connection upgrade handshake. We are only interested in the first 2 steps. On chain2 we will submit MsgConnectionOpenInit and then on chain1 we will submit MsgConnectionOpenTry. This message contains the information needed by chain1 to verify that chain2 has completed ConnOpenTry (i.e. a proof for a certain height of chain2 that it has stored a connection end with STATE_OPEN). We execute the following hermes command (see hermes documentation for more details):

> hermes --config config.toml tx conn-init \
--dst-chain chain2 \
--src-chain chain1 \
--dst-client 07-tendermint-0 \
--src-client 07-tendermint-0
> hermes --config config.toml tx conn-try \
--dst-chain chain1 \
--src-chain chain2 \
--dst-client 07-tendermint-0 \
--src-client 07-tendermint-0 \
--src-connection connection-0

Note that we initiate the handshake on the light client that is already created and not on a new one.

Building MsgUpdateClient

To create the update the on-chain client for chain2 the relayer submits MsgUpdateClient on chain1 (see the message proto definition).

In hermes the MsgUpdateClient is created and sent as followed:

The parameters needed to construct MsgUpdateClient are:

header

The header is obtained by calling build_header.

client_id

This the identifer of the light client that hermes wants to update.

signer

It is the address of the relayer that submits the message. Hermes retrieves it here.

Submitting MsgUpdateClient

From the hermes log we can retrieve the hash of the transaction that executed MsgUpdateClient on chain1:

2023-03-10T12:19:05.060447Z DEBUG ThreadId(25) send_messages_and_wait_commit{chain=chain1 tracking_id=ConnectionOpenTry}:send_tx_with_account_sequence_retry{chain=chain1 account.sequence=1}: tx was successfully broadcasted, increasing account sequence number response=Response { code: Ok, data: b"", log: "[]", hash: Hash::Sha256(2EFB1D433CC5AABDADC192AD108F486B663FC1988DCE7A0FF4C936B1B1C6A5DA) } account.sequence.old=1 account.sequence.new=2

And we can use this hash to get the transaction information using CometBFT's /tx RPC endpoint:

http://localhost:27000/tx?hash=0x2EFB1D433CC5AABDADC192AD108F486B663FC1988DCE7A0FF4C936B1B1C6A5DA

See sample JSON result.

The transaction was successfully executed and included in the block at height 443 of chain1. The value in the field result.tx is a base64-encoded string of the bytes of the messages that were executed as part of the transaction. We can also retrieve chain1's block 443 by hitting http://localhost:27000/block?height=443 on the browser and see that the result.block.data.txs array contains one transaction whose base64-encoded string matches the one obtained from the /tx endpoint (see the sample JSON result). This transaction contains two messages (MsgUpdateClient and MsgConnectionOpenTry) and if we decode these bytes we can retrieve back the message data that the relayer submitted:

> simd tx decode CssZCucHCiMvaWJjLmNvcmUuY2xpZW50LnYxLk1zZ1VwZGF0ZUNsaWVudBK/BwoPMDctdGVuZGVybWludC0wEvwGCiYvaWJjLmxpZ2h0Y2xpZW50cy50ZW5kZXJtaW50LnYxLkhlYWRlchLRBgrKBAqNAwoCCAsSBmNoYWluMhiSAiIMCLPArKAGENDhtssDKkgKIEGhSddbbVj0sGrVakxbcDeurLkWohTseitAWHRqqM8CEiQIARIgdoCZ7HRXaRNxB3avE1p1wxYQCf4mb0Aoe0Lxt8UP87oyIOfZObTlDlbEjPkTVIXiCR5GxdFv6itfCBTuLkPLBo+cOiDjsMRCmPwcFJr79MiZb7kkJ65B5GSbk0yklZkbeFK4VUIgRt7WE9jHiTQzsYgYzw/40ukY+aPOgkytdv3awfG6+vVKIEbe1hPYx4k0M7GIGM8P+NLpGPmjzoJMrXb92sHxuvr1UiAEgJG8fdwoP3e/v5HXPETaWMPfipy8hnQF2Lfz2q2iL1ogfT4FBaetd09IKeP1Q+ZpOh28eNL4hKch0+BnGrHwBd1iINVCZD/ag2Zx3r8Epmd+OTGWMf43QZh5YJnyF13jfIwyaiDjsMRCmPwcFJr79MiZb7kkJ65B5GSbk0yklZkbeFK4VXIUs7poqK9ORIoVBe1pHvyBbKE0v8oStwEIkgIaSAog9NsFABnF0Amg8q4tq3Fok7APdqsBeIt8h6AVYVgKg1oSJAgBEiAKvFfSFHDCEiUHBwzYPQCTZT0SFiDUQkfyMDI2eua0wiJoCAISFLO6aKivTkSKFQXtaR78gWyhNL/KGgwIuMCsoAYQsL7D1QMiQCfkJaZyqG5rO4sJyxjNhCd0mK6TKd2y1Y8D7JRgEaEBd+JpwZTjJgaBZ4XLEbbsi4DptEH53j5MJcjo3M/JnwASfgo8ChSzumior05EihUF7Wke/IFsoTS/yhIiCiC3e6obDJDYpHT8hcuANyuMmEMMW3FXyDjOEjgoMdVkOhgKEjwKFLO6aKivTkSKFQXtaR78gWyhNL/KEiIKILd7qhsMkNikdPyFy4A3K4yYQwxbcVfIOM4SOCgx1WQ6GAoYChoCEAYifgo8ChSzumior05EihUF7Wke/IFsoTS/yhIiCiC3e6obDJDYpHT8hcuANyuMmEMMW3FXyDjOEjgoMdVkOhgKEjwKFLO6aKivTkSKFQXtaR78gWyhNL/KEiIKILd7qhsMkNikdPyFy4A3K4yYQwxbcVfIOM4SOCgx1WQ6GAoYChotY29zbW9zMW04d3BneDZybDl4NHdjdzN1dGVsbHBnMzA5amdldHc5cWN6ZTdlCqURCiwvaWJjLmNvcmUuY29ubmVjdGlvbi52MS5Nc2dDb25uZWN0aW9uT3BlblRyeRL0EAoPMDctdGVuZGVybWludC0wGqkBCisvaWJjLmxpZ2h0Y2xpZW50cy50ZW5kZXJtaW50LnYxLkNsaWVudFN0YXRlEnoKBmNoYWluMRIECAEQAxoECIDqSSIECIDfbioCCCgyADoDELgDQhkKCQgBGAEgASoBABIMCgIAARAhGAQgDDABQhkKCQgBGAEgASoBABIMCgIAARAgGAEgATABSgd1cGdyYWRlShB1cGdyYWRlZElCQ1N0YXRlUAFYASImCg8wNy10ZW5kZXJtaW50LTASDGNvbm5lY3Rpb24tMBoFCgNpYmMyIwoBMRINT1JERVJfT1JERVJFRBIPT1JERVJfVU5PUkRFUkVEOgMQkgJCiQQKhQIKggIKGGNvbm5lY3Rpb25zL2Nvbm5lY3Rpb24tMBJSCg8wNy10ZW5kZXJtaW50LTASIwoBMRINT1JERVJfT1JERVJFRBIPT1JERVJfVU5PUkRFUkVEGAEiGAoPMDctdGVuZGVybWludC0wGgUKA2liYxoMCAEYASABKgQAAooEIioIARImBAaiBCDH++vn4c7YphLkOgKYVeN0sumRCQRuIoMeVoRF6ZBJZyAiLAgBEgUGDKIEIBohIIvnJCnCHC5Gm2vj+FdulB29OIPf3mZWhajiYEGZGSr4IioIARImChyiBCBljoiefyYfiSqLXlRjFy8F8rf8HqY0/a78FiTvOwvk5iAK/gEK+wEKA2liYxIgIJ2E1/rWWUUqDGcKdvRFPs80VnRtqChYMx21HkSkaZYaCQgBGAEgASoBACInCAESAQEaIMUFwP1Isc8rZWGfErMUThnGC05rYlJe6Ta+k9BBYj6/IicIARIBARogQZHPfXC+ja61o/C7ZHNxGC1S8L4w92bL88qqyTY8/9IiJQgBEiEBRw4qcxAUpiTyaNx7yYQtxbFlBZ64x+CxXHNOHRtCU+QiJQgBEiEBAlCiFy2Axrwoiuqe53lrcQZeJX6sDOPLwzPITwsWhn4iJwgBEgEBGiA7wWTS1gDFzYWA0RKW6l21JcqTKs0mO5MenIlt4qXi9kqeBQqaAwqXAwojY2xpZW50cy8wNy10ZW5kZXJtaW50LTAvY2xpZW50U3RhdGUSqQEKKy9pYmMubGlnaHRjbGllbnRzLnRlbmRlcm1pbnQudjEuQ2xpZW50U3RhdGUSegoGY2hhaW4xEgQIARADGgQIgOpJIgQIgN9uKgIIKDIAOgMQuANCGQoJCAEYASABKgEAEgwKAgABECEYBCAMMAFCGQoJCAEYASABKgEAEgwKAgABECAYASABMAFKB3VwZ3JhZGVKEHVwZ3JhZGVkSUJDU3RhdGVQAVgBGgwIARgBIAEqBAACogQiLAgBEgUCBKIEIBohIFuh8ORb/TdRBDTWJ9Q3kQrwOE1kKJC+keImM9H5bozcIiwIARIFBAaiBCAaISC6zwWKV9uwgJQHgkT0BCQGmzA3vqrDEHwacp5lDktOViIsCAESBQgQogQgGiEgqilTcun0efa+tPDif100FVGloJTTNMdhz5YQYN1vJQgiLAgBEgUKHKIEIBohIGkB3q9PMOcG51KXbvzNeUPGuMCE6CSW+euwBpXR1XDICv4BCvsBCgNpYmMSICCdhNf61llFKgxnCnb0RT7PNFZ0bagoWDMdtR5EpGmWGgkIARgBIAEqAQAiJwgBEgEBGiDFBcD9SLHPK2VhnxKzFE4ZxgtOa2JSXuk2vpPQQWI+vyInCAESAQEaIEGRz31wvo2utaPwu2RzcRgtUvC+MPdmy/PKqsk2PP/SIiUIARIhAUcOKnMQFKYk8mjce8mELcWxZQWeuMfgsVxzTh0bQlPkIiUIARIhAQJQohctgMa8KIrqnud5a3EGXiV+rAzjy8MzyE8LFoZ+IicIARIBARogO8Fk0tYAxc2FgNESlupdtSXKkyrNJjuTHpyJbeKl4vZSgQUK/QIK+gIKLWNsaWVudHMvMDctdGVuZGVybWludC0wL2NvbnNlbnN1c1N0YXRlcy8wLTQ0MBKGAQouL2liYy5saWdodGNsaWVudHMudGVuZGVybWludC52MS5Db25zZW5zdXNTdGF0ZRJUCgwIqcCsoAYQwJ76xAISIgogeGZ0Sxsbdr+P6GvjdpV70AA1eWwkCbkZrZlMlARvvrIaIFV2lsv5jjpOdKE3AacmyrOB6SaQGz2d7sfNpuUtqaHnGgwIARgBIAEqBAACogQiLAgBEgUEBqIEIBohIPJYuKhww8GJigZdn9XPTCGX505lRUFQ7kHUbwerblspIioIARImBgqiBCBV1gOxgB9JjDGsdfRCVq47OYqktCkeupHuh0x3Z4ujvCAiKggBEiYIEKIEIM7aENaF2ZTxzk9ReXgO4ORpIVN/llWaJxI20QGiVfNmICIsCAESBQocogQgGiEgaQHer08w5wbnUpdu/M15Q8a4wIToJJb567AGldHVcMgK/gEK+wEKA2liYxIgIJ2E1/rWWUUqDGcKdvRFPs80VnRtqChYMx21HkSkaZYaCQgBGAEgASoBACInCAESAQEaIMUFwP1Isc8rZWGfErMUThnGC05rYlJe6Ta+k9BBYj6/IicIARIBARogQZHPfXC+ja61o/C7ZHNxGC1S8L4w92bL88qqyTY8/9IiJQgBEiEBRw4qcxAUpiTyaNx7yYQtxbFlBZ64x+CxXHNOHRtCU+QiJQgBEiEBAlCiFy2Axrwoiuqe53lrcQZeJX6sDOPLwzPITwsWhn4iJwgBEgEBGiA7wWTS1gDFzYWA0RKW6l21JcqTKs0mO5MenIlt4qXi9loDELgDYi1jb3Ntb3MxbTh3cGd4NnJsOXg0d2N3M3V0ZWxscGczMDlqZ2V0dzlxY3plN2USN2hlcm1lcyAxLjMuMCs5OTdlNTQxYiAoaHR0cHM6Ly9oZXJtZXMuaW5mb3JtYWwuc3lzdGVtcykSZgpQCkYKHy9jb3Ntb3MuY3J5cHRvLnNlY3AyNTZrMS5QdWJLZXkSIwohA6PuXfV1hiYSuYNBoLlgvDX8bisF+y4OLZXUQeOFpAG/EgQKAggBGAESEgoMCgVzdGFrZRIDMTYzELnyCRpA9ZT+C/gnOgZ7gOsKGIAwI+a4YcM4Oq/OosctONfnbkFotgsqJ/233y2TV+cb5oMY2ztjCN8PBQ2YipUJGYlcGg==

We see here the message corresponding to MsgUpdateClient, since MsgConnectopnOpenTry is not relevant in this discussion:

{
  "@type":"/ibc.core.client.v1.MsgUpdateClient",
  "client_id":"07-tendermint-0",
  "client_message":{
    "@type":"/ibc.lightclients.tendermint.v1.Header",
    "signed_header":{
      "header":{
        "version":{
          "block":"11",
          "app":"0"
        },
        "chain_id":"chain2",
        "height":"274",
        "time":"2023-03-10T12:18:59.963490Z",
        "last_block_id":{
          "hash":"QaFJ11ttWPSwatVqTFtwN66suRaiFOx6K0BYdGqozwI=",
          "part_set_header":{
            "total":1,
            "hash":"doCZ7HRXaRNxB3avE1p1wxYQCf4mb0Aoe0Lxt8UP87o="
          }
        },
        "last_commit_hash":"59k5tOUOVsSM+RNUheIJHkbF0W/qK18IFO4uQ8sGj5w=",
        "data_hash":"47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=",
        "validators_hash":"Rt7WE9jHiTQzsYgYzw/40ukY+aPOgkytdv3awfG6+vU=",
        "next_validators_hash":"Rt7WE9jHiTQzsYgYzw/40ukY+aPOgkytdv3awfG6+vU=",
        "consensus_hash":"BICRvH3cKD93v7+R1zxE2ljD34qcvIZ0Bdi389qtoi8=",
        "app_hash":"fT4FBaetd09IKeP1Q+ZpOh28eNL4hKch0+BnGrHwBd0=",
        "last_results_hash":"1UJkP9qDZnHevwSmZ345MZYx/jdBmHlgmfIXXeN8jDI=",
        "evidence_hash":"47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=",
        "proposer_address":"s7poqK9ORIoVBe1pHvyBbKE0v8o="
      },
      "commit":{
        "height":"274",
        "round":0,
        "block_id":{
          "hash":"9NsFABnF0Amg8q4tq3Fok7APdqsBeIt8h6AVYVgKg1o=",
          "part_set_header":{
            "total":1,
            "hash":"CrxX0hRwwhIlBwcM2D0Ak2U9EhYg1EJH8jAyNnrmtMI="
          }
        },
        "signatures":[
          {
            "block_id_flag":"BLOCK_ID_FLAG_COMMIT",
            "validator_address":"s7poqK9ORIoVBe1pHvyBbKE0v8o=",
            "timestamp":"2023-03-10T12:19:04.984670Z",
            "signature":"J+QlpnKobms7iwnLGM2EJ3SYrpMp3bLVjwPslGARoQF34mnBlOMmBoFnhcsRtuyLgOm0QfnePkwlyOjcz8mfAA=="
          }
        ]
      }
    },
    "validator_set":{
      "validators":[
        {
          "address":"s7poqK9ORIoVBe1pHvyBbKE0v8o=",
          "pub_key":{
            "ed25519":"t3uqGwyQ2KR0/IXLgDcrjJhDDFtxV8g4zhI4KDHVZDo="
          },
          "voting_power":"10",
          "proposer_priority":"0"
        }
      ],
      "proposer":{
        "address":"s7poqK9ORIoVBe1pHvyBbKE0v8o=",
        "pub_key":{
          "ed25519":"t3uqGwyQ2KR0/IXLgDcrjJhDDFtxV8g4zhI4KDHVZDo="
        },
        "voting_power":"10",
        "proposer_priority":"0"
      },
      "total_voting_power":"10"
    },
    "trusted_height":{
      "revision_number":"0",
      "revision_height":"6"
    },
    "trusted_validators":{
      "validators":[
        {
          "address":"s7poqK9ORIoVBe1pHvyBbKE0v8o=",
          "pub_key":{
            "ed25519":"t3uqGwyQ2KR0/IXLgDcrjJhDDFtxV8g4zhI4KDHVZDo="
          },
          "voting_power":"10",
          "proposer_priority":"0"
        }
      ],
      "proposer":{
        "address":"s7poqK9ORIoVBe1pHvyBbKE0v8o=",
        "pub_key":{
          "ed25519":"t3uqGwyQ2KR0/IXLgDcrjJhDDFtxV8g4zhI4KDHVZDo="
        },
        "voting_power":"10",
        "proposer_priority":"0"
      },
      "total_voting_power":"10"
    }
  },
  "signer":"cosmos1m8wpgx6rl9x4wcw3utellpg309jgetw9qcze7e"
}

The field client_message.signed_header.header.height is 274 (this is the height of chain2). The message contains the block header for height 274 in the object client_message.signed_header.header. The values are the base64-encoded string of the byte representation of the hashes. If we decode, for example, the value for app_hash, we can see that it matches the app hash found in the block at height 274. We can query chain2 locally for the block at height 274 by entering http://localhost:27010/block?height=274 on the browser (see the sample JSON result) and check that the value in the response for the field result.block.header.app_hash (7D3E0505A7AD774F4829E3F543E6693A1DBC78D2F884A721D3E0671AB1F005DD) matches.

Executing MsgClientUpdate

The message enters by the UpdateClient RPC handler method on the message server, and after unpacking the client message, the execution continues in ClientKeeper's UpdateClient function, to which the client message is passed in. The client message can be a either a consensus header or wrapper over two conflicting headers. In this tutorial we will explore what happens when the client message is a header, which is the case where the light client would get updated.

In UpdateClient function, after retrieving the client state using the unique client identifier (07-tendermint-0) and the isolated prefix store for the provided client identifier, we check that the client is active and verify that the client message is either valid header or misbehavior. In this tutorial the client message is a header for a CometBFT client, so the code executes the verify implementation for 07-tendermint. Assuming that no misbehavior is found, then the client state is updated with a call to client state's UpdateState function. Inside UpdateState the client and consensus states and any associated metadata is set.

After MsgUpdateClient successfully executes, the client state stored on chain1 for chain2 is updated and a new consensus state for height 274 is stored. Again, we can use ibc-go's either CLI or REST interface to check the updated client state (http://localhost:27001/ibc/core/client/v1/client_states/07-tendermint-0) or the new consensus state (http://localhost:27001/ibc/core/client/v1/consensus_states/07-tendermint-0/revision/0/height/274) associated with client ID 07-tendermint-0. We will use here the CLI:

> simd q ibc client state 07-tendermint-0 --node http://localhost:27000
client_state:
  '@type': /ibc.lightclients.tendermint.v1.ClientState
  allow_update_after_expiry: true
  allow_update_after_misbehaviour: true
  chain_id: chain2
  frozen_height:
    revision_height: "0"
    revision_number: "0"
  latest_height:
    revision_height: "274"
    revision_number: "0"
  max_clock_drift: 40s
  proof_specs:
  - inner_spec:
      child_order:
      - 0
      - 1
      child_size: 33
      empty_child: null
      hash: SHA256
      max_prefix_length: 12
      min_prefix_length: 4
    leaf_spec:
      hash: SHA256
      length: VAR_PROTO
      prefix: AA==
      prehash_key: NO_HASH
      prehash_value: SHA256
    max_depth: 0
    min_depth: 0
  - inner_spec:
      child_order:
      - 0
      - 1
      child_size: 32
      empty_child: null
      hash: SHA256
      max_prefix_length: 1
      min_prefix_length: 1
    leaf_spec:
      hash: SHA256
      length: VAR_PROTO
      prefix: AA==
      prehash_key: NO_HASH
      prehash_value: SHA256
    max_depth: 0
    min_depth: 0
  trust_level:
    denominator: "3"
    numerator: "1"
  trusting_period: 1209600s
  unbonding_period: 1814400s
  upgrade_path:
  - upgrade
  - upgradedIBCState
proof: CuwCCukCCiNjbGllbnRzLzA3LXRlbmRlcm1pbnQtMC9jbGllbnRTdGF0ZRKpAQorL2liYy5saWdodGNsaWVudHMudGVuZGVybWludC52MS5DbGllbnRTdGF0ZRJ6CgZjaGFpbjISBAgBEAMaBAiA6kkiBAiA324qAggoMgA6AxCSAkIZCgkIARgBIAEqAQASDAoCAAEQIRgEIAwwAUIZCgkIARgBIAEqAQASDAoCAAEQIBgBIAEwAUoHdXBncmFkZUoQdXBncmFkZWRJQkNTdGF0ZVABWAEaDAgBGAEgASoEAAL2BiIsCAESBQIE9gYgGiEgmsPysDsTrufN7LQG1sQzViMH9teO5oGrf2gU2LKFnk0iLAgBEgUGDPYGIBohIKakgv/cU6T6v5bzybW5rJcwFsZGRbzK2gMJhYCaRaMyIiwIARIFChz2BiAaISBcgyEv2sIOBiA2nr0T5J3SJMl6hydzHUwE9BWKx4VT7gr+AQr7AQoDaWJjEiDkuXYMwaogDUy88nS8kqcVpvDyxpnIanjxUPgn9elSbRoJCAEYASABKgEAIicIARIBARogxQXA/UixzytlYZ8SsxROGcYLTmtiUl7pNr6T0EFiPr8iJwgBEgEBGiBqlJrA3IAoCCv7bW5siDYWxq7hNhb85zk/05M4xcDm8SIlCAESIQFHDipzEBSmJPJo3HvJhC3FsWUFnrjH4LFcc04dG0JT5CIlCAESIQElI6WBNaK+iJFEO9Yq2pOjmRvICswL+f/oMXng/YJtbCInCAESAQEaIASj4Dp7c5IPqYbcY2zMSnQLN3UZZ0tmlmHUbvABxQdh
proof_height:
  revision_height: "466"
  revision_number: "0"

We see that client_state.latest_height.revision_height is now 274, as we expected. If we query all consensus states for client with identifier 07-tendermint-0:

> simd q ibc client consensus-states 07-tendermint-0 --node http://localhost:27000
consensus_states:
- consensus_state:
    '@type': /ibc.lightclients.tendermint.v1.ConsensusState
    next_validators_hash: 46DED613D8C7893433B18818CF0FF8D2E918F9A3CE824CAD76FDDAC1F1BAFAF5
    root:
      hash: fT4FBaetd09IKeP1Q+ZpOh28eNL4hKch0+BnGrHwBd0=
    timestamp: "2023-03-10T12:18:59.963490Z"
  height:
    revision_height: "274"
    revision_number: "0"
- consensus_state:
    '@type': /ibc.lightclients.tendermint.v1.ConsensusState
    next_validators_hash: 46DED613D8C7893433B18818CF0FF8D2E918F9A3CE824CAD76FDDAC1F1BAFAF5
    root:
      hash: EIP4I6oX9Nf8icn2zA11HBeAwjEfabYIUsw9TDd/2iI=
    timestamp: "2023-03-10T11:56:35.188345Z"
  height:
    revision_height: "6"
    revision_number: "0"
pagination:
  next_key: null
  total: "0"

We see that there are now two consensus states, one for height 6 and another one for height 274. We can also query the particular consensus state for height 274:

> simd q ibc client consensus-state 07-tendermint-0 0-274 --node http://localhost:27000
consensus_state:
  '@type': /ibc.lightclients.tendermint.v1.ConsensusState
  next_validators_hash: 46DED613D8C7893433B18818CF0FF8D2E918F9A3CE824CAD76FDDAC1F1BAFAF5
  root:
    hash: fT4FBaetd09IKeP1Q+ZpOh28eNL4hKch0+BnGrHwBd0=
  timestamp: "2023-03-10T12:18:59.963490Z"
proof: Cv8CCvwCCi1jbGllbnRzLzA3LXRlbmRlcm1pbnQtMC9jb25zZW5zdXNTdGF0ZXMvMC0yNzQShgEKLi9pYmMubGlnaHRjbGllbnRzLnRlbmRlcm1pbnQudjEuQ29uc2Vuc3VzU3RhdGUSVAoMCLPArKAGENDhtssDEiIKIH0+BQWnrXdPSCnj9UPmaTodvHjS+ISnIdPgZxqx8AXdGiBG3tYT2MeJNDOxiBjPD/jS6Rj5o86CTK12/drB8br69RoMCAEYASABKgQAAvYGIiwIARIFAgT2BiAaISCM5NtjNwyVZKVEfMC3ZOuvPkmElbSS6QckuafWd5ImoyIsCAESBQQI9gYgGiEgIAKIf+G3cH4ZJsNJiZjjjM216NKKysB9QtQHA/4km/siKggBEiYGDPYGIMXAFEQG6yYdw6qGq6fEfeJr27YXnZmIY4zzskluGa2bICIsCAESBQoc9gYgGiEgXIMhL9rCDgYgNp69E+Sd0iTJeocncx1MBPQViseFU+4K/gEK+wEKA2liYxIg5Ll2DMGqIA1MvPJ0vJKnFabw8saZyGp48VD4J/XpUm0aCQgBGAEgASoBACInCAESAQEaIMUFwP1Isc8rZWGfErMUThnGC05rYlJe6Ta+k9BBYj6/IicIARIBARogokyaFwCGsQBpGfGlndB2fFlyj2gS3MGilU1THntJ3gMiJQgBEiEBRw4qcxAUpiTyaNx7yYQtxbFlBZ64x+CxXHNOHRtCU+QiJQgBEiEB3JYvSeIWxkIcrgI3144jF0LMlCkOuXkuDIq6HK5GThMiJwgBEgEBGiAcZWOWZfxhERI5Mk+LEsEy8v7tX4KJf06aO2faGmTV2w==
proof_height:
  revision_height: "479"
  revision_number: "0"

And just for completeness we can also query what are the heights for all the consensus states stored of client 07-tendermint-0:

> simd q ibc client consensus-state-heights 07-tendermint-0 --node http://localhost:27000
consensus_state_heights:
- revision_height: "274"
  revision_number: "0"
- revision_height: "6"
  revision_number: "0"
pagination:
  next_key: null
  total: "0"

State verification

The transaction we just executed contained two messages (MsgUpdateClient and MsgConnectionOpenTry). When handling MsgConnectionOpenTry there are three verifications happening in ConnOpenTry. We will take a closer walkthrough to what happens in one if them (verification of counterparty connection state), since the information stored in the consensus state of the client is crutial for these verifications.

Function VerifyConnectionState state calls 07-tendermint VerifyMembership (implementation here). The function receives a proof (which in our situation is the proof_try field of MsgConnectionOpenTry) and a path and value in a merkle tree. The path in this case contains two keys (ibc and connections/connection-0). This is because this path corresponds to a merkle tree of a multi store. The first key (ibc) is the key in the multi store for the IBC store and the second key (connections/connection-0) is the path in the merkle tree where IBC stores its data. The value is the connection end that we expect the counterparty to have stored during ConnOpenInit. In this function we retrieve the [consensus state at the proof_height submitted in MsgConnectionOpenTry] https://github.com/cosmos/ibc-go/blob/v7.0.0/modules/light-clients/07-tendermint/client_state.go#L242) because we will use its root hash to verify the merkle tree proof. Inside VerifyMembership we call the merkle proof object's VerifyMembership, which calls verifyChainedMembershipProof. The proof contains actually two proofs: one to proof to verify that the connection end is stored in the IBC store, and another one to verify that the state of IBC store of the counterparty is as expected.

proof_try

It is obtained by performing an ABCI RPC query to the ibc store of chain2. This proof is used to verify that chain2 has stored a connection end with state STATE_TRYOPEN at the query height (144 in this tutorial). We can make the same ABCI RPC query on the browser by simply using Tendermint's /abci_query REST endpoint:

path: "store/ibc/key"
data: "connections/connection-0" (in hex: 0x636f6e6e656374696f6e732f636f6e6e656374696f6e2d30)
prove: true (so that the response contains the merkle proof)
height: 144
URL: http://localhost:27010/abci_query?path="store/ibc/key"&data=0x636f6e6e656374696f6e732f636f6e6e656374696f6e2d30&prove=true&height=144

See sample JSON response.

The response to this ABCI RPC query contains both the value stored at the queried path (connections/connection-0) and the proof needed to verify it. If we take the base64-encoded byte string from the value for the field result.response.value and we decode it, then we can inspect the information for the connection end stored in chain2:

{
  ClientId:07-tendermint-0 
  Versions:[
    identifier:"1" 
    features:"ORDER_ORDERED" 
    features:"ORDER_UNORDERED" 
  ]
  State:STATE_TRYOPEN 
  Counterparty:{
    ClientId:07-tendermint-0 
    ConnectionId:connection-0 
    Prefix:{
      KeyPrefix:[105 98 99]
    }
  } 
  DelayPeriod:0
}

We see that the connection end is in state STATE_TRYOPEN. The key_prefix is the byte representation of the string ibc.