Skip to content
This repository was archived by the owner on Nov 1, 2024. It is now read-only.

Real Time Messages #64

Merged
merged 13 commits into from
Apr 26, 2022
Merged

Real Time Messages #64

merged 13 commits into from
Apr 26, 2022

Conversation

dtemir
Copy link
Contributor

@dtemir dtemir commented Mar 12, 2022

Description

With this PR I'm trying to make the current generate_labgraph_monitor() include the message data content, such as it will send the quantitative data to the front-end. In other words, I'm trying to update the structure of the WebSocketsAPI Message from this:

"Attenuator": [{
		"name": "RandomMessage",
		"fields": {
			"timestamp": {
				"type": "float"
			},
			"data": {
				"type": "ndarray"
			}
		}
	}
]

to this:

"Attenuator": [{
		"name": "RandomMessage",
		"fields": {
			"timestamp": {
				"type": "float",
				"content": 1647047794.474803,
			},
			"data": {
				"type": "ndarray",
				"content": array([0.77135353, 0.61299748, 0.34146919, 0.46154968, 0.19577749],
			}
		}
	}
]

It will fix #53 once finished (should only be merged after #58)

How To Run

By running this from your LabGraph root directory

labgraph$ python labgraph/examples/labgraph_monitor_example.py 

you will start getting the following print result:

"name": "WSSenderNode",
    "nodes": {
        "Serializer": {
            "upstreams": {
                "NoiseGenerator": [
                    {
                        "name": "RandomMessage",
                        "fields": {
                            "timestamp": {
                                "type": "float",
                                "content": 1648234610.0660968
                            },
                            "data": {
                                "type": "ndarray",
                                "content": [
                                    0.43446801511555844,
                                    0.12821419443680548,
                                    0.15750558665376602,
                                    ...
                                ]
                            }
                        }
                    }
                ],
...
}

Type of change

Please delete options that are not relevant.

  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

Feature/Issue validation/testing

Once finished, a good test would be to visit each field in fields and check content based on the given type (i.e. using python's type())

Checklist:

  • Have you added tests that prove your fix is effective or that this feature works?
  • New and existing unit tests pass locally with these changes?
  • Has code been commented, particularly in hard-to-understand areas?
  • Have you made corresponding changes to the documentation?

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 12, 2022
@fzchriha fzchriha mentioned this pull request Mar 19, 2022
4 tasks
@dtemir
Copy link
Contributor Author

dtemir commented Mar 25, 2022

Update

I used the last commit to convert the graph to send the following serialized real-time message. Look for "data_content"

It works by running the graph simultaneously with the WebSockets node, which helps us subscribe to all of the topics and then put the information in the serialized message.

Click to view
  {
      "stream_batch": {
          "stream_id": "LABGRAPH.MONITOR",
          "labgraph.monitor": {
              "samples": [
                  {
                      "data": {
                          "name": "WSSenderNode",
                          "nodes": {
                              "Serializer": {
                                  "upstreams": {
                                      "NoiseGenerator": [
                                          {
                                              "name": "RandomMessage",
                                              "fields": {
                                                  "timestamp": "float",
                                                  "data": "ndarray",
                                                  "data_content": "0.9776421478209004"
                                              }
                                          }
                                      ],
                                      "RollingAverager": [
                                          {
                                              "name": "RandomMessage",
                                              "fields": {
                                                  "timestamp": "float",
                                                  "data": "ndarray",
                                                  "data_content": "0.5336320474273315"
                                              }
                                          }
                                      ],
                                      "Amplifier": [
                                          {
                                              "name": "RandomMessage",
                                              "fields": {
                                                  "timestamp": "float",
                                                  "data": "ndarray",
                                                  "data_content": "1.1731705773850805"
                                              }
                                          }
                                      ],
                                      "Attenuator": [
                                          {
                                              "name": "RandomMessage",
                                              "fields": {
                                                  "timestamp": "float",
                                                  "data": "ndarray",
                                                  "data_content": "1.738520901856136"
                                              }
                                          }
                                      ]
                                  }
                              },
                              "WSAPIServerNode": {
                                  "upstreams": {
                                      "Serializer": [
                                          {
                                              "name": "WSStreamMessage",
                                              "fields": {
                                                  "samples": "float64",
                                                  "stream_name": "str",
                                                  "stream_id": "str"
                                              }
                                          }
                                      ]
                                  }
                              },
                              "NoiseGenerator": {
                                  "upstreams": {}
                              },
                              "RollingAverager": {
                                  "upstreams": {
                                      "NoiseGenerator": [
                                          {
                                              "name": "RandomMessage",
                                              "fields": {
                                                  "timestamp": "float",
                                                  "data": "ndarray",
                                                  "data_content": "0.9776421478209004"
                                              }
                                          }
                                      ]
                                  }
                              },
                              "Amplifier": {
                                  "upstreams": {
                                      "NoiseGenerator": [
                                          {
                                              "name": "RandomMessage",
                                              "fields": {
                                                  "timestamp": "float",
                                                  "data": "ndarray",
                                                  "data_content": "0.5336320474273315"
                                              }
                                          }
                                      ]
                                  }
                              },
                              "Attenuator": {
                                  "upstreams": {
                                      "NoiseGenerator": [
                                          {
                                              "name": "RandomMessage",
                                              "fields": {
                                                  "timestamp": "float",
                                                  "data": "ndarray",
                                                  "data_content": "1.1731705773850805"
                                              }
                                          }
                                      ]
                                  }
                              }
                          }
                      },
                      "produced_timestamp_s": 1648231824.1736534,
                      "timestamp_s": 1648231824.1736543
                  }
              ],
              "batch_num": 21
          }
      }
  }

@dtemir
Copy link
Contributor Author

dtemir commented Mar 25, 2022

Updated Scheme

Now converts each message attribute like timestamp and data to dictionary to which we save the data

Click to view
{
    "stream_batch": {
        "stream_id": "LABGRAPH.MONITOR",
        "labgraph.monitor": {
            "samples": [
                {
                    "data": {
                        "name": "WSSenderNode",
                        "nodes": {
                            "Serializer": {
                                "upstreams": {
                                    "NoiseGenerator": [
                                        {
                                            "name": "RandomMessage",
                                            "fields": {
                                                "timestamp": {
                                                    "type": "float",
                                                    "content": 1648234610.0660968
                                                },
                                                "data": {
                                                    "type": "ndarray",
                                                    "content": [
                                                        0.43446801511555844,
                                                        0.12821419443680548,
                                                        0.15750558665376602,
                                                        0.918397185853851,
                                                        0.8904501800365764,
                                                        0.2128943509383302,
                                                        0.15037686911084303,
                                                        0.9984199676408996,
                                                        0.7987219432051146,
                                                        0.3053190512014433,
                                                        0.029643302165308527,
                                                        0.3149755806905876,
                                                        0.8256521752831728,
                                                        0.218946714839382,
                                                        0.16412791624963385,
                                                        0.47384232396820614,
                                                        0.17635652572097116,
                                                        0.3522408024198592,
                                                        0.6344382551421732,
                                                        0.13512293194088876,
                                                        0.9112768440888541,
                                                        0.5575263995355353,
                                                        0.8842641348610316,
                                                        0.8055040832166873,
                                                        0.6148514882974205,
                                                        0.6321438310582602,
                                                        0.4129430926205505,
                                                        0.9508687194889407,
                                                        0.8594184676315734,
                                                        0.3256104460576392,
                                                        0.024813557390513497,
                                                        0.37591037000000693,
                                                        0.009083227260460403,
                                                        0.9118595626643696,
                                                        0.10036942153710193,
                                                        0.15275562707579038,
                                                        0.8449439937066034,
                                                        0.5954815608941298,
                                                        0.4574501763192581,
                                                        0.8070403841107208,
                                                        0.20431767639607468,
                                                        0.8498068033373913,
                                                        0.3771679565768835,
                                                        0.1920609496332747,
                                                        0.9230910024016403,
                                                        0.7072461030369684,
                                                        0.5144158670506508,
                                                        0.6095415130808768,
                                                        0.3846348701840029,
                                                        0.3446549895913138,
                                                        0.4958869601818289,
                                                        0.14696837251787687,
                                                        0.4293475902764491,
                                                        0.2744988473857012,
                                                        0.33779206059418154,
                                                        0.0946687948868602,
                                                        0.0030355299113816736,
                                                        0.42065938294056204,
                                                        0.5755944310424821,
                                                        0.05969300889279572,
                                                        0.10669476370014852,
                                                        0.5544805987916578,
                                                        0.6856940018619432,
                                                        0.2204089804341236,
                                                        0.7230813333542027,
                                                        0.887454035473332,
                                                        0.9145841102843242,
                                                        0.16610386682539413,
                                                        0.5029274279405064,
                                                        0.03367008369429858,
                                                        0.5227868509233211,
                                                        0.5309931554499453,
                                                        0.584994200664403,
                                                        0.5299053653442589,
                                                        0.35784814095456496,
                                                        0.7374994485892563,
                                                        0.8519498298076369,
                                                        0.1705399689382,
                                                        0.43126360810803877,
                                                        0.5494419019187354,
                                                        0.9963149370731197,
                                                        0.18890279549704436,
                                                        0.08505362827722085,
                                                        0.750326097132459,
                                                        0.5381685356419156,
                                                        0.7822082142388115,
                                                        0.25850960902091047,
                                                        0.4126265432106131,
                                                        0.4292478088144258,
                                                        0.5425835702257386,
                                                        0.32350921166538027,
                                                        0.4025415090587563,
                                                        0.7233938181362229,
                                                        0.4140250254563149,
                                                        0.4699557764879505,
                                                        0.33300863599384767,
                                                        0.8397157915591548,
                                                        0.1620502014393731,
                                                        0.05265436323205286,
                                                        0.43196911527568416
                                                    ]
                                                }
                                            }
                                        }
                                    ],
                                    "RollingAverager": [
                                        {
                                            "name": "RandomMessage",
                                            "fields": {
                                                "timestamp": {
                                                    "type": "float",
                                                    "content": 1648234610.0709183
                                                },
                                                "data": {
                                                    "type": "ndarray",
                                                    "content": [
                                                        0.23204516321510554,
                                                        0.10419077713061531,
                                                        0.512400436968537,
                                                        0.5073259762855388,
                                                        0.49347308334341505,
                                                        0.20979415240894533,
                                                        0.4576573506367747,
                                                        0.9364192311267716,
                                                        0.568443179253336,
                                                        0.36011063955062217,
                                                        0.16597987634207306,
                                                        0.16351285745067007,
                                                        0.8033679184915117,
                                                        0.44194784928414793,
                                                        0.5062584023003878,
                                                        0.6245738352167437,
                                                        0.5089710386514623,
                                                        0.32308070894853913,
                                                        0.6996918502437273,
                                                        0.11385251621378084,
                                                        0.9412948327464826,
                                                        0.5798990960129069,
                                                        0.5760817490276379,
                                                        0.5477955624133428,
                                                        0.4869620267685492,
                                                        0.6758198286154529,
                                                        0.41229841319153593,
                                                        0.5501437228550873,
                                                        0.7475084974311523,
                                                        0.5667035513134195,
                                                        0.3627662655984002,
                                                        0.41390626151244503,
                                                        0.05960994050933671,
                                                        0.6821012941123827,
                                                        0.4782599669021897,
                                                        0.41543822610820785,
                                                        0.7281781030944501,
                                                        0.6876163849595391,
                                                        0.5149822855466222,
                                                        0.7575988542838552,
                                                        0.4718310589350277,
                                                        0.7880138070257856,
                                                        0.38452205372559967,
                                                        0.2667740359177369,
                                                        0.5657045700740826,
                                                        0.7008741188472803,
                                                        0.7471275551137775,
                                                        0.6654322206448653,
                                                        0.2685181124311307,
                                                        0.26579558378919926,
                                                        0.6786890490552794,
                                                        0.32658065675004166,
                                                        0.5392097557488564,
                                                        0.5431619383926776,
                                                        0.6613050958501463,
                                                        0.33605182636353975,
                                                        0.3867424407659782,
                                                        0.28493014069670464,
                                                        0.5092298527856076,
                                                        0.40977991590376533,
                                                        0.2051251921548219,
                                                        0.7486191062169261,
                                                        0.6931163934797315,
                                                        0.4460150307005046,
                                                        0.8187973992035893,
                                                        0.9407544339269771,
                                                        0.8243539376447067,
                                                        0.5159491406892626,
                                                        0.41103964899718426,
                                                        0.1916033241868964,
                                                        0.6230383101699932,
                                                        0.7263831819493962,
                                                        0.6288662926347055,
                                                        0.5070116810452929,
                                                        0.3105727485993886,
                                                        0.6552770319179548,
                                                        0.8856080073937337,
                                                        0.481292764319486,
                                                        0.5061356414122178,
                                                        0.5861614756938085,
                                                        0.8190883367738673,
                                                        0.30820739180910567,
                                                        0.49738298349922605,
                                                        0.47527935287348433,
                                                        0.358111480051862,
                                                        0.8881257496044034,
                                                        0.6211562243789921,
                                                        0.49947102864151866,
                                                        0.22955318689170096,
                                                        0.3857300266007372,
                                                        0.3971842781835192,
                                                        0.2726147635200227,
                                                        0.7315644998957413,
                                                        0.4107139579334986,
                                                        0.2530438288635256,
                                                        0.20208036503932392,
                                                        0.7095508203989023,
                                                        0.4240858044755369,
                                                        0.17555042808300103,
                                                        0.45780100739924484
                                                    ]
                                                }
                                            }
                                        }
                                    ],
                                    "Amplifier": [
                                        {
                                            "name": "RandomMessage",
                                            "fields": {
                                                "timestamp": {
                                                    "type": "float",
                                                    "content": 1648234610.070494
                                                },
                                                "data": {
                                                    "type": "ndarray",
                                                    "content": [
                                                        0.5213616181386701,
                                                        0.1538570333241666,
                                                        0.18900670398451921,
                                                        1.102076623024621,
                                                        1.0685402160438917,
                                                        0.25547322112599624,
                                                        0.18045224293301163,
                                                        1.1981039611690794,
                                                        0.9584663318461375,
                                                        0.366382861441732,
                                                        0.035571962598370234,
                                                        0.37797069682870515,
                                                        0.9907826103398073,
                                                        0.2627360578072584,
                                                        0.19695349949956062,
                                                        0.5686107887618473,
                                                        0.21162783086516537,
                                                        0.42268896290383107,
                                                        0.7613259061706078,
                                                        0.16214751832906651,
                                                        1.093532212906625,
                                                        0.6690316794426423,
                                                        1.061116961833238,
                                                        0.9666048998600247,
                                                        0.7378217859569046,
                                                        0.7585725972699122,
                                                        0.4955317111446606,
                                                        1.1410424633867289,
                                                        1.031302161157888,
                                                        0.39073253526916707,
                                                        0.029776268868616194,
                                                        0.4510924440000083,
                                                        0.010899872712552482,
                                                        1.0942314751972435,
                                                        0.12044330584452231,
                                                        0.18330675249094844,
                                                        1.013932792447924,
                                                        0.7145778730729558,
                                                        0.5489402115831097,
                                                        0.9684484609328649,
                                                        0.2451812116752896,
                                                        1.0197681640048695,
                                                        0.4526015478922602,
                                                        0.23047313955992962,
                                                        1.1077092028819684,
                                                        0.8486953236443621,
                                                        0.6172990404607809,
                                                        0.7314498156970521,
                                                        0.46156184422080343,
                                                        0.41358598750957654,
                                                        0.5950643522181946,
                                                        0.17636204702145222,
                                                        0.5152171083317388,
                                                        0.3293986168628414,
                                                        0.40535047271301783,
                                                        0.11360255386423224,
                                                        0.0036426358936580082,
                                                        0.5047912595286744,
                                                        0.6907133172509785,
                                                        0.07163161067135486,
                                                        0.12803371644017822,
                                                        0.6653767185499894,
                                                        0.8228328022343318,
                                                        0.2644907765209483,
                                                        0.8676976000250431,
                                                        1.0649448425679984,
                                                        1.097500932341189,
                                                        0.19932464019047294,
                                                        0.6035129135286076,
                                                        0.04040410043315829,
                                                        0.6273442211079853,
                                                        0.6371917865399344,
                                                        0.7019930407972835,
                                                        0.6358864384131107,
                                                        0.42941776914547797,
                                                        0.8849993383071075,
                                                        1.0223397957691642,
                                                        0.20464796272584,
                                                        0.5175163297296465,
                                                        0.6593302823024825,
                                                        1.1955779244877436,
                                                        0.22668335459645322,
                                                        0.10206435393266501,
                                                        0.9003913165589508,
                                                        0.6458022427702986,
                                                        0.9386498570865738,
                                                        0.31021153082509256,
                                                        0.4951518518527357,
                                                        0.515097370577311,
                                                        0.6511002842708863,
                                                        0.3882110539984563,
                                                        0.4830498108705075,
                                                        0.8680725817634675,
                                                        0.4968300305475779,
                                                        0.5639469317855406,
                                                        0.39961036319261717,
                                                        1.0076589498709856,
                                                        0.1944602417272477,
                                                        0.06318523587846343,
                                                        0.5183629383308209
                                                    ]
                                                }
                                            }
                                        }
                                    ],
                                    "Attenuator": [
                                        {
                                            "name": "RandomMessage",
                                            "fields": {
                                                "timestamp": {
                                                    "type": "float",
                                                    "content": 1648234610.0707996
                                                },
                                                "data": {
                                                    "type": "ndarray",
                                                    "content": [
                                                        0.772605525600477,
                                                        0.2280006620416982,
                                                        0.28008894171249343,
                                                        1.6331668058415931,
                                                        1.5834692208244956,
                                                        0.3785856407872332,
                                                        0.26741209008593025,
                                                        1.7754696710275393,
                                                        1.4203507859479332,
                                                        0.5429425822441463,
                                                        0.05271407388613037,
                                                        0.5601145898071253,
                                                        1.4682402631599138,
                                                        0.38934843489453647,
                                                        0.2918652940793166,
                                                        0.8426240483176537,
                                                        0.3136111785156027,
                                                        0.626382566318824,
                                                        1.1282084860603474,
                                                        0.24028632769457317,
                                                        1.620504848688459,
                                                        0.9914377168471764,
                                                        1.5724687040592538,
                                                        1.432411325886514,
                                                        1.0933777418710906,
                                                        1.1241283589540274,
                                                        0.7343281991249208,
                                                        1.6909102655172594,
                                                        1.5282861655964293,
                                                        0.5790263519178892,
                                                        0.04412543819736929,
                                                        0.6684736709911255,
                                                        0.016152516013980985,
                                                        1.6215410851331453,
                                                        0.17848487571694557,
                                                        0.2716421863964622,
                                                        1.50254650664451,
                                                        1.05893259879587,
                                                        0.8134742296672115,
                                                        1.4351432981339982,
                                                        0.3633339170421352,
                                                        1.5111939408858792,
                                                        0.6707100113071265,
                                                        0.341538032205375,
                                                        1.6415137231630268,
                                                        1.2576811828609074,
                                                        0.9147751445734922,
                                                        1.0839351222756939,
                                                        0.6839882700312063,
                                                        0.6128928715574126,
                                                        0.8818255709981374,
                                                        0.2613508307754707,
                                                        0.763499979538437,
                                                        0.488135648385409,
                                                        0.6006886662292532,
                                                        0.16834756872050155,
                                                        0.005398020339967306,
                                                        0.74804991932288,
                                                        1.0235677252559146,
                                                        0.10615084863732895,
                                                        0.1897331014469424,
                                                        0.9860214320972578,
                                                        1.2193555250982844,
                                                        0.39194875169367377,
                                                        1.2858406468872692,
                                                        1.5781412386381781,
                                                        1.6263860920673812,
                                                        0.29537908630344567,
                                                        0.8943454898504365,
                                                        0.05987481656785843,
                                                        0.9296610928360297,
                                                        0.9442541952082347,
                                                        1.0402831420336858,
                                                        0.9423198004608486,
                                                        0.636353980980209,
                                                        1.3114800843413335,
                                                        1.5150048407330852,
                                                        0.3032677153514785,
                                                        0.7669071945976204,
                                                        0.9770612211947125,
                                                        1.7717263385113537,
                                                        0.33592195173118733,
                                                        0.15124911591448614,
                                                        1.3342894493455166,
                                                        0.9570140260628166,
                                                        1.390984761744193,
                                                        0.45970231501909725,
                                                        0.7337652858269691,
                                                        0.7633225402190175,
                                                        0.9648651911578388,
                                                        0.5752897700624694,
                                                        0.7158312772451828,
                                                        1.2863963321410863,
                                                        0.7362521780098057,
                                                        0.8357126809573765,
                                                        0.5921824007530058,
                                                        1.4932493024141809,
                                                        0.28817053661229697,
                                                        0.0936341699842701,
                                                        0.768161783467479
                                                    ]
                                                }
                                            }
                                        }
                                    ]
                                }
                            },
                            "WSAPIServerNode": {
                                "upstreams": {
                                    "Serializer": [
                                        {
                                            "name": "WSStreamMessage",
                                            "fields": {
                                                "samples": {
                                                    "type": "float64"
                                                },
                                                "stream_name": {
                                                    "type": "str"
                                                },
                                                "stream_id": {
                                                    "type": "str"
                                                }
                                            }
                                        }
                                    ]
                                }
                            },
                            "NoiseGenerator": {
                                "upstreams": {}
                            },
                            "RollingAverager": {
                                "upstreams": {
                                    "NoiseGenerator": [
                                        {
                                            "name": "RandomMessage",
                                            "fields": {
                                                "timestamp": {
                                                    "type": "float",
                                                    "content": 1648234610.0660968
                                                },
                                                "data": {
                                                    "type": "ndarray",
                                                    "content": [
                                                        0.43446801511555844,
                                                        0.12821419443680548,
                                                        0.15750558665376602,
                                                        0.918397185853851,
                                                        0.8904501800365764,
                                                        0.2128943509383302,
                                                        0.15037686911084303,
                                                        0.9984199676408996,
                                                        0.7987219432051146,
                                                        0.3053190512014433,
                                                        0.029643302165308527,
                                                        0.3149755806905876,
                                                        0.8256521752831728,
                                                        0.218946714839382,
                                                        0.16412791624963385,
                                                        0.47384232396820614,
                                                        0.17635652572097116,
                                                        0.3522408024198592,
                                                        0.6344382551421732,
                                                        0.13512293194088876,
                                                        0.9112768440888541,
                                                        0.5575263995355353,
                                                        0.8842641348610316,
                                                        0.8055040832166873,
                                                        0.6148514882974205,
                                                        0.6321438310582602,
                                                        0.4129430926205505,
                                                        0.9508687194889407,
                                                        0.8594184676315734,
                                                        0.3256104460576392,
                                                        0.024813557390513497,
                                                        0.37591037000000693,
                                                        0.009083227260460403,
                                                        0.9118595626643696,
                                                        0.10036942153710193,
                                                        0.15275562707579038,
                                                        0.8449439937066034,
                                                        0.5954815608941298,
                                                        0.4574501763192581,
                                                        0.8070403841107208,
                                                        0.20431767639607468,
                                                        0.8498068033373913,
                                                        0.3771679565768835,
                                                        0.1920609496332747,
                                                        0.9230910024016403,
                                                        0.7072461030369684,
                                                        0.5144158670506508,
                                                        0.6095415130808768,
                                                        0.3846348701840029,
                                                        0.3446549895913138,
                                                        0.4958869601818289,
                                                        0.14696837251787687,
                                                        0.4293475902764491,
                                                        0.2744988473857012,
                                                        0.33779206059418154,
                                                        0.0946687948868602,
                                                        0.0030355299113816736,
                                                        0.42065938294056204,
                                                        0.5755944310424821,
                                                        0.05969300889279572,
                                                        0.10669476370014852,
                                                        0.5544805987916578,
                                                        0.6856940018619432,
                                                        0.2204089804341236,
                                                        0.7230813333542027,
                                                        0.887454035473332,
                                                        0.9145841102843242,
                                                        0.16610386682539413,
                                                        0.5029274279405064,
                                                        0.03367008369429858,
                                                        0.5227868509233211,
                                                        0.5309931554499453,
                                                        0.584994200664403,
                                                        0.5299053653442589,
                                                        0.35784814095456496,
                                                        0.7374994485892563,
                                                        0.8519498298076369,
                                                        0.1705399689382,
                                                        0.43126360810803877,
                                                        0.5494419019187354,
                                                        0.9963149370731197,
                                                        0.18890279549704436,
                                                        0.08505362827722085,
                                                        0.750326097132459,
                                                        0.5381685356419156,
                                                        0.7822082142388115,
                                                        0.25850960902091047,
                                                        0.4126265432106131,
                                                        0.4292478088144258,
                                                        0.5425835702257386,
                                                        0.32350921166538027,
                                                        0.4025415090587563,
                                                        0.7233938181362229,
                                                        0.4140250254563149,
                                                        0.4699557764879505,
                                                        0.33300863599384767,
                                                        0.8397157915591548,
                                                        0.1620502014393731,
                                                        0.05265436323205286,
                                                        0.43196911527568416
                                                    ]
                                                }
                                            }
                                        }
                                    ]
                                }
                            },
                            "Amplifier": {
                                "upstreams": {
                                    "NoiseGenerator": [
                                        {
                                            "name": "RandomMessage",
                                            "fields": {
                                                "timestamp": {
                                                    "type": "float",
                                                    "content": 1648234610.0709183
                                                },
                                                "data": {
                                                    "type": "ndarray",
                                                    "content": [
                                                        0.23204516321510554,
                                                        0.10419077713061531,
                                                        0.512400436968537,
                                                        0.5073259762855388,
                                                        0.49347308334341505,
                                                        0.20979415240894533,
                                                        0.4576573506367747,
                                                        0.9364192311267716,
                                                        0.568443179253336,
                                                        0.36011063955062217,
                                                        0.16597987634207306,
                                                        0.16351285745067007,
                                                        0.8033679184915117,
                                                        0.44194784928414793,
                                                        0.5062584023003878,
                                                        0.6245738352167437,
                                                        0.5089710386514623,
                                                        0.32308070894853913,
                                                        0.6996918502437273,
                                                        0.11385251621378084,
                                                        0.9412948327464826,
                                                        0.5798990960129069,
                                                        0.5760817490276379,
                                                        0.5477955624133428,
                                                        0.4869620267685492,
                                                        0.6758198286154529,
                                                        0.41229841319153593,
                                                        0.5501437228550873,
                                                        0.7475084974311523,
                                                        0.5667035513134195,
                                                        0.3627662655984002,
                                                        0.41390626151244503,
                                                        0.05960994050933671,
                                                        0.6821012941123827,
                                                        0.4782599669021897,
                                                        0.41543822610820785,
                                                        0.7281781030944501,
                                                        0.6876163849595391,
                                                        0.5149822855466222,
                                                        0.7575988542838552,
                                                        0.4718310589350277,
                                                        0.7880138070257856,
                                                        0.38452205372559967,
                                                        0.2667740359177369,
                                                        0.5657045700740826,
                                                        0.7008741188472803,
                                                        0.7471275551137775,
                                                        0.6654322206448653,
                                                        0.2685181124311307,
                                                        0.26579558378919926,
                                                        0.6786890490552794,
                                                        0.32658065675004166,
                                                        0.5392097557488564,
                                                        0.5431619383926776,
                                                        0.6613050958501463,
                                                        0.33605182636353975,
                                                        0.3867424407659782,
                                                        0.28493014069670464,
                                                        0.5092298527856076,
                                                        0.40977991590376533,
                                                        0.2051251921548219,
                                                        0.7486191062169261,
                                                        0.6931163934797315,
                                                        0.4460150307005046,
                                                        0.8187973992035893,
                                                        0.9407544339269771,
                                                        0.8243539376447067,
                                                        0.5159491406892626,
                                                        0.41103964899718426,
                                                        0.1916033241868964,
                                                        0.6230383101699932,
                                                        0.7263831819493962,
                                                        0.6288662926347055,
                                                        0.5070116810452929,
                                                        0.3105727485993886,
                                                        0.6552770319179548,
                                                        0.8856080073937337,
                                                        0.481292764319486,
                                                        0.5061356414122178,
                                                        0.5861614756938085,
                                                        0.8190883367738673,
                                                        0.30820739180910567,
                                                        0.49738298349922605,
                                                        0.47527935287348433,
                                                        0.358111480051862,
                                                        0.8881257496044034,
                                                        0.6211562243789921,
                                                        0.49947102864151866,
                                                        0.22955318689170096,
                                                        0.3857300266007372,
                                                        0.3971842781835192,
                                                        0.2726147635200227,
                                                        0.7315644998957413,
                                                        0.4107139579334986,
                                                        0.2530438288635256,
                                                        0.20208036503932392,
                                                        0.7095508203989023,
                                                        0.4240858044755369,
                                                        0.17555042808300103,
                                                        0.45780100739924484
                                                    ]
                                                }
                                            }
                                        }
                                    ]
                                }
                            },
                            "Attenuator": {
                                "upstreams": {
                                    "NoiseGenerator": [
                                        {
                                            "name": "RandomMessage",
                                            "fields": {
                                                "timestamp": {
                                                    "type": "float",
                                                    "content": 1648234610.070494
                                                },
                                                "data": {
                                                    "type": "ndarray",
                                                    "content": [
                                                        0.5213616181386701,
                                                        0.1538570333241666,
                                                        0.18900670398451921,
                                                        1.102076623024621,
                                                        1.0685402160438917,
                                                        0.25547322112599624,
                                                        0.18045224293301163,
                                                        1.1981039611690794,
                                                        0.9584663318461375,
                                                        0.366382861441732,
                                                        0.035571962598370234,
                                                        0.37797069682870515,
                                                        0.9907826103398073,
                                                        0.2627360578072584,
                                                        0.19695349949956062,
                                                        0.5686107887618473,
                                                        0.21162783086516537,
                                                        0.42268896290383107,
                                                        0.7613259061706078,
                                                        0.16214751832906651,
                                                        1.093532212906625,
                                                        0.6690316794426423,
                                                        1.061116961833238,
                                                        0.9666048998600247,
                                                        0.7378217859569046,
                                                        0.7585725972699122,
                                                        0.4955317111446606,
                                                        1.1410424633867289,
                                                        1.031302161157888,
                                                        0.39073253526916707,
                                                        0.029776268868616194,
                                                        0.4510924440000083,
                                                        0.010899872712552482,
                                                        1.0942314751972435,
                                                        0.12044330584452231,
                                                        0.18330675249094844,
                                                        1.013932792447924,
                                                        0.7145778730729558,
                                                        0.5489402115831097,
                                                        0.9684484609328649,
                                                        0.2451812116752896,
                                                        1.0197681640048695,
                                                        0.4526015478922602,
                                                        0.23047313955992962,
                                                        1.1077092028819684,
                                                        0.8486953236443621,
                                                        0.6172990404607809,
                                                        0.7314498156970521,
                                                        0.46156184422080343,
                                                        0.41358598750957654,
                                                        0.5950643522181946,
                                                        0.17636204702145222,
                                                        0.5152171083317388,
                                                        0.3293986168628414,
                                                        0.40535047271301783,
                                                        0.11360255386423224,
                                                        0.0036426358936580082,
                                                        0.5047912595286744,
                                                        0.6907133172509785,
                                                        0.07163161067135486,
                                                        0.12803371644017822,
                                                        0.6653767185499894,
                                                        0.8228328022343318,
                                                        0.2644907765209483,
                                                        0.8676976000250431,
                                                        1.0649448425679984,
                                                        1.097500932341189,
                                                        0.19932464019047294,
                                                        0.6035129135286076,
                                                        0.04040410043315829,
                                                        0.6273442211079853,
                                                        0.6371917865399344,
                                                        0.7019930407972835,
                                                        0.6358864384131107,
                                                        0.42941776914547797,
                                                        0.8849993383071075,
                                                        1.0223397957691642,
                                                        0.20464796272584,
                                                        0.5175163297296465,
                                                        0.6593302823024825,
                                                        1.1955779244877436,
                                                        0.22668335459645322,
                                                        0.10206435393266501,
                                                        0.9003913165589508,
                                                        0.6458022427702986,
                                                        0.9386498570865738,
                                                        0.31021153082509256,
                                                        0.4951518518527357,
                                                        0.515097370577311,
                                                        0.6511002842708863,
                                                        0.3882110539984563,
                                                        0.4830498108705075,
                                                        0.8680725817634675,
                                                        0.4968300305475779,
                                                        0.5639469317855406,
                                                        0.39961036319261717,
                                                        1.0076589498709856,
                                                        0.1944602417272477,
                                                        0.06318523587846343,
                                                        0.5183629383308209
                                                    ]
                                                }
                                            }
                                        }
                                    ]
                                }
                            }
                        }
                    },
                    "produced_timestamp_s": 1648234613.5180447,
                    "timestamp_s": 1648234613.5180461
                }
            ],
            "batch_num": 16
        }
    }
}

@dtemir
Copy link
Contributor Author

dtemir commented Mar 28, 2022

Added an Example

I just added an example of using the graph topology along with real-time messages to labgraph/examples/labgraph_monitor_example.py

I'm hoping to add some logic behind assigning values and adding INPUT channels in serialier_node.py. Right now it is very manual.

Follow-Up to Aymen's Comment

I'm going to do some refactoring to generate_labgraph_monitor() as per @bennaaym's suggestion

yeah, that's true, actually, the whole serialization process ( identify_graph_nodes, connect_to_upstream, serialize_graph) should be moved from generate_labgraph_monitor to the SerializerNode. currently I'm only sending the result which was ok for only sending the topology without real-time values

#58 (comment)

@dtemir
Copy link
Contributor Author

dtemir commented Mar 29, 2022

Updated README

I made sure to update the README.md with the new way we should run topology.

New Function Names

I also renamed functions to represent their purpose better.

  • generate_labgraph_monitor() is now generate_graph_topology()
  • run_server() is now run_topology() as it is only sending the topology over WebSockets.

This is an example of how things should be run:

# Initialize a Demo graph
graph = Demo()
# Serialize its topology
topology = generate_graph_topology(graph)
# Run the WebSockets API to send the topology to Front-End
run_topology(topology)

@dtemir dtemir mentioned this pull request Mar 29, 2022
6 tasks
dtemir added 3 commits March 29, 2022 14:42
fixed test case values with new graph topic names
updated graphviz support README
added graphviz example
updated labgraph_monitor_example.py with new approach
added a function that matches subscribers with their publisher grouping
@dtemir
Copy link
Contributor Author

dtemir commented Apr 8, 2022

@bennaaym do you think you could give me a quick review? I'm trying to finish up this PR, so would love some feedback. The most useful way to check it would be here: labgraph_monitor_example.py

Copy link
Contributor

@bennaaym bennaaym left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dtemir great work!! please take a look at the suggestions

dtemir and others added 3 commits April 9, 2022 18:05
moved the labgraph monitor example to extensions instead of core library
added abstraction to setting up graph topology
updated serializer note to only work with right dictionaries
fixed a  mistake in lg_monitor_server.py
@dtemir dtemir changed the title [WIP] Real Time Messages Real Time Messages Apr 13, 2022
fixed a mistake in the serializer node, checking whether the graph is in real-time messaging mode or not
dtemir added 2 commits April 14, 2022 12:44
fixed WebSockets "not in the supported list" warning with @bennaaym
added return types for LabgraphMonitor facade methods
@dtemir
Copy link
Contributor Author

dtemir commented Apr 18, 2022

@jfResearchEng this PR is ready for review. you can find the latest documentation with examples here: https://github.com/facebookresearch/labgraph/blob/4751118039e1bea0ea0468cc27ef44755d2e224f/extensions/yaml_support/README.md

@jfResearchEng
Copy link
Contributor

jfResearchEng commented Apr 19, 2022

Description

With this PR I'm trying to make the current generate_labgraph_monitor() include the message data content, such as it will send the quantitative data to the front-end. In other words, I'm trying to update the structure of the WebSocketsAPI Message from this:
...

Please make sure the description here is updated.
For example, labgraph$ python labgraph/examples/labgraph_monitor_example.py
the path seems to have changed.

data_1: Optional[Dict] = None
data_2: Optional[Dict] = None
data_3: Optional[Dict] = None
data_4: Optional[Dict] = None

class Serializer(lg.Node):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is Serializer a generic class? There seems to be some hard-coded parts, e.g. input topics in this class.

data_1: Optional[Dict] = None
data_2: Optional[Dict] = None
data_3: Optional[Dict] = None
data_4: Optional[Dict] = None

class Serializer(lg.Node):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is Serializer a generic class? There seems to be some hard-coded parts, e.g. input topics in this class.

@jfResearchEng
Copy link
Contributor

Does this PR need to be merged first before your PR is launched?

@jfResearchEng jfResearchEng mentioned this pull request Apr 20, 2022
5 tasks
Copy link
Contributor

@jfResearchEng jfResearchEng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merging this PR for now and for the existing improvements, please help create a new PR.

@jfResearchEng
Copy link
Contributor

jfResearchEng commented Apr 26, 2022

Thank you for contributing to this important PR on LabGraph Monitor!
image

@jfResearchEng jfResearchEng merged commit 49a62be into facebookresearch:main Apr 26, 2022
@dtemir dtemir mentioned this pull request Sep 6, 2022
3 tasks
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

LabGraph Monitor Improvement - Support graph attributes in YAML
4 participants