Graphs

About NetworkVisualizer Graphs

NetworkVisualizer uses two libraries to accomplish graphing of the data we receive. One is Google.DataTable.Net.Wrapper, a glorious C# library that allows the creation and serialization of Google-compatible JSON tables for use. Data is accessed using LINQ from our database and a function compiles this into a JSON string. A timed service runs every 5 minutes which runs graph generation, then puts the resulting string into a cache in our database for quicker access. Google's Visualization API, a JS library, gets loaded when the index page of our site is loaded. An AJAX request is made to a function that retrieves the graph from the cache, and then the Visualization API displays it.

Essentially:

• Graphs get generated and cached every 5 minutes in the UpdateGraphsService.cs file, found under Services
• When the page gets loaded, the site calls the cache to load the graph
• Each graph has an integer ID in the cache, ie. "Graph1" which can be called via AJAX (/Cache/GetLatestGraph/{id})

Adding More Graphs

---

More graphs are trivial to add as the framework is already in place. To add another graph, follow the steps below:

1. In the UpdateGraphsService.cs file, create a new function for the table
2. This function should take no arguments and use Google.DataTable.Net.Wrapper to generate and serialize a table

Example:

        DataTable dt = new DataTable();

        // Get most searched domains in database
        List<string> topDomains = new List<string>();
        using (var scope = _scopeFactory.CreateScope())
        {
            // Get context (database)
            var _context = scope.ServiceProvider.GetRequiredService<NetworkVisualizerContext>();
            topDomains = _context.Packet
                         .GroupBy(q => q.DestinationHostname)
                         .OrderByDescending(gp => gp.Count())
                         .Take(4)
                         .Select(g => g.Key).ToList();
        }

        // Add most searched domains as columns in datatable, with Time and Other category
        dt.AddColumn(new Column(ColumnType.String, "Time", "Time"));
        foreach (string domain in topDomains)
        {
            dt.AddColumn(new Column(ColumnType.Number, domain, domain));
        }
        dt.AddColumn(new Column(ColumnType.Number, "other sites", "other sites"));

        // Create datapoints for every hour, starting with 23 hours ago up to now
        for (int t = 1; t <= 24; t++)
        {
            Row r = dt.NewRow();
            
            DateTime targetDateTime = DateTime.UtcNow.Subtract(TimeSpan.FromDays(1)).AddHours(t);

            // Add rows, each with top search result numbers & a time with offset applied
            List<int> domainSearches = TopDomainSearches(topDomains, targetDateTime);
            r.AddCell(new Cell($"{targetDateTime.AddHours(Config.config.UTCHoursOffset).Hour}:00"));
            foreach (int s in domainSearches)
            {
                r.AddCell(new Cell(s));
            }

            dt.AddRow(r);
        }

        // Return datatable as json
        return dt.GetJson();

3. The function should return the serialized string of the table using dt.GetJson();
4. In the DoWork() function, add a new item to the cache with the new GraphID, an expiry time of 24 hours, and the returned string of the table generating function

Example:

   _context.Cache.Add(new Cache
   {
        ExpireTime = DateTime.UtcNow.AddDays(1),
        Key = "Graph1",
        Value = GenerateMainDatatableJson()
   });

5. On the Index page's drawChart() function, use the Google Visualization API along with an AJAX call to display the graph

Example:

    var jsonData = $.ajax({
        url: "/Caches/GetLatestGraph/1",
        dataType: "json",
        async: false
    }).responseText;
    var data = new google.visualization.DataTable(jsonData);

    var options = {
        fontName: "Trebuchet MS",
        colors: ['#e06666', '#f6b26b', '#ffd966', '#93c47d', '#76a5af'],
        isStacked: true,
        theme: 'maximized',
        vAxis: {
            gridlines: { color: '#F8F8F8' },
            minValue: 0
        },
        height: window.innerHeight / 1.5
    };

    var chart = new google.visualization.AreaChart(document.getElementById('chart_div'));
    chart.draw(data, options);

6. Note that for different types of charts and customization, you want to refer to Google Visualization's documentation found here
7. Make sure you have a corresponding div in your HTML to place the chart in, otherwise the chart will not be displayed

And that's it! You have added a new chart to the page. If you put the new graph under the drawChart() JavaScript function, the loading icon will function until the entire page is loaded and all charts will get redrawn on browser resize. Please note performance issues with adding many graphs as they will all get redrawn on page resize.