Cleaning the Data in React

After fetching the data (that can be read on this page: Wiki - Collecting the data in Reac), this data of course still had to be cleaned up. Fortunately, I had already written reusable code during the functional data course to clean up the data. But, I also had to write new code, since I would now use several instead of 2 APIs.

Jump to:

• Reuse my old code
• Writing new code
  • Getting the 10 most popular electric cars
  • Getting the amount of cars

Reuse my old code

So first I started by copying my previously written code to my fetchData.js file. These were the functions below:

// This function merges two datasets together:
export function mergeData(dataset1, dataset2, itemArr) {
  return dataset2.map((obj1) => {
    const filtered = dataset1.find(
      (obj2) => obj1[itemArr[0]] === obj2[itemArr[0]]
    );
    obj1[itemArr[1]] = filtered;
    return obj1;
  });
}

//This function has as parameters the total data set and an array
// of columns and changes each item of the column to a integer or float:
function toNumbers(dataArray, columnArr) {
  return dataArray.forEach((arrItem) => {
    columnArr.forEach((c) => {
      arrItem[c] = +arrItem[c];
    });
  });
}

// This function changes the values within an object that is inside
// an object to integers or floats:
export function toIntegersInObj(dataArray, objName, columnArr) {
  return dataArray.forEach((arrItem) => {
    columnArr.forEach((c) => {
      arrItem[objName][c] = +arrItem[objName][c];
    });
  });
}

With these above functions I could merge two datasets into 1 dataset (mergeData), convert the strings inside the outer object to integers (toNumbers) and convert the strings inside one object inside the outer object to integers (toIntegersInObj). For example, I had cleaned my first two APIs into 1 dataset.

Below are pictures of the 2 APIs before I cleaned them up and after they were merged into a usable dataset.
Before:

After:

Writing new code

Then I wanted to make some new graphs and I needed other cleaning functions for the new APIs.

Getting the 10 most popular electric cars

I wanted to make a bar chart with the 10 most popular electric cars. But to get this done I had to clean up the API - Electric vehicles with new cleanup functions. With a little help from Sergio's code (and thus Laurens' code) I came up with a function that I needed to count how much of a specific car brand was included in the total API of all electric vehicles in the Netherlands. I did adjust this function a bit so that I found it easier to read. This function is shown below:

// Code from Laurens & Sergio:
// https://vizhub.com/HappyPantss/673013ca00df472da1a3dfa7a6b55aaf?edit=files&file=index.js
function getCount(dataArray, newArray) {
  dataArray.forEach((merk) => {
    let brandItem = newArray.find((item) => item.brand === merk);
    if (brandItem === undefined) {
      newArray.push({
        brand: merk,
        value: 0,
      });
    }
    newArray.find((item) => item.brand === merk).value += 1;
  });
}

Then I had to filter all the motors from this dataset to keep only the electric cars. For this I wrote the function below:

function getItems(dataArray, column) {
  return dataArray.map((obj) => {
    if (!obj[column].includes("MOTORS")) {
      return obj[column];
    }
  });
}

But I did not find this function functional enough yet so I just described it to the function below:

// This function searches for items in a column of an array and returns the item if
// it does not contain the specific word.
function getItems(dataArray, column, word) {
  return dataArray.map((obj) => {
    if (!obj[column].includes(word)) {
      return obj[column];
    }
  });
}

Then I needed to sort the data from the largest values to the lowest values. For this I also wrote a new function:

// This function sorts an array from largest value to smallest value.
// Source: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sort
function sortArray(dataArray) {
  dataArray.sort((min, max) => {
    return max.value - min.value;
  });
}

Finally, I only had to take the first 10 values. I did this by slicing the array:

const first10Cars = uniqueBrands.slice(0, 10);

And so my dataset was ready to create the bar chart. Fortunately, I actually had no errors during the cleaning of this data. So in other words: Everything went exactly as I wanted. Below I have shown screenshots of the before and after of the API that I transformed into the dataset I needed for the bar chart.
Before:

After:

Getting the amount of cars

Finally, I wanted to make a pie chart with the number of non-electric cars compared to the number of other cars. For this I created an array containing two objects: one for the electric cars and one for the other cars. Per object I gave it a key, label, number and percentage. First I calculated the percentage:

let perc = (+(allCars[0].count - carBrands.length) / +allCars[0].count) * 100;
let perc2 = (carBrands.length / +allCars[0].count) * 100;

let twoDigits = perc.toFixed(2);
let newNum = twoDigits.replace(".", ",");

let twodigit = perc2.toFixed(2);
let secondnum = twodigit.replace(".", ",");

But I soon saw that I had repeatable code that I changed to the function: getPercentage:

let amountAllCars = +allCars[0].count;
let amountElec = carBrands.length;
let percNonElec = getPercentage(amountAllCars - amountElec, amountAllCars);
let percElec = getPercentage(amountElec, amountAllCars);

// This function changes the percentage to a number with two digits and
// replaces the dot to a comma:
function getPercentage(num, total) {
  let percentage = (num / total) * 100;
  return percentage.toFixed(2).replace(".", ",");
}

After this I created the object:

const pieArr = createArr(amountAllCars, amountElec, percNonElec, percElec);

function createArr(cars, elecs, percCars, percElec) {
  return [
    {
      keyNum: 1,
      name: "Niet elektrische auto's",
      value: cars - elecs,
      percentage: percCars,
    },
    {
      keyNum: 2,
      name: "Elektrische auto's",
      value: elecs,
      percentage: percElec,
    },
  ];
}

Below I have posted screenshots of the before and after of the APIs (All cars and the electric cars) that I transformed into the object I needed for the pie chart.
Before:

After: