Data File Formats – CREATE DATABASE dbName; GO
Data comes in numerous forms, as stated earlier. The most common types and the types that are important to know about and be comfortable with for the exam are as follows:
- JavaScript Object Notation (JSON)
- Apache Parquet file format
- Optimized Row Columnar (ORC)
- Extensible Markup Language (XML)
- Yet Another Markup Language (YAML)
- Comma‐separated values (CSV)
Let’s take a closer look at each of those formats, beginning with the most common, JSON.
JavaScript Object Notation
The JavaScript Object Notation (JSON) file format is very useful for capturing and storing data. In addition to that, JSON is often used for configuration files, which are loaded into process memory at runtime. JSON is also the default file structure for data transmissions between a client and a REST API. Take a look at the following example of a JSON file:
{“Session”: {“Scenario”: “TikTok”,”POWReading”: [{“ReadingDate”: “2021-07-30T09:40:25.6635″,”Counter”: 0, “AF3”: [{“THETA”: 9.681,”ALPHA”: 3.849,”BETA_L”: 2.582,”BETA_H”: 0.959, “GAMMA”: 0.738}]}]}}
The JSON snippet is an example of a single reading of a brainwave from a brain computer interface IoT device. It was captured while viewing TikTok. This is only a small snippet of the actual reading, as it reflects only the waves from the AF3 electrode of the BCI. There are four other electrodes on the BCI: AF4, T7, T8, and Pz, so five in total. The AF3 electrode, like the others tracks, the THETA, ALPHA, BETA and GAMMA values.
A common difficulty of working with JSON files has to do with getting all the opening and closing curly brackets { }, square brackets [ ], quotes “”, and commas to work out symmetrically. As you can see just in this small snippet, there is an abundance of each. If you want to see how this file was created, take a look at the C# code in the Chapter02/Source Code directory on GitHub: https://github.com/benperk/ADE. One benefit of JSON‐formatted files is that they are rather easy to read through and get an idea of what they contain. Additionally, the number of unnecessary characters are limited, which reduces the size of the file. Finally, there are many libraries that help efficiently parse the file, such as System.Text.Json and Netwtonsoft.Json. The best uses of JSON files include the following:
- Large, complex datasets
- When you are already working with JavaScript
- When you are communicating between systems over the Internet
Data comes in numerous forms, as stated earlier. The most common types and the types that are important to know about and be comfortable with for the exam are as follows: Let’s take a closer look at each of those formats, beginning with the most common, JSON. JavaScript Object Notation The JavaScript Object Notation (JSON)…
