Comma‐Separated Values – CREATE DATABASE dbName; GO

• by Terica Anderson
• 2024-07-25

A comma‐separated values (CSV) file is just that, a CVS file that contains data values separated by commas. Sometimes, the first row in a CVS file identifies the column names:

Scenario,Counter,Electrode,THETA,ALPHA,GAMMA
TikTok,5,AF3,9.681,3.849,0.738
TikTok,6,Pz,8.392,4.142,1.106

Loading that file into memory using Python would look something like the following. First you import the CVS library, then open the file and load it in memory via the reader object. Each value in each row is accessible through the enumerable row.

import csv
with open(‘brainjammer.csv’) as f:
    reader = csv.reader(f)
    for row in reader:
  #load each row into database

The best uses for CVS files include the following:

• Small datasets
• If you want a simple solution for importing and exporting data
• If you are working with Microsoft Excel

Count on the data used within a large‐scale data analytics solution to come in lots of different formats. We’ve covered six different types along with some insights about their use cases and the way in which you can access the data using code. See Table 2.1 for a comparison overview of these file formats.

TABLE 2.1  File comparison

	JSON	Parquet	ORC	XML	YAML	CSV
Columnar	No	Yes	Yes	No	No	No
Complex structure	Yes	Yes	Yes	Yes	Yes	No
Compressible	Yes	Yes	Yes	Yes	Yes	Yes
Human readable	Yes	No	No	Yes	Yes	Yes
Simplicity	Simple	Complex	Complex	Simple	Simple	Simple
Write	Fast	Slow	Slow	Slow	Fast	Fastest
Read	Fast	Fastest	Fastest	Slow	Fast	Fast
Dataset size	Small	Large	Large	Small	Small	Small

There are many other formats, but the ingestion of those formats into your data warehouse or data lake is just a matter of first identifying the existing format. Then you need to identify the end format and develop the code to make the transformation.

A comma‐separated values (CSV) file is just that, a CVS file that contains data values separated by commas. Sometimes, the first row in a CVS file identifies the column names: Scenario,Counter,Electrode,THETA,ALPHA,GAMMATikTok,5,AF3,9.681,3.849,0.738TikTok,6,Pz,8.392,4.142,1.106 Loading that file into memory using Python would look something like the following. First you import the CVS library, then open the file and…