However, once I was introduced to PostgreSQL and TimescaleDB, I found how efficient and fast it was to do my data munging directly within my database. Historically, I have done my data munging and modeling all within Python or R, these being excellent options for analysis. The first three steps of the analysis lifecycle (evaluate, clean, transform) comprise the “data munging” stages of analysis.
#DATA DOES NOT IMPORT DATA TO PGADMIN 4 DATA LOST SERIES#
I began this series of posts on data analysis by presenting the following summary of the analysis process: Data Analysis Lifecycle TimescaleDB and PostgreSQL cannot replace these tools entirely, but they can help your data munging/cleaning tasks be more efficient and, in turn, let Excel, R, and Python shine where they do best: in visualizations, modeling, and machine learning.Ĭleaning is a very important part of the analysis process and generally can be the most grueling from my experience! By cleaning data directly within my database, I am able to perform a lot of my cleaning tasks one time rather than repetitively within a script, saving me considerable time in the long run. In this blog post, I explore some classic data cleaning scenarios and show how you can perform them directly within your database using TimescaleDB and PostgreSQL, replacing the tasks that you may have done in Excel, R, or Python. Sometimes to properly evaluate your data, you may need to do some pre-cleaning before you get to the main data cleaning, and that’s a lot of cleaning! In order to accomplish all this work, you may use Excel, R, or Python, but are these the best tools for data cleaning tasks?
During analysis, you rarely - if ever - get to go directly from evaluating data to transforming and analyzing it.
0 kommentar(er)
