SQLite is the most used database engine in the world. SQLite is a C-language library that implements a small, fast, self-contained, high-reliability, full-featured, SQL database engine. Specifically, we’re going to play around with SQLite - in their own words: No no, I’m not going to make you learn SQL to work with this dataset, but we are going to use SQL inside R. Also - there are VERY few times where you actually need to pull in all of the specific columns and/or rows for your analysis, why waste time reading in data that you don’t need? Enter SQL
#OPEN SQLITE FILE IN R DOWNLOAD#
However, as mentioned before this is ~ 2 GB of data in memory, which can take a while to download and/or read in especially if you do this multiple times as opposed to in just one session. Library ( dplyr ) read_pbp_rds % purrr :: map_dfr ( read_pbp_rds ) all_pbp %>% write_rds ( "data/2000-2019_pbp_raw.rds" )
#OPEN SQLITE FILE IN R OFFLINE#
We can also save them to local storage for offline access or to at least not need to re-download them down the line. With a quick purrr call we can download all 20 years of data and pull them into memory. RDS files for every year between 2000-2019 on GitHub for ease of download.
I’ll be covering some tooling to work with relatively larger datasets using the same dplyr tools you know and love! Read in the dataīen and Sebastian were kind enough to provide all of the. However, the 2000-2019 data consists of ~2.25 GB of data, 291 variables and 903,998 rows… this is 263,063,418 observations! While this is not anywhere close to too much for R to handle, it can start to feel a little intimidating to read it all into memory. With more data comes excitement - we can do new analysis with very similar code!
🏈 Fast functions for scraping team rosters and highlight videos /fqbyE1pPHE 🏈 Includes Completion Probability and CPOE going back to 2006 🏈 Play-by-play of all NFL games going back to 2000 INTRODUCING: an R package for scraping NFL data faster ⚡ Ben Baldwin and Sebastian Carl released nflfastR yesterday to provide faster scraping of NFL play-by-play data as well as data all the way back to 2000! This builds upon the work of the nflscrapR team who paved the way for this data for NFL games between 20.
0 kommentar(er)
