GIS and Earth Observation University

Welcome to the Building Big Data Pipelines with R & Sparklyr & PowerBI course. In this course we will be creating a big data analytics solution using big data technologies for R.

In our use case we will be working with raw earthquake data, we will be applying big data processing techniques to extract transform and load the data into usable datasets. Once we have processed and cleaned the data, we will use it as a data source for building predictive analytics and visualizations.

The tools we will be using are the following:
For Data visualization we will be using Power BI Desktop is a powerful data visualization tool that lets you build advanced queries, models and reports. With Power BI Desktop, you can connect to multiple data sources and combine them into a data model. This data model lets you build visuals, and dashboards that you can share as reports with other people in your organization.

For our big data processing pipeline, Sparklyr is an open-source library that is used for processing big data in R, by providing an interface between R and Apache Spark. It allows you to take advantage of Spark's ability to process and analyze large datasets in a distributed and interactive manner. It also provides interfaces to Spark's distributed machine learning algorithms and much more.