Skip to content

2check91/data_engineering_spark_project

Repository files navigation

Data_Engineering_Project

Agenda

Introduction

5 Ss (stream store structure synthesize show)

Architecture Diagram

ER Diagram

Low latency reads and updates:

Scalability:

Generalization:

Extensibility:

Ad hoc queries:

Minimal maintenance:

Debuggability:

Future development

Link to my static website:

http://thewebsitebucket.s3-website-us-east-1.amazonaws.com/

About

Scraping glassdoor and sending processed output to postgres via Spark.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published