Big Data
Big Data with Hadoop & Spark
Build data pipelines with Apache Spark, PySpark, and Kafka.
Level: IntermediateBig Data
What You'll Learn
Apache Spark architecture
RDDs & DataFrames
Spark SQL
PySpark
Spark Structured Streaming
Kafka fundamentals
Data serialisation
Cluster tuning basics
Project: streaming pipeline
What's Included
🧭AI Tutor🔒
Interactive learning path with AI-generated lessons read aloud by your personal tutor
🧪Practice Lab🔒
Step-by-step lab instructions with AI guidance — run in your own IDE or sandbox.
🎙️Mock Interviews🔒
AI interviewer asks tailored questions, gives real-time feedback and a score report
📝Evaluation🔒
MCQ questions (theory, practical, code) to test your knowledge and earn a badge
⚔️Challenge AI🔒
Compete head-to-head against AI on real production problems — judged on correctness, performance, scalability, code quality, and edge cases
Course Syllabus
Apache Spark Core
- •Spark Architecture
- •RDDs & Transformations
- •Actions vs Transformations
- •DAG & Lazy Evaluation
- •SparkContext & SparkSession
DataFrames & Spark SQL
- •DataFrames API
- •Spark SQL Queries
- •UDFs
- •Joins & Aggregations
- •Reading Parquet, CSV, JSON
PySpark
- •PySpark Setup
- •DataFrame Operations
- •GroupBy & Window Functions
- •Pandas UDFs
- •MLlib Introduction
Kafka
- •Kafka Architecture
- •Topics, Partitions & Offsets
- •Producers & Consumers
- •Consumer Groups
- •Kafka Connect
Streaming
- •Spark Structured Streaming
- •Kafka-Spark Integration
- •Windowing Operations
- •Checkpointing
- •Project: Streaming Pipeline
You're all set
Everything in this course is unlocked.
Continue Learning →✓AI Tutor
✓Practice Lab
✓Mock Interviews
✓Evaluation
✓Challenge AI