← Data Science & Analytics

Fees by level

Start at any level, or take the complete programme. The fee you pay for a level is locked for you.

All fees are in Indian Rupees and include applicable taxes. See our pricing & payment terms.

The complete curriculum

01 Introduction To Data Engineering 6 lessons ▶

1. 1.1Introduction to Data engineering
2. 1.2What is SQL
3. 1.3Overview of Application Architecture and RDBMS
4. 1.4Overview of Database Technologies
5. 1.5Purpose Built Database
6. 1.6RDBMS vs Database Technologies

02 Setting Up the System 3 lessons ▶

1. 2.1Setting up VS code and Python
2. 2.2Setting up MYSQL Server and PostgreSQL
3. 2.3Setting up Jupyter Lab online

03 Introduction to SQL Queries 3 lessons ▶

1. 3.1Creating the table in SQL
2. 3.2Relationship Model In SQL
3. 3.3Starting with SQL

04 Manipulation Commands in SQL 7 lessons ▶

1. 4.1Insert Command
2. 4.2Update part1
3. 4.3Update part2
4. 4.4Delete command
5. 4.5Drop and Truncate
6. 4.6Alter Part1
7. 4.7Alter Part2

05 Retrieving Data in SQL 4 lessons ▶

1. 5.1Retrieving data using Select and where
2. 5.2Sorting and using Alias
3. 5.3Limit and Select with Expression
4. 5.4Logical Operators

06 Aggregation in SQL 2 lessons ▶

1. 6.1Aggregation function part1
2. 6.2Aggregation Function part2

07 Joins in SQL 7 lessons ▶

1. 7.1Types of joins
2. 7.2Inner join
3. 7.3Left Join
4. 7.4Right Join
5. 7.5Full Outer Join
6. 7.6Self join
7. 7.7Cross join

08 Set Operators in SQL 2 lessons ▶

1. 8.1Set Operator Part1mp4
2. 8.2Set Operator Part2mp4

09 Subqueries and Case Statements 7 lessons ▶

1. 9.1Subqueries Part4
2. 9.2Subqueries Part1
3. 9.3Subqueries Part2
4. 9.4Subqueries Part3
5. 9.5Case Statements Part1
6. 9.6Case Statements Part2
7. 9.7Case Statements Part3

10 CTEs and Views 4 lessons ▶

1. 10.1CTEs Part1mp4
2. 10.2CTEs Part2
3. 10.3Views Part1
4. 10.4View Part2

11 Advanced Topics 8 lessons ▶

1. 11.1Window Function
2. 11.2RowNumber(), Rank(), DenseRank()
3. 11.3NTILE(), Lead(), Lag()
4. 11.4Window Functions examples
5. 11.5Stored Procedure
6. 11.6Index
7. 11.7Mini Project Part1
8. 11.8Mini Project Part2

12 Introduction to Python Programming 5 lessons ▶

1. 12.1Introduction to Python
2. 12.2Variables and Data Types
3. 12.3Input and Output
4. 12.4Basic Operator
5. 12.5Constants and Type casting

13 Data Structures in Python 8 lessons ▶

1. 13.1Conditional Statements
2. 13.2Loop Part1
3. 13.3Loop Part2
4. 13.4Loop Part3
5. 13.5List
6. 13.6Tuple and Dictionary
7. 13.7Sets
8. 13.8Mini project

14 Functions, Modules & File Handling 11 lessons ▶

1. 14.1Functions Part1
2. 14.2Functions Part2
3. 14.3Functions Part3
4. 14.4Project Calculator Part1
5. 14.5Project Calculator Part2
6. 14.6Modules Part1
7. 14.7Modules part2
8. 14.8Module Part3
9. 14.9File Handling Part1
10. 14.10File Handling Part2
11. 14.11Error Handling

15 Python Collections for Data Engineering 10 lessons ▶

1. 15.1File I_O using Python
2. 15.2Read Data from CSV into python
3. 15.3Python Collectionsmp4
4. 15.4Processing Python List
5. 15.5Lambda Functions
6. 15.6Filter data in Python
7. 15.7Manipulating List using set()mp4
8. 15.8Sort() & Overview of json strings
9. 15.9Reading data from json files
10. 15.10Extracting column from json file

16 Data Processing using Pandas 10 lessons ▶

1. 16.1Overview of pandas for Data Processing
2. 16.2Pandas Dataframe functionalities
3. 16.3Aggregation using pandas
4. 16.4Joining data using pandas
5. 16.5Summarizing Joins
6. 16.6Aggregation on Join Results Part1
7. 16.7Aggregation on Join Results Part2
8. 16.8Aggregation on Join Results Part3
9. 16.9Sort data using pandas dataframe
10. 16.10Exporting data to JSON files

17 Project1: File Format Converter 12 lessons ▶

1. 17.1Overview of the files to be processed
2. 17.2Working with regular expressions
3. 17.3Getting Column names from the data
4. 17.4Generating File paths and writing dataframe
5. 17.5Recap of the Project
6. 17.6Writing pandas dataframe to JSON file
7. 17.7Converting csv to json
8. 17.8Modularize File Format Converter
9. 17.9Wrapping up the Project
10. 17.10Runtime Arguments part1
11. 17.11Runtime Arguments part2
12. 17.12Environment variables

18 Project2: File to Database Loader 5 lessons ▶

1. 18.1Configuring sql to python
2. 18.2Connecting to the database
3. 18.3Validate pandas and SQL integration
4. 18.4Write CSV data to tables in chunks
5. 18.5Project ready for deployment

19 Troubleshoot and Debugging Python Issues 6 lessons ▶

1. 19.1Troubleshooting and Debugging
2. 19.2Troubleshooting Database connectivity
3. 19.3Troubleshooting Credentials for Database connectivity
4. 19.4Troubleshooting Errors in Python
5. 19.5Overview of SDLC
6. 19.6Unit Testing & Debugging

20 Overview of Performance Tunning in Python 9 lessons ▶

1. 20.1Introduction to Performance of Python Applications
2. 20.2Setting up Database Loader
3. 20.3Cleaning the data and Performance Tunning
4. 20.4Performance Tunning
5. 20.5Loading Multi Files in Python
6. 20.6Develop applications for Multiprocessing
7. 20.7Multiprocessing using python
8. 20.8Introduction to Parallel Processing
9. 20.9Executing Parallel Processing

21 Getting started with GCP 5 lessons ▶

1. 21.1Introduction to Getting started with GCP
2. 21.2Signing Up for GCP Account
3. 21.3Overview of Google cloud platform
4. 21.4Overview of google cloud shell
5. 21.5Overview of Analytics services on GCP

22 Overview of Big Data and Data Lakes 9 lessons ▶

1. 22.1Database and Their Types
2. 22.2Technologies for Different Databases
3. 22.3Usecase of different databases
4. 22.4Volume of different database
5. 22.5Overview of Big data and its evaluation
6. 22.6Data Lake using Hadoop
7. 22.7Overview of Modern Data Lakes on Cloud
8. 22.8Implementation of Modern Data Lakes
9. 22.9Implementation + Advantages of Modern Data Lakes

23 Overview of Spark and Spark Architecture 8 lessons ▶

1. 23.1Overview of Data Processing
2. 23.2Setting up the Environment
3. 23.3Code Examples of different Libraries
4. 23.4Difference between libraries and Distributed Computing
5. 23.5Example of Distributed Computing
6. 23.6Overview of Apache Spark Documentation
7. 23.7Spark Key Features and Infrastructure
8. 23.8Overview of Spark clusters and Key terms

24 Setup Databricks Environment using GCP 5 lessons ▶

1. 24.1Overview of Databricks on GCP
2. 24.2Signing up for Databricks on GCP
3. 24.3Resolving Workspace errors and creating cluster
4. 24.4Setup Databricks
5. 24.5Databricks Features and Architecture

25 Basic Transformation using Spark SQL 7 lessons ▶

1. 25.1Processing Data in databricks using spark
2. 25.2Getting started with Spark SQL
3. 25.3Create views and retrieve them using Spark SQL
4. 25.4Spark SQL Query to compute Daily Product Revenue
5. 25.5Overview of pyspark and Reading json files
6. 25.6Get Schema details using pyspark
7. 25.7Writing a function using Pyspark Python

26 Create Delta tables using Spark SQL 4 lessons ▶

1. 26.1Understanding delta tables and Database
2. 26.2Creating Delta tables
3. 26.3Copy data from external file
4. 26.4Performing operations in delta tables

27 Pre-defined Functions in Spark SQL 29 lessons ▶

1. 27.1Overview of Functions
2. 27.2Describe Functions in Spark SQL
3. 27.3Case Conversion Functions
4. 27.4Extracting Data using Functions
5. 27.5Trimming and Padding Functions
6. 27.6Reverse and Concatenation functions
7. 27.7Date Manipulation Functions
8. 27.8Beginning Date or Time Functions
9. 27.9Extracting information using date_format Part1
10. 27.10Extracting information using date_format Part2
11. 27.11Calendar Functions
12. 27.12Dealing with Unix Timestamp
13. 27.13Mathematical Functions Part1
14. 27.14Mathematical Functions Part2
15. 27.15Miscellaneous Functions Part1
16. 27.16Miscellaneous Functions Part2
17. 27.17Miscellaneous Functions Part3
18. 27.18CASE Statements Part1
19. 27.19CASE Statements Part2
20. 27.20Word count Query part1
21. 27.21Word count query Part2
22. 27.22Practice Exercise1
23. 27.23Practice Exercise2
24. 27.24Practice Exercise3
25. 27.25Practice Exercise4
26. 27.26Practice Exercise5
27. 27.27Practice Exercise6
28. 27.28Practice Exercise7
29. 27.29Practice Exercise8-

28 Setup Spark Tables for Basic Transformations 1 lesson ▶

1. 28.1Introduction to transformation using Spark SQL

29 Filtering Data using Spark SQL 2 lessons ▶

1. 29.1Filtering using Spark SQL part1
2. 29.2Filtering using Spark SQL Part2

30 Aggregations using Spark SQL 2 lessons ▶

1. 30.1Aggregation using Spark SQL part1
2. 30.2Aggregation using Spark SQL part2

31 Joins using Spark SQL 2 lessons ▶

1. 31.1Joins using Spark SQL part1
2. 31.2Joins using Spark SQL part2

32 Sorting using Spark SQL 1 lesson ▶

1. 32.1Sorting using Spark SQL

33 Copy Query Results into Spark Tables 3 lessons ▶

1. 33.1Copy Query Results using CTAS and INSERT
2. 33.2Design Pipleline using CTAS and INSERT
3. 33.3Copy Query Results using MERGE

34 Ranking using Spark SQL Windowing Functions 1 lesson ▶

1. 34.1Ranking using Spark SQL

35 Processing JSON like data using Spark SQL 4 lessons ▶

1. 35.1Creating Tables with Array type Columns
2. 35.2Creating and Dealing with Array of Struct Type columns
3. 35.3Overview of Functions to Process Data in Spark SQL
4. 35.4Processing Delimited strings using Spark SQL

36 Getting Started with Pyspark DataFrame APIs 4 lessons ▶

1. 36.1Introduction and Setup pf Pyspark
2. 36.2Process schema details in json using pyspark
3. 36.3Transforming and Processing Schemas
4. 36.4Convert CSV to Parquet with schema

37 Create spark data frames using Pyspark 4 lessons ▶

1. 37.1Creating Spark DataFrame and Processing json-like data
2. 37.2Overview of Data Processing and DataFrame concepts
3. 37.3Advantages and Transformations of DataFrame
4. 37.4using withColumn() and Writing Dataframe to Delta Files

38 Basic Transformation using Pyspark 6 lessons ▶

1. 38.1Overview of Transformations and withColumn usecase
2. 38.2Filtering Data using PySpark DataFrame
3. 38.3Aggregations using Spark DataFrame
4. 38.4Aggregation using Spark DataFrame APIs
5. 38.5Sorting data using PySpark DataFrame
6. 38.6Dealing with Nulls while sorting the data

39 Joining Data using Spark DataFrame 2 lessons ▶

1. 39.1Inner Join over the Spark dataframe APIs
2. 39.2Left and Right Join using Spark dataframe APIs

40 Ranking using Pyspark 4 lessons ▶

1. 40.1Overview of ranking along with OrderBy functionality
2. 40.2Filter Based on rank using Spark dataframe APIs
3. 40.3Filter based on Rank per Partition using spark Dataframe
4. 40.4Difference Between Rank() and Dense_Rank()

41 Integration of Spark SQL and Pyspark data frame 3 lessons ▶

1. 41.1Introduction to Integration of Spark SQL and PySpark Dataframe APIs
2. 41.2Run Spark SQL queries and Create Spark Metastore Tables using DataFrames
3. 41.3Dataframe Transformations over Metastore Tables

42 ELT Data Pipelines using Databricks 3 lessons ▶

1. 42.1Overview of Databricks Workflows
2. 42.2Multi-Task Workflows using Databricks jobs
3. 42.3Working on Multi-Task job

43 Performance Tuning of Spark-Catalyst Optimizer 4 lessons ▶

1. 43.1Getting Started with Performance Tuning and Spark Catalyst Optimizer
2. 43.2Overview of Plans using Spark UI
3. 43.3Understanding of Spark Architecture with Example
4. 43.4Filter, Broadcast Joins & Aggregations

44 Performance Tuning of Spark Cluster Configuration 3 lessons ▶

1. 44.1Introduction to Databricks Cluster Configuration and its Types
2. 44.2Setting up All-Purpose Cluster and Overview of Auto-Scaling
3. 44.3Cluster Performance Tuning using Auto Scaling

45 Performance Tuning while inferring schema from csv or json files 3 lessons ▶

1. 45.1Overview of Inferring Schema from CSV or JSON files
2. 45.2Overview of CSV_JSON files
3. 45.3Overhead of Inferring Schema

46 Performance Tuning using Columnar File Format and Partitioning Strategy 2 lessons ▶

1. 46.1Introduction of Performance tuning while storing data and Side effects of CSV files in Data Lake
2. 46.2Restructure CSV Data and performance Boosting

47 Introduction to Linux Commands for Data Engineering 9 lessons ▶

1. 47.1Introduction to linux and Basic Navigation Commands
2. 47.2File and Folder Management Commands
3. 47.3Commands for Viewing and Editing Files
4. 47.4Commands to make modification inside a Vi Editor
5. 47.5Understanding of SED Command
6. 47.6Understanding of GREP Command
7. 47.7Understanding of AWK Command
8. 47.8Understanding of Find Command
9. 47.9Disk, Process & System Monitoring

Tools you'll use

Related courses