Oreilly – Data Engineering with Python and AWS Lambda LiveLessons 2023-9

Data Engineering with Python and AWS Lambda LiveLessons course. This course teaches you how to build complete and powerful data engineering pipelines using Python, the same language that data scientists use to build machine learning models. By adopting serverless data engineering in Python, you can build highly scalable distributed systems on AWS cloud infrastructure.

In this course, you will learn about the new serverless paradigm, which means using events and event-driven applications instead of expensive and complex servers. Some of the benefits of programming with AWS Lambda in Python include: no need for server management, continuous scalability, and scaling in a fraction of a second. There are many use cases including data processing, stream processing, IoT infrastructure, mobile and web applications. In this course, you’ll learn to use a new pattern in software architecture that makes it easier to write, maintain, and deploy your code.

AWS Lambda functions are the building blocks for creating advanced applications and services on AWS. In this course, you will learn to use Python to develop Lambda functions that communicate with key AWS services such as API Gateway, SQS, and CloudWatch functions. You’ll also learn how a new cloud development environment called Cloud9 can simplify writing, debugging, and deploying AWS Lambda functions.

What you will learn

• Perform data engineering tasks on AWS
• Development with Cloud9
• Writing AWS Lambda functions with Python
• Implementation of cloud data engineering patterns, i.e. serverless
• Designing event-driven architectures on the AWS platform using SQS, Python Lambda and other AWS technologies

This course is suitable for people who:

• Are you an aspiring data engineer using Python?
• You work with data and want to learn cloud data engineering patterns
• You are new to the AWS cloud and want to write functions in Python that do not require a server
• Are you a data scientist who needs an easier way to get data engineering results?
• Want to learn about serverless technology and how to do it in Python?

Course details

• Publisher: Oreilly
• Instructor: Noah Gift , Robert Jordan, Kennedy Behrman
• Training level: beginner to advanced
• Training duration: 6 hours 24 minutes

Course headings

• Introduction
  1. Data Engineering with Python and AWS Lambda LiveLessons: Introduction
• Lesson 1: Get Started with AWS Lambda
  1. 1.1 Create a Hello World AWS Lambda function in the console
  2. 1.2 Learn basic Lambda patterns
  3. 1.3 Learn Lambda Management console
  4. 1.4 Upload external code to AWS Lambda
• Lesson 2: Use Cloud9 to Develop Python Lambda Functions
  1. 2.1 Set up Cloud9
  2. 2.2 Develop with Cloud9
  3. 2.3 Launch Cloud9 and workspace configuration
  4. 2.4 Import Lambda functions
  5. 2.5 Invoke Lambda functions
  6. 2.6 Invoke Lambda functions inside API Gateway
  7. 2.7 Deploy a Lambda function
• Lesson 3: Create Timed Lambda Functions
  1. 3.1 Use AWS Lambda with Cloudwatch events
  2. 3.2 Use AWS Lambda to populate AWS SQS
  3. 3.3 Use AWS Cloudwatch logging with AWS Lambda
• Lesson 4: Create Event-Driven Lambdas
  1. 4.1 Create a Producer Lambda function
  2. 4.2 Enable SQS Trigger
  3. 4.3 Serverless data engineering architecture
• Lesson 5: Learn SAM Local
  1. 5.1 Install SAM Local
  2. 5.2 Use SAM Local to invoke functions locally
  3. 5.3 Use SAM to package and deploy Lambda
  4. 5.4 Use SAM with IAM
  5. 5.5 Use SAM Lambda environment variables
• Lesson 6: Learn AWS Glue
  1. 6.1 What is AWS Glue?
  2. 6.2 Use AWS Glue
• Lesson 7: Create State Machines with Step Functions
  1. 7.1 Learn step functions
  2. 7.2 Use Amazon States Language
  3. 7.3 Step functions demo
• Lesson 8: Use Step Functions with AWS Services
  1. 8.1 Learn integration with other AWS products
  2. 8.2 Use DynamoDB with step functions
  3. 8.3 Use AWS ECS/Fargate with step functions
  4. 8.4 Use AWS Callback Pattern
• Lesson 9: Serverless Relational Databases
  1. 9.1 Serverless relational databases
  2. 9.2 Use Aurora Serverless
  3. 9.3 Use Data API for Aurora Serverless
  4. 9.4 Use stored procedures to invoke Lambda
• Lesson 10: Build APIs with API Gateway
  1. 10.1 Use API Gateway
  2. 10.2 Integrate Lambda and API Gateway best practices
• Lesson 11: Authenticate APIs with AWS Cognito
  1. 11.1 Begin authentication
  2. 11.2 Use Cognito User Pools
  3. 11.3 Use Cognito authentication with API Gateway
  4. 11.4 Use Federated Identity
• Lesson 12: Use Serverless Datastores
  1. 12.1 Use DynamoDB for data engineering
  2. 12.2 Use Amazon Athena for data engineering
  3. 12.3 Use Amazon EMR for data engineering
  4. 12.4 Use Amazon EFS for data engineering
• Lesson 13: Create Serverless Business Intelligence and AutoML
  1. 13.1 Integrate Amazon Quicksite
  2. 13.2 Integrate Lambda with AI APIs
  3. 13.3 Integrate Lambda with Sagemaker
• Lesson 14: Create Serverless Data Streaming
  1. 14.1 Use Kinesis Streams
  2. 14.2 Use Computer Vision Streams
• Lesson 15: Case Studies
  1. 15.1 Compare AWS Lambda with Google Cloud Functions
  2. 15.2 Use GCP Cloud Functions with Pub Sub + Cloud Scheduler
  3. 15.3 Use Chalice framework
  4. 15.4 Push versus Pull Architecture
  5. 15.5 Principles of DevOps
  6. 15.6 Principles of cloud computing
  7. 15.7 Summary of serverless computing
  8. 15.8 Managing Packages in AWS Lambda
  9. 15.9 Multi-cloud solutions
• Lesson 16: Course Summary
  1. 16.1 Course summary

Course prerequisites

• Can write functions in Python and execute statements
• Have a basic understanding of AWS

Images of Data Engineering with Python and AWS Lambda LiveLessons course

Sample video of the course

download link

Download part 1 – 1 GB

Download part 2 – 637 MB

File(s) password: www.downloadly.ir

Size

1.6 GB