Oreilly – Kafka Essentials LiveLessons: A Quick-Start for Building Effective Data Pipelines 2023-7

Kafka Essentials LiveLessons: A Quick-Start for Building Effective Data Pipelines. This course teaches you how to use Apache Kafka, a popular streaming data processing platform, to build efficient data pipelines. You will get to know the main concepts of Kafka, its components and how to set up and use it. You will also learn how to use Kafka to move data between different systems, such as databases, storage systems, and data processing engines. Course schedule:

• Lesson 1: Basic concepts of Kafka
  • Why do you need a messenger?
  • Familiarity with Kafka components
  • Running Kafka in the LHM virtual machine
  • Getting to know how to use Kafka
• Lesson 2: View Kafka operations
  • Explore the features of the KafkaEsque user interface
  • Repeating the basic examples with KafkaEsque
• Lesson 3: Stream NOAA Weather Data with Kafka Python
  • Familiarity with components: Kafka, Python, NOAA-sdk
  • Building a Python generator using keys, synchronization, and serialization
  • Building a Python consumer using groups, partition rebalancing, commits, and offsets
• Lesson 4: Transfer Kafka topic data to external storage
  • Using a PySpark consumer to stream data to HDFS
  • Using a Python consumer to write data to MariaDB (MySQL)
  • Use a Python consumer to write data to HBASE
• Lesson 5: Streaming Edge Images with Kafka Python
  • Familiarity with the diagnosis of edge 3D printing problems
  • How to configure Kafka for image playback
  • How to configure real-time CNN image capture, playback, and analysis using Kafka producers and consumers

What you will learn:

• Core Kafka concepts, such as messenger, topic, partition, consumer, and producer
• How to set up and use Kafka
• How to use Kafka to move data between different systems
• How to use Kafka for real-time data stream processing

This course is suitable for people who:

• Looking to learn how to use Kafka to build an efficient data pipeline
• No previous experience with Apache Kafka
• They have basic knowledge of Python programming

Description of the Kafka Essentials LiveLessons course: A Quick-Start for Building Effective Data Pipelines

• Publisher: Oreilly
• Lecturer: Douglas Eadline
• Training level: beginner to advanced
• Training duration: 3 hours 16 minutes

Course headings

• Lesson 1: Kafka Background Concepts
  1. 1.1 Why Do I need a Message Broker?
  2. 1.2 Understanding Kafka Components
  3. 1.3 Running Kafka on the LHM Virtual Machine
  4. 1.4 Understand Basic Kafka Usage
• Lesson 2: Viewing Kafka Operations
  1. 2.1 Exploring the KafkaEsque UI Features
  2. 2.2 Replay Basic Examples with KafkaEsque
• Lesson 3: Streaming NOAA Weather Data with Kafka Python
  1. 3.1 Understand Components: Kafka, Python, NOAA-sdk
  2. 3.2 Construct a Python Producer using keys, synchronization, and serialization
  3. 3.3 Construct a Python Consumer using groups, partition rebalancing, commits and offsets
• Lesson 4: Moving Kafka Topic Data to External Storage
  1. 4.1 Use a PySpark consumer to stream data to HDFS
  2. 4.2 Use a Python consumer to write data to MariaDB (MySQL)
  3. 4.3 Use a Python consumer to write data to HBASE
• Lesson 5: Edge Image Streaming with Kafka Python
  1. 5.1 Understand detecting Edge 3D printing problems
  2. 5.2 Learn how to configure Kafka for Image streaming
  3. 5.3 Learn how to configure image capture, streaming, and real-time CNN analysis using Kafka producers and consumers

Kafka Essentials LiveLessons course images: A Quick-Start for Building Effective Data Pipelines

Sample video of the course

download link

Download file – 586 MB

File(s) password: www.downloadly.ir

Size

586 MB