Last updated 9/2017MP4 | Video: h264, 1280×720 | Audio: AAC, 44.1 KHzLanguage: English | Size: 1.38 GB | Duration: 10h 54m

Build scalable and efficient data processing platforms

What you’ll learn

Basic concepts of Scala

Analysing data using Spark in Scala

Creation of fast data processing using SMACK Stack

Requirements

Experience with Scala is essential

Basic knowledge of data processing concepts

Description

If you want to outrun your competitors by taking business decisions using your data, then this course is for you.

SMACK is an open source full stack for big data architecture. It is a combination of Spark, Mesos, Akka, Cassandra, and Kafka. This stack is the newest technique developers have begun to use to tackle critical real- analytics for big data.

SMACK: Getting Started with Scala, Spark, and the SMACK Stack gets you familiar with Scala and understanding the various features offered by it. You will also get to understand the process for data analysis using Spark. Finally, you will be introduced to the SMACK Stack which helps us to process data blazingly fast. Development using these technologies can be summarized as: More data: Less .

This Learning Path is a learner material and the curriculum is so planned to meet your learning needs. It starts with the basics of Apache Spark, one of the trending big data processing frameworks on the market today. We it moves on to Scala, which has emerged as an important tool for perfog various data analysis tasks efficiently. It will help you leverage popular Scala libraries and tools to perform core data analysis tasks with ease in Spark. In the last part, we will teach you how to integrate the SMACK stack to create a highly efficient data analysis system for fast data processing.

By the end of the course, you’ll be able to analyze and process data swiftly and efficiently as compared to other traditional data analytic systems.

About the Author

For this course, we have combined the best works of this esteemed author

Nishant Garg has over 16 years of software architecture and development experience in various technologies, such as Java Enterprise Edition, SOA, Spring, Hadoop, Hive, Flume, Sqoop, Oozie, Spark, YARN, Impala, Kafka, Storm, Solr/Lucene, NoSQL databases (such as HBase, Cassandra, and MongoDB), and MPP databases (such as GreenPlum). He received his MS in software systems from the Birla Institute of Technology and Science, Pilani, India, and is currently working as a senior technical architect for the Big Data R&D Labs with Impetus Infotech Pvt. Ltd. Nishant has also undertaken many speaking engagements on big data technologies and is also the author of Learning Apache Kafka & HBase Essestials, Packt Publishing.

Anatolii Kmetiuk has been working with Scala-based technologies for four years. He has experience in Deep Learning models for text processing. He is interested in Category Theory and Type-level programming in Scala. Another field of interest is Chaos and Complexity Theory and Artificial Life, and ways to implement them in programming languages.

Raul Estrada Aparicio is a programmer since 1996 and Java Developer since 2001. He loves functional languages such as Scala, Elixir, Clojure, and Haskell. He also loves all the topics related to Computer Science. With more than 12 years of experience in High Availability and Enterprise Software, he has designed and implemented architectures since 2003.His specialization is in systems integration and has participated in projects mainly related to the financial sector. He has been an enterprise architect for BEA Systems and Oracle Inc., but he also enjoys Mobile Programming and Game Development. He considers himself a programmer before an architect, eeer, or developer.

Overview

Section 1: Apache Spark Fundamentals

Lecture 1 Course Overview

Lecture 2 Spark Introduction

Lecture 3 Spark Components

Lecture 4 Getting Started

Lecture 5 Introduction to Hadoop

Lecture 6 Hadoop Processes and Components

Lecture 7 HDFS and YARN

Lecture 8 Map Reduce

Lecture 9 Introduction to Scala

Lecture 10 Scala Programming Fundamentals

Lecture 11 Objects in Scala

Lecture 12 Collections

Lecture 13 Spark Execution

Lecture 14 Understanding RDD

Lecture 15 RDD Operations

Lecture 16 Loading and Saving Data in Spark

Lecture 17 Managing Key-Value Pairs

Lecture 18 Accumulators

Lecture 19 Writing a Spark Application

Section 2: Spark for Data Analysis in Scala

Lecture 20 The Course Overview

Lecture 21 ing the Competition Dataset

Lecture 22 Installing Spark Notebook

Lecture 23 Spark Abstractions – RDD, DataFrame

Lecture 24 Loading CSV data into DataFrame

Lecture 25 Different types of widgets supported for Spark Notebook for DataFrame visualizat

Lecture 26 Statistical Functions Supported by Spark

Lecture 27 Operations on DataFrame

Lecture 28 Feature Transformers

Lecture 29 Feature Selectors

Lecture 30 Architecture

Lecture 31 Algorithms: Linear Regression and Regression Trees

Section 3: Fast Data Processing Systems with SMACK Stack

Lecture 32 The Course Overview

Lecture 33 Modern Data-Processing Challenges

Lecture 34 The Data-Processing Pipeline Architecture

Lecture 35 SMACK Technologies

Lecture 36 Understanding Data Expert Profiles and Chag the Data Center Operations

Lecture 37 Scala Collections

Lecture 38 Iterators in Scala

Lecture 39 More Functions with Scala

Lecture 40 Actor Model In a Nutshell

Lecture 41 Working with Actors

Lecture 42 Spark Concepts

Lecture 43 Resilient Distributed Datasets

Lecture 44 Spark in Cluster Mode

Lecture 45 Spark Streaming

Lecture 46 NoSQL

Lecture 47 Apache Cassandra Installation

Lecture 48 Backup and Compression

Lecture 49 Recovery Techniques

Lecture 50 Recovery Techniques – DBMS Optimization, Bloom Filter, and More

Lecture 51 The Spark Cassandra Connector

Lecture 52 Introduction to the Spark Cassandra Connector

Lecture 53 Cassandra and Spark Streaming Basics

Lecture 54 Functions with Cassandra

Lecture 55 Akka and Cassandra

Lecture 56 Introducing Kafka

Lecture 57 Installation

Lecture 58 Cluster

Lecture 59 Architecture

Lecture 60 Producers

Lecture 61 Consumers

Lecture 62 Integration and Administration

Lecture 63 Akka, Spark, and Kafka

Lecture 64 Kafka and Cassandra

Lecture 65 The Apache Mesos Architecture

Lecture 66 Resource Allocation

Lecture 67 Running a Mesos Cluster on a Private Data Center

Lecture 68 Scheduling and Managing the Frameworks

Lecture 69 Apache Aurora

Lecture 70 Singularity

Lecture 71 Apache Spark on Apache Mesos

Lecture 72 Apache Cassandra on Apache Mesos

Lecture 73 Apache Kafka on Apache Mesos

Data Analysts, Data Scientists, and Business Analysts can use this course to make highly precise and fast data models.