Public Cloud Additional Uses and Considerations

Name: Public Cloud Additional Uses and Considerations
Author: Chintan Thakkar

Big Data is larger, more complex data sets usually from new data sources and can be so voluminous that processing software is unable to handle it. Cloud…

Preview the first lesson free — get full access to all 4 lessons.

Request a demo Sign in

Course: On-Demand

(pulsating music) Welcome to Public Cloud Additional Uses and Considerations. In these lessons, you will learn about big data and how to run big data on the public cloud. You will also explore machine learning as well as services available on the public cloud that support machine learning use cases. The public cloud provides different ways for scaling cloud resources under load, and you will learn about vertical and horizontal scaling, along with various cost reduction techniques on the public cloud. Let's get started. Companies are continuously collecting large data sets. The public cloud is useful for helping them store big data, analyze it, and generate meaningful outcomes from the data sets. The following chart describes the growth of unstructured or undefined data compared to structured, clearly defined and searchable data. You need to review the big data architecture and ensure data is stored on the right platform. The public cloud provides various services to store big data and then run the analytics use case on it. It also provides services that can run big data analytics and generate visualizations. Sources of big data include social media, images, videos, security systems data, the internet of things devices data, server logs, and application audit logs. Big data analytics can be run on the Hadoop platform. The Hadoop platform supports parallel execution, can break up a job and execute it into multiple parts simultaneously, and support scale-out policies. The downside is that the Hadoop configuration and maintenance are very complex and require special skills. A better idea is to move the entire big data workflow to a public cloud. The public cloud provides a massive computing infrastructure with a pay-per-use model and has options for elastic storage. If you're going to receive large data sets, then you can increase or decrease the size of the storage as well. Let's take a few minutes to understand streaming architecture on the public cloud. A live data stream comes in via the Amazon Kinesis data point. The Elastic MapReduce has the big data analytics that it will run live and then generates the output. You can create the visualization from the information that it generates. The Elastic MapReduce is a managed service available with Amazon Web Services. You can configure and launch the cluster based on your requirements within a few minutes. An added plus is it's low cost because you only pay the per hour cost for the cloud resources. For example, if you want to run batch analytics, then you can create the cluster, run analytics, and terminate the cluster without paying extra. You can also use spot instances in Elastic MapReduce, which will provide additional savings for your organization. You also have an option to use the ondemand nodes with spot nodes. For example, if you want to run the analytics that produces the required output, use the set number of compute instances. You can add capacity to the existing cluster with spot instances, or you can add or remove the instances based on the need within the cluster. There are several different instance types available on the cluster, and you have an option to pick any instance device based on your analytics requirements. The Elastic MapReduce can directly read from a variety of different sources, such as simple storage services, Kinesis stream et cetera? You can use Elastic MapReduce file system or EMRFS with a storage service. Simple storage services or S3, provides durable and reliable storage for your unstructured data, as well as running analytics and storing the data. Here is another use case. You can process 100 terabytes of data using the Elastic MapReduce batch processing architecture by which the data is pushed into simple storage services. Amazon Web Services Elastic MapReduce runs the analytics and generates the output. And then the output is stored on the S3 as well. Let's apply this to a real life use case. Imagine you were working for a company that takes a service contract for big machines. All the machines are streaming the data to your data center. You have written several analytics that can analyze the batch data and generate an alert for outages. You need to migrate this analytic use case to the public cloud. So you should perform the following actions: Upload data to the S3. Elastics MapReduce sorts and aggregates the data. Load the data into Redshift, the managed data warehouse service available on the public cloud. And use a visualization tool for reading the input data from Redshift or another managed data warehouse service and display the visualization.

Beginner Provider : Chintan Thakkar 4 Lessons · 18m in Arabic, German, English, Spanish, French, Portuguese, Chinese

Course Description

Big Data is larger, more complex data sets usually from new data sources and can be so voluminous that processing software is unable to handle it. Cloud infrastructure can be designed to process and interpret Big Data in real-time. In these lessons you will learn the concept of Big Data and how to run Big Data use cases on the public cloud. You will also learn how to run machine learning use cases for the public cloud.

What You'll Learn

Run Big Data use cases on the public cloud
Apply machine learning use cases for the public cloud
Design generic machine learning for public cloud architecture
Scale cloud resources and define auto-scaling requirements
Reduce costs on public cloud infrastructure

Key Takeaways

Big Data refers to larger, more complex data sets, usually from new data sources, that can be so voluminous that processing software is unable to handle it.
Cloud infrastructure can be designed to process and interpret Big Data in real-time.
The course covers running both Big Data and machine learning use cases on the public cloud.
Scalable architecture involves defining auto-scaling requirements on the public cloud.
Public cloud infrastructure can be configured to reduce costs.

Frequently Asked Questions

What does this course cover?

It covers the concept of Big Data, how to run Big Data use cases on the public cloud, how to run machine learning use cases for the public cloud, how to scale cloud resources, and how to reduce costs on public cloud.

What is Big Data as defined in this course?

Big Data is larger, more complex data sets usually from new data sources that can be so voluminous that processing software is unable to handle it.

What skills will I gain from this course?

You will gain skills in Big Data, Big Data Analytics, Cloud Applications, Cloud Infrastructure, Cloud Services, and Public Cloud.

What lessons are included in this course?

The lessons are: How to Run Big Data on the Public Cloud; Generic Machine Learning and How to Design It for Public Cloud Architecture; Scalable architecture and Defining Auto-Scaling Requirements on the Public; and Cost Reduction on the Public Cloud Infrastructure.

How can cloud infrastructure handle Big Data?

Cloud infrastructure can be designed to process and interpret Big Data in real-time.