Neural Network Design

Name: Neural Network Design
Author: Gabriel Popoola

This chapter will introduce neural network design concepts. It will cover the importance of data in both the performance and selection of neural networks. It…

Preview the first lesson free — get full access to all 7 lessons.

Request a demo Sign in

Course: On-Demand

Welcome to Knowledge City's course on neural networks, neural network design. During these lessons, you'll learn the importance of data, types of neural networks, selecting the type of network to use, activation functions, loss functions and optimizers, hyper-parameters, layers, training, testing, and validation. What you will learn. Why is data important to neural networks? How does the quality of data correlate to the performance of the neural network? Why is it important to consider ethical issues with the data that you use in neural networks? So first, why is data so important to neural networks? If you don't remember anything else about neural networks, you should remember this, garbage in, garbage out. In case you're unfamiliar with this phrase, it essentially means that if you use poor quality or garbage data, then you will get poor quality or garbage results. The neural network is only as good as the data used to train it. Because of this, you wanna avoid as much of the following when choosing your data as possible, noisy data, data with correlated features and instances, data with missing values, data with duplicate features and instances, data with values that have varying orders of magnitude. Data, neural networks, and ethics. Though neural networks can be extremely efficient and useful in a number of systems, it is important to keep in mind that their performance is heavily based on data. Earlier, we discussed some examples of garbage data. However, there's another type of garbage data, biased data. This type of data may not contain any of the qualities to avoid that were previously discussed, but it can lead to unethical classifiers and regressors. Biased data is data that is skewed more in the direction of a particular class or feature. More and more neural network uses and applications involving humans are being explored and implemented. Because of this, the data used to train the networks contains a lot of information pertaining to demographics such as race, age, ethnicity, sexual orientation, and other features. In cases where the networks are being used to classify a certain individual based on one, some, or all of these features, it is easy to create a network that profiles instead of classifies. In order to avoid these ethical issues, when selecting data used to train a model, it is important to have equal and ideal situations or near-equal representation of all demographics across all classes. It's important to remember the importance of privacy when selecting data to use and try in your networks, as well. Just because you're able to obtain information about an individual or entity does not mean that they would want you to use that information to help classify themselves or others. Here are a few examples in Python using the single layer perceptron in the breast cancer dataset. So first, we just have our function definitions. Next, I'm going to be loading the breast cancer dataset and printing the accuracy. So as you can see here, the test score is about 90, 91%. And the training score is about 92, 93%. So now I have duplicate data. And what I'm doing here, I'm just creating a duplicate of all the features in the dataset. So instead of 30 features, you have 60 features, but there's two identical features for each feature. And now I'm going to train the model again. And what you'll see is the training and testing scores have both dropped. Now, this is particularly interesting because I'm not actually introducing any new data, just more of the same data, yet it caused the classifier to perform worse. Next, I'm going to do a noisy data example. And in this case, I'm just adding random noise to the input data. And then I'm going to train the model again. Now, as you can see here, the testing score and training score dropped again, not as much as with the duplicate data, but it was still a decrease. One important thing to note here is that the breast cancer data set is a very clean data set. So regardless of how poor and noisy the data set is, you'll still get fairly good results. But if this was a less clean dataset, which is likely what you'll be working with in practice, then these things could cause an even greater drop in performance. And lastly, we have a correlated data example. So here I'm just adding the same data by multiplying it by a factor of 2.7. So it's really just the same data, just scaled differently. And as you can see, the testing score dropped again as did the training score. And now I'm just going to apply all the results so you can see what happened. And so the blue line is for the testing results and the green line is for the training results. So as you can see the original, which is just the regular clean dataset, no duplicates, no noise, no correlation, outperforms all the other data sets. This concludes this lesson. Thank you.

Beginner Provider : Gabriel Popoola 7 Lessons · 29m in Arabic, German, English, Spanish, French, Portuguese, Urdu, Chinese

Course Description

This chapter will introduce neural network design concepts. It will cover the importance of data in both the performance and selection of neural networks. It will also cover what activation functions, loss functions, and optimizers are as well as how to select which ones to use. This chapter will also introduce you to training, testing, and validation for neural networks.

What You'll Learn

Understand the importance of data in neural network performance and selection
Identify different types of neural networks such as MLP and Convolutional networks
Determine the appropriate neural network type based on data type
Select activation functions, loss functions, and optimizers for a neural network
Configure neural network components including hyperparameters and input, output, and hidden layers
Apply training, testing, and validation to neural networks

Key Takeaways

Data plays a central role in both the performance and the selection of neural networks.
Choosing a neural network involves matching the network type to the type of data being used.
Activation functions, loss functions, and optimizers are core components, and the course explains how to select which ones to use.
Neural networks rely on training, testing, and validation as part of their design process.
Designing a neural network includes setting hyperparameters and configuring input, output, and hidden layers.

Frequently Asked Questions

What does this course cover?

It introduces neural network design concepts, including the importance of data in neural network performance and selection, what activation functions, loss functions, and optimizers are and how to select them, and training, testing, and validation for neural networks.

What skills will I gain from this course?

The course builds skills in Algorithm Design, Artificial Neural Networks, Deep Learning, Deep Learning Methods, Loss Functions, and Neural Engineering.

What topics are included in the lessons?

Lessons cover the Importance of Data; Types of Neural Networks (MLP, Convolutional, etc.); Determining Neural Network Type based on Data Type; Activation Functions, Loss Functions, and Optimizers; Hyperparameters; Layers (Input, Output, Hidden); and Training, Testing, Validation.

Will I learn how to choose the right neural network?

Yes. The course covers how to select a neural network, including determining the neural network type based on the data type.