Data analysts take in collected data and use it for specific purposes. Oftentimes, the purpose of consuming data is to make better business decisions, improve business efficiencies, or to monitor and mitigate potential risks. To consume data effectively, data that has been collected must be organized and curated for those specific purposes. And this lesson will explore the different steps in curating data for data consumption. When encountering a new set of data, a common approach is to gain a better understanding of what the specific data points are and how they are related to one another. You gain this understanding by determining which data points could be useful data attributes and which data could be useful data measures. This would then put you into a position to better understand the quality of the collected data set you have been given and then what specific actions you must conduct to increase the data quality for downstream consumption. Data attributes are descriptive values for a particular data point or data object. They tend to be qualitative information that provides additional context about a data subject. For example, in a data set about customers, the data values for information such as name, address, phone number, and gender could be data attributes about a customer. These data points provide the analyst with information to better describe the customer. The analyst can also use this information to draw patterns and relationships across their customers. Among customer data, a common attribute used to categorize and group customers is location. By leveraging customer location data, the analyst is able to better understand where their customers come from, their organization's influence in those regions and gain valuable insights into how to focus resources for further customer advertisement or expansion efforts. Data measures are specific data values that provide a measurement or metric for a particular data point or data object. For example, in a data set about customers, the values for the quantity of items the customer purchases per visit and the total dollar spent by the customer per visit are possible data measures. By using these types of data points, the data consumer can calculate customer metrics to, for example, compare whether one customer spent more money than another. Data measures can also be used to calculate critical key performance metrics for a company, such as total customer sales by totaling what all customers spent over a period of time. Both data attributes and data measures are often necessary to provide meaningful analytics. The qualitative data and data attributes provides descriptive context and the ability to categorize the subject of interest in meaningful ways. The data measures provide quantitative data to calculate metrics that make the subjects attributes meaningful. For example, data attributes about identity and characteristics such as age and gender support the data measures of customer sales in several insightful ways. The data point for customer sales provides information about how much revenue a company has received but does not provide additional insights into how or from whom that revenue comes. With information about specific customer identities, an analyst can understand the customers that were most valuable in contributing to company revenue. These insights provide an overall better understanding of the company's operation, but are also used to generate strategic insights such as committing resources and certain tactics and targeting specific customers to maximize the company's revenue. As we will continue to emphasize, select the most appropriate data attributes and data measures relevant for your analytic objectives. If a collected data set has not been tailored specifically for your analysis, there may be a lot of data values that are simply not relevant. For example, given a data set of all customer purchase transactions with a particular company, we might have information about customer identity, purchase price, purchase quantity, total sales and item descriptions. If the objective is to organize data for customer demographics and purchases to determine what types of customer identities contributed to company sales, then specific data values about purchase price, quantity, and item descriptions could be ignored. Data selection from collected data is important, because you don't want to spend time wrangling data that will ultimately not be used. The value of accurate, clean data can't be overstated. In data analytics, a common saying among data analysts is garbage in, garbage out. If the data sets being used to drive important operational or strategic insights are not organized in a clear, consistent way, then the final results derived from the data could be wrong or misleading. To avoid inaccurate outcomes from data analytics, it's important that the data used for analysis is scrubbed for data quality issues, such as missing data, inconsistent values, and misleading assumptions. In a general sense, clean data implies that the organized data tells a consistent story. Consider a use case where we're organizing data about customer identities and their overall purchases. If the data set has missing values for customer attributes such as gender, it would be difficult to analyze the impact of a customer's gender on sales. To continue with such an analysis, you would have to fill in the missing data point with the values such as not collected to ensure it's clear that downstream analytics derived from this data does not make poor assumptions by ignoring the missing fields. An example of a poor outcome could be a report that shows 95% of a company's customers are female. When in reality, most male customers simply did not provide gender information and the demographic breakout is closer to 50-50. Another instance that requires data scrubbing is duplicate data, a situation where you have two or more data records that communicate exactly the same information. Duplicate data issues are not always caused by the collected data itself, but rather how data points are selected and organized for analysis. Let's again consider a data set of customer identities and their purchases. The data set may include customer location data such as address, city, state, and zip code. If our analytic objective is to determine the list of states where we have customers, then we would first end up with a list of state names. The most obvious duplicates would be data points that are exactly the same, such as California with a capital C and California with a lower case C. We should ultimately display a unique list so that only one value of California shows up. The less obvious duplicates could be values that represent the same thing, such as California, CA, and Cali. These are duplicates that also need to be reconciled to one value. Data reconciliation requires the ability to find linkages or mappings across different data records. In our example, a reconciliation exercise might be to map all the different representations of California into one value in order to derive a more complete and accurate analysis downstream. Another example of data reconciliation that can impact a company's analytics is to reconcile customer identities across different data sets. The analytic objective here is to be able to identify and link the same customer across multiple collected data sources. For example, being able to reconcile that a Thomas Cavill who ordered a product online is the same as the Thomas Cav who purchased an item in the company's Los Angeles flagship store and the same Thomas Cavill who signed it for a newsletter. By reconciling that those are the same customer, the company has more information about that customer and how they engage with the company. Having data prepared in such a way allows a company to understand a customer's journey with the company, understand the operational flow of a typical customer's interactions and provide insights to areas where the company can improve customer engagement. Data reconciliation and data linkage in particular is an area in which companies invest considerable time to obtain a 360 degree view of their customers, their inventory and a variety of other aspects of their business.
