Information Technology

Python With Data Science

Instructor-Led 9 Agenda

Description

Covers theoretical and technical aspects of using Python in Applied Data Science projects and Data Logistics use cases.

Course Overview

Covers theoretical and technical aspects of using Python in Applied Data Science projects and Data Logistics use cases.

Course Objective

• NumPy, pandas, Matplotlib, scikit-learn • Python REPLs • Jupyter Notebooks • Data analytics life-cycle phases • Data repairing and normalizing • Data aggregation and grouping • Data visualization • Data science algorithms for supervised and unsupervised machine learning

Who Should Attend

Audience: Data Scientists, Software Developers, IT Architects, and Technical Managers. Participants should have the general knowledge of statistics and programming Also familiar with Python

Python for Data Science Lesson Id 1

Python for Data Science

Applied Data Science Lesson Id 2

Applied Data Science

Data Analytics Life-cycle Phases Lesson Id 3

Data Analytics Life-cycle Phases

Repairing and Normalizing Data Lesson Id 4

Repairing and Normalizing Data

Descriptive Statistics Computing Features in Python Lesson Id 5

Descriptive Statistics Computing Features in Python

Data Aggregation and Grouping Lesson Id 6

Data Aggregation and Grouping

Data Visualization with matplotlib Lesson Id 7

Data Visualization with matplotlib

Data Science and ML Algorithms in scikit-learn Lesson Id 8

Data Science and ML Algorithms in scikit-learn

Lab Exercises Lesson Id 9

Lab Exercises

Lesson Id	Title	Description
1	Python for Data Science	• Using Modules<br />• Listing Methods in a Module<br />• Creating Your Own Modules<br />• List Comprehension<br />• Dictionary Comprehension<br />• String Comprehension<br />• Python 2 vs Python 3<br />• Sets (Python 3+)<br />• Python Idioms<br />• Python Data Science “Ecosystem”<br />• NumPy<br />• NumPy Arrays<br />• NumPy Idioms<br />• pandas<br />• Data Wrangling with pandas' DataFrame<br />• SciPy<br />• Scikit-learn<br />• SciPy or scikit-learn?<br />• Matplotlib<br />• Python vs R<br />• Python on Apache Spark<br />• Python Dev Tools and REPLs<br />• Anaconda<br />• IPython<br />• Visual Studio Code<br />• Jupyter<br />• Jupyter Basic Commands<br />• Summary
2	Applied Data Science	• What is Data Science?<br />• Data Science Ecosystem<br />• Data Mining vs. Data Science<br />• Business Analytics vs. Data Science<br />• Data Science, Machine Learning, AI?<br />• Who is a Data Scientist?<br />• Data Science Skill Sets Venn Diagram<br />• Data Scientists at Work<br />• Examples of Data Science Projects<br />• An Example of a Data Product<br />• Applied Data Science at Google<br />• Data Science Gotchas<br />• Summary
3	Data Analytics Life-cycle Phases	• Big Data Analytics Pipeline<br />• Data Discovery Phase<br />• Data Harvesting Phase<br />• Data Priming Phase<br />• Data Logistics and Data Governance<br />• Exploratory Data Analysis<br />• Model Planning Phase<br />• Model Building Phase<br />• Communicating the Results<br />• Production Roll-out<br />• Summary
4	Repairing and Normalizing Data	• Repairing and Normalizing Data<br />• Dealing with the Missing Data<br />• Sample Data Set<br />• Getting Info on Null Data<br />• Dropping a Column<br />• Interpolating Missing Data in pandas<br />• Replacing the Missing Values with the Mean Value<br />• Scaling (Normalizing) the Data<br />• Data Preprocessing with scikit-learn<br />• Scaling with the scale() Function<br />• The MinMaxScaler Object<br />• Summary
5	Descriptive Statistics Computing Features in Python	• Descriptive Statistics<br />• Non-uniformity of a Probability Distribution<br />• Using NumPy for Calculating Descriptive Statistics Measures<br />• Finding Min and Max in NumPy<br />• Using pandas for Calculating Descriptive Statistics Measures<br />• Correlation<br />• Regression and Correlation<br />• Covariance<br />• Getting Pairwise Correlation and Covariance Measures<br />• Finding Min and Max in pandas DataFrame<br />• Summary
6	Data Aggregation and Grouping	• Data Aggregation and Grouping<br />• Sample Data Set<br />• The pandas.core.groupby.SeriesGroupBy Object<br />• Grouping by Two or More Columns<br />• Emulating the SQL's WHERE Clause<br />• The Pivot Tables<br />• Cross-Tabulation<br />• Summary
7	Data Visualization with matplotlib	• Data Visualization<br />• What is matplotlib?<br />• Getting Started with matplotlib<br />• The Plotting Window<br />• The Figure Options<br />• The matplotlib.pyplot.plot() Function<br />• The matplotlib.pyplot.bar() Function<br />• The matplotlib.pyplot.pie () Function<br />• Subplots<br />• Using the matplotlib.gridspec.GridSpec Object<br />• The matplotlib.pyplot.subplot() Function<br />• Hands-on Exercise<br />• Figures<br />• Saving Figures to File<br />• Visualization with pandas<br />• Working with matplotlib in Jupyter Notebooks<br />• Summary
8	Data Science and ML Algorithms in scikit-learn	• Data Science, Machine Learning, AI?<br />• Types of Machine Learning<br />• Terminology: Features and Observations<br />• Continuous and Categorical Features (Variables)<br />• Terminology: Axis<br />• The scikit-learn Package<br />• scikit-learn Estimators<br />• Models, Estimators, and Predictors<br />• Common Distance Metrics<br />• The Euclidean Metric<br />• The LIBSVM format<br />• Scaling of the Features<br />• The Curse of Dimensionality<br />• Supervised vs Unsupervised Machine Learning<br />• Supervised Machine Learning Algorithms<br />• Unsupervised Machine Learning Algorithms<br />• Choose the Right Algorithm<br />• Life-cycles of Machine Learning Development<br />• Data Split for Training and Test Data Sets<br />• Data Splitting in scikit-learn<br />• Hands-on Exercise<br />• Classification Examples<br />• Classifying with k-Nearest Neighbors (SL)<br />• k-Nearest Neighbors Algorithm<br />• k-Nearest Neighbors Algorithm<br />• The Error Rate<br />• Hands-on Exercise<br />• Dimensionality Reduction<br />• The Advantages of Dimensionality Reduction<br />• Principal component analysis (PCA)<br />• Hands-on Exercise<br />• Data Blending<br />• Decision Trees (SL)<br />• Decision Tree Terminology<br />• Decision Tree Classification in Context of Information Theory<br />• Information Entropy Defined<br />• The Shannon Entropy Formula<br />• The Simplified Decision Tree Algorithm<br />• Using Decision Trees<br />• Random Forests<br />• SVM<br />• Naive Bayes Classifier (SL)<br />• Naive Bayesian Probabilistic Model in a Nutshell<br />• Bayes Formula<br />• Classification of Documents with Naive Bayes<br />• Unsupervised Learning Type: Clustering<br />• Clustering Examples<br />• k-Means Clustering (UL)<br />• k-Means Clustering in a Nutshell<br />• k-Means Characteristics<br />• Regression Analysis<br />• Simple Linear Regression Model<br />• Linear vs Non-Linear Regression<br />• Linear Regression Illustration<br />• Major Underlying Assumptions for Regression Analysis<br />• Least-Squares Method (LSM)<br />• Locally Weighted Linear Regression<br />• Regression Models in Excel<br />• Multiple Regression Analysis<br />• Logistic Regression<br />• Regression vs Classification<br />• Time-Series Analysis<br />• Decomposing Time-Series<br />• Summary
9	Lab Exercises	Lab 1 - Learning the Lab Environment <br />Lab 2 - Using Jupyter Notebook <br />Lab 3 - Repairing and Normalizing Data <br />Lab 4 - Computing Descriptive Statistics <br />Lab 5 - Data Grouping and Aggregation <br />Lab 6 - Data Visualization with matplotlib <br />Lab 7 - Data Splitting <br />Lab 8 - k-Nearest Neighbors Algorithm <br />Lab 9 - The k-means Algorithm <br />Lab 10 - The Random Forest Algorithm