Machine Learning Course (Fall 2024)

Codes and Projects for Machine Learning Course, University of Isfahan.

Contents:

Supervised Learning

Chapter : Regression

• Linear regression
• Gradient descent algorithm
• Multi-variable linear regression
• Polynomial regression
• Normal equation
• Locally weighted regression
• Probabilistic interpretation
• Download slides in Persian

Chapter : Logistic Regression

• Classification and logistic regression
• Probabilistic interpretation
• Logistic regression cost function
• Logistic regression and gradient descent
• Multi-class logistic regression
• Advanced optimization methods

Chapter : Regularization

• Overfitting and Regularization
• L2-Regularization (Ridge)
• L1-Regularization (Lasso)
• Regression with regularization
• Classification with regularization

Chapter : Neural Networks

• Milti-class logistic regression
• Softmax classifier
• Training softmax classifier
• Geometric interpretation
• Non-linear classification
• Neural Networks
• Training neural networks: Backpropagation
• Training neural networks: advanced optimization methods
• Gradient checking
• Mini-batch gradient descent

Chapter : Support Vector Machines

• Motivation: optimal decision boundary
• Support vectors and margin
• Objective function formulation: primal and dual
• Non-linear classification: soft margin
• Non-linear classification: kernel trick
• Multi-class SVM

Unsupervided Learning

Chapter : Clustering

• Supervised vs unsupervised learning
• Clustering
• K-Means clustering algorithm (demo)
• Determining number of clusters: Elbow method
• Postprocessing methods: Merge and Split clusters
• Bisectioning clustering
• Hierarchical clustering
• Application 1: Clustering digits
• Application 2: Image Compression

Chapter : Dimensionality Reduction and PCA

• Introduction to PCA
• PCA implementation in python
• PCA Applications
• Singular Value Decomposition (SVD)

Chapter : Anomally Detection

• Intoduction to anomaly detection
• Some applications (security, manufacturing, fraud detection)
• Anoamly detection using probabilitic modelling
• Uni-variate normal distribution for anomaly detection
• Multi-variate normal distribution for anomaly detection
• Evaluation measures (TP, FP, TN, FN, Precision, Recall, F-score)
• Anomaly detection as one-class classification
• Classification vs anomaly detection

Chapter : Recommender Systems

• Introduction to recommender systems
• Collaborative filtering approach
• User-based collaborative filtering
• Item-based collaborative filtering
• Similarity measures (Pearson, Cosine, Euclidian)
• Cold start problem
• Singular value decomposition
• Content-based recommendation
• Cost function and minimization