Machine Learning Basics

Machine Learning (ML) is the core engine behind modern AI systems.
Instead of writing rules manually, ML allows systems to learn patterns from data and improve their performance over time.
This module explains how learning happens, different types of ML, and common challenges faced while building models.

1. What Is Learning?

In Machine Learning, learning means improving performance on a task using experience (data).

Formal Definition

A program is said to learn from experience E, with respect to a task T and a performance measure P, if its performance on T, as measured by P, improves with experience E.

Simple Explanation

• Task → What the model is supposed to do
• Experience → Data provided to the model
• Performance → How well the model does the task

Example

Task: Predict house prices
Experience: Past house sale data
Performance: Prediction accuracy

As the model sees more data, it becomes better at predicting prices — this is learning.

2. Types of Machine Learning

Machine Learning is classified based on how data is provided and how feedback is given.

2.1 Supervised Learning

Supervised learning uses labeled data, meaning each input comes with a correct output.

How It Works

• Model is trained using input-output pairs
• Learns a mapping from input to output
• Feedback is given during training

Example Dataset

Input (Features)	Output (Label)
House size, location	Price
Email text	Spam / Not Spam

---

Types of Supervised Learning

1. Classification

• Output is categorical
• Example: Spam detection, disease prediction

2. Regression

• Output is continuous
• Example: Stock price prediction, temperature prediction

Real-World Example

Email spam filter:

• Input → Email content
• Output → Spam or Not Spam
• Model learns from labeled emails

2.2 Unsupervised Learning

Unsupervised learning works with unlabeled data.

How It Works

• No predefined output
• Model discovers patterns and structure
• Used for exploration and analysis

Common Tasks

1. Clustering

• Group similar data points

Example:
Customer segmentation based on buying behavior

2. Dimensionality Reduction

• Reduce number of features
• Preserve important information

Example:
PCA for data visualization

Real-World Example

E-commerce platform grouping customers into segments without knowing categories beforehand.

2.3 Semi-Supervised Learning

Semi-supervised learning uses both labeled and unlabeled data.

Why It Exists

• Labeling data is expensive
• Large amounts of unlabeled data are available

How It Works

• Train initial model on small labeled dataset
• Use it to learn from unlabeled data

Example

Image classification:

• Few labeled images
• Thousands of unlabeled images

Used in:

• Speech recognition
• Medical imaging

2.4 Reinforcement Learning

Reinforcement learning (RL) is based on trial and error.

How It Works

• Agent interacts with environment
• Takes actions
• Receives rewards or penalties
• Learns optimal strategy

Key Components

• Agent
• Environment
• Actions
• Rewards

Real-World Examples

• Game-playing AI (Chess, Go)
• Robotics
• Recommendation systems

Example:
A robot learning to walk by trial and error.

3. Bias-Variance Tradeoff

This concept explains why models fail to generalize.

Bias

Bias is error due to overly simplistic assumptions.

High bias leads to:

• Underfitting
• Poor training and testing performance

Example:
Using a straight line to model curved data.

Variance

Variance is error due to sensitivity to training data.

High variance leads to:

• Overfitting
• Excellent training performance
• Poor testing performance

Example:
Model memorizes training data.

Tradeoff

• Low bias → High variance
• Low variance → High bias

Goal:
Find balance that minimizes total error.

4. Overfitting & Underfitting

Underfitting

Occurs when:

• Model is too simple
• Cannot capture patterns

Symptoms:

• Low training accuracy
• Low testing accuracy

Example:
Linear model for complex data

Overfitting

Occurs when:

• Model is too complex
• Learns noise instead of patterns

Symptoms:

• High training accuracy
• Low testing accuracy

Example:
Decision tree with unlimited depth

How to Prevent Overfitting

• More data
• Regularization
• Simpler models
• Cross-validation

5. Train-Test Split

Train-test split evaluates model generalization ability.

Why Split Data?

To test model on unseen data.

Common Split Ratios

• 70% training / 30% testing
• 80% training / 20% testing

Example

• Train data → Model learns
• Test data → Model is evaluated

Important Rule:
Test data must not influence training.

6. Cross-Validation

Cross-validation provides a more reliable performance estimate.

K-Fold Cross-Validation

• Data is divided into K equal parts
• Model trains K times
• Each fold is used once as test set

Benefits

• Reduces bias in evaluation
• Works well with small datasets
• Detects overfitting

Example

5-fold cross-validation:

• Train on 4 folds
• Test on 1 fold
• Repeat 5 times
• Average results

Why These Concepts Matter

Without understanding these basics:

• Models fail silently
• Performance metrics mislead
• Production systems break

With strong fundamentals:

• You choose correct algorithms
• You detect problems early
• You build reliable ML systems