Log In

Don't have an account? Sign up now

Lost Password?

Sign Up

Prev Next
Data Science > Supervised Learning Algorithms

Supervised Learning Algorithms

Supervised Learning is a branch of Machine Learning where a model learns from labeled data.
Labeled data means that for every input, the correct output is already known.

Example:

  • Input: customer details
  • Output (label): whether the customer will churn (Yes/No)

The goal of supervised learning is to learn a mapping function that maps inputs to correct outputs, so the model can predict outputs for new, unseen data.

Supervised learning problems are mainly divided into:

  1. Regression → Predicting continuous values
  2. Classification → Predicting categories or classes

1. Regression

Regression algorithms are used when the target variable is continuous, meaning it can take any numerical value.

Examples:

  • House price prediction
  • Salary prediction
  • Temperature forecasting
  • Sales prediction

1.1 Linear Regression

What Is Linear Regression?

Linear Regression is the simplest and most fundamental regression algorithm.
It models the relationship between:

  • One independent variable (feature)
  • One dependent variable (target)

using a straight line.

Mathematical Intuition

The relationship is expressed as:

y = mx + c

Where:

  • y = predicted output
  • x = input feature
  • m = slope (weight)
  • c = intercept (bias)

How Linear Regression Works

  • The model tries to find the best-fit line
  • This line minimizes the error between predicted and actual values
  • Error is measured using Mean Squared Error (MSE)

Real-World Example

Predicting salary based on years of experience:

  • Experience = input
  • Salary = output

As experience increases, salary generally increases in a linear manner.

Key Assumptions

  • Linear relationship
  • No multicollinearity
  • Homoscedasticity
  • Errors are normally distributed

1.2 Multiple Linear Regression

What Is Multiple Linear Regression?

Multiple Linear Regression extends linear regression to multiple input variables.

Instead of one feature, we use many features to predict the target.

Mathematical Representation

y = b₀ + b₁x₁ + b₂x₂ + ... + bₙxₙ

Real-World Example

House price prediction based on:

  • Area
  • Number of bedrooms
  • Location
  • Age of the house

Each feature contributes some weight to the final price.

Why It’s Important

Most real-world problems depend on multiple factors, not just one.

1.3 Polynomial Regression

What Is Polynomial Regression?

Polynomial Regression is used when the relationship between variables is non-linear, but still modeled using linear regression techniques.

How It Works

  • Input features are transformed into polynomial terms
  • Model still remains linear in parameters

Example

Predicting car speed vs fuel consumption:

  • Fuel efficiency increases up to a point
  • Then decreases as speed increases

This curved relationship cannot be captured by straight lines.

Risk

  • High-degree polynomials may cause overfitting

1.4 Ridge & Lasso Regression

These are regularized regression techniques used to solve overfitting.

Why Regularization Is Needed

When a model:

  • Is too complex
  • Fits noise instead of patterns
  • Performs poorly on new data

Ridge Regression

  • Adds L2 penalty
  • Shrinks coefficients but does not eliminate them

Used when:
Many features contribute small effects.

Lasso Regression

  • Adds L1 penalty
  • Can reduce some coefficients to zero

Used when:
Feature selection is required.

Real-World Use

  • High-dimensional datasets
  • Preventing overfitting
  • Improving model generalization

2. Classification

Classification algorithms are used when the target variable is categorical.

Examples:

  • Spam vs Not Spam
  • Fraud vs Non-Fraud
  • Pass vs Fail
  • Disease Yes or No

2.1 Logistic Regression

What Is Logistic Regression?

Despite its name, Logistic Regression is a classification algorithm, not regression.

It predicts probabilities, which are then converted into class labels.

How It Works

  • Uses sigmoid function
  • Output is between 0 and 1
Sigmoid(x) = 1 / (1 + e⁻ˣ)

Decision Boundary

  • If probability ≥ 0.5 → Class 1
  • Else → Class 0

Real-World Example

Predicting whether a customer will buy a product:

  • Yes (1)
  • No (0)

Advantages

  • Simple
  • Interpretable
  • Efficient

2.2 K-Nearest Neighbors (KNN)

What Is KNN?

KNN is a distance-based algorithm.

It classifies a data point based on the majority class of its nearest neighbors.

Leave a Comment