Edge Detection & Gradient Filters

These filters find boundaries between regions by detecting rapid changes in pixel brightness. Edge detection is foundational to object detection, segmentation, and feature extraction.

Classification of Edge Detection Filters

graph TD
    A[Edge Detection & Gradient Filters] --> B[First-Order Derivative]
    A --> C[Second-Order Derivative]
    A --> D[Multi-Stage Pipeline]

    B --> B1[Sobel Filter]
    B --> B2[Prewitt Filter]
    B --> B3[Roberts Cross Filter]
    B --> B4[Scharr Filter]

    C --> C1[Laplacian of Gaussian - LoG]

    D --> D1[Canny Edge Detector]

    style A fill:#4A90D9,color:#fff
    style B fill:#7BB661,color:#fff
    style C fill:#E8A04C,color:#fff
    style D fill:#C586C0,color:#fff

1. Sobel Filter

Spatial
The Sobel filter is a classic, widely used and edge detection filter used in image processing to highlight boundaries and sharp intensity changes. It operates in the spatial domain to calculate an approximation of the image gradient—meaning it finds the areas where the pixel intensity changes the most rapidly.

How it works?

Unlike the Laplacian filter (which uses a single kernel to find the second derivative), the Sobel filter uses a first-order derivative approach. It works by calculating the gradient of image intensity in two specific directions, horizontally and vertically.

To do this, it convolves (slides) two separate $3 \times 3$ kernels across the original image:

One kernel detects changes along the X-axis (highlighting vertical edges).
One kernel detects changes along the Y-axis (highlighting horizontal edges).
The Sobel Formulas and Kernels
Let the original image be $A$ .
- The X-Direction Kernel ( $G_{x}$ ): This kernel calculates the horizontal derivative. The left column is negative and the right column is positive, while the center column is zero. As this slides across an image, it subtracts the pixels on the left from the pixels on the right.
- The Y-Direction Kernel ( $G_{y}$ ): This kernel calculates the vertical derivative. It subtracts the pixels on the top from the pixels on the bottom.

G_{x} = \underset{The X-Direction Kernel}{\underset{⏟}{\begin{array}{ccc} - 1 & 0 & + 1 \\ - 2 & 0 & + 2 \\ - 1 & 0 & + 1 \end{array}}} * A G_{y} = \underset{The Y-Direction Kernel}{\underset{⏟}{\begin{array}{ccc} - 1 & - 2 & - 1 \\ 0 & 0 & 0 \\ + 1 & + 2 & + 1 \end{array}}} * A

Calculating the Final Edge Output: Once you have the horizontal ( $G_{x}$ ) and vertical ( $G_{y}$ ) gradient approximations, you combine them to find the absolute edge strength and the direction of the edge at every single pixel.
- Gradient Magnitude (Edge Strength): To find the total magnitude of the edge at a specific point, you calculate the hypotenuse of the horizontal and vertical vectors using the Pythagorean theorem:
$G = \sqrt{G_{x}^{2} + G_{y}^{2}}$
- Gradient Direction (Edge Orientation): The Sobel filter doesn't just tell you where the edge is; it tells you which way it is pointing. The angle of the edge relative to the pixel grid is calculated using the arctangent:
$θ = \arctan (\frac{G_{y}}{G_{x}})$

Key Keatures

Built-in Noise Smoothing

Referring to the concept in linear algebra called matrix separability, A 2D Sobel filter is separable if it can be written as the product of a "column" and a "row".

[\begin{matrix} - 1 & 0 & 1 \\ - 2 & 0 & 2 \\ - 1 & 0 & 1 \end{matrix}] = [\begin{matrix} 1 \\ 2 \\ 1 \end{matrix}] * [\begin{matrix} - 1 & 0 & 1 \end{matrix}]

The Row Vector $[\begin{matrix} - 1 & 0 & 1 \end{matrix}]$ : This is the actual derivative part. As it moves horizontally, it calculates the difference between the left pixel and the right pixel.
The Column Vector $[\begin{matrix} 1 \\ 2 \\ 1 \end{matrix}]$ : This acts vertically (perpendicular to the derivative). This is a 1D Gaussian smoothing filter. It averages the pixels from the row above, the current row, and the row below.
By using the weights 1, 2, 1, the Sobel filter is applying a weighted average. It Looks at the pixels above and below me to help smooth out any random specks of noise, but give double priority (the 2) to the exact row I am currently calculating.

When to use

General-purpose edge detection where some noise tolerance is needed.
Because it computes $X$ and $Y$ separately, it is incredibly useful for computer vision tasks where you need to know the orientation (edge direction — horizontal or vertical) of an object.
As a gradient computation step inside more complex pipelines (e.g., Canny uses Sobel internally).
The $3 \times 3$ kernels contain only small integers (0, 1, and 2), which makes the convolution math extremely fast for computers to process, even in real-time video applications.

When to avoid

When you need very thin, precise edges — Sobel produces thick, somewhat blurry edge responses.
When the image has significant noise — consider denoising first.

Caution

Sobel detects edges per direction. If you only compute $G_{x}$ , you will miss horizontal edges entirely. Always compute the full magnitude unless you intentionally want directional edges.

2. Prewitt Filter

Spatial
Identical in concept to Sobel, the Prewitt Filter is another classic, first-order derivative edge-detection operator in image processing. The main and only difference between both is Prewitt Filter uses a simpler, uniform kernel without the center-weighted smoothin

How it works
Like the Sobel filter, the Prewitt operator works in the spatial domain to calculate the image gradient. It slides two $3 \times 3$ convolution kernels across the original image to detect the rate of brightness change in two distinct directions:

An X-direction kernel to detect vertical edges.
A Y-direction kernel to detect horizontal edges.
The Prewitt Formulas and Kernels

G_{x} = \underset{The X-Direction Kernel}{\underset{⏟}{\begin{array}{ccc} - 1 & 0 & + 1 \\ - 1 & 0 & + 1 \\ - 1 & 0 & + 1 \end{array}}} * A G_{y} = \underset{The Y-Direction Kernel}{\underset{⏟}{\begin{array}{ccc} - 1 & - 1 & - 1 \\ 0 & 0 & 0 \\ + 1 & + 1 & + 1 \end{array}}} * A

Gradient Magnitude (Edge Strength):

G = \sqrt{G_{x}^{2} + G_{y}^{2}}

Gradient Direction (Edge Orientation):

θ = \arctan (\frac{G_{y}}{G_{x}})

Key Features

Uniform Averaging (Box Blur):

Prewitt filter can be decomposed into a 1D derivative $[\begin{matrix} - 1 & 0 & 1 \end{matrix}]$ and a 1D smoothing vector $[\begin{matrix} 1 \\ 1 \\ 1 \end{matrix}]$ . This means it applies a uniform, equal-weight average perpendicular to the edge direction. 👎 It treats the row above, the current row, and the row below with exact equal importance. If you are trying to find the precise location of a horizontal pixel change, blurring everything equally across three rows can "smear" the edge vertically.

When to use

When you need a fast, simple gradient estimate and Sobel's extra smoothing is not required.
Low-noise images where the difference between Sobel and Prewitt is negligible.
Extreme Computational Simplicity: Filter not need to perform any multiplication (like multiplying by $2$ in Sobel); the computer only had to perform basic addition and subtraction to find the edges.

When to avoid

On noisy images — It treats the adjacent rows/columns with the exact same importance as the central pixel row/column, therefore random noise in the background can easily throw off the edge calculation.
It lacks the Gaussian "center-weighted" anchor of the Sobel filter.
Default to Sobel unless you have a specific reason to prefer Prewitt.
Slightly Sharper, Less Accurate Edges: Because it does not apply a Gaussian blur, the edges detected by Prewitt can sometimes appear slightly thinner or more jagged than Sobel

3. Roberts Cross Filter

How it works
Uses two tiny 2×2 diagonal kernels to compute gradients at 45° and 135°.

G_{x} = \begin{array}{cc} + 1 & 0 \\ 0 & - 1 \end{array} G_{y} = \begin{array}{cc} 0 & + 1 \\ - 1 & 0 \end{array}

When to use

Extremely fast edge detection on clean, low-noise images.
Embedded or real-time systems with tight compute budgets.

When to avoid

Any image with noise — Roberts is the most noise-sensitive of all gradient filters.
When you need accurate edge localization — the 2×2 kernel gives coarse results.

Caution

Roberts Cross is rarely used in modern pipelines. It is included for historical completeness and for very constrained environments. In almost every other case, Sobel or Canny will produce better results.

4. Canny Edge Detector

Canny is best described as an "Optimal Edge Detection Algorithm" that integrates both smoothing and sharpening filters into a complete pipeline.

How it works?

A multi-stage algorithm, not a simple convolution:

Gaussian Smoothing — reduces noise before edge detection.
Sobel Gradient — computes edge magnitude and direction at every pixel.
Non-Maximum Suppression — thins edges to 1-pixel-wide lines by keeping only local maxima in the gradient direction.
Double Thresholding — classifies pixels into
- Strong edges (above high threshold)
- Weak edges (between low and high)
- Non-edges (using both threshold)
Hysteresis — promotes weak edge pixels that are connected to strong edge pixels; discards isolated weak ones.

graph LR
	A[Input Image] --> B[1\.
Gaussian
Smoothing]
	B --> C[2\.
Gradient
Calculation
Sobel]
	C --> D[3\.
Non-Maximum
Suppression]
	D --> E[4\.
Double
Thresholding]
    E --> F[5\.
Edge
Tracking by
Hysteresis]
    F --> G[Final Edge Map]

When to use

When you need clean, thin, connected edge maps — it is the gold standard for edge detection.
Object boundary detection, contour extraction, lane detection.
Any application where edge precision and connectivity matter.

Parameters to tune

Low threshold: controls which weak edges are kept if connected to strong ones. Too low → noisy edges appear.
High threshold: controls which edges are classified as strong. Too high → real edges are missed.
A common starting ratio is high:low = 2:1 or 3:1.
$σ$ of the Gaussian: larger $σ$ suppresses more noise but also blurs fine edges.

When to avoid

Real-time pipelines where compute time is critical — Canny is more expensive than Sobel alone.
When you only need a rough gradient magnitude and not precise edge lines.

Caution

The two thresholds in Canny need to be tuned per-image or per-dataset. A fixed threshold that works well on one lighting condition may fail completely on another. Consider using automatic threshold selection (e.g., based on the median pixel intensity of the image).

5. Scharr Filter

Spatial
The Scharr filter is a direct refinement of Sobel. Sobel's 1-2-1 weighted smoothing kernel is a reasonable approximation of a Gaussian, but it introduces a subtle directional bias — the gradient magnitude it reports can vary slightly depending on the angle of the edge. The Scharr filter corrects this by replacing those weights with 3-10-3, which is the mathematically optimal weighting for near-perfect rotational symmetry. In practice, this means Scharr reports the same edge strength regardless of whether the edge is vertical, horizontal, or diagonal.

How it works?

Exactly like Sobel: two kernels slide across the image — one for horizontal change ( $G_{x}$ ), one for vertical ( $G_{y}$ ). The only difference is the kernel weights.

G_{x} = \underset{The X-Direction Kernel}{\underset{⏟}{\begin{array}{ccc} - 3 & 0 & + 3 \\ - 10 & 0 & + 10 \\ - 3 & 0 & + 3 \end{array}}} * A G_{y} = \underset{The Y-Direction Kernel}{\underset{⏟}{\begin{array}{ccc} - 3 & - 10 & - 3 \\ 0 & 0 & 0 \\ + 3 & + 10 & + 3 \end{array}}} * A

Gradient magnitude and direction are computed the same way as Sobel:

G = \sqrt{G_{x}^{2} + G_{y}^{2}} θ = \arctan (\frac{G_{y}}{G_{x}})

Step-by-Step Example

A 3×3 patch with a sharp left-to-right transition (dark left, bright right — a vertical edge):

Image Patch = [\begin{matrix} 50 & 50 & 200 \\ 50 & 50 & 200 \\ 50 & 50 & 200 \end{matrix}]

Applying $G_{x}$ (Scharr):

G_{x} = (- 3) (50) + (0) (50) + (3) (200) + (- 10) (50) + (0) (50) + (10) (200) + (- 3) (50) + (0) (50) + (3) (200)

= - 150 + 600 - 500 + 2000 - 150 + 600 = 2400

Applying $G_{x}$ (Sobel) on the same patch — for comparison:

G_{x}^{S o b e l} = (- 1) (50) + (0) (50) + (1) (200) + (- 2) (50) + (0) (50) + (2) (200) + (- 1) (50) + (0) (50) + (1) (200)

= - 50 + 200 - 100 + 400 - 50 + 200 = 600

Both correctly detect the vertical edge, and $G_{y} = 0$ for both (the patch has no top-to-bottom variation). The Scharr value is $4 \times$ larger because its weights are heavier — this is expected and not the point. The real advantage is directional accuracy: on a diagonal ( $45 °$ ) edge, Sobel's magnitude drifts by ~5% depending on the exact angle; Scharr stays consistent.

Sobel vs Scharr at a glance

	Sobel	Scharr
Smoothing weights	1 - 2 - 1	3 - 10 - 3
Rotational accuracy	~95% isotropic	~99.8% isotropic
Noise sensitivity	Moderate	Slightly higher (larger weights)
Kernel sizes available	3×3, 5×5, 7×7, …	Fixed 3×3 only
Best use case	General-purpose edge detection	Gradient direction accuracy

When to use

When gradient direction accuracy is critical — e.g., computing optical flow, or feeding gradients into feature descriptors (HOG, SIFT).
As a drop-in replacement for Sobel in standard pipelines where you want slightly better isotropy at no extra cost.
In 3×3-only contexts where Sobel's rotational bias is measurable.

When to avoid

When you need multi-scale gradients — Scharr only exists as a 3×3 kernel; use Sobel with larger kernel sizes instead.
On heavily noisy images — the stronger center weights amplify noise slightly more than Sobel.

Tip

In OpenCV, use cv2.Scharr(img, cv2.CV_64F, 1, 0) for the X gradient and cv2.Scharr(img, cv2.CV_64F, 0, 1) for Y. It is equivalent to cv2.Sobel() with ksize=-1.

6. Laplacian of Gaussian (LoG) — Edge Detection Mode

Spatial Second-Order Derivative

Cross-reference

The LoG filter is covered in detail in II — Sharpening Filters → LoG from an image sharpening and enhancement perspective. This section focuses on how the same filter is used as a precision edge detector through zero-crossing detection.

The key difference: Sharpening vs Edge Detection

The filter output is identical in both cases — what changes is entirely what you do with the result.

Mode	What you do with the LoG output
Sharpening	Add or subtract the LoG response to the original image to enhance edges
Edge Detection	Find zero-crossings — locations where the output flips sign — to locate boundaries

How Zero-Crossings Work

The Laplacian of an image is positive on one side of an edge and negative on the other. Exactly at the edge, it passes through zero. Finding where the LoG output flips sign gives a precise, thin edge map.

graph LR
    A[Input Image] --> B["Gaussian Blur (Reduce Noise)"]
    B --> C["Apply Laplacian (Second Derivative)"]
    C --> D["Find Zero-Crossings (Sign flips)"]
    D --> E[Edge Map]

Step-by-Step (1D illustration)

A simplified 1D intensity profile crossing from dark to bright:

Original intensities: 10, 10, 10, 80, 80, 80

After Gaussian smoothing (the sharp jump softens into a ramp):

Smoothed: 10, 15, 35, 55, 75, 80

First derivative (gradient) — peaks at the transition:

1st derivative: 5, 20, 20, 20, 5

Second derivative (Laplacian) — crosses zero at the edge:

2nd derivative: + 15, 0, 0, - 15

The zero-crossing sits exactly at the midpoint of the intensity transition — this is the detected edge. No magnitude thresholding needed to localize it precisely.

Why use LoG instead of Sobel?

	Sobel / Scharr	LoG (Zero-Crossing)
Derivative order	First (gradient)	Second (Laplacian)
Edge localization	Peak of gradient magnitude	Zero-crossing — more precise
Directionality	Separate X and Y kernels	Isotropic — equal response in all directions
Scale control	Kernel size	$σ$ of the Gaussian
Noise sensitivity	Moderate	Higher — Gaussian pre-smoothing is mandatory
Edge connectivity	Not guaranteed	Naturally produces closed, connected contours

Parameters to tune

$σ$ : controls which edge scales are detected. Larger $σ$ finds coarse, broad boundaries; smaller $σ$ finds fine textures and narrow features.
Rule of thumb: kernel size $\approx 6 σ \times 6 σ$ (same as the Gaussian blur guide in File I).

When to use

When sub-pixel edge precision is needed — zero-crossings can be localized more accurately than gradient magnitude peaks.
For scale-space edge detection — computing LoG at multiple $σ$ values finds edges at different scales simultaneously.
When closed contour maps are needed — LoG naturally produces connected edge loops rather than fragmented gradient responses.

When to avoid

Images with heavy random noise — the second derivative amplifies noise aggressively; the Gaussian pre-smoothing must be strong, but too large a $σ$ will blur real edges away.
When edge direction information is needed — LoG tells you where an edge is, not which way it points.
When speed is a priority — Sobel or Scharr is significantly faster and easier to tune.

Caution

Do not threshold the raw LoG output looking for large positive values — that is the sharpening use case. For edge detection, you must detect sign changes (zero-crossings) in the output. Naively thresholding the magnitude will not produce clean edges and is the most common misuse of LoG as an edge detector.