2026-06-30

Wiener Process

Wiener Process is a specific case of [[Stochastic Process|Stochastic Process]], and is the mathematical model of [[Brownian Motion|Brownian Motion]]. It is the most fundamental stochastic process in stochastic calculus and [[Diffusion Model|Diffusion Model]].

1. Definition and Properties

Wiener Process $W_{t}$ (or $B_{t}$ , $ω_{t}$ ) satisfies the following properties:

Starts at zero: $W_{0} = 0$ (almost surely)
Independent increments: For any $0 \leq t_{1} < t_{2} < t_{3} < t_{4}$ , increments $W_{t_{2}} - W_{t_{1}}$ and $W_{t_{4}} - W_{t_{3}}$ are independent
Gaussian increments: $W_{t} - W_{s} \sim N (0, t - s)$ , where $0 \leq s < t$
Continuous paths: $t \mapsto W_{t}$ is almost surely continuous

[!NOTE] Intuitive Understanding
Imagine a tiny particle moving randomly in liquid due to molecular collisions:

Each step’s direction and magnitude are random

Overall, displacement variance grows linearly with time: $Var (W_{t}) = t$

2. Basic Properties

2.1 Moments and Covariance

Mean: $E [W_{t}] = 0$
Variance: $Var (W_{t}) = t$
Covariance: $Cov (W_{s}, W_{t}) = min (s, t)$

2.2 Self-Similarity (Scale Invariance)

For any constant $c > 0$ , the process $\frac{1}{\sqrt{c}} W_{c t}$ has the same finite-dimensional distributions as $W_{t}$ .

2.3 Quadratic Variation

For partition of interval $[0, t]$ where $0 = t_{0} < t_{1} < \dots < t_{n} = t$ :

lim_{max Δ t_{i} \to 0} \sum_{i = 1}^{n} (W_{t_{i}} - W_{t_{i - 1}})^{2} = t (almost surely)

This property shows that the paths of Wiener Process are almost everywhere non-differentiable.

2.4 Markov Property

Wiener Process is a [[Markov Process|Markov Process]]:

E [f (W_{t}) ∣ F_{s}] = E [f (W_{t}) ∣ W_{s}]

2.5 [[Martingale]] Property

Wiener Process is a [[Martingale|Martingale]]:

E [W_{t} ∣ F_{s}] = W_{s}, \forall s < t

3. Role in Stochastic Differential Equations ([[Stochastic Differential Equation (SDE)|SDE]])

In diffusion models, the forward [[Stochastic Differential Equation (SDE)|SDE]] is written as:

d x = f (t) x d t + g (t) d W_{t}

$d W_{t}$ : Infinitesimal increment of Wiener Process, representing random noise injection
$f (t) x d t$ : Deterministic drift term, controlling signal decay
$g (t) d W_{t}$ : Stochastic diffusion term, controlling noise intensity

Since $d W_{t} \sim N (0, d t)$ , this term is equivalent to adding Gaussian noise with mean zero and variance $g (t)^{2} d t$ at the current time step.

4. Why Not Ordinary Gaussian Noise?

Ordinary Gaussian noise $ϵ \sim N (0, σ^{2} I)$ is a one-time independent perturbation.
The increment $d W_{t}$ of Wiener Process is a continuous stochastic process:

Temporal pointwise correlation: Future changes of Wiener Process are probabilistically independent of past changes, but the path itself is continuous
The [[Stochastic Differential Equation (SDE)|SDE]] form allows us to continuously inject noise at infinitesimally fine time scales, which is the theoretical foundation of the reverse denoising process in diffusion models

5. Specific Role in Diffusion Models

Role	Explanation
Forward Noising	Use $d W_{t}$ to gradually diffuse data into pure noise
Reverse Denoising	By estimating $\nabla_{x} \log p_{t} (x)$ ([[Score Function]]) to reverse the effect of Wiener process
[[Probability Flow ODE]]	Eliminate the stochastic term $d W_{t}$ , obtain deterministic ODE, but still maintain the same marginal distribution

6. Important Variants

Variant	[[Stochastic Differential Equation (SDE)\|SDE]] Form	Features
Brownian Motion with Drift	$d X_{t} = μ d t + σ d W_{t}$	$X_{t} = μ t + σ W_{t}$
Geometric Brownian Motion	$d X_{t} = μ X_{t} d t + σ X_{t} d W_{t}$	Used in financial modeling (Black-Scholes)
Ornstein-Uhlenbeck Process	$d X_{t} = - θ X_{t} d t + σ d W_{t}$	Mean-reverting process
Brownian Bridge	$d B_{t} = - \frac{B_{t}}{1 - t} d t + d W_{t}$	Conditioned on $B_{0} = B_{1} = 0$

7. Analogical Understanding

Concept	Analogy
Wiener Process $W_{t}$	Random motion trajectory of ink particles in a cup of water
$d W_{t}$	Irregular jumps of particles within each tiny time step
$g (t) d W_{t}$ in [[Stochastic Differential Equation (SDE)\|SDE]]	Random stirring with time-varying intensity

9. Numerical Simulation

9.1 Simulating Wiener Process

import numpy as np

def simulate_wiener(T, N, n_paths=1):
    """
    Simulate Wiener process paths.
    T: Total time
    N: Number of time steps
    n_paths: Number of independent paths
    """
    dt = T / N
    t = np.linspace(0, T, N+1)
    
    # Generate increments: dW ~ N(0, dt)
    dW = np.sqrt(dt) * np.random.randn(n_paths, N)
    
    # Cumulative sum to get Wiener process
    W = np.zeros((n_paths, N+1))
    W[:, 1:] = np.cumsum(dW, axis=1)
    
    return t, W

9.2 Key Observations from Simulation

Variance grows linearly: $Var (W_{t}) = t$ — paths spread out over time
Paths are continuous but nowhere differentiable: Zoom in and they stay rough
Self-similarity: Rescaled paths look statistically identical
Crossing behavior: $W_{t}$ crosses zero infinitely often near $t = 0$

9.3 Using Wiener Process in Diffusion Models

The forward noising process in [[Diffusion Model|DDPM]] is a discretized version:

def forward_diffusion(x_0, T, noise_schedule):
    """
    DDPM forward process = discrete approximation of SDE.
    """
    x = x_0
    for t in range(T):
        beta = noise_schedule[t]
        # dW ~ sqrt(dt) * N(0, I) → sqrt(beta) * N(0, I)
        x = sqrt(1 - beta) * x + sqrt(beta) * torch.randn_like(x)
    return x

[!NOTE] Continuous vs Discrete
[[Diffusion Model|DDPM]] uses discrete time; the SDE formulation via [[Stochastic Differential Equation (SDE)|SDE]] generalizes this to continuous time. Wiener Process is the bridge between them.

10. Wiener Process and [[Gaussian Process]]

Wiener Process is a specific [[Gaussian Process]] with:

Mean function: $m (t) = 0$
Covariance kernel: $k (s, t) = min (s, t)$

This kernel makes Wiener Process a non-stationary Gaussian Process (variance grows with time). In contrast:

Process	Kernel	Stationarity
Wiener Process	$min (s, t)$	Non-stationary
Ornstein-Uhlenbeck	$\exp(-\theta	t-s
Brownian Bridge	$min (s, t) - s t / T$	Non-stationary (pinned)

11. Summary

Wiener Process provides a continuous-time, continuous-path stochastic noise model. In diffusion models, it enables the forward noising process to proceed smoothly at any time scale, and through the mathematically reversible [[Stochastic Differential Equation (SDE)|SDE]]/ODE framework, achieves controllable generation from pure noise to real data.

Core Formula Cards

[!QUOTE] Wiener Process Increment Distribution
$W_{t} - W_{s} \sim N (0, t - s), 0 \leq s < t$

[!QUOTE] Variance Grows with Time
$Var (W_{t}) = t$

[!QUOTE] Covariance Function
$Cov (W_{s}, W_{t}) = min (s, t)$

[!QUOTE] Quadratic Variation
$[W]_{t} = t$

[!QUOTE] Forward Diffusion [[Stochastic Differential Equation (SDE)|SDE]]
$d x = f (t) x d t + g (t) d W_{t}$

[!QUOTE] Self-Similarity
$\frac{1}{\sqrt{c}} W_{c t} \overset{d}{=} W_{t}$

[[Diffusion Model]]
[[DDIM]]
[[DPM-Solver]]
[[Flow Matching]]
[[Stochastic Process]]
[[Brownian Motion]]
[[Stochastic Differential Equation (SDE)]]
[[Itô Integral]]
[[Itô’s Lemma]]
[[Fokker-Planck Equation]]
[[Kolmogorov Equations]]
[[Langevin Dynamics]]
[[Score Function]]
[[Probability Flow ODE]]
[[Markov Process]]
[[Martingale]]
[[Gaussian Process]]
[[Ornstein-Uhlenbeck Process]]
[[Poisson Process]]

Dataview Query

1
2
3

LIST
FROM #Wiener Process OR #brownian_motion
SORT file.ctime DESC

References

Book：Brownian Motion and Stochastic Calculus - Karatzas & Shreve
Book：Stochastic Differential Equations - Bernt Øksendal
Video：MIT 18.S096 Topics in Mathematics with Applications in Finance

ChungMG

Mathematics & Machine Learning

Wiener Process

1. Definition and Properties

2. Basic Properties

2.1 Moments and Covariance

2.2 Self-Similarity (Scale Invariance)

2.3 Quadratic Variation

2.4 Markov Property

2.5 [[Martingale]] Property

3. Role in Stochastic Differential Equations ([[Stochastic Differential Equation (SDE)|SDE]])

4. Why Not Ordinary Gaussian Noise?

5. Specific Role in Diffusion Models

6. Important Variants

7. Analogical Understanding

9. Numerical Simulation

9.1 Simulating Wiener Process

9.2 Key Observations from Simulation

9.3 Using Wiener Process in Diffusion Models

10. Wiener Process and [[Gaussian Process]]

11. Summary

Core Formula Cards

Dataview Query

References

Wiener Process

1. Definition and Properties

2. Basic Properties

2.1 Moments and Covariance

2.2 Self-Similarity (Scale Invariance)

2.3 Quadratic Variation

2.4 Markov Property

2.5 [[Martingale]] Property

3. Role in Stochastic Differential Equations ([[Stochastic Differential Equation (SDE)|SDE]])

4. Why Not Ordinary Gaussian Noise?

5. Specific Role in Diffusion Models

6. Important Variants

7. Analogical Understanding

9. Numerical Simulation

9.1 Simulating Wiener Process

9.2 Key Observations from Simulation

9.3 Using Wiener Process in Diffusion Models

10. Wiener Process and [[Gaussian Process]]

11. Summary

Core Formula Cards

Related Concepts

Dataview Query

References