Time Series Analysis

A time series is a sequence of data points indexed in time order — stock prices, weather readings, sensor data, sales figures. Time series analysis extracts patterns to understand the past and forecast the future.

import pandas as pd
import matplotlib.pyplot as plt

# Load a simple time series
df = pd.read_csv('data.csv', parse_dates=['date'], index_col='date')
df['value'].plot(figsize=(12, 4), title='Time Series')
plt.show()

Components of a Time Series

Every time series can be decomposed into four components:

from statsmodels.tsa.seasonal import seasonal_decompose

result = seasonal_decompose(df['value'], model='additive', period=12)
result.plot()
plt.tight_layout()
plt.show()
# Plots: original, trend, seasonal, residual

Stationarity

A time series is stationary if its statistical properties (mean, variance, autocorrelation) do not change over time. Most classical forecasting models require stationary data.

The Augmented Dickey-Fuller (ADF) test checks for stationarity — a p-value below 0.05 means the series is stationary.

from statsmodels.tsa.stattools import adfuller

def check_stationarity(series):
    result = adfuller(series.dropna())
    print(f'ADF Statistic : {result[0]:.4f}')
    print(f'p-value       : {result[1]:.4f}')
    print('Stationary ✓' if result[1] < 0.05 else 'Not stationary — try differencing')

check_stationarity(df['value'])

Making a Series Stationary

If the series is not stationary, differencing (subtracting the previous value) often helps:

# First-order differencing
df['diff1'] = df['value'].diff()
check_stationarity(df['diff1'])

ARIMA

ARIMA(p, d, q) combines three components:

• AR(p) — AutoRegressive: the value depends on its own past p values
• I(d) — Integrated: d rounds of differencing to make the series stationary
• MA(q) — Moving Average: the value depends on the past q forecast errors

from statsmodels.tsa.arima.model import ARIMA
import warnings
warnings.filterwarnings('ignore')

# Fit ARIMA(1,1,1) — a common starting point
model = ARIMA(df['value'], order=(1, 1, 1))
result = model.fit()
print(result.summary())

# Forecast next 12 steps
forecast = result.forecast(steps=12)
print(forecast)

Choosing p, d, q with ACF/PACF

from statsmodels.graphics.tsaplots import plot_acf, plot_pacf

fig, axes = plt.subplots(1, 2, figsize=(12, 4))
plot_acf(df['value'].diff().dropna(), ax=axes[0])   # guides q
plot_pacf(df['value'].diff().dropna(), ax=axes[1])  # guides p
plt.show()

Moving Average Explorer

Adjust the window size to see how SMA (Simple Moving Average) and EMA (Exponential Moving Average) smooth a noisy signal. Switch signal presets to explore different patterns.

Recurrent Neural Networks

A recurrent neural network (RNN) feeds its previous output back in as an input, so it carries a hidden state from one step to the next. That hidden state acts as internal memory, which lets the network handle temporal behaviour.

Why RNNs?

• ANNs and CNNs treat each input on its own. Previous or next inputs are not needed.
• Time series data points depend on the points around them, as in video or speech.
• Feed-forward networks cannot take previous inputs into account.

Architecture

The current state is a function of the previous state and the current input:

$$h_t = f(h_{t-1}, x_t)$$

With a tanh activation, this becomes:

$$h_t = \tanh(W_{hh}\,h_{t-1} + W_{xh}\,x_t)$$

and the output is:

$$y_t = W_{hy}\,h_t$$

Here \(h_t\) is the current state, \(h_{t-1}\) the previous state and \(x_t\) the input. \(W_{hh}\) is the weight on the recurrent connection, \(W_{xh}\) the weight on the input and \(W_{hy}\) the weight on the output layer.

How an RNN Is Trained

1. One time step of the input is given to the network.
2. The current state is computed from the current input and the previous state.
3. The current \(h_t\) becomes \(h_{t-1}\) for the next time step.
4. You can run as many time steps as the problem needs, combining information from all earlier states.
5. After the last step, the final state is used to compute the output.
6. The output is compared with the target and the error is calculated.
7. The error is backpropagated through time to update the weights.

RNN Architectures

Vanishing and Exploding Gradients

RNNs often suffer from vanishing and exploding gradients. Long-term dependencies are hard to capture because the gradients are multiplied at every step, so they can shrink or grow exponentially with the number of steps.

Gradient clipping handles exploding gradients by capping the maximum value of the gradient during backpropagation.

LSTM Forecasting

Long Short-Term Memory (LSTM) networks are a type of RNN that can learn long-range dependencies in sequential data — making them powerful for time series forecasting when patterns are complex or non-linear.

Gates

To fight the vanishing gradient problem, some RNNs use gates, each with a well-defined purpose. A gate \(\Gamma\) is computed as:

$$\Gamma = \sigma(W x_t + U h_{t-1} + b)$$

where \(W\), \(U\) and \(b\) are coefficients specific to the gate and \(\sigma\) is the sigmoid function.

Gated Recurrent Units (GRU) and Long Short-Term Memory units (LSTM) both address the vanishing gradient problem, and the LSTM can be seen as a generalisation of the GRU.

Inside an LSTM Cell

• Forget gate: removes information that is no longer useful from the cell state. The input \(x_t\) and the previous output \(h_{t-1}\) are multiplied by weight matrices, a bias is added, and a sigmoid squashes the result towards 0 or 1. Where it is 0, that piece of information is forgotten. Where it is 1, it is kept for later.
• Input gate: adds useful new information to the cell state. A sigmoid over \(h_{t-1}\) and \(x_t\) decides which values to let in, and a tanh creates a vector of candidate values between -1 and +1. Multiplying the two gives the useful new information.
• Output gate: picks what to show from the cell state. A tanh is applied to the cell, a sigmoid over \(h_{t-1}\) and \(x_t\) decides which parts to pass on, and the product is the output and the input to the next cell.

Prepare Sequences

import numpy as np
from sklearn.preprocessing import MinMaxScaler

scaler = MinMaxScaler()
scaled = scaler.fit_transform(df[['value']])

def create_sequences(data, window=30):
    X, y = [], []
    for i in range(len(data) - window):
        X.append(data[i:i+window])
        y.append(data[i+window])
    return np.array(X), np.array(y)

X, y = create_sequences(scaled)
split = int(len(X) * 0.8)
X_train, X_test = X[:split], X[split:]
y_train, y_test = y[:split], y[split:]

Build and Train the LSTM

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import LSTM, Dense, Dropout

model = Sequential([
    LSTM(64, return_sequences=True, input_shape=(30, 1)),
    Dropout(0.2),
    LSTM(32),
    Dense(1)
])
model.compile(optimizer='adam', loss='mse')
model.fit(X_train, y_train, epochs=20, batch_size=32,
          validation_split=0.1, verbose=1)

# Predict and inverse-transform
preds = scaler.inverse_transform(model.predict(X_test))