A non-typical estimation/forecasting setup using ForecastX

[SultanOrazbayev]

Hello, I have a problem in which I am trying to predict two variables $y_1$ and $y_2$ (as independent, single-variable predictions) as a function of exogenous variables $X$ and lags of these two variables to be forecasted.

Consider prediction of $y_{1,t}$: $y_{1,t} = f(X, y_{1,t-1}, y_{2,t-1})$.

Estimating the coefficients for this problem is not an issue, since all values are known in the training data.
However, forecasting for $t+2$ requires iteration:

• generate the forecast of $y_{1,t+1}$
• generate the forecast of $y_{2,t+1}$
• pass these forecasted values to the forecaster.

Initially, I thought this is something I could do using ForecastX, however it is not clear to me how to handle the iteration above. Kindly note that I am trying to use single-variable forecasters (each column will have its own forecaster), so from the perspective of variable $y_1$, forecasting variable $y_2$ is referencing a separate forecaster (not the one for $y_1$).

Here's the small code that I started with:

from pandas import DataFrame
from numpy import random, array, zeros_like

# construct the raw data
x = array([0,1,2,1,2,2,3,2,1,3,0])
y1 = zeros_like(x, dtype='float')
y2 = zeros_like(x, dtype='float')
y1[0] = 2
y2[0] = 4

for position, x_value in enumerate(x[1:], start=1):
    y1[position] = 0.1* y1[position-1] + 0.2* y2[position-1] + 1 + 2 * x_value + 0.1 * random.normal()
    y2[position] = 0.3* y2[position-1] + 0.4* y1[position-1]+ 3 + 4 * x_value + 0.2 * random.normal()

df = DataFrame({"y1": y1, "y2": y2, "x": x})

# construct X, y and fh
from sktime.forecasting.base import ForecastingHorizon


y = df[["y1", "y2"]].iloc[:-2]
X = df[["x"]].iloc[:-2]
X_pred = df[["x"]].iloc[-2:]

fh = ForecastingHorizon(values=[1,2], is_relative=True)

# testing a simple forecaster (no lags)
from sktime.forecasting.compose import make_reduction
from sklearn.linear_model import LinearRegression

forecaster = make_reduction(LinearRegression(), window_length=1, strategy="recursive")
forecaster.fit_predict(y, X=X, X_pred=X_pred, fh=fh)
forecaster.get_fitted_params()

# attempt at using ForecastX (fails)
from sktime.forecasting.compose import ForecastX
from pandas import concat

yy = df[["y1", "y2"]].iloc[:-2]
XX = concat([df[["x"]].iloc[:-2], df[["y1", "y2"]].shift(-1).iloc[:-2]])
XX_pred = concat([df[["x"]].iloc[-2:], df[["y1", "y2"]].shift(-2).iloc[:-2]])


forecasterX = ForecastX(forecaster_y=forecaster)
forecasterX.fit_predict(yy, X=XX, X_pred=XX_pred, fh=fh)
forecasterX.get_fitted_params()

The specific error raised is TypeError, however, it seems the core issue is I am trying to perform a non-typical estimation/forecasting operation. There is probably a way to implement it, but it's not clear to me how I should proceed. Any pointers will be appreciated, including if this is not currently feasible using sktime.

[fkiraly]

Silly question: from the description above, I thought the first use of make_reduction already does what you want. ForecastX is there for creating forecasts of X, which do not seem to figure as part of your requirement.

Can you explain why the first instance, the code that runs, is not enough?

[SultanOrazbayev]

Thank you for the quick follow-up, @fkiraly ! If I understand correctly, what the first output does is estimate $y_{1,t} = f (X_t, y_{1,t-1})$ (so doesn't include $y_{2,t-1}$ as an input into prediction).

[fkiraly]

Yes, I think it does not, if you use make_reduction, because the reducers accessed through make_reduction are purely univariate, so they will broadcast across components of y. Forecasts of one variable will not use the past of another variable.

[SultanOrazbayev]

Got it, thanks, @fkiraly ! Is there any clever way of using some additional transformers on $X$ that could access (all) $y$ values?

[fkiraly]

all transformers used in the pipeline get to see y, transformers on X are passed the y seen up to that point in the pipeline. If you want to use it as an input to a transformer, you can use YtoX

[SultanOrazbayev]

Thank you, @fkiraly ! I will investigate YtoX.