Allow y_lag = 0 for classification and inference tasks

[dj-gauthier]

@wilsonrljr
In the current implementation of SysIdentPy, you are required to have a lagged value of the variable y. That is, the model must be of the form y_n = F(y_{n-1}, x_{n-1}, etc.). However, there are machine learning models where y is a class or another variable that you are trying to infer (based only on x) and hence y should not appear in the model F. By allowing no lag for y, you will increase the utility of SysIdentPy to these problems. I suspect that this would be a simple change - just allow all of the modules to evaluate with y_lag = 0 and verifying that any loop structure will still work. Making this change will also simplify going to MIMO forecasting problems. Thanks for your consideration!

[wilsonrljr]

Hey @dj-gauthier ! Lets discuss about that. When you say y_lag = 0 you are talking about adding y values without any lag? There are some affine models that do that, but I'm not sure if thats the case. Or are you talking about NFIR models?

Currently, you can build FIR and NFIR models in SysIdentPy (e.g. y(k) = F(x(k-1), x(k-2), ..., x(k-n)), where your model have only input regressors (but all of them are lagged regressors).

[dj-gauthier]

I don’t know about FIR and NFIR models. Do you have documentation or a citation? That looks like what I want as long as there can be many inputs such as x1 at lagged values, x2 at lagged values, etc

…

On Mon, Oct 16, 2023 at 3:17 PM Wilson Rocha ***@***.***> wrote: Hey @dj-gauthier <https://github.com/dj-gauthier> ! Lets discuss about that. When you say y_lag = 0 you are talking about adding y values without any lag? There are some affine models that do that, but I'm not sure if thats the case. Or are you talking about NFIR models? Currently, you can build FIR and NFIR models in SysIdentPy (e.g. y(k) = F(x(k-1), x(k-2), ..., x(k-n)), where your model have only input regressors (but all of them are lagged regressors). — Reply to this email directly, view it on GitHub <#121 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALEY24SFH6E4CGEDVJ6L5T3X7WB6JAVCNFSM6AAAAAA6COCQJWVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TEOJWGQ2DM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[wilsonrljr]

You can use as many input as you want. Its basically the same as the NARX, but the regressor matrix is built using only lagged regressors of the input. The following example show how to use it:

import pandas as pd
from sysidentpy.utils.generate_data import get_siso_data
from sysidentpy.metrics import root_relative_squared_error
from sysidentpy.basis_function._basis_function import Polynomial
from sysidentpy.utils.display_results import results
from sysidentpy.utils.plotting import plot_results
from sysidentpy.model_structure_selection import FROLS

# generating simulated data
x_train, x_test, y_train, y_test = get_siso_data(
    n=1000, colored_noise=False, sigma=0.001, train_percentage=90
)

basis_function = Polynomial(degree=2)
model = FROLS(
    xlag=10,
    ylag=10,
    order_selection=True,
    # k=5,
    # L=1,
    basis_function=basis_function,
    model_type="NFIR"  # Set NFIR here
)

model.fit(X=x_train, y=y_train)

yhat = model.predict(X=x_test, y=y_test[:model.max_lag, :])
rrse = root_relative_squared_error(y_test, yhat)
print(rrse)
r = pd.DataFrame(
    results(
        model.final_model, model.theta, model.err,
        model.n_terms, err_precision=8, dtype='sci'
        ),
    columns=['Regressors', 'Parameters', 'ERR'])
print(r)

plot_results(y=y_test, yhat = yhat, n=1000)
ee = compute_residues_autocorrelation(y_test, yhat)
plot_residues_correlation(data=ee, title="Residues", ylabel="$e^2$")
x1e = compute_cross_correlation(y_test, yhat, x_test)
plot_residues_correlation(data=x1e, title="Residues", ylabel="$x_1e$")

The following code will result in a model like

Regressors Parameters ERR
0 x1(k-2) 9.0007E-01 9.58505175E-01
1 x1(k-3) 1.8077E-01 3.68053254E-02
2 x1(k-3)x1(k-1) 8.9468E-02 2.90543638E-03
3 x1(k-4) 3.5427E-02 1.47615523E-03
4 x1(k-4)x1(k-1) 1.6901E-02 1.07296092E-04
5 x1(k-4)x1(k-2) 1.8556E-02 1.13643948E-04
6 x1(k-5) 6.6682E-03 5.79016056E-05
7 x1(k-5)x1(k-2) 2.9090E-03 4.69087365E-06
8 x1(k-5)x1(k-3) 3.6004E-03 4.22777823E-06
9 x1(k-5)x1(k-1) 2.6654E-03 2.93639283E-06

with only input regressors. You can have x1, x2, x3,..., xn regressors. This is another case where I don't have an example in the docs, but I opened an issue to create that a month or two ago.

I don't have a good citation in hand, but I found this two that could help a little:
https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=c3f8631e809a6a288064a0d64ccfbac86d878861

https://www.chee.uh.edu/sites/chbe/files/faculty/nikolaou/firpaper.pdf

I usually use this book here (https://link.springer.com/book/10.1007/978-3-662-04323-3, and the Billings one) and I know it is a good resource for NFIR models (and NARMAX in general)

[wilsonrljr]

Its important to mention that I have only implemented one-step ahead and infinity-steps ahead prediction for NFIR models. If you try to set steps-ahead=5, for example, you'll probably get an error message (and not one explicitly made to handle this issue).