Modifying behaviour of special derivatives

[Crown421]

I recently learned of about this package in the SciML slack channel, and I like it a lot. However, I have one question:
The documentation states that the derivative of $u/|u|$ is deliberately set to NaN.

I am coming to a similar problem from the perspective of RBF kernels, in which case it is necessary to consider this from the "right side" derivative perspective, such that $\tfrac{d^2}{dxdy} \sqrt{(x-y)^2}^2 = -2$ and $\tfrac{d^2}{dxdy} \sqrt{(x-y)^2} = 0$. Due to the mentioned behaviour, this returns NaN when using this package.

I was wondering if there was an option to deliberately modify this behaviour?

[brianguenter]

I am open to adding such an option. Let me look at the code to see what would be the best way to allow users to define this.

[brianguenter]

Maybe you could do this by wrapping the function in a conditional

ifelse x-y == 0 ? -2 : $\tfrac{d^2}{dxdy} \sqrt{(x-y)^2}^2$

FD doesn't support conditionals yet and it will be a while before it does.

[Crown421]

Interesting, that could certainly be a way to do it (not just with FD, but probably with any choice of AD framework). It at least lets me do something on my end now.

However, using the conditional approach might need a lot of special casing or type piracy. For added context, the problem appears when using the Euclidean distance from Distances.jl (which is exactly $\sqrt((x-y)^2)$) in some of the stationary kernels. This came up in JuliaGaussianProcesses/KernelFunctions.jl#517
KernelFunctions already defines a special ChainRules.jl rule for this distances, but also making it possible to use more recent/ faster AD frameworks would be fantastic.

I will play around with the idea above, and beyond that, happy to try to help on this package, if I can.

[brianguenter]

It seems that allowing users to define custom derivative rules would increase the range of problems FD can be applied to. If you have any insight about how to do this then let's continue this discussion.