Implementing an opcua aggregator

[roin93]

Hello,
I want to build some kind of repeater for opcua.
I have a server which provide me a whole bunch of objects and variables (Siemens SPS for example). Because of the network security and the potential load I want to subscribe all nodes with my script and provide the gathered nodes at another server (my script / accessable server).

I currently have plenty of problems. I tried to export_xml and import it at the server but first of all it will take ages to create the xml and secondly the xml importing is not fully working because of already existing nodes (default nodes) or missing datatypes.

Therefore I thought I just iterate recursivly through all children of my most basic nodes and gather the browse_name and the path for example.
Then I want to create the same node at my reperater server and somehow link them together via a callback function to update the values when needed.
I currently struggle to do so because I need to define the datatypes first i guess. Furthermore I am not sure how to create the mentioned callback function to identify the right node. I cannot be sure that my new node at my repeater server will have the same node id because i am planning to subscribe to multiple servers and provide them all together at the same repeater server.

Is there an implementation of that somewhere or can someone give me some advise how to accomplish that task?
Would you do it the same way I do?

Also: Is there a way to multithread / multiprocess the opcua client and server? Just doing the iteration over one of my servers takes several minutes.

Best regards.

[swamper123]

Hi @roin93 ,

First, I recommened to switch to opcua-asyncio, since this repo is in maintainance mode (we also have a sync wrapper, which should work).

I tried to export_xml and import it at the server but first of all it will take ages to create the xml and secondly the xml importing is not fully working because of already existing nodes (default nodes) or missing datatypes.

The duration of parsing big xml files realy needs time. This is a known behaviour and we can't speed this up at the moment (even in the other repo). PRs welcome. Luckly the xmlimporter works fine there so far (otherwise please address issues to me 😅 ).
The xml_exporter has known issues as well, especially with complex types, in both repos. We try to fix them at the moment.

If I understand you correctly, you try to develope an OPC UA Server Aggregator?

About the problematic, finding the right node: I would use the browsepath to identify the right node. Mayabe browse the path first, then get the ID node and use this NodeId for later calls (depending on your server philosophy).

[roin93]

I recommened to switch to opcua-asyncio

I will check that out when I have a solution to my general problem. Thanks for the recommendation.

If I understand you correctly, you try to develope an OPC UA Server Aggregator?

As I read here, yes. I want to have an OPC UA Server Aggregator. In the first step i just wanted to gather all information about one server and provide these informations on my own server.
If this is successful I would try to aggregate multiple servers into one server.

I currently face the problematic that I try to copy Siemens specific data types but there seem to be multiple variables with the same NodeId which causes problems. Im not fully sure if that are only some references or something similiar. I tried to find that out with uaExpert but I am not really sucessful with that.

[roin93]

I tried to convert my code to use opcua-asyncio. First i got plenty of error messages that I did not use any awaits... ok, added async and await to different places.
That finally did not work as I expected so I wanted to use the sync wrapper. The examples are not intuitive to me. The examples look like the same code I wrote for this library. Im not sure what I missed to do to use the sync version...
Im really confused by now.

[zerox1212]

Isn't there a section in the spec about one server talking to another server? I thought there was some guidelines that a node should only ever exist in one server?

[AndreasHeine]

data aggregation is a commen usecase! there are already products out there which did that (https://www.sterfive.com/#/product-aggregator), its like crawling some parts of the addressspace and rebuild them in the new server (subscribe and update bidirectional) passing thru methodcalles and results also passing thru events (subscribe and re-emit).
but the term repeater is missleading --> aggregation

you need to check typedefinition and all other references and rebuild the addressspace inkl. custom models and xml (all namespaces > Index 0) if its a variable or property you need tu subscribe them and you need to find a way to wirte thru the new server (bidirectional read and write) thats a lot of work and many pitfalls if you have customtypes and methods

[roin93]

the term repeater is missleading --> aggregation

Ok, I guess I mean an aggregator

I see you understand what I currently try to do. I started by copying the datatype objects / variables but now I am facing the problem that multiple variables use the same NodeId which confuses me. I guess that is just a reference but I did not find an example how to use the add_reference function to create the same structure as on the original server.

you need to check typedefinition and all other references and rebuild the addressspace

Can you provide an example of such a thing in python?
I processed some of the type definitions I guess but I am afraid that I forgot many things to process because my resulting tree unfortunally looks different from the original.

I already wrote a function which starts at a specific node at the original server and adds all its childs recursivly to the new server. The problem is that I have trouble to add all needed type definitions in the correct tree. Especially the use of add_reference is not intuitive to me.

[AndreasHeine]

sorry cant provide an example... thats stuff i get paid for!