BinaryDecoder ServiceResultException with complex types

[symarchand]

Hello,

We have an issue while using complex types with optional fields. It works fine for complex types with only mandatory fields.
The problem happens when decoding values using a BinaryDecoder. The read variable's type is MeasIdDataType (from machine vision companion specification).

When we call a method returning complex types, the decoding of the output arguments throws an Opc.Ua.ServiceResultException: 'MaxStringLength 65535 < 190296809'. The source of the excpetion is in Opc.Ua.BinaryDecoder.ReadString.

Here are 2 relevant methods we use.

//In MachineVisionClient.MachineVisionProxy
public ResultDataType GetResultById(string resultId, int timeout)
        {
            ResultIdDataType resultIdDataType = new ResultIdDataType
            {
                Id = resultId
            };

            ResultDataType result = new ResultDataType();

            Console.WriteLine("Calling GetResultById");
            IList<object> outputs = Session.Call(
            ResultManagement.NodeId,
            ResultManagement.GetResultById,
            new ExtensionObject(resultIdDataType),
            timeout);
            if (outputs != null)
            {
                ExtensionObject extensionObject = outputs[1] as ExtensionObject;
                byte[] extensionObjectBody = extensionObject.Body as byte[];
                ServiceMessageContext serviceMessageContext = new ServiceMessageContext();

                BinaryDecoder binaryDecoder = new BinaryDecoder(extensionObjectBody, serviceMessageContext);

				//The following line throws an Exception
                result.Decode(binaryDecoder);
            }

            return result;
        }
		
		//In Opc.Ua.VDMA.MachineVisionDataTypes.cs
public virtual void Decode(IDecoder decoder)
        {
            decoder.PushNamespace(Opc.Ua.VDMA.MachineVision.Namespaces.MachineVisionXsd);

            ResultId = (ResultIdDataType)decoder.ReadEncodeable("ResultId", typeof(ResultIdDataType));
            HasTransferableDataOnFile = decoder.ReadBoolean("HasTransferableDataOnFile");
            IsPartial = decoder.ReadBoolean("IsPartial");
            IsSimulated = decoder.ReadBoolean("IsSimulated");
            ResultState = decoder.ReadInt32("ResultState");
			
			//The following line throws an Opc.ua.ServiceReultException: 'MaxStringKength 65535 < 1920296809' 
			//The exception originates from Opc.Ua.BinaryDecoder.ReadString
            MeasId = (MeasIdDataType)decoder.ReadEncodeable("MeasId", typeof(MeasIdDataType));
            PartId = (PartIdDataType)decoder.ReadEncodeable("PartId", typeof(PartIdDataType));
            ExternalRecipeId = (RecipeIdExternalDataType)decoder.ReadEncodeable("ExternalRecipeId", typeof(RecipeIdExternalDataType));
            InternalRecipeId = (RecipeIdInternalDataType)decoder.ReadEncodeable("InternalRecipeId", typeof(RecipeIdInternalDataType));
            ProductId = (ProductIdDataType)decoder.ReadEncodeable("ProductId", typeof(ProductIdDataType));
            ExternalConfigurationId = (ConfigurationIdDataType)decoder.ReadEncodeable("ExternalConfigurationId", typeof(ConfigurationIdDataType));
            InternalConfigurationId = (ConfigurationIdDataType)decoder.ReadEncodeable("InternalConfigurationId", typeof(ConfigurationIdDataType));
            JobId = (JobIdDataType)decoder.ReadEncodeable("JobId", typeof(JobIdDataType));
            CreationTime = decoder.ReadDateTime("CreationTime");
            ProcessingTimes = (ProcessingTimesDataType)decoder.ReadEncodeable("ProcessingTimes", typeof(ProcessingTimesDataType));
            ResultContent = decoder.ReadVariantArray("ResultContent");

            decoder.PopNamespace();
        }

Let me know if you need more precisions.

[mregen]

Hi @symarchand, to analyze this I need to look at the decode function of MeasIdDataType. It may not properly implement the decode of the optional fields parameter. Is the code created with ModelCompiler from the VDMA spec?

[eoursel]

Hi @mregen yes the code was created by the model compiler but as i understand the model compiler and the model design XSD format can't handle optional fields. Am I wrong? The wrong idea is to manually modify the generated source code, the right idea is to modify the model compiler

[eoursel]

@mregen, Of course another solution is to use the complex types client to avoid compiling vdma specifications on the client side. What we are looking for is a version of the aggregation server which is able to load the types of the connected servers without requiring to load the DLL which contain the compiled code from the model compiler.

[mregen]

Hi @eoursel and @symarchand, could you provide the ModelCompiler input file for the VDMA spec (per email if there is (c) issues). I'd like to understand whats going wrong and what needs to be done in ModelCompiler. Did you open a Mantis issue for fixing ModelCompiler?

[eoursel]

Hi @mregen

No i did not opened a MANTIS issue for the model compiler but have opened a github issue OPCFoundation/UA-ModelCompiler#48. I raised the question during the last TAC meeting. There will be no official support from OPC Foundation to fix the model compiler.

I generated the ModelDesign XML file from the MachineVision NodeSet by using ASMD from @mpostol, but not with the last version, i should try again with the last version he published recently. We are still missing a NodeSet2 to ModelDesign generator knowing that most companion specifications working groups only provide the NodeSet2 file.

[mpostol]

Hi @eoursel, You can also check out UAModelDesign Export. Description is here: Address Space Prototyping Tool (asp.exe).
ASMD has the same component used but not sure if up to date.
Good luck and let me know how it works.
Let me know in any case questions or comments you may have.
Mariusz

[eoursel]

Hi @eoursel, You can also check out UAModelDesign Export. Description is here: Address Space Prototyping Tool (asp.exe).
ASMD has the same component used but not sure if up to date.
Good luck and let me know how it works.
Let me know in any case questions or comments you may have.
Mariusz

Thanks Mariusz @mpostol , I tried UAModelDesign Export. I notice some validation errors when parsing the Machine Vision NodeSet. The problem is that we don't have the description of the source of the error. Most important question is what is the meaning of:

Trace: Information, Error Focus:XML, ErrorID: P0-0001010000 Info: The XML attribute or element is not supported and neglected. Extensions is omitted during the import

PS C:\Users\eoursel\code\mpostol\Release> .\asp .\XMLModels\Opc.Ua.MachineVision.NodeSet2.xml
Address Space Prototyping (asp.exe) 5.1.0.36523
Copyright(c) 2019 Mariusz Postol

Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Entering AddressSpaceContext creator - starting creation the OPC UA Address Space.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Address Space - the OPC UA defined has been uploaded.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Entering AddressSpaceContext.ImportNodeSet - starting import http://opcfoundation.org/UA/.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information AddressSpaceContext.ImportNodeSet - context for imported model is created and starting import nodes.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Finishing AddressSpaceContext.ImportNodeSet - imported 3737 nodes.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Address Space - has bee created successfully.

Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Entering AddressSpaceContextService.ImportUANodeSet - importing form file
Trace: Information, Error Focus:XML, ErrorID: P0-0001010000 Info: The XML attribute or element is not supported and neglected. Extensions is omitted during the import
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Entering AddressSpaceContext.ImportNodeSet - starting import http://opcfoundation.org/UA/MachineVision.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information AddressSpaceContext.ImportNodeSet - context for imported model is created and starting import nodes.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Finishing AddressSpaceContext.ImportNodeSet - imported 790 nodes.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Entering AddressSpaceContext.ValidateAndExportModel - starting for the http://opcfoundation.org/UA/MachineVision namespace.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information AddressSpaceContext.ValidateAndExportModel - selected 60 nodes to be added to the model.
Trace: Information, Error Focus:NodeClass, ErrorID: P3-0503020000 Info: Unexpected value of the InverseName. If ReferenceType 1:FromTransition is not symmetric and not abstract the InverseName shall be specified.
Trace: Information, Error Focus:NodeClass, ErrorID: P3-0503020000 Info: Unexpected value of the InverseName. If ReferenceType 1:ToTransition is not symmetric and not abstract the InverseName shall be specified.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Finishing Validator.ValidateExportModel - the model contains 60 nodes.
Press Enter to close this window.......

Regards
Eric

[eoursel]

Hi @mpostol

I have some questions about the generated ModelDesign file. i will open an issue on your repository.

[mpostol]

Hi @eoursel, I have noticed and scheduled your issue
To get more check out the project https://github.com/mpostol/OPC-UA-OOI/tree/master/SemanticData/BuildingErrorsHandling.
There is file providing all model errors and warnings supported: https://github.com/mpostol/OPC-UA-OOI/blob/master/SemanticData/BuildingErrorsHandling/BuildErrors.cs
It is auto-generated file form xml. Generally it allows leveraging this list to other languages. The list is to be expanded as a result of: mpostol/ASMD#158
The testing of the validation engine is covered by Unit Tests: https://github.com/mpostol/OPC-UA-OOI/tree/master/SemanticData/Tests/USNodeSetValidationUnitTestProject

[eoursel]

Hi @mregen @mpostol

Let's take the very simple example

<?xml version="1.0" encoding="utf-8"?>
<opc:ModelDesign
  xmlns:uax="http://opcfoundation.org/UA/2008/02/Types.xsd"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:ua="http://opcfoundation.org/UA/"
  xmlns:opc="http://opcfoundation.org/UA/ModelDesign.xsd"
  xmlns:xsd="http://www.w3.org/2001/XMLSchema"
  xmlns="http://opcfoundation.org/tests/ComplexTypesInterop"
  TargetNamespace="http://opcfoundation.org/tests/ComplexTypesInterop"
  TargetXmlNamespace="http://opcfoundation.org/tests/ComplexTypesInterop"
  TargetVersion="1.00"
  TargetPublicationDate="2020-09-09T00:00:00Z">

  <opc:Namespaces>
    <opc:Namespace Name="ComplexTypesInterop" Prefix="ComplexTypesInterop" XmlPrefix="ComplexTypesInterop">http://opcfoundation.org/tests/ComplexTypesInterop</opc:Namespace>
    <opc:Namespace Name="OpcUa" Version="1.04.3" PublicationDate="2019-09-09T00:00:00Z" Prefix="Opc.Ua" XmlNamespace="http://opcfoundation.org/UA/2008/02/Types.xsd">http://opcfoundation.org/UA/</opc:Namespace>
   </opc:Namespaces>
  <opc:DataType SymbolicName="ComplexTypeWithOptionsType" BaseType="ua:Structure">
    <opc:Fields>
      <opc:Field Name="X" DataType="ua:Double"/>
      <opc:Field Name="Y" DataType="ua:Double" BitMask="0001"/>
      <opc:Field Name="Z" DataType="ua:Double" BitMask="0002"/>
    </opc:Fields>
  </opc:DataType>
</opc:ModelDesign>

where Y and Z are optional. This is approximately the kind of structures we find in machine vision. The problem is that the modelCompiler completely ignores the BitMask attribute.

The following code is generated

  <UADataType NodeId="ns=1;i=15001" BrowseName="1:ComplexTypeWithOptionsType">
    <DisplayName>ComplexTypeWithOptionsType</DisplayName>
    <References>
      <Reference ReferenceType="HasSubtype" IsForward="false">i=22</Reference>
    </References>
    <Definition Name="1:ComplexTypeWithOptionsType">
      <Field Name="X" DataType="i=11" />
      <Field Name="Y" DataType="i=11" />
      <Field Name="Z" DataType="i=11" />
    </Definition>
  </UADataType>

In the NodeSet2 generated file the IsOptional attribute is missing on Fields Y and Z. Also according to part 6 the way encoding/decoding is done result in decoding errors if the encoding is done correctly according to Part 6 which is the case for node-opcua. Honestly, i am not comfortable to fix the model compiler but if someone has an idea. Refer to issue #48 in model compile project.

Thanks.

[mregen]

This is unfortunate but the latest modelcompiler doesn't seem to produce the proper code for encoding/decoding structure with optional fields.
I reopened:

From my understanding this is the modelcompiler input that should produce the EncodingMask:

<opc:DataType SymbolicName="ComplexTypeWithOptionsType" BaseType="ua:Structure">
    <opc:Fields>
      <opc:Field Name="X" DataType="ua:Double" IsOptional="true" />
      <opc:Field Name="Y" DataType="ua:Double" IsOptional="true" />
      <opc:Field Name="Z" DataType="ua:Double" IsOptional="true" />
    </opc:Fields>
  </opc:DataType>

but out is shown --> OPCFoundation/UA-ModelCompiler#92

not having an Encodingmask field

[mregen]

due to the dependency on modelcompiler, removed milestone

[mregen]

Please check recent ModelCompiler updates. Complex types have been improved in 1.4.371. Please reopen if there are still issues.