2 Specialized Decodes
When performance is of highest priority and one is interested in a limited part of the ASN.1 encoded message, before one decide what to do with the rest of it, one may want to decode only this small part. The situation may be a server that has to decide to which addressee it will send a message. The addressee may be interested in the entire message, but the server may be a bottleneck that one want to spare any unnecessary load. Instead of making two complete decodes (the normal case of decode), one in the server and one in the addressee, it is only necessary to make one specialized decode(in the server) and another complete decode(in the addressee). The following specialized decodes exclusive decode and selected decode support to solve this and similar problems.
So far this functionality is only provided when using the optimized BER_BIN version, that is when compiling with the options
ber_bin
andoptimize
. It does also work using thedriver
option. We have no intent to make this available on the default BER version, but maybe in the PER_BIN version (per_bin
).2.1 Exclusive Decode
The basic idea with exclusive decode is that you specify which parts of the message you want to exclude from being decoded. These parts remain encoded and are returned in the value structure as binaries. They may be decoded in turn by passing them to a certain
decode_part/2
function. The performance gain is high when the message is large and you can do an exclusive decode and later on one or several decodes of the parts or a second complete decode instead of two or more complete decodes.2.1.1 How To Make It Work
In order to make exclusive decode work you have to do the following:
- First,decide the name of the function for the exclusive decode.
- Second, write instructions that must consist of the name of the exclusive decode function, the name of the ASN.1 specification and a notation that tells which parts of the message structure will be excluded from decode. These instructions shall be included in a configuration file.
- Third, compile with the additional option
asn1config
. The compiler searches for a configuration file with the same name as the ASN.1 spec but with the extension .asn1config. This configuration file is not the same as used for compilation of a set of files. See section Writing an Exclusive Decode Instruction.
2.1.2 User Interface
The run-time user interface for exclusive decode consists of two different functions. First, the function for an exclusive decode, whose name the user decides in the configuration file. Second, the compiler generates a
decode_part/2
function when exclusive decode is chosen. This function decodes the parts that were left undecoded during the exclusive decode. Both functions are described below.If the exclusive decode function has for example got the name
decode_exclusive
and an ASN.1 encoded messageBin
shall be exclusive decoded, the call is:{ok,Excl_Message} = 'MyModule':decode_exclusive(Bin)The result
Excl_Message
has the same structure as an complete decode would have, except for the parts of the top-type that were not decoded. The undecoded parts will be on their place in the structure on the format{Type_Key,Undecoded_Value}
.Each undecoded part that shall be decoded must be fed into the
decode_part/2
function,like:{ok,Part_Message} = 'MyModule':decode_part(Type_Key,Undecoded_Value)2.1.3 Writing an Exclusive Decode Instruction
This instruction is written in the configuration file on the format:
Exclusive_Decode_Instruction = {exclusive_decode,{Module_Name,Decode_Instructions}}. Module_Name = atom() Decode_Instructions = [Decode_Instruction]+ Decode_Instruction = {Exclusive_Decode_Function_Name,Type_List} Exclusive_Decode_Function_Name = atom() Type_List = [Top_Type,Element_List] Element_List = [Element]+ Element = {Name,parts} | {Name,undecoded} | {Name,Element_List} Top_Type = atom() Name = atom()Observe that the instruction must be a valid Erlang term ended by a dot.
In the
Type_List
the "path" from the top type to each undecoded sub-components is described. The top type of the path is an atom, the name of it. The action on each component/type that follows will be described by one of{Name,parts}, {Name,undecoded}, {Name,Element_List}
The use and effect of the actions are:
{Name,undecoded}
Tells that the element will be left undecoded during the exclusive decode. The type of Name may be any ASN.1 type. The value of element Name will be returned as a tuple,as mentioned above , in the value structure of the top type.
{Name,parts}
The type of Name may be one of SEQUENCE OF or SET OF. The action implies that the different components of Name will be left undecoded. The value of Name will be returned as a tuple, as above , where the second element is a list of binaries. That is because the representation of a SEQUENCE OF/ SET OF in Erlang is a list of its internal type. Any of the elements of this list or the entire list can be decoded by thedecode_part
function.
{Name,Element_List}
This action is used when one or more of the sub-types of Name will be exclusive decoded.
Name in the actions above may be a component name of a SEQUENCE or a SET or a name of an alternative in a CHOICE.
2.1.4 Example
In the examples below we use the definitions from the following ASN.1 spec:
GUI DEFINITIONS AUTOMATIC TAGS ::= BEGIN Action ::= SEQUENCE { number INTEGER DEFAULT 15, handle [0] Handle DEFAULT {number 12, on TRUE} } Key ::= [11] EXPLICIT Button Handle ::= [12] Key Button ::= SEQUENCE { number INTEGER, on BOOLEAN } Window ::= CHOICE { vsn INTEGER, status E } Status ::= SEQUENCE { state INTEGER, buttonList SEQUENCE OF Button, enabled BOOLEAN OPTIONAL, actions CHOICE { possibleActions SEQUENCE OF Action, noOfActions INTEGER } } ENDIf
Button
is a top type and we want to exclude componentnumber
from decode the Type_List in the instruction in the configuration file will be['Button',[{number,undecoded}]]
. If we call the decode functiondecode_Button_exclusive
the Decode_Instruction will be{decode_Button_exclusive,['Button',[{number,undecoded}]]}
.We also have another top type
Window
whose sub component actions in typeStatus
and the parts of componentbuttonList
shall be left undecoded. For this type we name the functiondecode__Window_exclusive
. The whole Exclusive_Decode_Instruction configuration is as follows:{exclusive_decode,{'GUI', [{decode_Window_exclusive,['Window',[{status,[{buttonList,parts},{actions,undecoded}]}]]}, {decode_Button_exclusive,['Button',[{number,undecoded}]]}]}}.
Figure symbolizes the bytes of a Window:status message. The components buttonList and actions are excluded from decode. Only state and enabled are decoded when decode__Window_exclusive is called.
Compiling GUI.asn including the configuration file is done like:
unix> erlc -bber_bin +optimize +asn1config GUI.asn erlang> asn1ct:compile('GUI',[ber_bin,optimize,asn1config]).The module can be used like:
1> Button_Msg = {'Button',123,true}. {'Button',123,true} 2> {ok,Button_Bytes} = 'GUI':encode('Button',Button_Msg). {ok,[<<48>>, [6], [<<128>>, [1], 123], [<<129>>, [1], 255]]} 3> {ok,Exclusive_Msg_Button} = 'GUI':decode_Button_exclusive(list_to_binary(Button_Bytes)). {ok,{'Button',{'Button_number',<<128,1,123>>}, true}} 4> 'GUI':decode_part('Button_number',<<128,1,123>>). {ok,123} 5> Window_Msg = {'Window',{status,{'Status',35,[{'Button',3,true},{'Button',4,false},{'Button',5,true},{'Button',6,true},{'Button',7,false},{'Button',8,true},{'Button',9,true},{'Button',10,false},{'Button',11,true},{'Button',12,true},{'Button',13,false},{'Button',14,true}],false,{possibleActions,[{'Action',16,{'Button',17,true}}]}}}}. {'Window',{status,{'Status',35, [{'Button',3,true}, {'Button',4,false}, {'Button',5,true}, {'Button',6,true}, {'Button',7,false}, {'Button',8,true}, {'Button',9,true}, {'Button',10,false}, {'Button',11,true}, {'Button',12,true}, {'Button',13,false}, {'Button',14,true}], false, {possibleActions,[{'Action',16,{'Button',17,true}}]}}}} 6> {ok,Window_Bytes}='GUI':encode('Window',Window_Msg). {ok,[<<161>>, [127], [<<128>>, ... 8> {ok,{status,{'Status',Int,{Type_Key_SeqOf,Val_SEQOF},BoolOpt,{Type_Key_Choice,Val_Choice}}}}='GUI':decode_Window_status_exclusive(list_to_binary(Window_Bytes)). {ok,{status,{'Status',35, {'Status_buttonList',[<<48,6,128,1,3,129,1,255>>, <<48,6,128,1,4,129,1,0>>, <<48,6,128,1,5,129,1,255>>, <<48,6,128,1,6,129,1,255>>, <<48,6,128,1,7,129,1,0>>, <<48,6,128,1,8,129,1,255>>, <<48,6,128,1,9,129,1,255>>, <<48,6,128,1,10,129,1,0>>, <<48,6,128,1,11,129,1,255>>, <<48,6,128,1,12,129,1,255>>, <<48,6,128,1,13,129,1,0>>, <<48,6,128,1,14,129,1,255>>]}, false, {'Status_actions',<<163,21,160,19,48,17,2,1,16,160,12,172,10,171,8,48,6,128,1,...>>}}}} 10> 'GUI':decode_part(Type_Key_SeqOf,Val_SEQOF). {ok,[{'Button',3,true}, {'Button',4,false}, {'Button',5,true}, {'Button',6,true}, {'Button',7,false}, {'Button',8,true}, {'Button',9,true}, {'Button',10,false}, {'Button',11,true}, {'Button',12,true}, {'Button',13,false}, {'Button',14,true}]} 11> 'GUI':decode_part(Type_Key_SeqOf,hd(Val_SEQOF)). {ok,{'Button',3,true}} 12> 'GUI':decode_part(Type_Key_Choice,Val_Choice). {ok,{possibleActions,[{'Action',16,{'Button',17,true}}]}}2.2 Selective Decode
This specialized decode decodes one single subtype of a constructed value. It is the fastest method to extract one sub value. The typical use of this decode is when one want to inspect, for instance a version number,to be able to decide what to do with the entire value. The result is returned as
{ok,Value}
or{error,Reason}
.2.2.1 How To Make It Work
The following steps are necessary:
- Write instructions in the configuration file. Including the name of a user function, the name of the ASN.1 specification and a notation that tells which part of the type will be decoded.
- Compile with the additional option
asn1config
. The compiler searches for a configuration file with the same name as the ASN.1 spec but with the extension .asn1config. In the same file you can provide configuration specs for exclusive decode as well. The generated Erlang module has the usual functionality for encode/decode preserved and the specialized decode functionality added.
2.2.2 User Interface
The only new user interface function is the one provided by the user in the configuration file. You can invoke that function by the
ModuleName:FunctionName
notation.So, if you have the following spec
{selective_decode,{'ModuleName',[{selected_decode_Window,TypeList}]}}
in the con-fig file, you do the selective decode by{ok,Result}='ModuleName':selected_decode_Window(EncodedBinary).
2.2.3 Writing a Selective Decode Instruction
It is possible to describe one or many selective decode functions in a configuration file, you have to use the following notation:
Selective_Decode_Instruction = {selective_decode,{Module_Name,Decode_Instructions}}. Module_Name = atom() Decode_Instructions = [Decode_Instruction]+ Decode_Instruction = {Selective_Decode_Function_Name,Type_List} Selective_Decode_Function_Name = atom() Type_List = [Top_Type|Element_List] Element_List = Name|List_Selector Name = atom() List_Selector = [integer()]Observe that the instruction must be a valid Erlang term ended by a dot.
The
Module_Name
is the same as the name of the ASN.1 spec, but without the extension. ADecode_Instruction
is a tuple with your chosen function name and the components from the top type that leads to the single type you want to decode. Notice that you have to choose a name of your function that will not be the same as any of the generated functions. The first element of theType_List
is the top type of the encoded message. In theElement_List
it is followed by each of the component names that leads to selected type. Each of the names in theElement_List
must be constructed types except the last name, which can be any type.The List_Selector makes it possible to choose one of the encoded components in a SEQUENCE OF/ SET OF. It is also possible to go further in that component and pick a sub type of that to decode. So in the
Type_List
:['Window',status,buttonList,[1],number]
the componentbuttonList
has to be a SEQUENCE OF or SET OF type. In this example componentnumber
of the first of the encoded elements in the SEQUENCE OFbuttonList
is selected. This apply on the ASN.1 spec above.2.2.4 Another Example
In this example we use the same ASN.1 spec as above. A valid selective decode instruction is:
{selective_decode,{'GUI',[{selected_decode_Window1,['Window',status,buttonList,[1],number]}, {selected_decode_Action,['Action',handle,number]}, {selected_decode_Window2,['Window',status,actions,possibleActions,[1],handle,number]}]}}.The first
Decode_Instruction
,{selected_decode_Window1,['Window',status,buttonList,[1],number]}
is commented in the previous section. The instruction{selected_decode_Action,['Action',handle,number]}
picks the componentnumber
in thehandle
component of the typeAction
. If we have the valueValAction = {'Action',17,{'Button',4711,false}}
the internal value 4711 should be picked byselected_decode_Action
. In an Erlang terminal it looks like:ValAction = {'Action',17,{'Button',4711,false}}. {'Action',17,{'Button',4711,false}} 7> {ok,Bytes}='GUI':encode('Action',ValAction). ... 8> BinBytes = list_to_binary(Bytes). <<48,18,2,1,17,160,13,172,11,171,9,48,7,128,2,18,103,129,1,0>> 9> 'GUI':selected_decode_Action(BinBytes). {ok,4711} 10>The third instruction,
['Window',status,actions,possibleActions,[1],handle,number]
, which is a little more complicated,
- starts with type Window.
- Picks component status of
Window
that is of typeStatus
.
- Then takes component actions of type
Status
.
- Then possibleActions of the internal defined CHOICE type.
- Thereafter it goes into the first component of the SEQUENCE OF by [1]. That component is of type
Action
.
- The instruction next picks component handle.
- And finally component number of the type
Button
.
The following figures shows which components are in the TypeList
['Window',status,actions,possibleActions,[1],handle,number]
. And which part of a message that will be decoded by selected_decode_Window2.
The elements specified in the config file for selective decode of a sub-value in a Window message
Figure symbolizes the bytes of a Window:status message. Only the marked element is decoded when selected_decode_Window2 is called.With the following example you can examine that both
selected_decode_Window2
andselected_decode_Window1
decodes the intended sub-value of the valueVal
1> Val = {'Window',{status,{'Status',12, [{'Button',13,true}, {'Button',14,false}, {'Button',15,true}, {'Button',16,false}], true, {possibleActions,[{'Action',17,{'Button',18,false}}, {'Action',19,{'Button',20,true}}, {'Action',21,{'Button',22,false}}]}}}} 2> {ok,Bytes}='GUI':encode('Window',Val). ... 3> Bin = list_to_binary(Bytes). <<161,101,128,1,12,161,32,48,6,128,1,13,129,1,255,48,6,128,1,14,129,1,0,48,6,128,1,15,129,...>> 4> 'GUI':selected_decode_Window1(Bin). {ok,13} 5> 'GUI':selected_decode_Window2(Bin). {ok,18}Observe that the value feed into the selective decode functions must be a binary.
2.3 Performance
To give an indication on the possible performance gain using the specialized decodes, some measures have been performed. The relative figures in the outcome between selective, exclusive and complete decode (the normal case) depends on the structure of the type, the size of the message and on what level the selective and exclusive decodes are specified.
2.3.1 ASN.1 Specifications, Messages and Configuration
The specs GUI and MEDIA-GATEWAY-CONTROL was used in the test.
For the GUI spec the configuration looked like:
{selective_decode,{'GUI',[{selected_decode_Window1,['Window',status,buttonList,[1],number]}, {selected_decode_Window2,['Window',status,actions,possibleActions,[1],handle,number]}]}}. {exclusive_decode,{'GUI',[{decode_Window_status_exclusive,['Window',[{status,[{buttonList,parts},{actions,undecoded}]}]]}]}}.The MEDIA-GATEWAY-CONTROL configuration was:
{exclusive_decode, {'MEDIA-GATEWAY-CONTROL', [{decode_MegacoMessage_exclusive,['MegacoMessage',[{authHeader,undecoded},{mess,[{mId,undecoded},{messageBody,undecoded}]}]]}]}}. {selective_decode, {'MEDIA-GATEWAY-CONTROL', [{decode_MegacoMessage_selective,['MegacoMessage',mess,version]}]}}.The corresponding values were:
{'Window',{status,{'Status',12, [{'Button',13,true}, {'Button',14,false}, {'Button',15,true}, {'Button',16,false}, {'Button',13,true}, {'Button',14,false}, {'Button',15,true}, {'Button',16,false}, {'Button',13,true}, {'Button',14,false}, {'Button',15,true}, {'Button',16,false}], true, {possibleActions, [{'Action',17,{'Button',18,false}}, {'Action',19,{'Button',20,true}}, {'Action',21,{'Button',22,false}}, {'Action',17,{'Button',18,false}}, {'Action',19,{'Button',20,true}}, {'Action',21,{'Button',22,false}}, {'Action',17,{'Button',18,false}}, {'Action',19,{'Button',20,true}}, {'Action',21,{'Button',22,false}}, {'Action',17,{'Button',18,false}}, {'Action',19,{'Button',20,true}}, {'Action',21,{'Button',22,false}}, {'Action',17,{'Button',18,false}}, {'Action',19,{'Button',20,true}}, {'Action',21,{'Button',22,false}}, {'Action',17,{'Button',18,false}}, {'Action',19,{'Button',20,true}}, {'Action',21,{'Button',22,false}}]}}}} {'MegacoMessage',asn1_NOVALUE, {'Message',1, {ip4Address, {'IP4Address',[125,125,125,111],55555}}, {transactions, [{transactionReply, {'TransactionReply',50007,asn1_NOVALUE, {actionReplies, [{'ActionReply',0,asn1_NOVALUE,asn1_NOVALUE, [{auditValueReply,{auditResult,{'AuditResult', {'TerminationID',[],[255,255,255]}, [{mediaDescriptor, {'MediaDescriptor',asn1_NOVALUE, {multiStream, [{'StreamDescriptor',1, {'StreamParms', {'LocalControlDescriptor',sendRecv,asn1_NOVALUE,asn1_NOVALUE, [{'PropertyParm',[0,11,0,7],[[52,48]],asn1_NOVALUE}]}, {'LocalRemoteDescriptor', [[{'PropertyParm',[0,0,176,1],[[48]],asn1_NOVALUE}, {'PropertyParm',[0,0,176,8],[[73,78,32,73,80,52,32,49,50,53,46,49,50,53,46,49,50,53,46,49,49,49]],asn1_NOVALUE}, {'PropertyParm',[0,0,176,15],[[97,117,100,105,111,32,49,49,49,49,32,82,84,80,47,65,86,80,32,32,52]],asn1_NOVALUE}, {'PropertyParm',[0,0,176,12], [[112,116,105,109,101,58,51,48]],asn1_NOVALUE}]]}, {'LocalRemoteDescriptor', [[{'PropertyParm',[0,0,176,1],[[48]],asn1_NOVALUE}, {'PropertyParm',[0,0,176,8],[[73,78,32,73,80,52,32,49,50,52,46,49,50,52,46,49,50,52,46,50,50,50]],asn1_NOVALUE}, {'PropertyParm',[0,0,176,15],[[97,117,100,105,111,32,50,50,50,50,32,82,84,80,47,65,86,80,32,32,52]],asn1_NOVALUE}, {'PropertyParm',[0,0,176,12],[[112,116,105,109,101,58,51,48]],asn1_NOVALUE}]]}}}]}}}, {packagesDescriptor, [{'PackagesItem',[0,11],1}, {'PackagesItem',[0,11],1}]}, {statisticsDescriptor, [{'StatisticsParameter',[0,12,0,4],[[49,50,48,48]]}, {'StatisticsParameter',[0,11,0,2],[[54,50,51,48,48]]}, {'StatisticsParameter',[0,12,0,5],[[55,48,48]]}, {'StatisticsParameter',[0,11,0,3],[[52,53,49,48,48]]}, {'StatisticsParameter',[0,12,0,6],[[48,46,50]]}, {'StatisticsParameter',[0,12,0,7],[[50,48]]}, {'StatisticsParameter',[0,12,0,8],[[52,48]]}]}]}}}]}]}}}]}}}The size of the encoded values was 458 bytes for GUI and 464 bytes for MEDIA-GATEWAY-CONTROL.
2.3.2 Results
The ASN.1 specs in the test are compiled with the options
ber_bin, optimize, driver
andasn1config
. If thedriver
option had been omitted there should have been higher values fordecode
anddecode_part
.The test program runs 10000 decodes on the value, resulting in a printout with the elapsed time in microseconds for the total number of decodes.
Function Time (microseconds) Kind of Decode ASN.1 spec % of time vs. complete decode decode_MegacoMessage_selective/1
374045
selective
MEDIA-GATEWAY-CONTROL
8.3 decode_MegacoMessage_exclusive/1
621107
exclusive
MEDIA-GATEWAY-CONTROL
13.8 decode/2
4507457
complete
MEDIA-GATEWAY-CONTROL
100 selected_decode_Window1/1
449585
selective
GUI
7.6 selected_decode_Window2/1
890666
selective
GUI
15.1 decode_Window_status_exclusive/1
1251878
exclusive
GUI
21.3 decode/2
5889197
complete
GUI
100 Results of complete, exclusive and selective decode Another interesting question is what the relation is between a complete decode, an exclusive decode followed by
decode_part
of the excluded parts and a selective decode followed by a complete decode. Some situations may be compared to this simulation, e.g. inspect a sub-value and later on look at the entire value. The following table shows figures from this test. The number of loops and time unit is the same as in the previous test.
Actions Function Time (microseconds) ASN.1 spec % of time vs. complete decode complete
decode/2
4507457
MEDIA-GATEWAY-CONTROL
100 selective and complete
decode_MegacoMessage_selective/1
4881502
MEDIA-GATEWAY-CONTROL
108.3 exclusive and decode_part
decode_MegacoMessage_exclusive/1
5481034
MEDIA-GATEWAY-CONTROL
112.3 complete
decode/2
5889197
GUI
100 selective and complete
selected_decode_Window1/1
6337636
GUI
107.6 selective and complete
selected_decode_Window2/1
6795319
GUI
115.4 exclusive and decode_part
decode_Window_status_exclusive/1
6249200
GUI
106.1 Results of complete, exclusive + decode_part and selective + complete decodes Other ASN.1 types and values can differ much from these figures. Therefore it is important that you, in every case where you intend to use either of these decodes, perform some tests that shows if you will benefit your purpose.
2.3.3 Comments
Generally speaking the gain of selective and exclusive decode in advance of complete decode is greater the bigger value and the less deep in the structure you have to decode. One should also prefer selective decode instead of exclusive decode if you are interested in just one single sub-value.
Another observation is that the exclusive decode followed by decode_part decodes is very attractive if the parts will be sent to different servers for decoding or if one in some cases not is interested in all parts.
The fastest selective decode are when the decoded type is a primitive type and not so deep in the structure of the top type. The
selected_decode_Window2
decodes a big constructed value, which explains why this operation is relatively slow.It may vary from case to case which combination of selective/complete decode or exclusive/part decode is the fastest.