opp_vec2csv.pl merge-by flag not sufficient for same event, time, module

[minnie-jo]

If event, time, node is all same or duplicated, --merge-by etm merges the values in a chunk.
Would there be a way to input a flag parameter to merge by the columns created?

[minnie-jo]

test.csv

This is how it looks like after ./opp_vec2csv.pl -F id:vector=id -F posX:vector=posx -F posY:vector=posy -F age:vector=age ${FILE_PATH}.vec | sort -k2 -n > test.csv

[dbuse]

Not sure about the solution as I'm not an active user of the Perl scripts. But the title mismatched the called script, so I fixed that :D

Maybe @heinovski or @sommer can help with this?

[minnie-jo]

I don't know if this is a feature request or a bug, I would say a slight bug + feature request.

I'm right now printing out the contents of a neighbor table, which is extracting signals in a same event, time and node. What I found out that in the opp_vec2csv.pl, there are several flag options to merge the output of multiple vectors. However, if there are multiple vectors with same event, time, or module, all vectors are merged by one row.

Could there be a feature that joins the rows also by generated columns?

These are references and examples of this request.
For example:

Output of ./opp_vec2csv.pl -F id:vector=id -F posX:vector=posx -F posY:vector=posy -F age:vector=age ${FILE_PATH}.vec | sort -k2 -n > test.csv

event	time	node	age	id	posx	posy
22	6	Scenario.node[0].appl				119.12773144385
22	6	Scenario.node[0].appl			127.01227578646	
22	6	Scenario.node[0].appl		0	
22	6	Scenario.node[0].appl	1	
22	6	Scenario.node[0].appl				518.51675007978
22	6	Scenario.node[0].appl			526.6
22	6	Scenario.node[0].appl		9			
22	6	Scenario.node[0].appl	1		

Output of ./opp_vec2csv.pl --merge-by etm -F id:vector=id -F posX:vector=posx -F posY:vector=posy -F age:vector=age ${FILE_PATH}.vec | sort -k2 -n > test.csv would result in

22	6	Scenario.node[0].appl	1	9	526.6	518.51675007978

So I am working around by using a long bash shell script something like this:

for (( runNum = 0; runNum < $RUNNUMBER; runNum++ ))
do
    FILE_PATH="${VEC_FILE_PATH}-#${runNum}"
    ./opp_vec2csv.pl -F "id:vector"=id ${FILE_PATH}.vec > ${VEC_FILE_PATH}Id-\#$runNum.csv
    ./opp_vec2csv.pl -F "posX:vector"=posx ${FILE_PATH}.vec > ${VEC_FILE_PATH}PosX-\#$runNum.csv
    ./opp_vec2csv.pl -F "posY:vector"=posy ${FILE_PATH}.vec > ${VEC_FILE_PATH}PosY-\#$runNum.csv
    ./opp_vec2csv.pl -F "age:vector"=age ${FILE_PATH}.vec > ${VEC_FILE_PATH}Age-\#$runNum.csv
    cut -d $'\t' -f 4 ${VEC_FILE_PATH}PosX-\#$runNum.csv | paste ${VEC_FILE_PATH}Id-\#$runNum.csv - > results/temp1-#$runNum.csv
    cut -d $'\t' -f 4 ${VEC_FILE_PATH}PosY-\#$runNum.csv | paste results/temp1-\#$runNum.csv - > results/temp2-#$runNum.csv
    cut -d $'\t' -f 4 ${VEC_FILE_PATH}Age-\#$runNum.csv | paste results/temp2-\#$runNum.csv - | sort -k2 -n > ${FILE_PATH}.csv
    rm ${VEC_FILE_PATH}Id-\#$runNum.csv ${VEC_FILE_PATH}PosX-\#$runNum.csv ${VEC_FILE_PATH}PosY-\#$runNum.csv ${VEC_FILE_PATH}Age-\#$runNum.csv results/temp1-\#$runNum.csv results/temp2-\#$runNum.csv
done

[sommer]

The OMNeT++ manual does not seem to give any guarantees wrt. the order of data in an output vector file, so I would be hesitant to rely on the order of data to survive from the time it was written to the time it is read. This, however, is what this feature would need to rely on. That said, I would be happy to receive a pull request to retrofit the desired feature in a stable way.

As a quick workaround, it should also be possible to use cOutVector::recordWithTimestamp(simtime_t t, double value) to both enforce an order of data and to make merging by timestamp a possibility.