VNL-PASTE(1)

NAME
vnl-paste - combine log files record-by-record

SYNOPSIS
vnl-paste concatenates records from the given data files. Comments are stripped to make sure things line up

 $ cat a.vnl
 ## gathered from sensor model xxx
 # humidity temperature
 90 25
 80 20
 81 19
 82 18
 70 15

 $ cat b.vnl
 # position
 10
 20
 ## we moved the sensor
 150
 160
 170

 $ vnl-paste a.vnl b.vnl
 # humidity temperature position
 90 25  10
 80 20  20
 81 19  150
 82 18  160
 70 15  170

We can also give it more than 2 files and annotate the field labels

 $ cat c.vnl
 # position
 12
 18
 155
 168
 190

 $ vnl-paste --vnl-suffix2 _b a.vnl b.vnl c.vnl | vnl-align
 # humidity temperature position_b position
 90         25           10         12
 80         20           20         18
 81         19          150        155
 82         18          160        168
 70         15          170        190

 $ vnl-paste --vnl-suffix ,_b,_c a.vnl b.vnl c.vnl | vnl-align
 # humidity temperature position_b position_c
 90         25           10         12
 80         20           20         18
 81         19          150        155
 82         18          160        168
 70         15          170        190

 $ vnl-paste --vnl-autosuffix a.vnl b.vnl c.vnl | vnl-align
 # humidity_a temperature_a position_b position_c
 90           25             10         12
 80           20             20         18
 81           19            150        155
 82           18            160        168
 70           15            170        190

DESCRIPTION
  Usage: vnl-paste
                  [ --vnl-[pre|suf]fix[1|2] xxx    |
                    --vnl-[pre|suf]fix xxx,yyy,zzz |
                    --vnl-autoprefix               |
                    --vnl-autosuffix ]
                  logfile1 logfile2 ...

This tool is essentially a join using the line number as the key. This tool concatenates the data in the given vnlog files record-by-record. "vnl-paste" is a wrapper around the GNU coreutils "paste" tool. Since this is a wrapper, the behavior of the underlying tool is preserved; consult the paste(1) manpage for detail. The differences from GNU coreutils "paste" are

•   The input and output to this tool are vnlog files, complete with a legend

•   By default we call the "paste" tool to do the actual work. If the underlying tool has a different name or lives in an odd path, this can be specified by passing "--vnl-tool TOOL"

•   None of the "paste" options make sense in a vnlog context, so this tool has only the vnlog-specific "--vnl-..." options.

Note that all non-legend comments are stripped out, since it's not obvious where they should end up.

  Field names in the output
The field name logic works the same way as in "vnl-join", with all the suffix and prefix options from that tool being available here.

By default, the field names in the output match those in the input. This is what you want most of the time. It is possible, however that a column name adjustment is needed. One common use case for this is if the files being pasted have identically-named columns, which would produce duplicate columns in the output. Example: we fixed a bug in a program, and want to compare the results before and after the fix. The program produces an x-y trajectory, so both the bugged and the bug-fixed programs produce a vnlog with a legend

 # x y

Pasting these will produce a vnlog with a legend

 # x y x y

which is confusing, and not what you want. Instead, we invoke "vnl-paste" as

 vnl-paste --vnl-suffix1 _buggy --vnl-suffix2 _fixed buggy.vnl fixed.vnl

And in the output we get a legend

 # x_buggy y_buggy x_fixed y_fixed

Much better.

Note that "vnl-paste" provides several ways of specifying this. The above works only for 2-way pastes. An alternate syntax is available for N-way pastes, a comma-separated list. The same could be expressed like this:

 vnl-paste --vnl-suffix _buggy,_fixed buggy.vnl fixed.vnl

Finally, if passing in structured filenames, "vnl-paste" can infer the desired syntax from the filenames. The same as above could be expressed even simpler:

 vnl-paste --vnl-autosuffix buggy.vnl fixed.vnl

This works by looking at the set of passed in filenames, and stripping out the common leading and trailing strings.

BUGS AND CAVEATS
If the input data files has a mismatched number of lines, the extra data will be output without the null-column markers. This will result in mislabeled data. Use this tool only if you're sure you have a matching number of records. For instance:

 $ vnl-paste <(echo '# x'; seq 3) <(echo '# y'; seq 5)
 # x y
 1      1
 2      2
 3      3
        4
        5

 $ vnl-paste <(echo '# x'; seq 3) <(echo '# y'; seq 5) | vnl-align
 # x y
 1   1
 2   2
 3   3
 4
 5

COMPATIBILITY
I use GNU/Linux-based systems exclusively, but everything has been tested functional on FreeBSD and OSX in addition to Debian, Ubuntu and CentOS. I can imagine there's something I missed when testing on non-Linux systems, so please let me know if you find any issues.

SEE ALSO
paste(1)

REPOSITORY
https://github.com/dkogan/vnlog/

AUTHOR
Dima Kogan "<dima@secretsauce.net>"

LICENSE AND COPYRIGHT
Copyright 2018 Dima Kogan "<dima@secretsauce.net>"

This library is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 2.1 of the License, or (at your option) any later version.