Discussion:
bug#30430: [column] accounting for ansi color codes
(too old to reply)
Boruch Baum
2018-02-12 06:07:15 UTC
Permalink
The `column -t' command doesn't align input properly when that input
includes ansi color codes.

An argument that it should do so, is that `column' serves to make input
more readable, and easier to find and compare fields of input data - the
same goal as colorization.

I encountered this when trying to columnate log output that had been fed
through the colorization program `ccze'.

Example: Compare the output of the following two one-liners:

awk '{sub("install|configure|upgrade|trigproc","& _",$3);print}' \
/var/log/dpkg.log | column -t

awk '{sub("install|configure|upgrade|trigproc","& _",$3);print}' \
/var/log/dpkg.log | ccze -m ansi -o noscroll >&1 | column -t
--
hkp://keys.gnupg.net
CA45 09B5 5351 7C11 A9D1 7286 0036 9E45 1595 8BC0
Bernhard Voelker
2018-02-12 07:23:29 UTC
Permalink
Post by Boruch Baum
The `column -t' command doesn't align input properly when that input
includes ansi color codes.
An argument that it should do so, is that `column' serves to make input
more readable, and easier to find and compare fields of input data - the
same goal as colorization.
I encountered this when trying to columnate log output that had been fed
through the colorization program `ccze'.
awk '{sub("install|configure|upgrade|trigproc","& _",$3);print}' \
/var/log/dpkg.log | column -t
awk '{sub("install|configure|upgrade|trigproc","& _",$3);print}' \
/var/log/dpkg.log | ccze -m ansi -o noscroll >&1 | column -t
Unfortunately, you did not show us neither the input - and I don't
have dpkg -, nor the output, so I'd have to guess what you think
might be going wrong.

Would you provide a simple reproducer, please?

In the following example, column(1) produces correct output:

$ printf "%s\n" \
"this_is_a_long_string 99" \
"short abcdefghijklmnop" \
| ccze -m ansi -o noscroll \
| column -t
this_is_a_long_string 99
short abcdefghijklmnop

FWIW: what happens if you switch the two formatting commands, i.e., use

$ ... | column -t | ccze ...

instead of

$ ... | ccze .... | column -t

?
At least when the input uses a different separator, then you'd have to
do that anyway - otherwise the separation of the columns is messed;
compare:

$ head -n3 /etc/passwd | ccze -m ansi -o noscroll | column -t -s:

$ head -n3 /etc/passwd | column -t -s: | ccze -m ansi -o noscroll

Is that the same what's happening to you?

Have a nice day,
Berny
Boruch Baum
2018-02-12 12:22:22 UTC
Permalink
Post by Bernhard Voelker
Post by Boruch Baum
...
Unfortunately, you did not show us neither the input
Easy... A sample input would be:

2018-02-09 11:10:27 status unpacked libefivar0:amd64 0.15-3
2018-02-09 11:10:28 status half-configured libefivar0:amd64 0.15-3
2018-02-09 11:10:28 status installed libefivar0:amd64 0.15-3
2018-02-09 11:10:28 trigproc man-db:amd64 2.7.0.2-5 <none>
2018-02-09 11:10:28 status half-configured man-db:amd64 2.7.0.2-5
2018-02-09 11:10:29 status installed man-db:amd64 2.7.0.2-5
2018-02-09 11:10:29 configure efibootmgr:amd64 0.11.0-3 <none>
2018-02-09 11:10:29 status unpacked efibootmgr:amd64 0.11.0-3
2018-02-09 11:10:29 status half-configured efibootmgr:amd64 0.11.0-3
2018-02-09 11:10:29 status installed efibootmgr:amd64 0.11.0-3
Post by Bernhard Voelker
- and I don't have dpkg -, nor the output,
Also easy. First, an easily readable sample output, without piping
Post by Bernhard Voelker
Post by Boruch Baum
awk '{sub("install|configure|upgrade|trigproc","& _",$3);print}' \
/var/log/dpkg.log | column -t
2018-02-09 11:10:27 status unpacked libefivar0:amd64
0.15-3
2018-02-09 11:10:28 status half-configured libefivar0:amd64
0.15-3
2018-02-09 11:10:28 status installed libefivar0:amd64
0.15-3
2018-02-09 11:10:28 trigproc _ man-db:amd64
2.7.0.2-5 <none>
2018-02-09 11:10:28 status half-configured man-db:amd64
2.7.0.2-5
2018-02-09 11:10:29 status installed man-db:amd64
2.7.0.2-5
2018-02-09 11:10:29 configure _ efibootmgr:amd64
0.11.0-3 <none>
2018-02-09 11:10:29 status unpacked efibootmgr:amd64
0.11.0-3
2018-02-09 11:10:29 status half-configured efibootmgr:amd64
0.11.0-3
2018-02-09 11:10:29 status installed efibootmgr:amd64 0.11.0-3


For the colorized output, ie. the actual raw output and the subject of
the bug report, here are two versions. First, in-line, with the
colorization codes stripped out so that it might be easily readable by
all, and; second, in raw format, as an mail attachment file, to avoid
having the control sequences somehow possibly messing with some hapless
reader's TTY.
Post by Bernhard Voelker
Post by Boruch Baum
awk '{sub("install|configure|upgrade|trigproc","& _",$3);print}' \
/var/log/dpkg.log | ccze -m ansi -o noscroll >&1 | column -t
2018-02-09 11:10:27 status unpacked libefivar0:amd64 0.15-3
2018-02-09 11:10:28 status half-configured libefivar0:amd64 0.15-3
2018-02-09 11:10:28 status installed libefivar0:amd64 0.15-3
2018-02-09 11:10:28 trigproc _ man-db:amd64
2.7.0.2-5 <none>
2018-02-09 11:10:28 status half-configured man-db:amd64
2.7.0.2-5
2018-02-09 11:10:29 status installed man-db:amd64
2.7.0.2-5
2018-02-09 11:10:29 configure _ efibootmgr:amd64
0.11.0-3 <none>
2018-02-09 11:10:29 status unpacked efibootmgr:amd64
0.11.0-3
2018-02-09 11:10:29 status half-configured efibootmgr:amd64
0.11.0-3
2018-02-09 11:10:29 status installed efibootmgr:amd64
0.11.0-3
Post by Bernhard Voelker
FWIW: what happens if you switch the two formatting commands, i.e., use
$ ... | column -t | ccze ...
instead of
$ ... | ccze .... | column -t
?
! The colorization is not usefully applied, ie. the entire output is
monochrome, but in a color different that the input or normal STDOUT.
Post by Bernhard Voelker
At least when the input uses a different separator,
That would be a misunderstanding in what `ccze' does. It is specifically
intended for log files only. See, for example, man (1) ccze.
Post by Bernhard Voelker
then you'd have to do that anyway - otherwise the separation of the
columns is messed;
As above, the colorization is not usefully applied, ie. the entire
output is monochrome, but in a color different that the input or normal
STDOUT.
Post by Bernhard Voelker
$ head -n3 /etc/passwd | column -t -s: | ccze -m ansi -o noscroll
A reasonably colorized output is applied.
--
hkp://keys.gnupg.net
CA45 09B5 5351 7C11 A9D1 7286 0036 9E45 1595 8BC0
Pádraig Brady
2018-02-17 21:33:48 UTC
Permalink
Post by Boruch Baum
The `column -t' command doesn't align input properly when that input
includes ansi color codes.
An argument that it should do so, is that `column' serves to make input
more readable, and easier to find and compare fields of input data - the
same goal as colorization.
I encountered this when trying to columnate log output that had been fed
through the colorization program `ccze'.
awk '{sub("install|configure|upgrade|trigproc","& _",$3);print}' \
/var/log/dpkg.log | column -t
awk '{sub("install|configure|upgrade|trigproc","& _",$3);print}' \
/var/log/dpkg.log | ccze -m ansi -o noscroll >&1 | column -t
Note column(1) is part of the util-linux project.
We've similar considerations though with wc -L
which we've at least mentioned at:
http://www.pixelbeat.org/docs/coreutils-gotchas.html#wc
Assaf Gordon
2018-10-30 02:45:17 UTC
Permalink
tags 30430 notabug
close 30430
stop

(triaging old bugs)
Post by Pádraig Brady
Post by Boruch Baum
The `column -t' command doesn't align input properly when that input
includes ansi color codes.
Note column(1) is part of the util-linux project.
We've similar considerations though with wc -L
http://www.pixelbeat.org/docs/coreutils-gotchas.html#wc
Given the above, I'm closing this bug.

-assaf

Continue reading on narkive:
Loading...