_piped=True swallows exceptions/errors

[mattixpet]

Using _piped=True seems to swallow exceptions and log nothing even when there are errors in the piped commands. A very simple example is straight from the docs:

for line in sh.tr(sh.tail("-f", "test.log", _piped=True), "[:upper:]", "[:lower:]", _iter=True):
    print(line)

If test.log does not exist, tail understandably throws an exception in a normal shell and run on it's own using sh in python

/usr/bin/tail: cannot open ‘test.log’ for reading: No such file or directory
/usr/bin/tail: no files remaining

However, if you use it in piping you get no error and no logging.

A simple workaround for the logging is adding _err=myCallback and having myCallback manually printing out the errors, but that still doesn't handle the disappearance of the exceptions.

Is this intended behavior? This should at least be mentioned in the docs, if not fixed.

[amoffat]

ok I think I have a fix for this in the release-1.2 branch now. The issue is not really a bug, per-se, but not ideal and definitely not documented.

The behavior in release-1.2 is that if a process is launched that is not waited, and its used as stdin to a process that is waited on, the outer process, and its termination, will do some exception checking on the inner process, so any exceptions that occurred in the inner process will bubble up to the main thread and can be caught. In the cases where a process is launched that is not waited on, but is not used as stdin...there's no good way to bubble up those exceptions to the main thread, so we just cause them to raise inside of its internal threads, which will at least print something to the screen.