Kill the entire process group when a command times out #237
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In cases where the command being executed spawns subcommands we need to ensure they are all killed, so that the timeout occurs when it should instead of waiting for the subprocesses to exit.
|
Thanks for reporting this Derek, I'll review and it will probably go with the 1.2 release |
You have 1.11 release, wouldn't 1.2 be lower? |
|
Proposed to include as a patch in Fedora/EPEL https://bugzilla.redhat.com/show_bug.cgi?id=1306405 |
|
I would propose that instead a I think timeout calling |
This was referenced Nov 27, 2016
This was referenced Dec 1, 2016
0-wiz-0
added a commit
to NetBSD/pkgsrc-wip
that referenced
this pull request
Dec 12, 2016
* added `_out` and `_out_bufsize` validator [#346](amoffat/sh#346) * bugfix for internal stdout thread running when it shouldn't [#346](amoffat/sh#346) * regression bugfix on timeout [#344](amoffat/sh#344) * regression bugfix on `_ok_code=None` * further improvements on cpu usage * regression in cpu usage [#339](amoffat/sh#339) * fd leak regression and fix for flawed fd leak detection test [#337](amoffat/sh#337) * support for `io.StringIO` in python2 * added support for using raw file descriptors for `_in`, `_out`, and `_err` * removed `.close()`ing `_out` handler if FIFO detected * composed commands no longer propagate `_bg` * better support for using `sys.stdin` and `sys.stdout` for `_in` and `_out` * bugfix where `which()` would not stop searching at the first valid executable found in PATH * added `_long_prefix` for programs whose long arguments start with something other than `--` [#278](amoffat/sh#278) * added `_log_msg` for advanced configuration of log message [#311](amoffat/sh#311) * added `sh.contrib.sudo` * added `_arg_preprocess` for advanced command wrapping * alter callable `_in` arguments to signify completion with falsy chunk * bugfix where pipes passed into `_out` or `_err` were not flushed on process end [#252](amoffat/sh#252) * deprecated `with sh.args(**kwargs)` in favor of `sh2 = sh(**kwargs)` * made `sh.pushd` thread safe * added `.kill_group()` and `.signal_group()` methods for better process control [#237](amoffat/sh#237) * added `new_session` special keyword argument for controlling spawned process session [#266](amoffat/sh#266) * bugfix better handling for EINTR on system calls [#292](amoffat/sh#292) * bugfix where with-contexts were not threadsafe [#247](amoffat/sh#195) * `_uid` new special keyword param for specifying the user id of the process [#133](amoffat/sh#133) * bugfix where exceptions were swallowed by processes that weren't waited on [#309](amoffat/sh#309) * bugfix where processes that dupd their stdout/stderr to a long running child process would cause sh to hang [#310](amoffat/sh#310) * improved logging output [#323](amoffat/sh#323) * bugfix for python3+ where binary data was passed into a process's stdin [#325](amoffat/sh#325) * Introduced execution contexts which allow baking of common special keyword arguments into all commands [#269](amoffat/sh#269) * `Command` and `which` now can take an optional `paths` parameter which specifies the search paths [#226](amoffat/sh#226) * `_preexec_fn` option for executing a function after the child process forks but before it execs [#260](amoffat/sh#260) * `_fg` reintroduced, with limited functionality. hurrah! [#92](amoffat/sh#92) * bugfix where a command would block if passed a fd for stdin that wasn't yet ready to read [#253](amoffat/sh#253) * `_long_sep` can now take `None` which splits the long form arguments into individual arguments [#258](amoffat/sh#258) * making `_piped` perform "direct" piping by default (linking fds together). this fixes memory problems [#270](amoffat/sh#270) * bugfix where calling `next()` on an iterable process that has raised `StopIteration`, hangs [#273](amoffat/sh#273) * `sh.cd` called with no arguments no changes into the user's home directory, like native `cd` [#275](amoffat/sh#275) * `sh.glob` removed entirely. the rationale is correctness over hand-holding. [#279](amoffat/sh#279) * added `_truncate_exc`, defaulting to `True`, which tells our exceptions to truncate output. * bugfix for exceptions whose messages contained unicode * `_done` callback no longer assumes you want your command put in the background. * `_done` callback is now called asynchronously in a separate thread. * `_done` callback is called regardless of exception, which is necessary in order to release held resources, for example a process pool
This was referenced Dec 16, 2016
This was referenced Jan 4, 2017
This was referenced Mar 30, 2017
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In cases where the command being executed spawns subcommands
we need to ensure they are all killed, so that the timeout
occurs when it should instead of waiting for the subprocesses to exit.
The problem I'm trying the fix can be reproduced by setting a timeout on a git clone, git calls subprocesses which prevent the timeout from working. Killing the process group should be safe because os.setsid() was called earlier so the commands have been started in their own group.