Skip to content

Poe the Poet ‐ An intro & cookbook

Jump to bottom
William B edited this page May 31, 2024 · 3 revisions

Poe the Poet is a fully featured task runner that integrates with Poetry as a plugin and is flexible enough to be run as a standalone CLI. We define our tasks in pyproject.toml and call them with the poe command.

Running a task

To run a task simply call poe <task_name>.

If you're not sure what to run, just call poe to get a list of available tasks that have been defined for the project.

Defining tasks

There are two main ways to define a task.

As a fully quantified table

Full tables provide a more verbose and structured way of defining tasks. They are most appropriate when tasks are not simple one liners and require documentation or additional configuration, such as defining arguments that a user can pass to the task via CLI.

# Define a task called greet-me
[tool.poe.tasks.greet-me]
help = "Prints 'Hello {username}'" # Document it
cmd  = "echo 'Hello ${username}'" # Use a shell-less subprocess to execute the command

	# Define args for the greet-me task
	[tool.poe.tasks.greet-me.args.username]
	help    = "The name of the user to greet." # Document it
	options = ["-u", "--username"] # Provide options to pass the argument in from CLI
	type     = "string"
	required = true

As keys in the [tool.poe.tasks] table

Simple one liner tasks can be defined as keys under the [tool.poe.tasks] table. When a task is defined this way, i.e. without a task type specified, it defaults to running as a cmd task.

[tool.poe.tasks]
hello      = "echo Hello" 
search-csv = "python scripts/search_csv.py"

Inline Tables - Why styling matters

When tasks become more complex, e.g. the greet-me task we defined before, we can use Inline Table styling to achieve the same result. However doing so can quickly make tasks difficult to read.

To demonstrate this, let's re-write the greet-me task as an inline table under the [tool.poe.tasks] table.

[tool.poe.tasks]
greet-me = { help = "Prints 'hello {username}", cmd = "echo 'Hello", args = { help = "The name of the user to greet.", options = ["-u", "--username"], type = "string",},}

As you can see, it is an absolute mess. Naturally you might expect the ability to JSON-ify the structure but due to TOML's multi-line restrictions, when using inline tables specifically, we cannot embark on such a sanity restoring journey.

[tool.poe.tasks] # Wouldn't this be nice?
	greet-me = { 
	help = "Prints 'hello {username}", 
	cmd  = "echo 'Hello", 
	args = { 
		help    = "The name of the user to greet.", 
		options = ["-u", "--username"], 
		type    = "string"
	},
}

The only exception to the multi-line restriction is when a value is a multiline string.

# Even more of a mess
[tool.poe.tasks]
greet-me = { help = """
	Prints 'hello {username}
""", cmd = "echo 'Hello", args = { help = "The name of the user to greet.", options = ["-u", "--username"], type = "string",},}

Why should I care? It sounds like we shouldn't use them

There are instances where inline tables can be used, albeit sparingly, to make a task definition more succinct. Below is a typical task to run our apps, allowing the user to specify the port and host that the app should run on.

[tool.poe.tasks.run-dev]
help  = "Runs the application."
shell = "flask run -p ${port} --host=${host}"

	[tool.poe.tasks.run-dev.args.port]
	help    = "The port to run the app on"
	options = ["-p", "--port"]
	default = "localhost"
	type    = "string"
	
	[tool.poe.tasks.run-dev.args.host]
	help    = "The host to run the app on"
	options = ["-h", "--host"]
	default = "6012"
	type    = "string"

Due to the simplicity of this task, defining the type and help sections aren't necessary as the reader can easily understand what these parameters are for. We can re-write this using inline tables, saving both space and making it easier to read at a glance.

[tool.poe.tasks.run-dev]
help      = "Runs the application."
shell     = "flask run -p ${port} --host=${host}"
args.port = { options = ["--port", "-p"], default = "6012"}
args.host = { options = ["--host", "-h"], default = "localhost"}

At the end of the day their use is subjective, and depends on your judgement to balance readability and consistency.

Task cookbook

What follows are some basic examples of how to use each of the task types available in Poe the Poet. They are relatively high level, so it's encouraged to checkout the corresponding documentation linked in the headings to learn more about them.

Command tasks - cmd

Used for simple commands that are executed as a subprocess without a shell.

[tool.poe.tasks.test]
help = "Runs the suite of unit tests for both python and JS code."
cmd  = "./scripts/run_tests.sh"

Shell tasks - shell

for scripts to be executed with via an external interpreter (such as sh).

[tool.poe.tasks.run-dev]
help      = "Runs the application."
shell     = "flask run -p ${port} --host=${host}"
args.port = { options = ["--port", "-p"], default = "6012"}
args.host = { options = ["--host", "-h"], default = "localhost"}

[tool.poe.tasks.babel]
help = "Compiles app translations from fr.csv"
# Task bodies can be multi-line
shell = """ 
	python scripts/generate_en_translations.py
	csv2po app/translations/csv/en.csv app/translations/en/LC_MESSAGES/messages.po
	csv2po app/translations/csv/fr.csv app/translations/fr/LC_MESSAGES/messages.po
	pybabel compile -d app/translations
"""

Script tasks - script

Used to invoke a callable python function in a given script.

[tool.poe.tasks.search-csv]
help         = "Searches translations for the specified list of strings"
script       = "scripts.search_csv:search_translation_strings(search_terms=keywords)"
print_result = true
	
	[tool.poe.tasks.search-csv.args.keywords]
	help     = "Comma separated list of keywords to search by."
	options  = ["-k", "--keywords"]
	type     = "string"
	required = true

Note that we do not reference the keywords argument with ${} like in the Shell example this is intentional as it is not permitted when passing arguments as python params.

Styling tip We could leverage an inline-table for the argument here, but we need to inform the user what input is expected. Stuffing that into a one-liner will get messy, so using a sub-table is more appropriate here.

Sequence tasks - sequence

Used to compose multiple tasks into a sequence of tasks. We can combine any number or type of task into a sequence.

For this example, let's take the babel task from the Shell Tasks section and split it into a sequence of tasks.

[tool.poe.tasks]
# Define the tasks we want to use in this sequence
_generate-translations = "python scripts/generate_en_translations.py"
_convert2po-en         = "csv2po app/translations/csv/en.csv app/translations/en/LC_MESSAGES/messages.po"
_convert2po-fr         = "csv2po app/translations/csv/fr.csv app/translations/fr/LC_MESSAGES/messages.po"
_babel-compile         = "pybabel compile -d app/translations"
# The task that gets called is just an array of other tasks
babel                  = ["_generate-translations", "_convert2po-en", "_convert2po-fr", "_babel-compile"]

By default, when a task fails, the execution of the sequence is stopped. See the Continue sequence on task failure docs to learn more about configuring this behaviour.

Tasks prefixed with an _ are considered "private" or "helper tasks" and cannot be called directly via the CLI.

Expression tasks - expr

Used to evaluate python expressions within the task.

Let's build upon the search-csv script task. We know that the user should input a comma separated list of words for the keywords argument. We can convert this to an expression task to validate if the keywords argument is in the correct format before calling the python function.

[tool.poe.tasks.search-csv]
    expr    = """(
        scripts.search_csv.search_translation_strings(search_terms=keywords)
        if re.fullmatch(r'^([^,]+)(,[^,]+)*$', keywords)
        else f'Invalid search terms: {keywords}. Please provide a comma separated list of keywords.'
    )"""
    assert  = true
    imports = ["scripts.search_csv", "re"]

        [tool.poe.tasks.search-csv.args.keywords]
        help = "Comma separated list of keywords to search by. e.g: user,service,contact"
        options = ["-k", "--keywords"]
        required = true

Switch tasks - switch

For running different tasks depending on a control value.

Typically when switching branches you'll want to run poetry install before you spin up the app to ensure you have the needed dependencies. However if you forget to check and update the lock file with poetry check --lock and/or poetry lock --no-update then you're hit with a stark reminder of your forgetfulness:

Error: poetry.lock is not consistent with pyproject.toml. Run '`poetry' lock to fix it.

Let's create a switch task to make poetry install a bit better so we can work smarter, not harder.

[tool.poe.tasks]
# Captures the output of poetry check --lock avoiding non-zero exit codes that could prematurely fail a task utilizing this helper task.
_lock-state-output = "echo $(poetry check --lock 2>&1)"

[tool.poe.tasks.build]
control.expr = """(
	'inconsistent'
	if ${LOCK_STATE}.find('poetry.lock is not consistent with pyproject.toml') != -1
	else 'consistent'
)"""
uses = { LOCK_STATE = "_lock-state-output" }

	# If the lock file is out of date, update it before we call poetry install
	[[tool.poe.tasks.build.switch]]
	case = "inconsistent"
	shell = """
		poetry lock --no-update
		poetry install
	"""
	# Otherwise just install the deps!
	[[tool.poe.tasks.build.switch]]
	case = "consistent"
	cmd = "poetry install"

Reference tasks - ref 

For defining a task as an alias of another task, such as in a sequence task.

We won't go into this one because it's rarely used on its own.

Clone this wiki locally