Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FR(python): update owlbot_main() to automatically sort python imports using isort with --fss option #1401

Closed
parthea opened this issue Apr 2, 2022 · 0 comments · Fixed by #1402
Labels
lang: python Issues specific to Python. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@parthea
Copy link
Contributor

parthea commented Apr 2, 2022

There are at least 2 benefits of using isort to automatically sort python imports in googleapis python-* repos.

  • Maintainers and contributors no longer have to resolve lint errors related to out of order imports
  • File sizes will decrease as imports are automatically grouped together. Smaller file sizes improve code readability.

For example, The __init__.py file in the python-dataproc repo decreased by 40% after running isort

partheniou@partheniou-vm-2:~/git/python-dataproc/google/cloud/dataproc$ ls -la __init__.py 
-rw-r----- 1 partheniou primarygroup 13586 Apr  2 16:39 __init__.py

partheniou@partheniou-vm-2:~/git/python-dataproc/google/cloud/dataproc$ isort --fss __init__.py 
Fixing /usr/local/google/home/partheniou/git/python-dataproc/google/cloud/dataproc/__init__.py

partheniou@partheniou-vm-2:~/git/python-dataproc/google/cloud/dataproc$ black __init__.py 
reformatted __init__.py

All done! ✨ 🍰 ✨
1 file reformatted.
partheniou@partheniou-vm-2:~/git/python-dataproc/google/cloud/dataproc$ ls -la __init__.py 
-rw-r----- 1 partheniou primarygroup 8194 Apr  2 16:39 __init__.py

Note: The lint nox session in the noxfile in the python samples templated files expects strict alphabetical ordering for imports so we need to use the -fss option of isort . From the docs on the --fss option:

Don't sort straight-style imports (like import sys) before from-style imports (like from itertools import groupby). Instead, sort the imports by module, independent of import style.

The lint session expects strict alphabetical order:

import glob
import os
from pathlib import Path
import sys
from typing import Callable, Dict, List, Optional

not

import glob
import os
import sys
from pathlib import Path
from typing import Callable, Dict, List, Optional

Without the --fss option, here is the error that we see in lint

partheniou@partheniou-vm-2:~/git/python-dataproc/samples/snippets$ nox -s lint
nox > Running session lint
nox > Creating virtual environment (virtualenv) using python3.9 in .nox/lint
nox > python -m pip install flake8 flake8-import-order
nox > flake8 --show-source --builtin=gettext --max-complexity=20 --import-order-style=google '--exclude=.nox,.cache,env,lib,generated_pb2,*_pb2.py,*_pb2_grpc.py' --ignore=E121,E123,E126,E203,E226,E24,E266,E501,E704,W503,W504,I202 --max-line-length=88 --application-import-names list_clusters,submit_job,update_cluster_test,update_cluster,quickstart,pyspark_sort,submit_job_to_cluster,instantiate_inline_workflow_template,create_cluster,submit_job_test,.nox,instantiate_inline_workflow_template_test,dataproc_e2e_donttest,single_job_workflow,pyspark_sort_gcs,create_cluster_test,noxfile,noxfile_config .
./noxfile.py:20:1: I100 Import statements are in the wrong order. 'from pathlib import Path' should be before 'import sys'
from pathlib import Path
^
@parthea parthea added type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. lang: python Issues specific to Python. labels Apr 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lang: python Issues specific to Python. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant