Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Code Review PR for Initial Commit #10

Closed
wants to merge 70 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
776a407
Create the setup-webapp action (#1)
ktmud Apr 11, 2020
bd1bd45
Add GitHub Actions build icon
ktmud Apr 11, 2020
98e0b4e
Add link to the badge icon
ktmud Apr 11, 2020
105a2a3
Add renovate.json (#2)
renovate[bot] Apr 11, 2020
e0e972c
Pin dependencies (#3)
renovate[bot] Apr 11, 2020
244e97f
Update @actions/cache commit hash to 78809b9 (#4)
renovate[bot] Apr 11, 2020
92c2d94
Add LICENSE
ktmud Apr 11, 2020
4b8c1c0
Minor update to README
ktmud Apr 11, 2020
748b9f7
Fix default run commands
ktmud Apr 11, 2020
f0cde56
Change how @actions/cache are imported
ktmud Apr 11, 2020
86221b8
Add codecov badge
ktmud Apr 11, 2020
0ef973d
Update dependency @actions/core to v1.2.3 (#5)
renovate[bot] Apr 11, 2020
0e639ef
Update dependency @types/jest to v25 (#6)
renovate[bot] Apr 11, 2020
c7960dc
Update dependency prettier to v2 (#7)
renovate[bot] Apr 11, 2020
53334a2
Update readme
ktmud Apr 11, 2020
50d9302
Change some defaults
ktmud Apr 11, 2020
9f6bc83
Fix reverted save and restore
ktmud Apr 11, 2020
6edbda4
Minor update
ktmud Apr 11, 2020
716b5ae
Use a patch script to load modules
ktmud Apr 12, 2020
e9dd325
Update badge icon path
ktmud Apr 12, 2020
6c9f39c
Update persistent storage in patch
ktmud Apr 12, 2020
853d25d
Add log groups
ktmud Apr 12, 2020
f41787f
Optimize console logs
ktmud Apr 12, 2020
d78992b
Add groups to bash commands, too
ktmud Apr 12, 2020
0ce476f
Optimize output
ktmud Apr 12, 2020
da85eaf
Add yarn install
ktmud Apr 12, 2020
6e0860a
Update docs
ktmud Apr 12, 2020
d7070f4
update doc
ktmud Apr 12, 2020
713b7d4
Pin dependencies (#9)
renovate[bot] Apr 12, 2020
a9dc3db
Fix pip cache path
ktmud Apr 12, 2020
5db7e72
Update docs and some defaults
ktmud Apr 12, 2020
d7c370a
Read cache state based on unique key to avoid collision
ktmud Apr 12, 2020
e9a7924
Better test cases
ktmud Apr 12, 2020
093acef
Fix a typo
ktmud Apr 12, 2020
fbcd5b0
Change default hashfiles for npm and yarn
ktmud Apr 12, 2020
e9cdd76
Update docs
ktmud Apr 12, 2020
1d9cb40
Include path in hash key
ktmud Apr 13, 2020
8b4fdcf
Allow vanila strings in cache configs
ktmud Apr 14, 2020
ddf7d7f
Add debug log for state persist
ktmud Apr 14, 2020
d053c79
Update typescript-eslint monorepo to v2.28.0 (#11)
renovate[bot] Apr 14, 2020
542e215
Fix code quality issue identified by Codacy
ktmud Apr 14, 2020
5a1ac90
Exit on bash errors
ktmud Apr 15, 2020
adc6f73
Pin dependency prettier-plugin-packagejson to 2.2.0 (#13)
renovate[bot] Apr 15, 2020
bcf1d01
Update dependency eslint to v7 (#26)
renovate[bot] May 8, 2020
9354908
Update typescript-eslint monorepo to v3 (major) (#31)
renovate[bot] Jun 29, 2020
a899f35
Update dependency jest to v26 (#25)
renovate[bot] Jun 29, 2020
a0a864e
Update dependency prettier to v2.0.5 (#19)
renovate[bot] Jun 29, 2020
94e268e
Update dependency ts-jest to v26 (#29)
renovate[bot] Jun 29, 2020
90b77de
Update dependency @types/node to v12.12.47 (#15)
renovate[bot] Jun 29, 2020
b384315
Update dependency @zeit/ncc to v0.22.3 (#30)
renovate[bot] Jun 29, 2020
8948297
Update dependency @actions/core to v1.2.4 (#23)
renovate[bot] Jun 29, 2020
0dc1a0a
Update typescript-eslint monorepo to v3.5.0 (#36)
renovate[bot] Jun 29, 2020
c0e6f22
Update dependency ts-jest to v26.1.4 (#43)
renovate[bot] Jul 29, 2020
860ed86
Update dependency eslint-plugin-jest to v23.19.0 (#40)
renovate[bot] Jul 29, 2020
6d0fd16
Update typescript-eslint monorepo to v3.7.1 (#41)
renovate[bot] Jul 29, 2020
e136cf8
Update dependency @types/jest to v26 (#35)
renovate[bot] Jul 29, 2020
2c24103
Update @actions/cache commit hash to d29c1df (#47)
renovate[bot] Jul 29, 2020
45c514f
Update dependency tempy to v0.6.0 (#44)
renovate[bot] Jul 29, 2020
9109c07
Update dependency @types/node to v12.12.53 (#42)
renovate[bot] Jul 29, 2020
3745d67
Update dependency eslint to v7.5.0 (#38)
renovate[bot] Jul 29, 2020
39e26aa
Update dependency prettier-plugin-packagejson to v2.2.5 (#16)
renovate[bot] Jul 29, 2020
622d6c8
Update dependency typescript to v3.9.7 (#27)
renovate[bot] Jul 29, 2020
c974a3e
Update dependency js-yaml to v3.14.0 (#32)
renovate[bot] Jul 29, 2020
d28f64d
Update dependency @actions/exec to v1.0.4 (#21)
renovate[bot] Jul 29, 2020
b90713b
Update dependency @types/uuid to v7.0.4 (#20)
renovate[bot] Jul 29, 2020
b782557
Update dependency eslint-plugin-jest to v23.20.0 (#48)
renovate[bot] Dec 7, 2020
24800c3
Update typescript-eslint monorepo to v3.10.1 (#52)
renovate[bot] Dec 7, 2020
8b9ac14
Update dependency typescript to v4 (#59)
renovate[bot] Dec 7, 2020
3e08c26
Update dependency @actions/core to v1.2.6 (#61)
renovate[bot] Dec 7, 2020
064315d
Update dependency tempy to v0.7.1 (#64)
renovate[bot] Jan 16, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
indent_size = 2
3 changes: 3 additions & 0 deletions .eslintignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
dist/
lib/
node_modules/
26 changes: 26 additions & 0 deletions .eslintrc.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
module.exports = {
plugins: ['jest', '@typescript-eslint'],
extends: ['plugin:jest/all'],
parser: '@typescript-eslint/parser',
parserOptions: {
ecmaVersion: 9,
sourceType: 'module',
},
rules: {
'eslint-comments/no-use': 'off',
'import/no-namespace': 'off',
'no-unused-vars': 'off',
'no-console': 'off',
'jest/prefer-expect-assertions': 'off',
'jest/no-disabled-tests': 'warn',
'jest/no-focused-tests': 'error',
'jest/no-identical-title': 'error',
'jest/prefer-to-have-length': 'warn',
'jest/valid-expect': 'error',
},
env: {
node: true,
es6: true,
'jest/globals': true,
},
};
34 changes: 34 additions & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
name: Tests
on:
pull_request:
paths-ignore:
- '**.md'
push:
branches:
- master
paths-ignore:
- '**.md'
jobs:
test:
strategy:
matrix:
os: [ubuntu-latest, macOS-latest]
name: Test on ${{ matrix.os }}
runs-on: ${{ matrix.os }}

steps:
- uses: actions/checkout@v1
- uses: actions/setup-node@v1
with:
node-version: '12.x'
- name: Install dependencies
run: npm ci
- name: Run prettier format check
run: npm run format-check
- name: Build
run: npm run build
- name: Run tests
run: npm run test
- name: Upload code coverage
run: |
bash <(curl -s https://codecov.io/bash)
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,3 @@ node_modules

!dist
!dist/cache

3 changes: 3 additions & 0 deletions .prettierignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
dist/
lib/
node_modules/
11 changes: 11 additions & 0 deletions .prettierrc.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"printWidth": 80,
"tabWidth": 2,
"useTabs": false,
"semi": true,
"singleQuote": true,
"trailingComma": "all",
"bracketSpacing": true,
"arrowParens": "avoid",
"parser": "typescript"
}
22 changes: 22 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@

The MIT License (MIT)

Copyright (c) 2018 GitHub, Inc. and contributors

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
212 changes: 212 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,212 @@
# cached-dependencies

[![](https://github.com/ktmud/cached-dependencies/workflows/Tests/badge.svg)](https://github.com/ktmud/cached-dependencies/actions?query=workflow%3ATests) [![codecov](https://codecov.io/gh/ktmud/cached-dependencies/branch/master/graph/badge.svg)](https://codecov.io/gh/ktmud/cached-dependencies)

Enable **multi-layer cache** and **shortcut commands** in any workflows.

Manage multiple cache targets in one step. Use either the built-in cache configs for npm, yarn, and pip, or write your own. Create a bash command library to easily reduce redudencies across workflows. Most useful for building webapps that require multi-stage building processes.

This is your all-in-one action for everything related to setting up dependencies with cache.

## Inputs

- **run**: bash commands to run, allows shortcut commands
- **caches**: path to a JS module that defines cache targets, defaults to `.github/workflows/caches.js`
- **bashlib**: path to a BASH scripts that defines shortcut commands, defaults to `.github/workflows/bashlib.sh`
- **parallel**: whether to run the commands in parallel with node subprocesses

## Examples

Following workflow sets up dependencies for a typical Python web app with both `~/.pip` and `~/.npm` cache configured in one simple step:

```yaml
jobs:
build_and_test:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v2
- name: Install dependencies
uses: ktmud/cached-dependencies@v1
with:
run: |
npm-install
npm run build

pip-install
python ./bin/manager.py fill_test_data
```

Here we used predefined `npm-install` and `pip-install` commands to install dependencies with correponding caches.

You may also replace `npm-install` with `yarn-install` to install npm pacakges with `yarn.lock`.

```yaml
- name: Install dependencies
uses: ktmud/cached-dependencies@v1
with:
run: |
yarn-install
yarn build

pip-install
python ./bin/manager.py fill_test_data
```

See below for more details.

## Usage

### Cache configs

Under the hood, we use [@actions/cache](https://github.com/marketplace/actions/cache) to manage cache storage. But instead of defining only one cache at a time and specify them in workflow YAMLs, you manage all caches in a spearate JS file: `.github/workflows/caches.js`.

Here is [the default configuration](https://github.com/ktmud/cached-dependencies/blob/master/src/cache/caches.ts) for Linux:

```js
module.exports = {
pip: {
path: [`${process.env.HOME}/.cache/pip`],
hashFiles: ['requirements*.txt'],
keyPrefix: 'pip-',
restoreKeys: 'pip-',
},
npm: {
path: [`${HOME}/.npm`],
hashFiles: [
`package-lock.json`,
`*/*/package-lock.json`,
`!node_modules/*/package-lock.json`,
],
},
yarn: {
path: [`${HOME}/.npm`],
// */* is for supporting lerna monorepo with depth=2
hashFiles: [`yarn.lock`, `*/*/yarn.lock`, `!node_modules/*/yarn.lock`],
},
}
```

In which `hashFiles` and `keyPrefix` will be used to compute the primary cache key used in [@actions/cache](https://github.com/marketplace/actions/cache). `keyPrefix` will default to `${cacheName}-` and `restoreKeys` will default to `keyPrefix` if not specified.

It is recommended to always use absolute paths in these configs so you can share them across different worflows more easily (in case you the action is called from different working directories).

#### Speficy when to restore and save

With the predefined `cache-store` and `cache-save` bash commands, you have full flexibility on when to restore and save cache:

```yaml
steps:
- uses: actions/checkout@v2
- uses: ktmud/cached-dependencies@v1
with:
run: |
cache-restore npm
npm install
cache-save npm

cache-restore pip
pip install -r requirements.txt
cache-save pip
```

### Shortcut commands

All predefined shortcut commands can be found [here](https://github.com/ktmud/cached-dependencies/blob/master/src/scripts/bashlib.sh). You can also customize them or add new ones in `.github/workflows/bashlib.sh`.

For example, if you want to install additional packages for before saving `pip` cache, simply add this to the `bashlib.sh` file:

```bash
# override the default `pip-install` command
pip-install() {
cd $GITHUB_WORKSPACE

cache-restore pip

echo "::group::pip install"
pip install -r requirements.txt # prod requirements
pip install -r requirements-dev.txt # dev requirements
pip install -e ".[postgres,mysql]" # current pacakge with some extras
echo "::endgroup::"

cache-save pip
}
```

### Default setup command

When `run` is not provided:

```yaml
jobs:
name: Build
steps:
- name: Install dependencies
uses: ktmud/cached-depdencies@v1
```

You must provide a `default-setup-command` in the bashlib. For example,

```bash
default-setup-command() {
pip-install & npm-install
}
```

This will start installing pip and npm dependencies at the same time.

### Customize config locations

Both the two config files, `.github/workflows/bashlib.sh` and `.github/workflows/caches.js`, can be placed in other locations:

```yaml
- uses: ktmud/cached-dependencies@v1
with:
caches: ${{ github.workspace }}/.github/configs/caches.js
bashlib: ${{ github.workspace }}/.github/configs/bashlib.sh
```

### Run commands in parallel

When `parallel` is set to `true`, the `run` input will be split into an array of commands and passed to `Promise.all(...)` to execute in parallel. For example,

```yaml
- uses: ktmud/cached-dependencies@v1
with:
parallel: true
run: |
pip-install
npm-install
```

is equivalent to

```yaml
- uses: ktmud/cached-dependencies@v1
with:
run: |
pip-install & npm-install
```

If one or more of your commands must spread across multiple lines, you can add a new line between the parallel commands. Each command within a parallel group will still run sequentially.

```yaml
- uses: ktmud/cached-dependencies@v1
with:
run: |
cache-restore pip
pip install requirements*.txt
# additional pip packages
pip install package1 package2 pacakge2
cache-save pip

npm-install

cache-restore cypress
cd cypress/ && npm install
cache-save cypress
```

## License

This project is released under [the MIT License](LICENSE).
Loading