feat: package manager support #960

0618 · 2024-01-30T22:57:27Z

Problem

Branch poc/package-manager-support has the following major changes:

add a 'poc-e2e-flow-test' to run package_manager_sanity_checks.test.ts check against supported Package Managers
refactored npmPackageManagerController to packageManagerContollerFactory and packageManagerController. And moved them to cli-core
refactored backed-deploy to use packageManagerContollerFactory

Related PR:

TODO:

Issue number, if available:

Changes

Corresponding docs PR, if applicable:

Validation

Checklist

If this PR includes a functional change to the runtime behavior of the code, I have added or updated automated test coverage for this change.
If this PR requires a change to the Project Architecture README, I have included that update in this PR.
If this PR requires a docs update, I have linked to that docs PR above.
If this PR modifies E2E tests, makes changes to resource provisioning, or makes SDK calls, I have run the PR checks with the run-e2e label set.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

* feat: add env ci support * test: update unit tests * feat: create amplify --yes option * chore: remove un-used code * feat: support Panage Manager env var * chore: add comments * chore: remove env.CI * chore: update changeset * chore: rename PACKAGE_MANAGER_EXECUTABLE * fix: tests * fix npm -> npx

* test: init e2e flow test * fix GH hash * use dynamic pkg manager * setup nodejs with npm * setup more pkg managers * install pkg managers * fix yarn1 and change step name * fix yarn1 * fix yarn1 * split yarn1 into 2 steps * fix yarn1 * fix yarn1 win * exclude bun on windows * yarn1 windows * setup yarn3 * yarn3 * yarn3 yarnPath * yarn use 3rd party action * try yarn3 * set yarn 3.6.x * set yarn berry * set yarn stable and pass --yes * feat: add env ci support * test: update unit tests * feat: create amplify --yes option * chore: remove un-used code * feat: support Panage Manager env var * chore: add comments * chore: remove env.CI * chore: update changeset * chore: rename PACKAGE_MANAGER_EXECUTABLE * fix: tests * use Env Var PACKAGE_MANAGER_EXECUTABLE * fix npm -> npx, remove bun * fix npm -> npx * change -- --yes * chore: update package.lock * chore: remove @Alpha * fix: clean up yarn * fix: change yarn1 to yarn * chore: rename yarn step * chore: use env * fix: yarn-stable env var * add comments * chore: update sample app with new imports (#541) * feat: use "use client"; directive in generated react components (#540) * fix: flatten prop types auth (#534) * fix: flatten auth login with types * chore: update snapshots * chore: update api * chore: add changeset * chore: add TODO --------- Co-authored-by: Amplifiyer <51211245+Amplifiyer@users.noreply.github.com> Co-authored-by: Spencer Stolworthy <sstol@amazon.com> Co-authored-by: awsluja <110861985+awsluja@users.noreply.github.com>

* fix: npm_project_initializer to use env var * fix: pnpm init

* feat: setup GH workflow for create amplify * chore: changeset * fix: add pkg type for modern yarn * fix npm install * feat: test Node v20 and Node v19 * feat: test Node v20 and yarn-stable with Node 19 * feat: exclude yarn-stable * feat: include node 20 and exclude node 19 * feat: use node 20 for others, use node 19 for yarn-stable * chore: add TODO to use Node 20

* chore: change branch to poc/pms-create-amplify * fix: gitIgnore test * fix: refactor e2e for pms * chore: update package.lock * chore: update package.lock * temp: refactor * temp: run 1 test * temp: fix npx * temp: install packages for yarn * temp: update initial_project_file_generator * temp: install ts for yarn * temp: ignore node_modules and yarn.lock * temp: create yarn.lock for yarn stable * temp: fix yarn-stable init * temp: fix --help * fix: assert for gitignore * chore: change yarn stable install * fix: not install yarn stable globally * add @yarnpkg/sdks base * yarn stable use node-modules * remove emoji * remove @yarnpkg/sdks * yarn stable use node 18.18 * yarn stable use node 20 * chore: re-enable all tests * Revert "chore: re-enable all tests" This reverts commit f7a167b. * chore: uncomment test * chore: enable all initialStatues * test: change concurrencyLevel for yarn and yarn stable * add yarn-stable to test fails fast * chore: update changeset * chore: change workflow trigger * chore: yarn not install typescript in root folder * chore: update package.lock * Revert "chore: yarn not install typescript in root folder" This reverts commit 0d83dd0. * chore: remove comment * chore: fix typo * chore: rename yarn-classic and yarn modern

* chore: update package.lock * init * feat: dynamically get package manager * chore: change workflow trigger * chore: change workflow trigger * chore: rename npm functions * chore: update output log * fix: pnpm cache clear command * chore: refactor packageManager * fix: remove pnpm store * chore: create package_manager file * chore: change workflow trigger * chore: refactor PackageManager Controller and Initializer * fix: update types * chore: refactor to use factory * chore: remove try catch from the controllers * chore: change type * chore: use abstract class * fix tests * fix: PackageManagerBase * chore: refactor PackageManagerControllerFactory * chore: refactor packageJsonExists * chore: cleanup code * fix: projectJsonExists * remove package_manager file * chore: refactor package_manager props into package manager controllers * chore: rename packageManagerControllerFactory * chore: make projectRoot, userAgent, getPackageManagerName private * chore: refactor ensureInitialized * chore: refactor PackageManagerController to extend PackageManagerControllerFactory * Revert "chore: refactor PackageManagerController to extend PackageManagerControllerFactory" This reverts commit 7ddc64f. * chore: refactor to inject PackageManagerControllerFactory to xPackageManagerController * chore: move initialProjectFileGenerator to packageManagerController * fix: initializeAMplifyFolder * fix: generateInitialProjectFiles * fix: template path * chore: refactor getPackageManagerController yarn-modern * chore: create getWelcomeMessage * Revert "chore: refactor getPackageManagerController yarn-modern" This reverts commit 28da8a9. * chore: refactor generateInitialProjectFiles for yarn-modern * chore: cleanup code * chore: remove PackageManagerControllerFactory index * chore: remove ensureInitialized * Revert "chore: remove ensureInitialized" This reverts commit 96483ce. * Revert "Revert "chore: remove ensureInitialized"" This reverts commit 7c59c80. * temp: add initializeProject to PackageManagerController * temp: revert InitialProjectFileGenerator * fix: InitialProjectFileGenerator * test: restore initial_project_file_generator * move installDependencies and getWelcomeMessage to PackageManagerController * add addLockFile * add JSDocs and resolve some review comments * fix: create_amplify.test * chore: add addTypescript * chore: refactor generateInitialProjectFiles * Update packages/create-amplify/src/package-manager-controller/package_manager_controller_factory.ts Co-authored-by: Edward Foyle <foyleef@amazon.com> * chore: refactor generateInitialProjectFiles again * chore: refactor welcomeMessage * test: add package_manager_controller_factory * test: fix test types * chore: comments update * chore: handle process.env.npm_config_user_agent undefined * chore: move addLockFile and addTypescript into initializeTsConfig * test: refactor packageManagerControllerFactory * fix: yarn initializeTsConfig * fix: yarn modern initializeTsConfig * fix: amplify_project_creator test * chore: refactor contructor * chore: update package.lock * fix: pnpm init, remove --debug, etc * test: add test for NpmPackageManagerController * fix build error * chore: refactor NpmPackageManagerController * test: add test for xPackageManagerController * chore: convert fs to fsp * fix yarn modern --------- Co-authored-by: Edward Foyle <foyleef@amazon.com>

sobolk · 2024-01-31T18:33:03Z

scripts/components/dependencies_validator.ts

+      /**
+       * TODO: This is a temporary fix for execa version inconsistency. Remove this once execa version is fixed.
+       * Issue: https://github.com/aws-amplify/amplify-backend/issues/962
+       */
+      if (
+        dependencyVersionUsage.allVersions.size > 1 /* Issue-962 Start */ &&
+        dependencyName !== 'execa' /* Issue-962 End */
+      ) {


as for this one.

we should lift this as an option, so that it can be configured here

amplify-backend/scripts/check_dependencies.ts

Lines 5 to 18 in 9f1a78b

await glob('packages/*'),

{

'aws-amplify': { denyAll: true },

'@aws-amplify/datastore': { denyAll: true },

'@aws-amplify/core': { denyAll: true },

'@aws-amplify/cli-core': {

allowList: [

'@aws-amplify/backend-cli',

'@aws-amplify/sandbox',

'create-amplify',

],

},

},

[['aws-cdk', 'aws-cdk-lib']]

.

Can be a follow up.

…er_controller_factory.ts Co-authored-by: Kamil Sobol <sobkamil@amazon.com>

0618 · 2024-02-01T00:09:47Z

.changeset/new-timers-warn.md

+'@aws-amplify/plugin-types': minor
+'@aws-amplify/cli-core': minor
+'@aws-amplify/sandbox': minor
+'@aws-amplify/backend-cli': minor


are all of the versions minor?

given where we are today (0.x versions and added feature with api breaks) yes that seems ok.

0618 · 2024-02-01T00:11:30Z

.changeset/new-timers-warn.md

+'@aws-amplify/backend-cli': minor
+---
+
+Support more package managers, such as yarn (version 1 and version 2+), pnpm.


Do we also want to add several examples in the changelog?

I'm thinking of adding:

For example the following commands:
yarn create amplify
yarn amplify pipeline-deploy
pnpm create amplify
pnpm amplify pipeline-deploy

yes why not

however if you do you may want to split changeset into two files and add this only for CLI and create amplify

* add package-manager-tests to health-checks * remove poc-e2e-flow-test * add run_package_manager_tests_tests condition * fix run_package_manager_tests_tests condition * remove amplify-backend_windows-latest_8-core * use re-usable action for run-e2e * Revert "remove amplify-backend_windows-latest_8-core" This reverts commit 547834a. * Revert "use re-usable action for run-e2e" This reverts commit 3159e01. * revert re-usable action

.changeset/new-timers-warn.md

sobolk

Looks good.

sobolk · 2024-02-01T14:46:27Z

packages/cli-core/src/package-manager-controller/yarn_modern_package_manager_controller.test.ts

+      // const writeFileMock = mock.fn(() => Promise.resolve());
+      // mock.method(fsp, 'writeFile', writeFileMock);


please remove commented out code

Done 010277b

Co-authored-by: Kamil Sobol <sobkamil@amazon.com>

sobolk

Ship it!

                                  .                   .
                              _..-''"""\          _.--'"""\
                              |         L         |        \
                  _           / _.-.---.\.        / .-.----.\
                _/-|---     _/<"---'"""""\\`.    /-'--''"""""\
               |       \     |            L`.`-. |            L
               /_.-.---.L    |            \  \  `|            J`.
             _/--'""""  \    F            \L  \  |             L
               L      `. L  J  _.---.-"""-.\`. L_/ _.--|"""""--.\ `.
               |        \+. /-"__.--'""""   \ `./'"---'""""""   \`. `.
               F   _____ \ `F"        `.     \  \                L `.
              /.-'"_|---'"\ |           `    JL |                 L  `.`.
             <-'""         \|    _.-.------._ A J    _.-.-----`.--|   ``.`.
              L         `. |/.-'"_.-`---'""\."| /-'"---'"""""   \`.\.  \ `.`.
              |  _.------\.<'"""            L\ L\                `.`\`. \  `.
         _.-'//'"--'"""   L\|       ________\ `.F     ___.-------._L \ `-\   \`.
        /___| F             F _.--'"_|-------L  /_.-'"_.-|-'"""""""\  L   L   `.`.
            | F  _.-'|"""""/'"-'"""          J <'"""                L J   |     `.`.
            |/-'-''/|""\ )-|\                 F \                   |  L .'"""`\""-\\_
             F`-'-'-(`-')  | \                F  \                  |___`"""`.""`.-'"
------------/        `-'---|  F               L   L             __     |"""""`-'"__________
          .'_         |    |__L          __  J__  |    _.--'""""   `".----'".'
         '""""""""""""|--._+--F _.-'""||"   """___/.-'"   ||-'"/""""" \_. .'
         J------------(___\__/'_____.--------'-------'""""""""           /
         `-.                                        _.__.__.__.____     J_.-._
    .'`-._ (-`--`---.'--._`---._.-'`-._.-'_.-'``-._'               `-''-'

.changeset/new-timers-warn.md

edwardfoyle · 2024-02-01T17:35:09Z

.changeset/new-timers-warn.md

@@ -0,0 +1,11 @@
+---
+'@aws-amplify/integration-tests': minor


nit: this package is private so it doesn't need to be included in changesets

Removed 276b82a

edwardfoyle · 2024-02-01T17:54:30Z

packages/cli-core/src/package-manager-controller/execute_with_debugger_logger.test.ts

@@ -42,7 +42,7 @@ void describe(() => {
      throw new Error('test error');
    });

-    await assert.rejects(
+    assert.throws(


I guess if it works it works, but it seems like this should be reverted because executeWithDebugLogger is still returning a promise

I think it works because executeWithDebugLogger return a promise, but itself is not async because it doesn't have the async key word

edwardfoyle · 2024-02-01T17:58:14Z

packages/cli-core/src/package-manager-controller/yarn_modern_package_manager_controller.ts

+  };
+
+  /**
+   * addTypescript - initializes a tsconfig.json file in the project root


nit: either update or remove this jsdoc

Good catch. Done 223f9eb

edwardfoyle · 2024-02-01T18:09:11Z

packages/create-amplify/src/get_project_root.ts

 import { AmplifyPrompter, LogLevel } from '@aws-amplify/cli-core';
 import { printer } from './printer.js';

 /**
 * Returns the project root directory.
 */
 export const getProjectRoot = async () => {
-  const useDefault = process.env.npm_config_yes === 'true';
+  const argv = await yargs(process.argv.slice(2)).options({


is this our only usage of yargs in this package? If so, I'd suggest we remove it an just do a check for process.argv.some(arg => arg === '--yes'

I'd rather not.
This will break this:

amplify-backend/packages/integration-tests/src/test-live-dependency-health-checks/health_checks.test.ts

Line 45 in 348717b

await execa('npm', ['create', 'amplify', '--yes', '--', '--help'], {

And. we'll loose --help output :(

having said that, we should probably refactor how do we use yargs in this package.

but then, looking at sources on main I don't see yargs usage in that package's src but I can see yargs in dependencies.
So I wonder how this works now...

bcd0740fa72c:testapp004 sobkamil$ npm create amplify@latest -- --help ? Where should we create your project? (.)

bcd0740fa72c:testapp004 sobkamil$ npm create amplify@0.4.4 -- --help Options: --help Show help [boolean] --version Show version number [boolean] --debug [boolean] [default: false]

Looks like it regressed...

btw. that regression happened on main so if we want to fix it it can be a follow up.

Can't reproduce the regression on amplify@0.6.0, which is this PR ✌️

I'm good with cutting a ticket to track and doing this in a follow up

Issue to remove yargs in a follow-up PR #972

Co-authored-by: Edward Foyle <foyleef@amazon.com>

0618 and others added 30 commits October 30, 2023 11:51

Merge branch 'main' into poc/package-manager-support

b155096

Merge branch 'main' into poc/package-manager-support

3f37820

Merge branch 'main' into poc/package-manager-support

3799a9c

Merge branch 'main' into poc/package-manager-support

3940efa

fix: pnpm init and env var (#563)

6c0036f

* fix: npm_project_initializer to use env var * fix: pnpm init

Merge branch 'main' into poc/package-manager-support

6cb29c6

Merge branch 'main' into poc/package-manager-support

7634c37

Merge branch 'main' into poc/package-manager-support

cb56b3d

Merge branch 'main' into poc/package-manager-support

cbed3f4

chore: update package.lock

6750713

Merge branch 'update-package-lock' into poc/package-manager-support

43d8da3

Merge branch 'main' into poc/package-manager-support

4b30f2a

chore: update package.lock

d5eab4a

Merge branch 'update-package-lock' into poc/package-manager-support

e2bbafc

Merge branch 'main' into poc/package-manager-support

c1edfdf

Merge branch 'main' into poc/package-manager-support

5dbbc62

Merge branch 'main' into poc/package-manager-support

8943ac5

Merge branch 'main' into poc/package-manager-support

ddbc3c5

Merge branch 'main' into poc/package-manager-support

fd3defe

Merge branch 'main' into poc/package-manager-support and use main

6bff8b4

fix(create_amplify.test): after merge main

4c77c70

fix the concurrency for yarn classic and modern, and fix expect

0413a9e

Merge branch 'main' into poc/package-manager-support

af81b8e

Merge branch 'main' into poc/package-manager-support

b6a0050

Merge branch 'main' into poc/package-manager-support

2f51f94

0618 added 4 commits January 30, 2024 21:41

fix getCommand, cdk_deployer.test, amplify_project_creator.test

e6f143e

fix initial_project_file_generator.test

51d6dad

fix executeWithDebugLogger test

99c1b0b

fix YarnModernPackageManagerController test

4bfda61

sobolk reviewed Jan 31, 2024

View reviewed changes

0618 and others added 5 commits January 31, 2024 10:47

fix packageManagerControllerFactory test

4ee0cf2

fix lint error

ead5864

Update packages/cli-core/src/package-manager-controller/package_manag…

c4a03ad

…er_controller_factory.ts Co-authored-by: Kamil Sobol <sobkamil@amazon.com>

remove never used packageManagerExecutable

e344b5b

update changeset

a4011c4

0618 commented Feb 1, 2024

View reviewed changes

0618 added the run-e2e Label that will include e2e tests in PR checks workflow label Feb 1, 2024

sobolk reviewed Feb 1, 2024

View reviewed changes

.changeset/new-timers-warn.md Outdated Show resolved Hide resolved

sobolk previously approved these changes Feb 1, 2024

View reviewed changes

Update .changeset/new-timers-warn.md

1f297f0

Co-authored-by: Kamil Sobol <sobkamil@amazon.com>

0618 dismissed sobolk’s stale review via 1f297f0 February 1, 2024 15:33

0618 added 2 commits February 1, 2024 07:34

clean up commented code

010277b

use amplify-backend_windows-latest_8-core

4f85b3d

sobolk previously approved these changes Feb 1, 2024

View reviewed changes

edwardfoyle reviewed Feb 1, 2024

View reviewed changes

Update .changeset/new-timers-warn.md

e98d642

Co-authored-by: Edward Foyle <foyleef@amazon.com>

0618 dismissed sobolk’s stale review via e98d642 February 1, 2024 19:11

0618 added 2 commits February 1, 2024 11:20

remove inaccurate JSdoc

223f9eb

remove integration-tests from changeset

276b82a

edwardfoyle approved these changes Feb 1, 2024

View reviewed changes

0618 mentioned this pull request Feb 1, 2024

remove yargs from create-amplify #972

Open

sobolk approved these changes Feb 1, 2024

View reviewed changes

sobolk merged commit b73d76a into main Feb 1, 2024
33 of 36 checks passed

sobolk deleted the poc/package-manager-support branch February 1, 2024 20:00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: package manager support #960

feat: package manager support #960

0618 commented Jan 30, 2024 •

edited

Loading

sobolk Jan 31, 2024

sobolk Jan 31, 2024

0618 Feb 1, 2024

sobolk Feb 1, 2024

0618 Feb 1, 2024

sobolk Feb 1, 2024

sobolk Feb 1, 2024

sobolk left a comment

sobolk Feb 1, 2024

0618 Feb 1, 2024

sobolk left a comment

edwardfoyle Feb 1, 2024

0618 Feb 1, 2024

edwardfoyle Feb 1, 2024

0618 Feb 1, 2024

edwardfoyle Feb 1, 2024

0618 Feb 1, 2024

edwardfoyle Feb 1, 2024

sobolk Feb 1, 2024

sobolk Feb 1, 2024

sobolk Feb 1, 2024

sobolk Feb 1, 2024

sobolk Feb 1, 2024

0618 Feb 1, 2024

edwardfoyle Feb 1, 2024

0618 Feb 1, 2024 •

edited

Loading

	await glob('packages/*'),
	{
	'aws-amplify': { denyAll: true },
	'@aws-amplify/datastore': { denyAll: true },
	'@aws-amplify/core': { denyAll: true },
	'@aws-amplify/cli-core': {
	allowList: [
	'@aws-amplify/backend-cli',
	'@aws-amplify/sandbox',
	'create-amplify',
	],
	},
	},
	[['aws-cdk', 'aws-cdk-lib']]

		// const writeFileMock = mock.fn(() => Promise.resolve());
		// mock.method(fsp, 'writeFile', writeFileMock);

feat: package manager support #960

feat: package manager support #960

Conversation

0618 commented Jan 30, 2024 • edited Loading

Problem

Changes

Validation

Checklist

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sobolk left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sobolk left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

0618 Feb 1, 2024 • edited Loading

Choose a reason for hiding this comment

0618 commented Jan 30, 2024 •

edited

Loading

0618 Feb 1, 2024 •

edited

Loading