Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cli(extra-headers): Enable sending additional HTTP Headers via the CLI #3732

Merged
merged 14 commits into from
Jan 19, 2018

Conversation

rupesh1
Copy link
Contributor

@rupesh1 rupesh1 commented Nov 3, 2017

I can across #2746 which mentioned a SO question, that had a link to commit by @fdn - after taking a look it seemed abandoned but functional. So based off their initial commit I've:

  • merged in master (had to resolve some conflicts - initial commit was v1.6.0)
  • fixed unit tests
  • updated some references and function locations to match the current codebase
  • added unit tests and basic docs

I've tested the CLI locally and it's working as expected for me.

P.s. I had to run yarn install-all a few times for it to run cleanly hence adding yarn-error.log to the gitignore

@googlebot
Copy link

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed, please reply here (e.g. I signed it!) and we'll verify. Thanks.


  • If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check your existing CLA data and verify that your email is set on your git commits.
  • If your company signed a CLA, they designated a Point of Contact who decides which employees are authorized to participate. You may need to contact the Point of Contact for your company and ask to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the project maintainer to go/cla#troubleshoot.
  • In order to pass this check, please resolve this problem and have the pull request author add another comment and the bot will run again.

@@ -14,7 +14,8 @@ import {GetValidOutputOptions, OutputMode} from './printer';
export interface Flags {
port: number, chromeFlags: string, output: any, outputPath: string, saveArtifacts: boolean,
saveAssets: boolean, view: boolean, maxWaitForLoad: number, logLevel: string,
hostname: string, blockedUrlPatterns: string[], enableErrorReporting: boolean
hostname: string, blockedUrlPatterns: string[], enableErrorReporting: boolean,
extraHeaders: any
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: Hmm this should probably be of type string

@rupesh1
Copy link
Contributor Author

rupesh1 commented Nov 6, 2017

I signed it!

@googlebot
Copy link

So there's good news and bad news.

👍 The good news is that everyone that needs to sign a CLA (the pull request submitter and all commit authors) have done so. Everything is all good there.

😕 The bad news is that it appears that one or more commits were authored by someone other than the pull request submitter. We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that here in the pull request.

Note to project maintainer: This is a terminal state, meaning the cla/google commit status will not change from this State. It's up to you to confirm consent of the commit author(s) and merge this pull request when appropriate.

@fdn
Copy link

fdn commented Nov 7, 2017

Merge away. Glad this commit is going to see the light of day! Good stuff @rupesh1

@johntron
Copy link

@paulirish This PR looks useful for both CLI and programmatic use. If the conflicts are resolved, is there a reason it can't be merged? I know there were other issues where we were discussing using Chrome's setCookie.

@johnboxall
Copy link
Contributor

This feature would be handy for us: we have conditionals in our code the can be activated/deactivated using HTTP Headers. Specifically cookies.

Our current workflow is:

  1. Launch an instance of Chrome: ./node_modules/.bin/chrome-debug. Note the port number!
  2. In the Chrome instance, set the state you want to test. For example, navigate to a page and set cookies using the devtools.
  3. Clear cached images / files via chrome://settings/clearBrowserData?search=cache
  4. In another terminal, run Lighthouse, providing the --port argument you noted.

We could script this behaviour using the Chrome Debug Protocol, but ... wouldn't it be sweet if you could just pass an option into 💡 🏠 ?

❤️

Copy link
Collaborator

@patrickhulce patrickhulce left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the PR @rupesh1!!

sorry for the long delay but it seems to be quite popular these days! 🎉 we like the idea and just need some 🚲 🏠 on the CLI usage a bit

setExtraHTTPHeaders(jsonHeaders) {
let headers;
try {
headers = JSON.parse(jsonHeaders);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wdyt about handling this in the CLI instead?

this way we could fail early and also do something fancy like accept files :)

let extraHeaders
if (argv.extraHeaders) {
  if (argv.extraHeaders.substr(0, 1) === '{') {
    extraHeaders = JSON.parse(argv.extraHeaders)
  } else {
    extraHeaders = JSON.parse(fs.readFileSync(argv.extraHeaders))
  }
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rupesh1 holler if you can make this change. if you've moved on, we can pick it up;

log.warn('Driver', 'Invalid header JSON');
headers = {};
}
return this.sendCommand('Network.setExtraHTTPHeaders', {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could probably skip this entirely if headers is invalid/empty instead

@rupesh1
Copy link
Contributor Author

rupesh1 commented Jan 18, 2018

@patrickhulce @paulirish I've had a go at implementing your suggestions (everything passes and runs locally for me node --version 8.9.1, yarn --version 1.3.2), but now bumped in to some typings issue with yargs when the CI runs yarn unit-cli (weirdly yarn type-check passes on TravisCI):

/home/travis/build/GoogleChrome/lighthouse/lighthouse-cli/cli-flags.js:137
      .coerce('extra-headers', /** @param {string} arg */ (arg) => {
       ^
TypeError: y.help(...).version(...).showHelpOnFail(...).usage(...).example(...).example(...).example(...).example(...).example(...).example(...).example(...).example(...).group(...).describe(...).group(...).describe(...).alias(...).group(...).describe(...).boolean(...).choices(...).array(...).string(...).default(...).default(...).default(...).default(...).default(...).default(...).check(...).coerce is not a function

I can't reproduce the error locally so any help is appreciated.

@patrickhulce
Copy link
Collaborator

Thanks for updating @rupesh1!! It looks like you might have a newer version of yargs installed locally in lighthouse-cli/node_modules we're quite behind at v3 which doesn't have the .coerce option which is why travis is failing

how about we move the parsing logic over to bin.js for now instead?

@rupesh1
Copy link
Contributor Author

rupesh1 commented Jan 18, 2018

@patrickhulce thanks for the tip, CI is looking much better now

Copy link
Collaborator

@patrickhulce patrickhulce left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this looks awesome! great job @rupesh1! 🎉 💯

everything a-ok by me, @paulirish over to you for any opinions on the CLI usage

@paulirish
Copy link
Member

I'm not 100% on using a file rather than a string, but neither seems like the clear winner. Let's do this.

@paulirish paulirish merged commit 8901034 into GoogleChrome:master Jan 19, 2018
@paulirish
Copy link
Member

thanks @rupesh1 !

@rupesh1
Copy link
Contributor Author

rupesh1 commented Jan 19, 2018

Awesome! Really appreciate everyone's help with this PR 👍

@JoelTW
Copy link

JoelTW commented Apr 16, 2018

This is great. Is there any example of the format required for the json file?

@patrickhulce
Copy link
Collaborator

@JoelTW it should just be key/value header name/value

{
  "Cookie": "token=12345acdfe",
  "x-secret-header": "it's a secret"
}

@Pagan-Idel
Copy link

Pagan-Idel commented Jul 8, 2022

Is there an example we can use these extra headers through programmatical Lighthouse?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants