Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

User Agent adjustments #861

Closed
gmetais opened this issue Feb 4, 2021 · 3 comments · Fixed by #862
Closed

User Agent adjustments #861

gmetais opened this issue Feb 4, 2021 · 3 comments · Fixed by #862
Milestone

Comments

@gmetais
Copy link
Contributor

gmetais commented Feb 4, 2021

Hi @macbre,
Yeah, me again...

I've noticed a few issues regarding the user agent sent by Phantomas. By default, Phantomas sends a user agent that looks like Phantomas/2.1.0 (HeadlessChrome/88.0.4298.0). This is OK most of the time.

However, there are some websites that filter users regarding their user agent. You might remember the airline company's website https://www.airfrance.com that was responding with a 403 error (#764). This is not the case when tested with the --phone or the --tablet options, because the user-agent is overridden by Page.emulate().

There is another issue regarding Google Fonts. This service reads user agents to deliver a compatible font format. And when called from Phantomas in desktop mode, it responds with TTF fonts. Example page: https://www.worldelse.com/

I'd like to improve the way it is handled, but we should exchange about the right solution first.

Solution 1:
Change the desktop user agent template, so that it simulates a real Chrome desktop user. For example:
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<ChromiumVersion> Safari/537.36

Solution 1 (extra):
Also change the version for phone and tablet. They seem to be stuck on old browser versions:
Mozilla/5.0 (Linux; Android 5.0; SM-G900P Build/LRX21T) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3765.0 Mobile Safari/537.36
Mozilla/5.0 (Linux; U; en-us; KFAPWI Build/JDQ39) AppleWebKit/535.19 (KHTML, like Gecko) Silk/3.13 Safari/535.19 Silk-Accelerated=true
I do not see it as a problem now, but it could become one if Puppeteer never updates the list. Some websites might trigger a popup saying the device is not supported one day.
We could have a template that auto updates with Chromium's version, just like desktops.

Solution 2:
Allow the user to specify a user-agent. It used to be possible in Phantomas v1, but this option is not working anymore.

Solution 3:
Do all of the above.

@macbre macbre added this to the v2.2 milestone Feb 4, 2021
@macbre
Copy link
Owner

macbre commented Feb 4, 2021

@gmetais - thanks for bringing up this topic. I vote for solution 3.

@gmetais
Copy link
Contributor Author

gmetais commented Feb 4, 2021

Ok! Do you want to keep Silk browser for Tablet, or should I replace with Chrome?

@macbre
Copy link
Owner

macbre commented Feb 4, 2021

Let's use Chrome.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants