Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(unused-css-rules): update to support new coverage format #2518

Merged
merged 8 commits into from
Aug 18, 2017
Merged

fix(unused-css-rules): update to support new coverage format #2518

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
4608a68
fix(unused-css-rules): update to support new coverage format
patrickhulce Jun 16, 2017
c56b19f
re-enable smokehouse tests
patrickhulce Aug 1, 2017
b669d5e
Merge branch 'master' into css_coverage
patrickhulce Aug 16, 2017
210c514
feedback
patrickhulce Aug 16, 2017
57de9c9
add tooltips to url
patrickhulce Aug 17, 2017
a0b414f
last two feedbacks
patrickhulce Aug 17, 2017
237a94f
clearer total byte logic
patrickhulce Aug 18, 2017
1e35aa9
adjust multipliers
patrickhulce Aug 18, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 10 additions & 11 deletions lighthouse-cli/test/smokehouse/byte-efficiency/expectations.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,17 +13,16 @@ module.exports = [
initialUrl: 'http://localhost:10200/byte-efficiency/tester.html',
url: 'http://localhost:10200/byte-efficiency/tester.html',
audits: {
// TODO: re-enable once CSS protocol has stabilized
// 'unused-css-rules': {
// score: false,
// extendedInfo: {
// value: {
// results: {
// length: 2
// }
// }
// }
// },
'unused-css-rules': {
score: '<100',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we make this more precise since we control the test page?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, the score gets bucketed by the generic byte efficiency audit though so it's not that useful. I added an assertion on the "wastedKb" instead which is a little more explicit and useful

extendedInfo: {
value: {
results: {
length: 2
}
}
}
},
'offscreen-images': {
score: '<100',
extendedInfo: {
Expand Down
2 changes: 1 addition & 1 deletion lighthouse-cli/test/smokehouse/byte-efficiency/run-tests.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ node lighthouse-cli/test/fixtures/static-server.js &

sleep 0.5s

config="lighthouse-core/config/default.js"
config="lighthouse-core/config/full-config.js"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need to update byte-config.js instead (after #2732)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

expectations="lighthouse-cli/test/smokehouse/byte-efficiency/expectations.js"

yarn smokehouse -- --config-path=$config --expectations-path=$expectations
Expand Down
98 changes: 58 additions & 40 deletions lighthouse-core/audits/byte-efficiency/unused-css-rules.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,25 +39,22 @@ class UnusedCSSRules extends ByteEfficiencyAudit {
indexed[record.url] = record;
return indexed;
}, {});

return styles.reduce((indexed, stylesheet) => {
indexed[stylesheet.header.styleSheetId] = Object.assign({
used: [],
unused: [],
usedRules: [],
networkRecord: indexedNetworkRecords[stylesheet.header.sourceURL],
}, stylesheet);
return indexed;
}, {});
}

/**
* Counts the number of unused rules and adds count information to sheets.
* Adds used rules to their corresponding stylesheet.
* @param {!Array.<{styleSheetId: string, used: boolean}>} rules The output of the CSSUsage gatherer.
* @param {!Object} indexedStylesheets Stylesheet information indexed by id.
* @return {number} The number of unused rules.
*/
static countUnusedRules(rules, indexedStylesheets) {
let unused = 0;

static indexUsedRules(rules, indexedStylesheets) {
rules.forEach(rule => {
const stylesheetInfo = indexedStylesheets[rule.styleSheetId];

Expand All @@ -66,14 +63,44 @@ class UnusedCSSRules extends ByteEfficiencyAudit {
}

if (rule.used) {
stylesheetInfo.used.push(rule);
} else {
unused++;
stylesheetInfo.unused.push(rule);
stylesheetInfo.usedRules.push(rule);
}
});
}

/**
* @param {!Object} stylesheetInfo
* @param {boolean} isInline
* @return {{debugString: string|undefined, wastedBytes: number, totalBytes: number}}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wastedPercent, too

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

*/
static computeUsage(stylesheetInfo, isInline) {
let usedUncompressedBytes = 0;
const totalUncompressedBytes = stylesheetInfo.content.length;

for (const usedRule of stylesheetInfo.usedRules) {
usedUncompressedBytes += usedRule.endOffset - usedRule.startOffset;
}

// If we don't know for sure how many bytes this sheet used on the network,
// we can guess it was roughly the size of the content gzipped.
const totalBytes = stylesheetInfo.networkRecord ?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will this be weird for inline stylesheets that came in a page that was gzipped?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will be weird for inline stylesheets that came in a page that was not gzipped since we'll underestimate the savings.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will be weird for inline stylesheets that came in a page that was not gzipped since we'll underestimate the savings.

ah, yeah, that was based on how it was (I believe) filtering out networkRecords for non-stylesheets before

stylesheetInfo.networkRecord.transferSize :
Math.round(stylesheetInfo.content.length / 3);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since this is the full run, not the default run, instead of estimating like this can we just actually run gzip on it like response-compression does?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we don't actually know it was gzipped anyway I'm inclined to not introduce that complexity here too, but I don't feel that strongly.


return unused;
const percentUnused = (totalUncompressedBytes - usedUncompressedBytes) / totalUncompressedBytes;
const wastedBytes = Math.round(percentUnused * totalBytes);

let debugString;
if (!isInline && !stylesheetInfo.networkRecord) {
debugString = `Unable to find network record for ${stylesheetInfo.header.sourceURL}`;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get this error on both paulirish.com and theverge.com.. can we avoid that?

}

return {
debugString,
wastedBytes,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not worth logging debugString anymore for network records not found (it'll show up in HTTP Archive data that way)? Or we could sentry it, maybe. Otherwise we won't have any feedback on how often we're failing at finding network records...or it might not matter :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah didn't seem worth it anymore, the cases where it happens now are wholly expected and there's no sense being noisy

wastedPercent: percentUnused * 100,
totalBytes,
};
}

/**
Expand Down Expand Up @@ -115,40 +142,25 @@ class UnusedCSSRules extends ByteEfficiencyAudit {
/**
* @param {!Object} stylesheetInfo The stylesheetInfo object.
* @param {string} pageUrl The URL of the page, used to identify inline styles.
* @return {{url: string, label: string, code: string}} The result for the details object.
* @return {?{url: string, wastedBytes: number, totalBytes: number}}
*/
static mapSheetToResult(stylesheetInfo, pageUrl) {
const numUsed = stylesheetInfo.used.length;
const numUnused = stylesheetInfo.unused.length;

if ((numUsed === 0 && numUnused === 0) || stylesheetInfo.isDuplicate) {
if (stylesheetInfo.isDuplicate) {
return null;
}

let url = stylesheetInfo.header.sourceURL;
let isInline = false;
if (!url || url === pageUrl) {
const contentPreview = UnusedCSSRules.determineContentPreview(stylesheetInfo.content);
url = '*inline*```' + contentPreview + '```';
url = {type: 'code', text: contentPreview};
isInline = true;
} else {
url = URL.getURLDisplayName(url);
}

// If we don't know for sure how many bytes this sheet used on the network,
// we can guess it was roughly the size of the content gzipped.
const totalBytes = stylesheetInfo.networkRecord ?
stylesheetInfo.networkRecord.transferSize :
Math.round(stylesheetInfo.content.length / 3);

const percentUnused = numUnused / (numUsed + numUnused);
const wastedBytes = Math.round(percentUnused * totalBytes);

return {
url,
numUnused,
wastedBytes,
wastedPercent: percentUnused * 100,
totalBytes,
};
const usage = UnusedCSSRules.computeUsage(stylesheetInfo, isInline);
return Object.assign({url}, usage);
}

/**
Expand All @@ -163,21 +175,27 @@ class UnusedCSSRules extends ByteEfficiencyAudit {
const devtoolsLogs = artifacts.devtoolsLogs[ByteEfficiencyAudit.DEFAULT_PASS];
return artifacts.requestNetworkRecords(devtoolsLogs).then(networkRecords => {
const indexedSheets = UnusedCSSRules.indexStylesheetsById(styles, networkRecords);
UnusedCSSRules.countUnusedRules(usage, indexedSheets);
const results = Object.keys(indexedSheets).map(sheetId => {
return UnusedCSSRules.mapSheetToResult(indexedSheets[sheetId], pageUrl);
}).filter(sheet => sheet && sheet.wastedBytes > 1024);
UnusedCSSRules.indexUsedRules(usage, indexedSheets);

const results = Object.keys(indexedSheets)
.map(sheetId => UnusedCSSRules.mapSheetToResult(indexedSheets[sheetId], pageUrl))
.filter(sheet => sheet && sheet.wastedBytes > 1024);

const headings = [
{key: 'url', itemType: 'url', text: 'URL'},
{key: 'numUnused', itemType: 'url', text: 'Unused Rules'},
{key: 'totalKb', itemType: 'text', text: 'Original'},
{key: 'potentialSavings', itemType: 'text', text: 'Potential Savings'},
];

const debugString = results
.map(result => result.debugString)
.filter(Boolean)
.pop();

return {
debugString,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add to return type

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

results,
headings
headings,
};
});
}
Expand Down
3 changes: 0 additions & 3 deletions lighthouse-core/config/default.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@ module.exports = {
'manifest',
'chrome-console-messages',
'image-usage',
// 'css-usage',
'accessibility',
'dobetterweb/all-event-listeners',
'dobetterweb/anchors-with-no-rel-noopener',
Expand Down Expand Up @@ -123,7 +122,6 @@ module.exports = {
'accessibility/video-caption',
'accessibility/video-description',
'byte-efficiency/total-byte-weight',
// 'byte-efficiency/unused-css-rules',
'byte-efficiency/offscreen-images',
'byte-efficiency/uses-webp-images',
'byte-efficiency/uses-optimized-images',
Expand Down Expand Up @@ -229,7 +227,6 @@ module.exports = {
{id: 'estimated-input-latency', weight: 1, group: 'perf-metric'},
{id: 'link-blocking-first-paint', weight: 0, group: 'perf-hint'},
{id: 'script-blocking-first-paint', weight: 0, group: 'perf-hint'},
// {id: 'unused-css-rules', weight: 0},
{id: 'uses-responsive-images', weight: 0, group: 'perf-hint'},
{id: 'offscreen-images', weight: 0, group: 'perf-hint'},
{id: 'uses-optimized-images', weight: 0, group: 'perf-hint'},
Expand Down
11 changes: 9 additions & 2 deletions lighthouse-core/config/full-config.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,17 +12,24 @@ module.exports = {
passName: 'extraPass',
gatherers: [
'styles',
'css-usage',
]
},
],
audits: [
'byte-efficiency/unused-css-rules',
'dobetterweb/no-old-flexbox',
],
categories: {
'performance': {
audits: [
{id: 'unused-css-rules', weight: 0, group: 'perf-hint'},
],
},
'best-practices': {
audits: [
{id: 'no-old-flexbox', weight: 1},
]
}
],
},
},
};
50 changes: 18 additions & 32 deletions lighthouse-core/test/audits/byte-efficiency/unused-css-rules-test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,26 +98,22 @@ describe('Best Practices: unused css rules audit', () => {
baseSheet = {
header: {sourceURL: baseUrl},
content: 'dummy',
used: [{dummy: 1}],
unused: [],
usedRules: [],
};
});

it('correctly computes potentialSavings', () => {
assert.equal(map({used: [], unused: [1, 2]}).wastedPercent, 100);
assert.equal(map({used: [1, 2], unused: [1, 2]}).wastedPercent, 50);
assert.equal(map({used: [1, 2], unused: []}).wastedPercent, 0);
assert.equal(map({usedRules: []}).wastedPercent, 100);
assert.equal(map({usedRules: [{startOffset: 0, endOffset: 3}]}).wastedPercent, 40);
assert.equal(map({usedRules: [{startOffset: 0, endOffset: 5}]}).wastedPercent, 0);
});

it('correctly computes url', () => {
assert.equal(map({header: {sourceURL: ''}}).url, '*inline*```dummy```');
assert.equal(map({header: {sourceURL: 'a'}}, 'http://g.co/a').url, '*inline*```dummy```');
const expectedPreview = {type: 'code', text: 'dummy'};
assert.deepEqual(map({header: {sourceURL: ''}}).url, expectedPreview);
assert.deepEqual(map({header: {sourceURL: 'a'}}, 'http://g.co/a').url, expectedPreview);
assert.equal(map({header: {sourceURL: 'foobar'}}).url, '/foobar');
});

it('does not give content preview when url is present', () => {
assert.ok(!/dummy/.test(map({header: {sourceURL: 'foobar'}}).url));
});
});

describe('#audit', () => {
Expand Down Expand Up @@ -175,13 +171,8 @@ describe('Best Practices: unused css rules audit', () => {
requestNetworkRecords,
URL: {finalUrl: ''},
CSSUsage: [
{styleSheetId: 'a', used: true},
{styleSheetId: 'a', used: false},
{styleSheetId: 'a', used: false},
{styleSheetId: 'a', used: false},
{styleSheetId: 'b', used: true},
{styleSheetId: 'b', used: false},
{styleSheetId: 'c', used: false},
{styleSheetId: 'a', used: true, startOffset: 0, endOffset: 11}, // 44 * 1 / 4
{styleSheetId: 'b', used: true, startOffset: 0, endOffset: 3075}, // 2050 * 3 / 2
],
Styles: [
{
Expand All @@ -199,6 +190,8 @@ describe('Best Practices: unused css rules audit', () => {
]
}).then(result => {
assert.equal(result.results.length, 2);
assert.ok(!result.results[0].debugString, 'present network record errored');
assert.ok(result.results[1].debugString, 'missing network record did not error');
assert.equal(result.results[0].totalBytes, 10 * 1024);
assert.equal(result.results[1].totalBytes, 2050);
assert.equal(result.results[0].wastedPercent, 75);
Expand All @@ -212,10 +205,7 @@ describe('Best Practices: unused css rules audit', () => {
requestNetworkRecords,
URL: {finalUrl: ''},
CSSUsage: [
{styleSheetId: 'a', used: true},
{styleSheetId: 'a', used: true},
{styleSheetId: 'a', used: false},
{styleSheetId: 'b', used: false},
{styleSheetId: 'a', used: true, startOffset: 0, endOffset: 33}, // 44 * 3 / 4
],
Styles: [
{
Expand All @@ -239,12 +229,8 @@ describe('Best Practices: unused css rules audit', () => {
requestNetworkRecords,
URL: {finalUrl: ''},
CSSUsage: [
{styleSheetId: 'a', used: true},
{styleSheetId: 'a', used: true},
{styleSheetId: 'a', used: false},
{styleSheetId: 'b', used: true},
{styleSheetId: 'b', used: false},
{styleSheetId: 'b', used: false},
{styleSheetId: 'a', used: true, startOffset: 0, endOffset: 8000}, // 4000 * 3 / 2
{styleSheetId: 'b', used: true, startOffset: 0, endOffset: 500}, // 500 * 3 / 3
],
Styles: [
{
Expand All @@ -256,21 +242,21 @@ describe('Best Practices: unused css rules audit', () => {
content: `${generate('123', 500)}`
},
{
header: {styleSheetId: 'c', sourceURL: 'c.css'},
header: {styleSheetId: 'c', sourceURL: 'file://c.css'},
content: '@import url(http://googlefonts.com?myfont)'
},
{
header: {styleSheetId: 'd', sourceURL: 'd.css'},
header: {styleSheetId: 'd', sourceURL: 'file://d.css'},
content: '/* nothing to see here */'
},
{
header: {styleSheetId: 'e', sourceURL: 'e.css'},
header: {styleSheetId: 'e', sourceURL: 'file://e.css'},
content: ' '
},
]
}).then(result => {
assert.equal(result.results.length, 1);
assert.equal(result.results[0].numUnused, 1);
assert.equal(Math.floor(result.results[0].wastedPercent), 33);
});
});
});
Expand Down