feat(backports): Add output backport fixes for debian based distros #1273

BreadGenie · 2021-07-24T11:14:18Z

- Backported fix utility
- Tests
- Docs (docs: backport-fix #1344)

codecov-commenter · 2021-07-24T11:18:39Z

Codecov Report

Merging #1273 (77746ca) into main (82de345) will decrease coverage by 0.03%.
The diff coverage is 81.17%.

@@            Coverage Diff             @@
##             main    #1273      +/-   ##
==========================================
- Coverage   79.35%   79.32%   -0.04%     
==========================================
  Files         271      274       +3     
  Lines        4902     4996      +94     
  Branches      591      608      +17     
==========================================
+ Hits         3890     3963      +73     
- Misses        865      875      +10     
- Partials      147      158      +11

Flag	Coverage Δ
longtests	`79.32% <81.17%> (-0.04%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
cve_bin_tool/backports/debian_backports.py	`78.57% <78.57%> (ø)`
cve_bin_tool/backports/__init__.py	`82.14% <82.14%> (ø)`
test/test_backports.py	`84.61% <84.61%> (ø)`
cve_bin_tool/cli.py	`73.29% <100.00%> (-1.71%)`	⬇️
cve_bin_tool/merge.py	`79.50% <0.00%> (-0.33%)`	⬇️
cve_bin_tool/cvedb.py	`82.30% <0.00%> (+0.29%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 82de345...77746ca. Read the comment docs.

requirements.txt

BreadGenie · 2021-07-28T13:05:54Z

Sample output

breadgenie@breadlamp:~/Code/cve-bin-tool$ python3 -m cve_bin_tool.cli test/condensed-downloads/pspp-1.2.0-lp151.2.1.x86_64.rpm.tar.gz -b 
[18:35:05] INFO     cve_bin_tool.CVEDB - Using cached CVE data (<24h old). Use -u now to update immediately.                                                                                           cvedb.py:304
           INFO     cve_bin_tool.CVEDB - Check database is using latest schema                                                                                                                         cvedb.py:316
           INFO     cve_bin_tool.CVEDB - There are 167181 CVE entries in the database                                                                                                                  cvedb.py:334
[18:35:06] INFO     cve_bin_tool.VersionScanner - Checkers: accountsservice, avahi, bash, bind, binutils, bolt, bubblewrap, busybox, bzip2, cronie, cryptsetup, cups, curl, dbus, dnsmasq,    version_scanner.py:91
                    dovecot, dpkg, enscript, expat, ffmpeg, freeradius, ftp, gcc, gimp, glibc, gnomeshell, gnupg, gnutls, gpgme, gstreamer, gupnp, haproxy, hostapd, hunspell, icecast, icu,                       
                    irssi, kbd, kerberos, kexectools, libarchive, libbpg, libcurl, libdb, libgcrypt, libical, libjpeg_turbo, liblas, libnss, libsndfile, libsoup, libssh2, libtiff, libvirt,                       
                    libxslt, lighttpd, logrotate, lua, mariadb, mdadm, memcached, mtr, mysql, ncurses, nessus, netpbm, nginx, node, ntp, openafs, openjpeg, openldap, openssh, openssl,                            
                    openswan, openvpn, p7zip, png, polarssl_fedora, postgresql, pspp, python, qt, radare2, rsyslog, samba, sqlite, strongswan, subversion, sudo, syslogng, systemd, tcpdump,                       
                    trousers, varnish, webkitgtk, wireshark, wpa_supplicant, xerces, xml2, zlib, zsh                                                                                                               
           WARNING  cve_bin_tool.VersionScanner - pspp was detected with version UNKNOWN in file test/condensed-downloads/pspp-1.2.0-lp151.2.1.x86_64.rpm.tar.gz contains                    version_scanner.py:210
                    /usr/bin/pspp-dump-sav                                                                                                                                                                         
           INFO     cve_bin_tool.VersionScanner - test/condensed-downloads/pspp-1.2.0-lp151.2.1.x86_64.rpm.tar.gz contains /usr/bin/psppire contains pspp 1.2.0                              version_scanner.py:218
           INFO     cve_bin_tool.CVEScanner - Known CVEs in ProductInfo(vendor='gnu', product='pspp', version='1.2.0')                                                                           cve_scanner.py:195
[18:35:06]                                                                                                                                                                                       cve_scanner.py:199
           ╭─────────────────────────────────────────────────────────────────────────── 2 CVE(s) in gnu.pspp v1.2.0 ───────────────────────────────────────────────────────────────────────────╮ cve_scanner.py:200
           │                                                                                                                                                                                   │                   
           │ CVE-2018-20230  CVE-2019-9211                                                                                                                                                     │                   
           │                                                                                                                                                                                   │                   
           ╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯                   
                                                                                                                                                                                                 cve_scanner.py:216
           INFO     cve_bin_tool.VersionScanner - test/condensed-downloads/pspp-1.2.0-lp151.2.1.x86_64.rpm.tar.gz contains /usr/bin/pspp contains pspp 1.2.0                                 version_scanner.py:218
           WARNING  cve_bin_tool.VersionScanner - pspp was detected with version UNKNOWN in file test/condensed-downloads/pspp-1.2.0-lp151.2.1.x86_64.rpm.tar.gz contains                    version_scanner.py:210
                    /usr/lib64/pspp/libpspp-core-1.2.0.so                                                                                                                                                          
           INFO     cve_bin_tool.VersionScanner - test/condensed-downloads/pspp-1.2.0-lp151.2.1.x86_64.rpm.tar.gz contains /usr/lib64/pspp/libpspp-1.2.0.so is pspp 1.2.0                    version_scanner.py:218
           INFO     cve_bin_tool -                                                                                                                                                                       cli.py:415
           INFO     cve_bin_tool - Overall CVE summary:                                                                                                                                                  cli.py:416
           INFO     cve_bin_tool - There are 3 files with known CVEs detected                                                                                                                            cli.py:422
           INFO     cve_bin_tool - Known CVEs in ('pspp', '1.2.0'):                                                                                                                                      cli.py:432
╔══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╗
║                                                   CVE BINARY TOOL                                                    ║
╚══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╝

 • cve-bin-tool Report Generated: 2021-07-28  18:35:06                                                                  
 • Time of last update of CVE Data: 2021-07-28  18:21:33                                                                
╭─────────────────╮
│  NewFound CVEs  │
╰─────────────────╯
┏━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┓
┃ Vendor ┃ Product ┃ Version ┃ CVE Number     ┃ Severity ┃ Score (CVSS Version) ┃
┡━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━┩
│ gnu    │ pspp    │ 1.2.0   │ CVE-2018-20230 │ HIGH     │ 7.8 (v3)             │
│ gnu    │ pspp    │ 1.2.0   │ CVE-2019-9211  │ MEDIUM   │ 6.5 (v3)             │
└────────┴─────────┴─────────┴────────────────┴──────────┴──────────────────────┘
           INFO     cve_bin_tool - pspp: CVE-2018-20230 has backported fix in v1.2.0-3 release.                                                                                              debian_backports.py:56
           INFO     cve_bin_tool - pspp: CVE-2019-9211 has backported fix in v1.2.0-4 release.                                                                                               debian_backports.py:56

BreadGenie · 2021-07-31T13:26:24Z

The commit 869b899 removes requests library and replaces it with urllib. But since we are going to use requests anyway soon should I revert the commit?

requests performed better than urllib on my machine.

terriko · 2021-08-03T23:24:37Z

I honestly can't decide whether we should be using requests or not. I'm going to open an issue about it and maybe we can talk about it in the meeting tomorrow too.

terriko

I'm so glad that this was possible for debian-based systems. I have a few suggestions, but also don't forget to add documentation of the new flag into the manual.

cve_bin_tool/cli.py

terriko · 2021-08-06T00:25:26Z

cve_bin_tool/backports/__init__.py

+        self.all_cve_data = all_cve_data
+
+    def check_backport(self):
+        if distro.id() in DEBIAN_DISTROS:


It may be useful for users to be able to list backport information even if the machine being used for scanning isn't running debian itself. I'm thinking particularly of folk who might be running cve-bin-tool in a separate CI system or running it on a downloads server, but it might also be important for other debian-based distros whose names we don't know yet.

Let's figure out if there's a nice way to do that. Could backports/debian_backports.py take an argument (say from a --backported-fixes=debian option, and have the default be "local_system" that grabs distro.id() or something) instead of distro.id()?

If the user specify an option debian how will the system recognise whether which version of debian the user wants? 🤔
Maybe something like debian-bullseye, debian-buster and so on for the choices?
Or maybe stick to the latest version?

I like having debian-bullseye etc. as options. Not sure the best way to communicate the availability of these options to users, but maybe just documenting it in Manual.md?

The help message got very messy from the long list of options. So I'll add these options in MANUAL.md.

Co-authored-by: Terri Oda <terri@toybox.ca>

terriko

Needs a quick update because the previous PR caused an argument conflict in cli.py

terriko

Looks like we're missing SPDX license lines at the top of the new files.
Can you also file a separate issue for the "get data by explicitly specifying the distro" option so we don't forget that as future work?
Other than those two things, this is looking ready to merge.

BreadGenie · 2021-08-14T15:00:35Z

I have added the explicit specification of distro info in this PR

terriko

I like your solution for specifying distros!

feat(backports): Add output backport fixes for debian based distros

2489fcf

BreadGenie commented Jul 24, 2021

View reviewed changes

requirements.txt Outdated Show resolved Hide resolved

BreadGenie and others added 4 commits July 26, 2021 12:11

feat(backports): Output a message if utility doesn't support the distro

6d7f912

fix: Some 'no backfixes' not logging

b1e8199

feat: Add more Ubuntu-Debian Mappings

1c101f8

Merge branch 'main' into backport-debian

8446f15

refactor: Use urllib instead of requests

869b899

terriko mentioned this pull request Aug 3, 2021

Refactor: requests library vs urlib* #1311

Closed

terriko requested changes Aug 6, 2021

View reviewed changes

refactor: change help text for backport flag

76840ad

Co-authored-by: Terri Oda <terri@toybox.ca>

BreadGenie force-pushed the backport-debian branch from 56924a4 to 76840ad Compare August 6, 2021 10:06

terriko requested changes Aug 11, 2021

View reviewed changes

Merge branch 'main' into backport-debian

5d49436

terriko requested changes Aug 12, 2021

View reviewed changes

BreadGenie added 2 commits August 13, 2021 19:16

chore: add spdx header

4676aa4

chore: add spdx header

bea6e84

BreadGenie mentioned this pull request Aug 13, 2021

Backported fixes - explicit specifying distros #1339

Closed

feat(backports): Add explicit specifying of distro info

6c0236d

test: Update tests for explicit distro info specification

77746ca

terriko approved these changes Aug 17, 2021

View reviewed changes

terriko merged commit 030fe34 into intel:main Aug 17, 2021

terriko mentioned this pull request Aug 17, 2021

fix: group arguments in cli #1340

Merged

BreadGenie deleted the backport-debian branch August 18, 2021 02:42

BreadGenie mentioned this pull request Aug 18, 2021

docs: backport-fix #1344

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(backports): Add output backport fixes for debian based distros #1273

feat(backports): Add output backport fixes for debian based distros #1273

BreadGenie commented Jul 24, 2021 •

edited

Loading

codecov-commenter commented Jul 24, 2021 •

edited

Loading

BreadGenie commented Jul 28, 2021

BreadGenie commented Jul 31, 2021 •

edited

Loading

terriko commented Aug 3, 2021

terriko left a comment

terriko Aug 6, 2021

BreadGenie Aug 6, 2021 •

edited

Loading

terriko Aug 10, 2021

BreadGenie Aug 14, 2021

terriko left a comment

terriko left a comment

BreadGenie commented Aug 14, 2021

terriko left a comment

feat(backports): Add output backport fixes for debian based distros #1273

feat(backports): Add output backport fixes for debian based distros #1273

Conversation

BreadGenie commented Jul 24, 2021 • edited Loading

codecov-commenter commented Jul 24, 2021 • edited Loading

Codecov Report

BreadGenie commented Jul 28, 2021

Sample output

BreadGenie commented Jul 31, 2021 • edited Loading

terriko commented Aug 3, 2021

terriko left a comment

Choose a reason for hiding this comment

terriko Aug 6, 2021

Choose a reason for hiding this comment

BreadGenie Aug 6, 2021 • edited Loading

Choose a reason for hiding this comment

terriko Aug 10, 2021

Choose a reason for hiding this comment

BreadGenie Aug 14, 2021

Choose a reason for hiding this comment

terriko left a comment

Choose a reason for hiding this comment

terriko left a comment

Choose a reason for hiding this comment

BreadGenie commented Aug 14, 2021

terriko left a comment

Choose a reason for hiding this comment

BreadGenie commented Jul 24, 2021 •

edited

Loading

codecov-commenter commented Jul 24, 2021 •

edited

Loading

BreadGenie commented Jul 31, 2021 •

edited

Loading

BreadGenie Aug 6, 2021 •

edited

Loading