Fixed exception in append_tokens

[willmcgugan]

Fixes #3014

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.25%. Comparing base (05428ec) to head (e96ae13).
Report is 4 commits behind head on master.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #3471   +/-   ##
=======================================
  Coverage   98.25%   98.25%           
=======================================
  Files          74       74           
  Lines        8070     8071    +1     
=======================================
+ Hits         7929     7930    +1     
  Misses        141      141           

Flag	Coverage Δ
unittests	98.25% <100.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.