Stuck on a logging issue in Python 3.85

[jvedman]

Running on Ubuntu 20.0.4 LTS.

First, I know you all aren't working on pushing up to Python 3.x, but I am. I have installed the correlator with all package version maximums blanked (so running the latest versions of everything in the standard Ubuntu repository) and have the correlator starting (apparently) correctly, but after that any activity that should generate an alert gives me the following errors in /var/log/syslog:

`Mar 10 14:29:57 python3-nas opencanary-correlator[31490]: Unhandled Error

Mar 10 14:29:57 python3-nas opencanary-correlator[31490]: Traceback (most recent call last):

Mar 10 14:29:57 python3-nas opencanary-correlator[31490]: File "/cor/lib/python3.8/site-packages/Twisted-21.2.0-py3.8.egg/twisted/python/log.py", line 101, in callWithLogger

Mar 10 14:29:57 python3-nas opencanary-correlator[31490]: return callWithContext({"system": lp}, func, *args, **kw)

Mar 10 14:29:57 python3-nas opencanary-correlator[31490]: File "/cor/lib/python3.8/site-packages/Twisted-21.2.0-py3.8.egg/twisted/python/log.py", line 85, in callWithContext

Mar 10 14:29:57 python3-nas opencanary-correlator[31490]: return context.call({ILogContext: newCtx}, func, *args, **kw)

Mar 10 14:29:57 python3-nas opencanary-correlator[31490]: File "/cor/lib/python3.8/site-packages/Twisted-21.2.0-py3.8.egg/twisted/python/context.py", line 118, in callWithContext

Mar 10 14:29:57 python3-nas opencanary-correlator[31490]: return self.currentContext().callWithContext(ctx, func, *args, **kw)

Mar 10 14:29:57 python3-nas opencanary-correlator[31490]: File "/cor/lib/python3.8/site-packages/Twisted-21.2.0-py3.8.egg/twisted/python/context.py", line 83, in callWithContext

Mar 10 14:29:57 python3-nas opencanary-correlator[31490]: return func(*args, **kw)

Mar 10 14:29:57 python3-nas opencanary-correlator[31490]: --- ---

Mar 10 14:29:57 python3-nas opencanary-correlator[31490]: File "/cor/lib/python3.8/site-packages/Twisted-21.2.0-py3.8.egg/twisted/internet/posixbase.py", line 687, in _doReadOrWrite

Mar 10 14:29:57 python3-nas opencanary-correlator[31490]: why = selectable.doRead()

Mar 10 14:29:57 python3-nas opencanary-correlator[31490]: File "/cor/lib/python3.8/site-packages/Twisted-21.2.0-py3.8.egg/twisted/internet/tcp.py", line 246, in doRead

Mar 10 14:29:57 python3-nas opencanary-correlator[31490]: return self._dataReceived(data)

Mar 10 14:29:57 python3-nas opencanary-correlator[31490]: File "/cor/lib/python3.8/site-packages/Twisted-21.2.0-py3.8.egg/twisted/internet/tcp.py", line 251, in _dataReceived

Mar 10 14:29:57 python3-nas opencanary-correlator[31490]: rval = self.protocol.dataReceived(data)

Mar 10 14:29:57 python3-nas opencanary-correlator[31490]: File "/cor/lib/python3.8/site-packages/Twisted-21.2.0-py3.8.egg/twisted/protocols/basic.py", line 536, in dataReceived

Mar 10 14:29:57 python3-nas opencanary-correlator[31490]: line, self._buffer = self._buffer.split(self.delimiter, 1)

Mar 10 14:29:57 python3-nas opencanary-correlator[31490]: builtins.TypeError: a bytes-like object is required, not 'str'`

I'm having trouble figuring out where the write (or maybe read?) is that is causing the problem. I've tried encoding everything place that I can find that looks like it is writing to a log. I was hoping that one of you who wrote the logging and incident pieces might have some insight to what I'm not seeing.

Oh, and I'm passing all of my code back along to you all. Once you accept my initial PR for syntax clean up I'll also share my SendGrid integration.

[manincharge]

@jvedman Did you ever get this working ?

[jvedman]

No, in fact I pretty much left it after this post. I'd love to get the project up to python 3.85 though if you have any ideas.

[manincharge]

We ended up just using a 2.7 virtual env for the correlator and the virtual 3.8 for the opencanary process. We run both on the same system.
I would be nice to get the correlator upgraded but other features are more important to us.