Replace MD5 with SHA256+

[matt-domsch-sp]

Is your feature request related to a problem? Please describe.

sidekiq-unique-jobs/lib/sidekiq_unique_jobs/lock_digest.rb

Line 54 in bdca185

digest = OpenSSL::Digest::MD5.hexdigest(dump_json(digestable_hash.sort))

uses OpenSSL MD5 digest algorithm. MD5 is disallowed under FIPS 140-2 and later (e.g. 140-3), so systems operating in FIPS mode do not have access to the OpenSSL MD5 digest algorithm. Given that FIPS 140-2 is required in many environments, and that MD5 use is discouraged anymore regardless, another digest algorithm would be appropriate to use.

Describe the solution you'd like
Replace use of MD5 with a FIPS 140-2 or -3 permitted algorithm, such as in the SHA2 or SHA3 family.

This does raise the question of forward compatibility - upgrading a system previously using MD5 digests to a system using a SHA2+ digest, with transactions in flight. This may call for a major version bump, and an operational requirement to drain all previous unique job records. I'm open to a less invasive solution that doesn't involve the use of MD5 except perhaps in a brief transition period.

Describe alternatives you've considered
Disabling FIPS mode is not an option for my use case.
Re-enabling MD5 while in FIPS mode invalidates any FIPS certification.

We intend to monkey-patch lock_digest.rb create_digest() effectively as follows:

    def create_digest
-      digest = OpenSSL::Digest::MD5.hexdigest(dump_json(digestable_hash.sort))
+      digest = OpenSSL::Digest.new("SHA3-256", dump_json(digestable_hash.sort)).hexdigest
       "#{lock_prefix}:#{digest}"
     end

Additional context
Failures at runtime when FIPS mode is enabled on a RHEL 9.4 Universal Base Image OS with FIPS mode enabled look like:

OpenSSL::Digest::DigestError,
  | Digest initialization failed: initialization error

/usr/local/lib/ruby/3.3.0/openssl/digest.rb:31:in `initialize'
/usr/local/lib/ruby/3.3.0/openssl/digest.rb:31:in `block (3 levels) in <class:Digest>'
/usr/local/lib/ruby/3.3.0/openssl/digest.rb:38:in `new'
/usr/local/lib/ruby/3.3.0/openssl/digest.rb:38:in `block (3 levels) in <class:Digest>'
/usr/src/app/seczetta_identity_suite/vendor/bundle/ruby/3.3.0/gems/sidekiq-unique-jobs-7.1.33/lib/sidekiq_unique_jobs/lock_digest.rb:54:in `create_digest'
/usr/src/app/seczetta_identity_suite/vendor/bundle/ruby/3.3.0/gems/sidekiq-unique-jobs-7.1.33/lib/sidekiq_unique_jobs/lock_digest.rb:48:in `lock_digest'
/usr/src/app/seczetta_identity_suite/vendor/bundle/ruby/3.3.0/gems/sidekiq-unique-jobs-7.1.33/lib/sidekiq_unique_jobs/lock_digest.rb:22:in `call'
/usr/src/app/seczetta_identity_suite/vendor/bundle/ruby/3.3.0/gems/sidekiq-unique-jobs-7.1.33/lib/sidekiq_unique_jobs/job.rb:52:in `add_lock_digest'
/usr/src/app/seczetta_identity_suite/vendor/bundle/ruby/3.3.0/gems/sidekiq-unique-jobs-7.1.33/lib/sidekiq_unique_jobs/job.rb:25:in `add_digest'
/usr/src/app/seczetta_identity_suite/vendor/bundle/ruby/3.3.0/gems/sidekiq-unique-jobs-7.1.33/lib/sidekiq_unique_jobs/job.rb:17:in `prepare'
/usr/src/app/seczetta_identity_suite/vendor/bundle/ruby/3.3.0/gems/sidekiq-unique-jobs-7.1.33/lib/sidekiq_unique_jobs/middleware.rb:38:in `call'
...

[ravindrae]

Facing the same issue when trying to run the application on RHEL 8 with FIPS enabled. Since FIPS does not allow non-FIPS digest algorithms like MD5, this has caused some challenges.

Cannot disable FIPS as security concern

2024-07-17 15:18:14 +0000, tid=39040 ERROR: Couldn't execute scheduled job! from  
Wed, Jul 17 2024 8:48:14 pm
2024-07-17 15:18:14 +0000, tid=39040 ERROR: Digest initialization failed: disabled for FIPS from  
Wed, Jul 17 2024 8:48:14 pm
2024-07-17 15:18:14 +0000, tid=39040 INFO: Finished JobClass duration_ms=1 error=#<OpenSSL::Digest::DigestError: Digest initialization failed: disabled for FIPS> from  

@matt-domsch-sp, I noticed that you mentioned a monkey-patched version of create_digest. Could you please share more details on how you managed to handle the old transactions that used MD5 digests? Any guidance or examples would be greatly appreciated.

Thanks in advance

[matt-domsch-sp]

In my case we made the transition while the application was otherwise naturally idle, and did not attempt to solve for both algorithms being present simultaneously. I recognize that may not be possible for all users of this library.