c/log_eviction_stm: do not request snapshot if already progressed

It is perfectly possible that the install snapshot request will reach
the follower right before the `log_eviction_stm` asks for the snapshot
creation. In this case an `stm_manager` would do the check and throw an
exception informing that the snapshot can not be taken. In order to
handle the situation gracefully added a check in log eviction stm to
skip taking snapshot if start offset already progressed.

Fixes: redpanda-data#14220

Signed-off-by: Michal Maslanka <michal@redpanda.com>
(cherry picked from commit 4433851)

src/v/cluster/log_eviction_stm.cc

@@ -183,12 +183,34 @@ log_eviction_stm::do_write_raft_snapshot(model::offset truncation_point) {
           max_collectible_offset,
           truncation_point);
     }
+    if (truncation_point <= _raft->last_snapshot_index()) {
+        vlog(
+          _log.trace,
+          "Skipping writing snapshot as Raft already progressed with the new "
+          "snapshot. Current raft snapshot index: {}, requested truncation "
+          "point: {}",
+          _raft->last_snapshot_index(),
+          truncation_point);
+        co_return;
+    }
     vlog(
       _log.debug,
       "Requesting raft snapshot with final offset: {}",
       truncation_point);
     auto snapshot_data = co_await _raft->stm_manager()->take_snapshot(
       truncation_point);
+    // we need to check snapshot index again as it may already progressed after
+    // snapshot is taken by stm_manager
+    if (truncation_point <= _raft->last_snapshot_index()) {
+        vlog(
+          _log.trace,
+          "Skipping writing snapshot as Raft already progressed with the new "
+          "snapshot. Current raft snapshot index: {}, requested truncation "
+          "point: {}",
+          _raft->last_snapshot_index(),
+          truncation_point);
+        co_return;
+    }
     co_await _raft->write_snapshot(
       raft::write_snapshot_cfg(truncation_point, std::move(snapshot_data)));
 }