Batching of circuits to overcome memory issues when using statevector…

… simulator (qiskit-community#209)

Currently batch_size parameter is ignored if the statevector simulator is used. This leads to unreasonable memory use due to the size of the transpiled circuits, up to 1TB of RAM for 800 by 800 kernel matrix and 20 qubits (see qiskit-terra, issue #6991). This pull request fixes this by transpiling and simulating circuits in batches, never storing the entire 800 circuits. The modification uses batch_size parameter that is already used in non-statevector case.

* initial attempt that passes the tests

* further improvement

* fix formatting

* added reno file

* Update releasenotes/notes/batch-circuits-statevector-522a842c6f68d954.yaml

Co-authored-by: Steve Wood <40241007+woodsp-ibm@users.noreply.github.com>

* added the word `transpiled` to the dictionary for spellchecker

* updated docstring to accurately reflect default batch size

Co-authored-by: Anton Dekusar <62334182+adekusar-drl@users.noreply.github.com>
Co-authored-by: Steve Wood <40241007+woodsp-ibm@users.noreply.github.com>
Co-authored-by: Manoel Marques <Manoel.Marques@ibm.com>

.pylintdict

@@ -165,6 +165,7 @@ toctree
 todo
 traceback
 transpilation
+transpiled
 uncompiled
 unitaries
 univariate

qiskit_machine_learning/kernels/quantum_kernel.py

@@ -61,7 +61,7 @@ def __init__(
                 the `ZZFeatureMap` is used with two qubits.
             enforce_psd: Project to closest positive semidefinite matrix if x = y.
                 Only enforced when not using the state vector simulator. Default True.
-            batch_size: Number of circuits to batch together for computation. Default 1000.
+            batch_size: Number of circuits to batch together for computation. Default 900.
             quantum_instance: Quantum Instance or Backend
             user_parameters: Iterable containing ``Parameter`` objects which correspond to
                  quantum gates on the feature map circuit which may be tuned. If users intend to
@@ -285,7 +285,7 @@ def _compute_overlap(self, idx, results, is_statevector_sim, measurement_basis)
         """
         if is_statevector_sim:
             # |<0|Psi^dagger(y) x Psi(x)|0>|^2, take the amplitude
-            v_a, v_b = [results.get_statevector(int(i)) for i in idx]
+            v_a, v_b = [results[int(i)] for i in idx]
             tmp = np.vdot(v_a, v_b)
             kernel_value = np.vdot(tmp, tmp).real  # pylint: disable=no-member
         else:
@@ -420,16 +420,21 @@ def evaluate(self, x_vec: np.ndarray, y_vec: np.ndarray = None) -> np.ndarray:
                 is_statevector_sim=is_statevector_sim,
             )
             parameterized_circuit = self._quantum_instance.transpile(parameterized_circuit)[0]
-            circuits = [
-                parameterized_circuit.assign_parameters({feature_map_params: x})
-                for x in to_be_computed_data
-            ]
+            statevectors = []
 
-            results = self._quantum_instance.execute(circuits)
+            for min_idx in range(0, len(to_be_computed_data), self._batch_size):
+                max_idx = min(min_idx + self._batch_size, len(to_be_computed_data))
+                circuits = [
+                    parameterized_circuit.assign_parameters({feature_map_params: x})
+                    for x in to_be_computed_data[min_idx:max_idx]
+                ]
+                results = self._quantum_instance.execute(circuits)
+                for j in range(max_idx - min_idx):
+                    statevectors.append(results.get_statevector(j))
 
             offset = 0 if is_symmetric else len(x_vec)
             matrix_elements = [
-                self._compute_overlap(idx, results, is_statevector_sim, measurement_basis)
+                self._compute_overlap(idx, statevectors, is_statevector_sim, measurement_basis)
                 for idx in list(zip(mus, nus + offset))
             ]
 

releasenotes/notes/batch-circuits-statevector-522a842c6f68d954.yaml

@@ -0,0 +1,8 @@
+---
+fixes:
+  - |
+    Fixed a bug in ``QuantumKernel`` where for statevector simulator
+    all circuits were constructed and transpiled at once, leading to high memory
+    usage. Now the circuits are batched similarly to how it was previously
+    done for non-statevector simulators (same flag is used for both now;
+    previously ``batch_size`` was silently ignored by statevector simulator)