Despite many sessions for a single subject, the reports did not get separated per session

[celprov]

What happened?

I tried running fMRIPrep v23.2.2 on our dataset, which contains 59 sessions for a single subject. Yet, all sessions got aggregated into a single report.

The tests implemented in fmriprep/reports/tests/test_reports.py are still passing, indicating the report separation still works under the circumstances of the test.

I also tried to see whether the BIDSLayout correctly detects that there are many sessions in my dataset, which seems to be the case as indicated by the output below.

>>> from bids.layout import BIDSLayout
>>> l = BIDSLayout("/cprovins/hcph/")
>>> l.get_sessions()
['001', '003', '004', '005', '006', '007', '008', '009', '010', '011', '013', '015', '016', '017', '018', '019', '020', '021', '022', '023', '024', '025', '026', '028', '037', '038', '039', '040', '041', '042', '043', '044', '045', '046', '047', '048', 'excl027', 'excl029', 'pilot001', 'pilot002', 'pilot003', 'pilot004', 'pilot005', 'pilot006', 'pilot007', 'pilot008', 'pilot009', 'pilot010', 'pilot011', 'pilot012', 'pilot013', 'pilot014', 'pilot015', 'pilot016', 'pilot017', 'pilot018', 'pilot019', 'pilot020', 'pilot021']
>>> ses_001 = l.get_sessions(subject='001')
>>> ses_001
['001', '003', '004', '005', '006', '007', '008', '009', '010', '011', '013', '015', '016', '017', '018', '019', '020', '021', '022', '023', '024', '025', '026', '028', '037', '038', '039', '040', '041', '042', '043', '044', '045', '046', '047', '048', 'excl027', 'excl029', 'pilot001', 'pilot002', 'pilot003', 'pilot004', 'pilot005', 'pilot006', 'pilot007', 'pilot008', 'pilot009', 'pilot010', 'pilot011', 'pilot012', 'pilot013', 'pilot014', 'pilot015', 'pilot016', 'pilot017', 'pilot018', 'pilot019', 'pilot020', 'pilot021']
>>> len(ses_001)

Not sure why the report separation is not working on this dataset specifically.

Linked to #3191

What command did you use?

#!/bin/bash

DATADIR="/cprovins/hcph/"
SUB="sub-001"
pushd $DATADIR/$SUB > /dev/null
ALL_SES=(`ls -d ses-*`)
popd > /dev/null

#Remove session with no fMRI
SES=()
for S in "${ALL_SES[@]}"; do
        if [ -d "$DATADIR/$SUB/$S/func" ]; then
                SES+=("$S")
        else
            	echo "Session '$S' has no functional scan."
        fi
done

echo "Submitting `basename $DATADIR` with ${#SES[@]} sessions"
# remove one since we are starting at 0
JOBS=`expr ${#SES[@]} - 1`
sbatch --array=0-$JOBS ss-fmriprep-new.sh $DATADIR ${SES[@]}


where ss-fmriprep-new.sh is:

#!/bin/bash

#SBATCH --partition=russpold
#SBATCH --mem=75GB
#SBATCH --cpus-per-task=16
#SBATCH --time=25:00:00
#SBATCH --job-name=fmriprep
#SBATCH --error="slurm-%A_%a.err"

# Submit a Single Session through fMRIPrep

ARGS=($@)
DATADIR=$1
CODEDIR="/cprovins/code"
DERIVATIVEDIR="/cprovins/hcph-derivatives"
STUDY=`basename $DATADIR`

if [[ -n $SLURM_ARRAY_TASK_ID ]]; then
  SES=${ARGS[`expr ${SLURM_ARRAY_TASK_ID} + 1`]}
else
  SES=$2
fi

echo "Processing: $SES"

IMG="/cprovins/singularity_images/fmriprep-23.2.2.simg"

WORKDIR="${L_SCRATCH}/fmriprep-23.2.2-wp/${STUDY}/${SES}"
mkdir -p ${WORKDIR}
OUTDIR="${DERIVATIVEDIR}"
mkdir -p $OUTDIR

PATCHES=""

BINDINGS="-B $DATADIR:/data:ro \
-B ${WORKDIR}:/work \
-B ${OUTDIR}/fmriprep-23.2.2-wp:/out \
-B $CODEDIR/license.txt:/opt/freesurfer/license.txt \
-B $CODEDIR/filter_file_$SES.json:/filter_file_$SES.json \
-B ${HOME}/.cache/templateflow/tpl-MNI152NLin6Asym/:${HOME}/.cache/templateflow/tpl-MNI152NLin6Asym/
$PATCHES"

FMRIPREP_CMD="/data /out participant --participant-label 001 \
-w /work --bids-filter-file /filter_file_$SES.json --skip_bids_validation \
--fs-subjects-dir /out/sourcedata/freesurfer --ignore fieldmaps \
--derivatives /out --output-spaces T1w fsnative func MNI152NLin2009cAsym  \
--nprocs 4 --mem 63G --omp-nthreads 2 -vv"

#Create json file to filter one session only
echo '{"bold": {"datatype": "func", "session": "'${SES#"ses-"}'", "suffix": "bold"}}' > $CODEDIR/filter_file_${SES}.json

SING_CMD="singularity run -e $BINDINGS $IMG $FMRIPREP_CMD"
echo $SING_CMD
$SING_CMD
echo "Completed with return code: $?"

What version of fMRIPrep are you running?

23.2.2

How are you running fMRIPrep?

Singularity

Is your data BIDS valid?

Yes

Are you reusing any previously computed results?

Anatomical derivatives

Please copy and paste any relevant log output.

No response

Additional information / screenshots

No response

[effigies]

Was this implemented in 23.2? Might need to try unstable.

[celprov]

Yes, it was implemented in 23.2. What does that mean to try unstable?

[effigies]

The nipreps/fmriprep:unstable image. That's a preview of 24.0.0.

[celprov]

Thank you @effigies running nipreps/fmriprep:unstable indeed separates the reports per sessions.