Skip to content

Commit

Permalink
io_uring/rsrc: optimise single entry advance
Browse files Browse the repository at this point in the history
Iterating within the first bvec entry should be essentially free, but we
use iov_iter_advance() for that, which shows up in benchmark profiles
taking up to 0.5% of CPU. Replace it with a hand coded version.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
  • Loading branch information
isilence authored and axboe committed Feb 22, 2023
1 parent edd4782 commit b000ae0
Showing 1 changed file with 4 additions and 1 deletion.
5 changes: 4 additions & 1 deletion io_uring/rsrc.c
Original file line number Diff line number Diff line change
Expand Up @@ -1364,7 +1364,10 @@ int io_import_fixed(int ddir, struct iov_iter *iter,
const struct bio_vec *bvec = imu->bvec;

if (offset <= bvec->bv_len) {
iov_iter_advance(iter, offset);
iter->bvec = bvec;
iter->nr_segs = bvec->bv_len;
iter->count -= offset;
iter->iov_offset = offset;
} else {
unsigned long seg_skip;

Expand Down

0 comments on commit b000ae0

Please sign in to comment.