I do not have any custom JS that triggers on field change.
Playing around a little more, I discovered that even the standard way of removing values also induces lag (i.e. manually selecting values and clicking the “Remove” button). The delay from the manual process is still not as much as calling the setFieldValue function.
But perhaps this points to a deeper issue with the list? Starting to think this may not be a DFFS issue after all…