After a few times coming across this problem, I have seen that when I fire up a terminal quickly during this specific bug and type htop I see that tumblerd (tumbler) is on the top of the processes and disappears as soon as desktop becomes responsive again, so I'm guessing that gvfs is not the culprit here I think it's more tumbler...