After clicking on something non-interactive, the current view was getting deactivated, but still received keyboard events. roots_seat_set_focus now changes both together in this case.