Just delete the key out of the appropriate directory in /etc/salt/pki/master/ https://github.com/saltstack/salt/issues/24561