git rm --cached -r --cached tells it to remove the paths from staging and the index without removing the files themselves and -r operates on directories recursively.