Remove empty folders after uninstall

[Ruben Arts]

When a package is uninstalled it might leave behind some empty folders.

lib/python3.10/site-packages via 🅒 haystack 
➜ tree attrs-23.1.0.dist-info/
attrs-23.1.0.dist-info/
└── licenses

2 directories, 0 files

I would expect these folders to be removed aswell.