The certificates are in the base configuration. The deletion removes them from being processed and from display (or is supposed to); however, they are present in case the system needs to be restored to defaults for some reason. For the deletion, you are making the correct action by choosing the certificate and then delete.
This should be fixed in 8.2R5.1; do you have a test system you can confirm? Unfortunately, it is also possible that you have found something we haven't come across yet.