Is there a way to remove custom properties in a similar way to adding them via the Add Custom Properties plug-in?

Thanks.

Yes. The way is writing another plugin. You can download the source code pack at Customer Area, in there you will find the AddCustomPropertiesPlugin source code. It will be easy to create your own DeleteCustomPropertiesPlugin based on that.