It has been my experience that orbs are spirits/energies that are still around in the spiritual realm because they choose to be good to those of us on the physical plane, or they choose to be evil and bring confusion and misfortune. I beleive that the spiritual and physical planes both deal with the eternal tug between good and evil. It's the same energy on whatever plane.
In your case, it seemed easy for you to determine immediately that the black orbs were negative/destructive and you easily brushed them away with recognition and positivity. The red orbs could have been high energy spirits. One of which you held onto for a bit and then gently released. The blue and gold seem to represent spirits of a higher, calmer, more evolved nature, because of their color and the energy they carried. Concentrate on those and internalize this elevated peacefulness. And find others to talk to about your deepest feelings and beliefs. When you meditate see the Robin's egg blue and gold colored orbs and invite them to swirl around you while you are in meditation. Hope that helped. Your sister in beleif, Gitana