Stay away from toothpicks - that's the right call, you risk doing serious damage. For dry earwax that's stuck inside the mesh, you should try compressed air first, like the kind you use to clean your keyboard - it blows away a lot of stuff without touching anything. Hold them at like 15-20 cm away and don't go overboard with the pressure.
If there's still dirt inside after that, a slightly damp microfiber cloth actually works really well, but the trick here is drying them right after with another dry cloth. Moisture inside the earbuds = problem. For the outside parts you can also use a bit of isopropyl alcohol (the kind I found at the pharmacy) on a cotton swab, swipe it quick and it dries on its own, but avoid letting it drip inside.
If the earwax is really crusted on and the compressed air isn't enough, you could try a plastic toothpick or an old really soft toothbrush, but move carefully and gently. Anyway, two years of buildup is normal - it's not like you did something wrong. It happens to everyone.