What can I use for garden insect pests that isn't toxic?

It's not like everything "non-toxic" works equally well, honestly. It really depends on what pests you've got and how bad the infestation already is. That said, there are tons of options that work pretty decently if you apply them regularly and stick with it.

Neem oil is your friend here. You spray it every one or two weeks on the plants and it kills larvae, mites and aphids no problem. It works better if you do it in the evening so it doesn't burn the leaves. Then there's potassium soap dissolved in water, which is really effective against aphids and mealybugs. Another option that always works is just water with vinegar or water with blended garlic sprayed on; sounds weird but people have been using it forever and insects hate it. For ants and other soil bugs, you dust raw diatomaceous earth (food-grade, the kind they sell for consumption) around the plants and it kills the crawling insects.

If you want something more "immediate," pyrethrin insecticides (a natural compound from chrysanthemum flowers) are really safe and work fast. Oh, and the simplest thing that people sometimes forget: keep your garden clean of dead weeds and debris where they hide. Introduce ladybugs or beneficial beetles if you can, they eat pests like crazy. Crop rotation also stops pests from adapting to one specific area. It's more work, but that way you're not fighting the same problem year after year.

What a lot of people don't mention is that these natural methods work better if you apply them preventively or when the pest first shows up; if you already have a heavy infestation, you'll probably need to combine several techniques or do more frequent applications for them to actually work. Plus, some pests like whiteflies or spider mites can be particularly resistant to certain "natural" options, so it's worth identifying exactly what pest you're dealing with before you commit to just one method!