Fabric glue and fusible web are two completely different tools, so it depends what you're actually trying to do. Fabric glue (like Aleene's Tacky Glue or similar brands) is water-based and temporary - it holds pieces in place while you sew them, then washes out or stays flexible depending on the type. It's perfect for positioning appliqués or keeping layers aligned before you hit them with your machine. Fusible web, on the other hand, uses heat to permanently bond fabrics together, and it's what you use when you want to skip stitching entirely or reinforce a seam from the inside without visible thread on top.
For tote bags and pouches, I'd say fabric glue is your safest bet if you're still learning. It won't damage your machine, and if you apply it sparingly in a thin line it won't bleed through or stain. Just let it dry for a minute or two before sewing - don't rush it. The downside is it takes longer than fusible web and you still need to stitch. Fusible web is faster if you want a permanent bond without stitching, but it can gunk up your iron if you're not careful, and it creates stiffness in the fabric that some people hate on bags.
For reinforcing seams without visible stitching, fusible web might actually be better since you're not trying to hide glue - you're creating a structural bond. But honestly, if you're doing sewn projects, a line of stitching is usually stronger anyway. Start with fabric glue for positioning, get comfortable with that, then experiment with fusible web once you understand what each one does.