What really helps is watching it with other people or at least sharing your thoughts as you go along. I'm not talking about not skipping episodes, but about changing your mindset: if you notice you're skipping, the show probably just isn't grabbing you right now. There's nothing wrong with that. The Office has a pace that doesn't appeal to everyone right away, and forcing yourself to watch it just because others say it gets better is boring in itself.
If you really want to give it another shot, try watching the episodes you find slower at double speed, so you get through them without missing the plot. Or just watch the episodes your friends mention as the best ones without worrying about skipping. A lot of people don't feel like waiting for a comedy to hit its stride, and that's totally fine. If there's another show that catches your interest in the meantime, watch that instead: you don't have to finish everything you start. Coming back to The Office later (maybe in a few months) sometimes works better than forcing yourself now.