I used to feel wary of writing switching PoVs until I realized several of my favorite stories had them. You just don't notice if it's well-written; it's not jarring, and it sometimes makes the story feel more dynamic. I mostly see switching-PoV stories that have one scene or part from one PoV, one scene or part from another and so forth, so I'm not entirely sure what yours would look like, but, as I said, as long as it's ~organic/a natural development of the narrative, it shouldn't be a problem.