Evenly spoon yogurt into six 3-ounce paper cups. Place in freezer for about 1 hour, until semi-frozen. (Or, if making in advance, remove 3-ounce cups of low-fat vanilla frozen yogurt from freezer about 15 minutes before serving.)
Preheat oven to 400 F. Remove green leaves from rhubarb and discard (they're toxic). Cut rhubarb into 1 1/2-inch lengths on the bias. Wash (don't dry).
In a shallow casserole, stir rhubarb and jelly until all pieces are coated. Bake until rhubarb is tender, about 35 minutes, stirring twice. Remove from oven; cool for 5 minutes. Spoon into 6 shallow bowls. Unmold yogurt and serve atop rhubarb.