These cookies make vibrant and unique additions to any celebration.
Sift flour, baking powder, and salt into a bowl. Cream butter and sugar with a mixer on medium-high until pale and fluffy. Reduce speed to medium; add eggs and vanilla, scraping down sides of bowl often. Reduce speed to low; gradually add flour, and beat to combine. Shape dough into 2 disks; wrap in plastic, and chill until firm, at least 1 hour or up to 2 days.
Heat oven to 325 degrees. Roll out dough on a lightly floured surface to 1/8 inch thick. Cut out shapes with 3-inch cutter; transfer to parchment-lined baking sheet. Cut out inner "windows" using smaller cutters. Reroll scraps and cut out. Transfer baking sheets to freezer until dough is firm, about 20 minutes.
Bake until slightly golden around the edges, 14 to 16 minutes, rotating pan halfway through. Let cool on sheets on wire racks.
To fill centers, place cooled cookies on parchment-lined baking sheets. Make hard-candy syrup: Stir sugar, corn syrup, and the water in a medium pan over high heat until sugar dissolves and mixture boils. Boil without stirring until it registers 300 degrees on a candy thermometer. Remove from heat; add food coloring and flavored oil (syrup will steam and bubble up). Transfer to a glass measuring cup; pour syrup into centers of cookies. Let cool and set completely before moving cookies, about 30 minutes.