Bring sugar and 1/2 cup water to a boil in a saucepan. Cook, stirring, until sugar dissolves. Remove from heat. Add rosemary sprig and let steep for 2 hours; remove sprig. Refrigerate simple syrup up to 2 weeks.
Combine lemon juice, gin, orange juice, and simple syrup in a punch bowl. Garnish bowl and glasses with orange slices and rosemary sprigs, if desired. Add ice cubes as needed to keep chilled.