Assemble ingredients, preheat oven to 400 degrees F (200 degrees C), and place drumsticks in a 7x11-inch baking dish.
Whisk ketchup, vinegar, water, brown sugar, butter, salt, Worcestershire sauce, mustard, and chili powder together in a bowl; pour mixture over drumsticks. Cover with aluminum foil.
Bake in the preheated oven until no longer pink at the bone and the juices run clear, about 1 hour, turning chicken about halfway through. An instant-read thermometer inserted near the bone should read 165 degrees F (74 degrees C).