Preheat the oven to 350°F. Line a square 8x8 inch baking pan with parchment paper so you can easily remove the cake from the pan. Alternatively, directly grease the pan.
To make the cinnamon crumb, whisk together the brown sugar, gluten-free flour, and cinnamon together in a medium bowl. Use a pastry blender or two forks to cut the chilled butter into the mixture until the butter breaks down into pea-sized pieces. Store in the refrigerator until you assemble the cake.
To make the cake batter, whisk together the gluten-free flour, baking powder, baking soda, and salt in a medium bowl.
In a large bowl using a handheld or stand mixer fitted with a paddle attachment, beat the butter and sugar together on high speed until smooth and creamy, 2-3 minutes. Scrape down the sides and bottom of the bowl as needed.
Add the eggs and vanilla and beat on medium-high speed until no yellow streaks remain and the batter is light and fluffy, another 2 minutes. With the mixture still running, beat in the sour cream. Scrape down the sides and bottom of the bowl as needed. The mixture will be lumpy but will smooth out as you add the dry ingredients.
With the mixer running on low speed, slowly add the dry ingredients to the wet ingredients until just combined then finish by adding the milk. Be careful not to over-mix. The batter will be thick.
Pour just over half of the batter into the prepared pan and smooth it out into an even layer. It doesn't have to be exact. Sprinkle 1 cup of crumb topping evenly over the batter.
Repeat the steps and spread the remaining batter on top of the crumb. I like to do this by placing dollops of batter evenly over the crumb and using a greased spatula to help spread it. It will be tricky because the thick batter is going on top of an unstable crumb. Do your best! Finish with the remaining crumb topping.
Bake for 32-37 minutes or until a toothpick inserted in the center comes out with a few moist crumbs. Start checking for doneness around 30 minutes and tent with foil if the top is browning too much.
While it’s baking make the icing. Whisk them together in a small bowl. Remove the cake from the oven and drizzle the icing over the warm cake. Cover and store at room temperature for up to 3 days or in the fridge for up to 1 week. Coffee cake freezes well wrapped tightly in plastic wrap for up to 3 months. Thaw overnight in the fridge before serving.