To change swatches size on shop/category page , visit swatches tab under woocommerce -> settings and define your preferable width and height.