The absence of net magnetization inside antiferromagnetic domains has made the control of their spatial distribution quite challenging. Here we experimentally demonstrate an optical method for controlling antiferromagnetic domain distributions in MnF2. Reduced crystalline symmetry can couple an order parameter with non-conjugate external stimuli. In the case of MnF2, time-reversal symmetry is macroscopically broken reflecting the different orientations of the two magnetic sublattices. Thus, it exhibits different absorption coefficients between two orthogonal linear polarizations below its antiferromagnetic transition temperature under an external magnetic field. Illumination with linearly polarized laser light under this condition selectively destructs the formation of a particular antiferromagnetic order via heating. As a result, the other antiferromagnetic order is favoured inside the laser spot, achieving spatially localized selection of an antiferromagnetic order. Applications to control of interface states at antiferromagnetic domain boundaries, exchange bias and control of spin currents are expected.