1) The aperture (slider) may be installed backwards, so the teeth of the catch are on the wrong side to catch the notches.
2) Your push button spring is missing or broken.
Note that the flat side of the rear sight aperture (part 110 in the schematic) faces forward, with the push button (part 112) on the left side from the shooter's viewpoint. If your rear sight is backwards, you won't be able to compress the plunger, nor will it catch on anything. The other possibility is that the aperture is installed correctly, but your spring (part 111) is broken or missing. If that were the case, however, your push button should still be able to move a bit.
Removing the rear sight is as simple as unscrewing one of the windage screws all the way, and tapping the sight out to that side. There's a tiny pin (part 107) at the front that will fall out and get lost if you aren't careful. Drive the pin out the bottom of the sight body if it doesn't fall out, and slide the aperture off to the front. The push button in the aperture will slide out to the left. The spring for it is another tiny part that's easily lost. Reassembly is pretty intuitive once you see how it fits together.