Alrighty then, I've got a solution for you.
When you add an adjustment layer, it has a colour in the Layers divider, like so:

In this one, it is white.
To erase parts of it, make sure your eraser is set as the same colour as the colour set in the adjustment layer. So, if it is white (which is will be 99% of the time), make sure your foreground colour is set to white, like so:

Now all you gotta do is fire away!