I have hit higher speeds on track without issue. I have much stiffer suspension, and is sitting about as low as reasonably can for street driving.
For one, it sounds like you need to look into at the very least, lowering the height of the car, and stiffening your suspension. Can be done using Dinan or similar springs if not wanting a full body coilover. I'd then look into better tires, and LCA/TS bushings, weather OE F8x or aftermarket using the original arms.