High frequency surface wave radar (HFSWR), used for coastal surveillance, operates in a challenging environment as the clutter signals returned from the ocean surface can be several orders of magnitude larger than returns from targets. Reliably detecting small boats in severe sea states, is therefore quite difficult. In this thesis we take a two-fold approach to improving the detection performance of Raytheon Canada's third generation HFSWR system. First, we consider the design of transmit waveforms with improved range resolution, thus reducing the area of the clutter cell. We develop an algorithm to design practical spectrally-compliant waveforms which achieve high bandwidths while simultaneously avoiding interference with concurrent communications users. We then propose a new detection algorithm based on the best known statistical model for sea clutter, the K-distribution. We show that both the proposed transmit waveforms and detection algorithm can lead to improved detection performance in a sea clutter environment.