I have found that you can directly connect the Atmel ICE to the board without needing JTAG capability on chip. JTAG is not available on all Atmel AVR MCUs. There is not an available header for the ICSP, but you don't need one. You just have to connect it manually to the correct pins on the dev board. It's a bit of a rats nest if you don't have a custom cable, but it can be done with some simple jumpers. Below is the pinout for the ICSP connector, as well as the ATMEGA168 for reference.
One thing to note is that the on board programmer must be powered up for this to work. These pins are not directly connected to the MCU. The ICSP bus flows through this programmer and it seems to act as a transceiver of sorts. The reset line must be also be tied via the correct jumper to the headers so that it can be accessed by the ICE.
No different than connecting the ICE to a chip on a breadboard. Not what I wanted from a purpose built dev board, but a work around none-the-less if you need access to DebugWIRE.