I used to use Dxtory with the x264 codec, but have since moved onto Nvidia Shadowplay/Share. A lot of people use the default settings (which are way overkill, 'cause YouTube compresses down to 7 Mbps), but mine is in the range of about 10-15 Mbps, which creates much more manageable file sizes...