Assuming the system on which you would be doing this is booting in legacy (MBR) mode, not UEFI mode, these are the installation steps for Ubuntu 14.04 LTS:

Boot from a live DVD/USB

On partition selection stage, choose "Something Else.."

Create/Select partition(s) on your target USB drive as mount point for root (/)

Make sure the target for Grub is also the same USB drive, not a 'Partition' or another drive (e.g. /dev/sdc or /dev/sdd etc., NOT /dev/sdc1 or /dev/sdd1..)

Install normally and finish.