Thud sound when i shift
 

I have a 03 IS300 5-speed and every time i shift there comes this "Thud Thud" from the back of the car. I picked up the car and checked my drive shaft and i could move it with my hands, and when i would shift the drive shaft shakes. Should the drive shaft be solid in place and i need a new drive shaft mount? Or is it something completely different?