I have made a quad with the kk 5.5 and I am facing a problem
The board is too responsive and has super quick reactions which causes imbalance
Do you guys out there know why this is happening and do I have change the pitch,roll, yaw gain pots 

Yes!

As yours is a 250 quad, you would need adjusting gains below 50%.
Also you can set expo for pitch, roll and yaw channels in your tx to regulate response to stick movements

I need to adjust all the gain pots (yaw roll and pitch)?
Will it be stable after trimming and act normal after trimming?
I will slowly lessen the gain and try and hopefully it will stop over reacting

Before that set expo curves for pitch , roll and yaw channels on your transmitter..   This will make controls smooth around the stick center

It's drifting in one direction and when I tell it to move in the opposite direction it becomes mad and super unstable
Sooraj sir I have sent you the video

Darshan post video if possible. As i remember old KK boards need to reverse gyro gain, else it might act weird.

Rcindia doesn't allow videos
Takes a lot of time and then says the procedure timed out
Just that It completely goes crazy and bangs into something

you can upload the vid on youtube and post the link here....

I just decreased the pitch and roll value and it was quite stable but it needs to be more stable so will be decreasing the value a bit more