Moved into a new rented house and it is a disgrace, what are my rights?

I guess you live in the UK. I don't know what the laws specify in the UK. However, the first thing I'd do is go after the agent. Promises were made that were not kept and now he wants to abandon you. It's not your fault the agent refunded the previous tenant's deposit without inspecting the property. Either he or the owner should clean the place, repair dangerous faults, and get rid of the vermin. If neither will take responsibility, contact your local health department. If nothing's done after that you're going to have to move, foot the bill, and sue for your money back plus damages.