I'd agree with ringing your bank - if they're any good it shouldn't be a hassle getting it sorted if my one experience is anything to go by.
My former energy supplier, having agreed that I was in credit, assured me both by phone and by e-mail that the letter they'd sent suggesting otherwise was yet another cock-up and could be ignored, they wouldn't take any further money and would sort things out when they eventually got around to doing the final bill.
That was in mid-October. I'd cancelled the direct debit of, of course, and come 1st November, no problem. However, come 1st December, a month and a half after I'd left them and was still waiting for their snail-paced accounts system to actually produce a final bill, the scumbags submitted a new direct debit and attempted to steal £50 from me!
I phoned the bank when I noticed this in my online statement and told them what the score was.
They did say that I should also tell the company that I disputed it, which I did later that day, but, as I said to the bank, "I thought I'd ring you first because I know I'll get someone with a brain at your end - the chances of tracking down the sole employee in that incompetent lot who isn't sharing a brain cell with a village idiot are pretty remote".
They thought that was fair enough and didn't quibble at all, they bounced it back straight away.