The only things that make leaving difficult are fear of change and uncertainty. Fear of whether you'll like your next job, fear you'll be/seem incompetent starting out, uncertainty of whether your benefits will be as good.
Change is hard. I left due to layoffs. I had notification 4 months before my severance date. I had a new job lined up 2 months before leaving.
I'm in a new industry. Yes, there's uncertainty and times when I question my competence, but it's exciting too. No week is like the previous. I'm paid better, my benefits are just as good, I travel more, I have better support from my superiors, and I'm in an industry that is growing. I like my job.
Change is hard, but sometimes change is necessary. Be brave.
Admittedly, it is easier to be brave and accept change when you have no choice. It's even easier when you're drawing salary on top of severance.