Ranking stuck on the 10th position – sitewides

I have many sitewide links to one of my websites. Google shows 530k links from 1400 domains (but my internal calculation is double than 530k). It did get me to where I am – ranking on very tough keywords, but it seems that since Penguin 4 last year in October it's only a downwards spiral.

I don't know if it's because there were domains that were lost or the insane backlinks/unique domains ratio. I ranked position 1,2 for many though keywords and now I'm 6-13.

Your thoughts?