Be very careful before removing your neighbor's fence. I had a very similiar situation at my house and did exactly that, resulting in two years of lawsuit, surveys, etc.
A couple of thoughts...
1) Your neighbor may have a survey that differs slightly from yours. Unless you'll can agree which one is correct (not likely) it would likely take a lawsuit to figure it out. This will not be cheap, trust me.
2) Your survey may be 100% complete, but you might not be able to prove it. In my case, my neighbor introduced her own NEW survey and clouded the issue. Even though we could "prove" mathematically that her's was wrong, our attorney pointed out that we still would have to convince a judge (also not very likely).. In other words, we could hae been 100% right and still lost in court.
3) We ended up in mediation (required by the courts here in TX) and the mediator was able to convince our neighbor that she had a pretty good chance of losing and to work on a compromise. We ultimately gave her an easement that was about 60' long X 6 - 10 inches wide.
4) We also have two fences back to back as she would not agree on the type of fencing, etc. I have a chain link and she has a wooden privacy fence. Great for me as my dogs can't get out and I don't have to look at her.
5) In Texas, and may be some other states, a fence is not considered a permanent structure and would not come into play on an adverse possession claim.
"Never argue with stupid people, they will drag you down to their level and then beat you with experience." - Mark Twain