One of the most effective methods for removing the rust layer is known as electrolysis. Here's a picture of my electrolysis rig, and although it's a bit crude in aesthetics, the design is essentially the same as those used by universities and professional laboratories to preserve iron artifacts. NOTE: Electrolysis can be a very dangerous process, involving electricity with exposed wires and the generation of potentially harmful, flammable, and/or explosive gasses. Don't try this at home. Or if you do, be safe, use adequate ventilation, and don't say I didn't warn you.
My rig consists of a plastic container (in my case, a 5 gallon bucket) filled with an electrolyte solution (I used Cascade dish soap, primarily sodium carbonate). In the solution are two electrodes. For a positive electrode, I used stainless steel flashing left over from a construction job (EDIT: I've recently learned that SS results in harmful byproducts, and should be avoided. Use iron rebar, instead). As you can see in the picture, I used two pieces on opposite sides of the tub to provide an even current distribution, and connected them together using four lead wires. The negative electrode will be the iron relic to be cleaned. In my setup, it is suspended by a crossbar using two connected wire leads. It is very important that the two electrodes not come into contact with each other! The positive and negative electrodes are connected to a battery charger (I used either 12V 6A or 12V 2A settings, depending on the object being cleaned). When the current is supplied by the battery charger, the relic will start to bubble as iron oxide is stripped away, leaving a rust-free artifact. If you're interested in learning more, I would highly recommend this video from relic hunter Beau Ouimette showing his electrolysis unit in action.
Here is a before and after showing the CS pentagonal cavity shell fragment after the first round of electrolysis. I will most likely run it through again to remove some of the remaining stubborn rust spots, but you can already see a tremendous improvement.
A more striking example is the lock plate from an Enfield rifle found last weekend. What was once a glob of rusted iron has been transformed to reveal the mostly-intact mechanism from inside the rifle. The next step will be to remove excess water from the iron by soaking in acetone, and finally sealing the relic with a hot wax to prevent further rusting.
I do hope this post has been helpful (or at least entertaining). Thanks for reading, and I look forward to showing you what I dig up next!