Partially revert last patch, and improve the implementation some more

- shorter code
- don't zero out columns, there's no reason to do so
- don't copy rows that get overwritten with zeros anyways

[[Imported from SVN: r4894]]