I'm not sure exactly how TDictionary works but in general when you are removing items from Lists it is always better to go from last item to first instead of always removing first item in a loop. Why?
Because when you remove first item from the list all other items needs to be shifted in order to fill the hole that you caused by removing that one item. This results in lots of unneeded data moving in the list internal array.