That handle or edit tool, allows you to better position the cursor for editing or copying. Touch it and you can slide it to the exact place that needs editing.
Not sure if there's a way to turn it off or not, but you can edit the png's, which can be found in the framework-res.apk file. They're named: text_select_handle_left.png, text_select_handle_middle.png and text_select_handle_right.png