Unfortunately there's currently no way to change the emoticons in the gTalk app.
It's really quite surprising how little love the gTalk app receives from Google. IMHO, if Google gave even a little effort into improving the app, google talk could/would become the predominate method of text communication and alleviate the plethora of new chat tools (whatsapp, kik, pingchat etc..) that seem to be trying to reinvent the wheel.
However; There are a lot of other 3rd party apps for gtalk, yahoo, AIM etc.. that have many more features. Some free, some paid. I use Trillian myself.
I totally agree with you. With the number of IM apps springing up and their popularity rapidly growing, its quite surprising that Google is not capitalising on this (well, neither are other big corps like Microsoft) and it seems it takes young, innovative companies to come up with ideas in this sphere. Like Microsoft, it seems Google is losing its edge on which way the consumer really wants apps to go