It should respond almost instantly, if only to show the charging screen. It may need to build up a bit of a charge (maybe 5-10%) before actually allowing you to power up. If you plug it in with no response, try another cable. If that doesn't work, it could be either a hardware defect in the phone's charging circuit, or a bad battery. Try leaving it plugged in but powered off for about an hour to see if you get a charge. If so, then additional battery monitoring may be needed to see if it's actually bad. If that doesn't work, then I would consider the phone dead and in need of repair.
You didn't say which phone you have. If it's an old one, then it may not be worth repairing due to age and how much things have advanced.
If you have additional questions, please register a free account here so you can reply to this thread and keep the conversation going.