Which phone model are you using? Some models require an application that runs on your PC or Mac in order to access storage via USB.
Some phones also prompt you to select a connection mode when plugged in. If yours gives you such a prompt, make sure to select MTP or File Transfer (these names may differ slightly depending on the phone model).