SQL Server will start with as much memory as needed, then slowly ramp up until it uses all allocated to it in the Server Properties:
It will not release any of this memory until the service is restarted; this is by design.
It is generally recommended to leave 2ish GB for the OS, and you need to be mindful of any other processing running on the same server as SQL. It is usually recommended to have SQL Server on it’s own server without anything else running there.
That said, 1.5 GB of RAM for SQL Server isn’t that much. If you don’t have more available, it may be time to add some or upgrade the server.
See also:
Technet, Brent Ozar