Few, a few, little and a little are used as adjectives to quantify a noun. It seems and most people use few/a few as same words with no difference in meaning; the same goes for little/a little too. However addition of ‘a’ makes a difference.

Few and a few are used before countable plural nouns.
a few refers to a small number
There were a few scratches on the front door of the car. (some scratches)

few refers to very small number, almost giving a negative tone to the statement.
Only few cities have such splendid trees. (hardly two or three cities)

Similarly, a little refers to a small amount and is used before uncountable nouns.
There is a little water left in the jug. (some water still left)

little denotes scarcity and gives negative tone to the statement.
We saw little at the museum as we were getting late for the concert. (hardly saw anything)

the few and the little are used for the small number/quantity already mentioned or specified in a statement.

Only a few people were invited to the party. The few people who attended the party were his special friends.

Kindly throw away the little oil left in the pan.