There are various reasons given for this practice.
1. Before formal tombstones were introduced, it was customary to mark
the site of a grave by piling stones on it. Because of the elements, and
also, people or animals passing by who might remove them, it became
customary to pile them up again every time someone went to visit.
2. There are some people who suggest that it recalls the ancient
practice of sealing the opening of a grave with a large stone. This was
particularly appropriate when a body was laid to rest in a catacomb
rather than being interred in the ground.
3. Many people think it is a sign of honour to the deceased to show that
he/she has had visitors.
I answered this question on another group and someone wrote to tell me
that in our climate in the UK it is not good for the marble to leave
stones resting on it. I knew a superintendant at one of the largest
Jewish cemeteries in England, who regularly used to go round taking
these pebbles off.
Rabbi Geoffrey Shisler
Bournemouth Hebrew Congregation