It is typically in reference to a visual manifestation, though it can be extended to those that affect the other senses.
It is actually quite a hard thing to completely and correctly define, as with the explosion of paranormal popularity, the word 'ghost' has been applied in a non-strict way to its definition.
Another aspect that gets quite tricky is the difference between a ghost and a spirit. There is a difference, but it is quite subtle. In paranormal terms, the differences are that a ghost is the earth bound aspect of a disembodied spirit. A ghost is an aspect of a deceased persons being, that has either chosen or has been forced to not be allowed to move on.
A spirit is not bound to our realm, and can freely move beyond our plain. Not everyone agrees with that distinction, and that is more than, ok as the paranormal field is very open to interpretation.
Regardless, the terms are interchanged quite frequently, and not many people will call you wrong for doing so.
Just about every culture has stories pertaining ghosts and hauntings in one form or another, and although they can go by different names, generally they are all manifestations of deceased people.
Ghosts can be classified into quite a number of categories, depending on how they manifest and the activity they create. These will be looked at another time, but for now I would love to see how you define 'ghosts'. This page has many people from different nationalities, cultures and belief structures, so I am more than interested in having a read of what you believe a ghost to be, and the way your cultures portray them.