There is a HUGE difference. A question mark tells whatever algorithm you are using that the value is missing. Replacing it with a zero says that the value is actually a zero.
Some methods are capable of handling missing values. The methods may replace the missing value with the mean of the other values of that attribute, they may ignore the instance altogether, or they may do something else. However, if you tell the method that the value is zero (which it truly isn't) you are dictating what happens to missing values, they are assumed to be zero.
This link provides a very detailed description of ".arff" files, and what is expected in them.
There is a HUGE difference. A question mark tells whatever algorithm you are using that the value is missing. Replacing it with a zero says that the value is actually a zero.
Some methods are capable of handling missing values. The methods may replace the missing value with the mean of the other values of that attribute, they may ignore the instance altogether, or they may do something else. However, if you tell the method that the value is zero (which it truly isn't) you are dictating what happens to missing values, they are assumed to be zero.
This link provides a very detailed description of ".arff" files, and what is expected in them.