Here is a nice Ed Jaynes quote, from his article “Probability Theory as Logic”:
“Then in studying probability theory, it was vaguely troubling to see reference to “gaussian random variables”, or “stochastic processes”, or “stationary time series”, or “disorder”, as if the property of being gaussian, random, stochastic, stationary, or disorderly is a real property, like the property of possessing mass or length, existing in Nature.”
Here are two plots I made to illustrate this point. The first signal is not stochastic. Probability has nothing to do with it, even though it looks ragged, because every point is known. The second plot is stochastic. Even though it doesn’t “look random” (whatever that means), the y-value is unknown, so probability theory could be used.