The molecular function of a protein is achieved by its dynamic structure that is ultimately determined by the amino acid sequence. Currently the most popular method to investigate the native dynamics is the Elastic Network Model (ENM) for which all pairwise distance information (i.e. O(N2) information for an N-residue protein) is required. When we consider the relationship among protein sequence, structure and function, how a protein sequence can encode this amount of information is not trivial. However, does a protein actually need this amount of information? To verify this, here we introduce the Contact Number Diffusion model (CND) that uses only O(N) restraints to capture native dynamics. The reduced number of restraints is a result of not treating sequentially non-local interactions specifically, but rather semi-specifically in the form of contact numbers. Surprisingly, the native dynamics of CND is found to be comparable to that obtained from ENM and a molecular dynamics simulation, and sometimes even better than ENM regarding fits to the experimentally observed conformational changes and thermal fluctuations. This tells us most of the specific non-local interactions are not necessary to capture native dynamics. In future, CND may provide a framework to associate protein sequence to its dynamics and hence function.