The thing is, databinding won't have set the field, unless you have changed to some temp value then back again to the original value.
But now, since you set the fields yourself, the IsChanged for the field is set to true, and consequently the IsDirty flag is set too.
You can do a check first and skip setting the value of a field if it matches the existing one.
You can do this in the routine setting the values into the fields, and you can do this inside the Set-ter of each the Field property.