Adjust the logic to add/remove tags or dependencies
This change makes sure we do not delete something we did not want to
just because it looked close to something we were looking for.
In addition, it makes the UI closer to the usual UI when updated by
eventsource