If you don't use some of these connections at all you can delete connection between activity and connected object in EntityConnection table.
In case if you still want to use some of the connections from the list for other Activity Types you can try to override method responsible for forming this dropdown.
In OOB it's defined in EntityConnectionLinksUtilities mixin. And loadEntityConnectionColumns is the method that loads these column.
The list is being formed with an ESQ and you can try to change the filter condition for it in overriden mixin.
If you don't use some of these connections at all you can delete connection between activity and connected object in EntityConnection table.
In case if you still want to use some of the connections from the list for other Activity Types you can try to override method responsible for forming this dropdown.
In OOB it's defined in EntityConnectionLinksUtilities mixin. And loadEntityConnectionColumns is the method that loads these column.
The list is being formed with an ESQ and you can try to change the filter condition for it in overriden mixin.