Am I overlooking something?
Nope, you are not overlooking anything. The only advantage and reason to use UnityEvent is that it allows you to use events in the Editor. That’s for drag and drop people or those making Editor plugins.
Another advantage of UnityEvent is that it prevents the problem of Unity Object not being freed due to the misuse of delegates or using anonymous delegates with Unity Objects. Although they get freed when the main script that’s holding them is destroyed. The reason for this is because UnityEvent is implemented with weak references therefore removing/minimizing this problem. These two things are still not worth it to use UnityEvent over native C# events.
You should always use native event and delegate over UnityEvent if you are not making an Editor plugin because of its fast performance and small memory usage. See this and this post post for more information.