OK, with some critical input from Tim Heuer [MSFT], I figured out what was going on and how to get around this problem.
Surprisingly, none of my three initial guesses were correct. This wasn’t about memory, threading, or system resources. Instead, it was about limitations in the Windows messaging system. Apparently it is a little like a stack overflow exception, in that when you make too many changes to the visual tree all at once, the asynchronous update queue gets so long that it trips a wire and the exception gets thrown.
In this case, the problem is that there are enough UIElements going into the data grid I am working with that allowing the grid to generate all its own columns all at once can in some cases exceed the limit. I was using a number of grids all at once, and all loading in response to page navigation events, which made it all the trickier to nail down.
Thankfully, the limitations I was running into were NOT limitations in the visual tree or the XAML UI subsystem itself, just in the messaging used to update it. This means that if I could spread out the same operations over multiple ticks of the dispatcher’s clock, I could accomplish the same end result.
What I ended up doing was to instruct my data grid not to autogenerate its own columns. Instead, I embedded the grid into a user control that, when the data was loaded, would parse out the columns needed and load them into a list. Then, I called the following method:
void LoadNextColumns(List<ColumnDisplaySetup> colDef, int startIdx, int numToLoad)
{
for (int idx = startIdx; idx < startIdx + numToLoad && idx < colDef.Count; idx++)
{
DataGridTextColumn newCol = new DataGridTextColumn();
newCol.Header = colDef[idx].Header;
newCol.Binding = new Binding() { Path = new PropertyPath(colDef[idx].Property) };
dgMainGrid.Columns.Add(newCol);
}
if (startIdx + numToLoad < colDef.Count)
{
Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () =>
{
LoadNextColumns(colDef, startIdx + numToLoad, numToLoad);
});
}
}
(ColumnDisplaySetup is a trivial type used to house the parsed-out configuration or a configuration loaded from a file.)
This method is called with the following arguments, respectively: list of columns, 0, and my arbitrary guess of 5 as a fairly safe number of columns to load at a time; but this number is based on testing and the expectation that a good number of grids could be loading simultaneously. I asked Tim for more information that might inform this part of the process, and will report back here if I learn more about how to determine how much is safe.
In practice, this seems to work adequately, although it results in the sort of progressive rendering you’d expect, with the columns visibly popping in. I expect this could be improved both by using the maximum possible value for numToLoad and by other UI sleight-of-hand. I may investigate hiding the grid while the columns are generated and only showing the result when everything is ready. Ultimately the decision will come down to which feels more ‘fast and fluid’.
Again, I will update this answer with more information if I get it, but I hope this helps someone facing similar problems in the future. After pouring more time than I’d care to admit into the bug hunt, I don’t want anyone else to have to kill themselves over this.