I've solved my problems with this. The main sticking point seems to be that I had a separate <<profile>> folder for view specifications as mentioned in the documentation. Moving them to the main <<profile>> folder made it work.
Here are the steps to creating sub-diagram types aka view specifications.
1) Have a working MDG with profile, toolboxes and diagram type defined. My simple "MyMDG" test defined MyClassDiagram, MyClass, MyObject, MyAssociation, MyGeneralization, MyInstantiation.
2) Add a new Class diagram(s) to the <<profile>> folder called View-MyClassesOnly
3) Drag on MyMDG::MyClassDiagram from <<diagram profile>> folder onto this new diagram as a link
4) Create a new View Specification from the Metamodel page of the MDG Technology Builder -> Profile toolbox. I named this Classes Only.
5) Drag on MyClass, MyObject and the three relationships I defined earlier.
6) Create an expose relationship from the <<View Specification>> to the elements/relationships you want to show in this sub-diagram.
7) Copy this diagram and call it View-MyObjectsOnly and expose a different subset of elements/relationships.

Publish the <<profile>> package as a profile as usual
9) Generate the MDG as usual
10) Create a new diagram, navigate to MyMDG and feast your eyes on a nested set of diagram types! I can now choose to create a MyClassDiagram with all elements and relationships, or a MyClassesOnly or MyObjectsOnly with a subset of elements/relationships otherwise available to a full MyClassDiagram (my names could be better).
https://i.postimg.cc/9QPgsN94/mdg-view-specification.png