No, you are looking in the right place but Diagram only appears for the first brand new filter on, effectively, a brand new search.
In the Search Builder, right-click on the body of the Search Builder tab and select the New Search context menu option. Type in a name - say, DiagSearch. The Editor Type radio button is selected by default. Click on the OK button. So, the Search Builder tab should be clear.
Right-click on the body of the Search Builder tab again and select the Add Filter option. On the Add Filters dialog, the Search On field defaults to Element. Click on the drop-down arrow and see that the second item on the list is Diagram. Select it! Quick!
This gives you a search on a small group of diagram properties, which you can enable and set values for as necessary. Click on the OK button.
In theory, you could select to add another filter to this search, in which case the drop-down arrow on the Search On field would show you Diagram and only Diagram - you can't mix any other object type search into the Diagram search. In practice, I'm not sure that adding another filter would give you anything you couldn't acheive with the first, single filter.
If you had selected Element for the first filter, or indeed pretty much any option other than Diagram, the Search would have to act initially on an element and thereafter you could add more filters on a number of element properties. So the Search On field would continue to offer several options, but not Diagram because you can't mix the Diagram filter into an Element search.
I hope that helps.