CAD tip # 6506:

By principle, it is not possible to show a single layer from an xref and hide (switch off) all other drawing layers (including the layer where the xref is inserted). By switching off (freezing) the xref insertion layer, you'll switch off the whole xref, regardless of the visibility of its individual layers.

Nevertheless, as a limited solution for this functionality (assumes an xref inserted in the layer 0, not switched off), you can use the following LISP utility - the "LIX" command (save it to the file lix.lsp and load it with APPLOAD):

(defun C:LIX ( / entlay)
 (setq entlay(cdr(assoc 8 (entget(car(nentsel "Select an Xref-object to isolate: "))))))
 (if (vl-string-search "|" entlay)
  (command "._-layer" "_s" "0" "_off" "*" "_n" "_on" entlay "")
  (princ "\nThis is not an Xref!")
 )
 (prin1)
)