We do not have this type of API at this time, but I can share your feedback with our voice of customer team.
One thing that you need to keep in mind when trying to cast a ray from the CV camera and get a hit from the depth image is that the CV camera and depth camera are not in the same place and do not have the same intrinsics or distortion coefficients.
There are a couple of different ways one could approach this - for example, the azure kinect api has a set of transform function that perform mesh reprojection of the images (Use Azure Kinect Sensor SDK image transformations | Microsoft Learn). Once you are able to transform the contents of one image to match the properties of the other then you can just query the depth image at the same pixel location as the color image to get your depth value.