# Virtual Objects shifted on CVCamera Image captured in MR Mode

**URL:** https://forum.magicleap.cloud/t/virtual-objects-shifted-on-cvcamera-image-captured-in-mr-mode/3105
**Category:** Unity Development
**Tags:** unity, mixed-reality-cam, focus-distance
**Created:** [January 31, 2024, 2:18pm UTC](https://forum.magicleap.cloud/t/virtual-objects-shifted-on-cvcamera-image-captured-in-mr-mode/3105 "2024-01-31T14:18:40Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![kocsisa](https://avatars.discourse-cdn.com/v4/letter/k/ccd318/32.png) [@kocsisa](https://forum.magicleap.cloud/u/kocsisa)
#### Post date: [January 31, 2024, 2:18pm UTC](https://forum.magicleap.cloud/t/virtual-objects-shifted-on-cvcamera-image-captured-in-mr-mode/3105/1 "2024-01-31T14:18:40Z")

</div>

When we have virtual objects on ML screen which are aligned to / overlapping real world objects, and we capture CVCamera Image in MR mode, than the virtual objects are on different location on the captured image than we can see them on the MLDevice.

See the 3 sphere-pairs marked on the attached captured image, which should cover the realworld object as we see them on the Device screen.

**Unity Editor version** : 2022.3.11f1  
**ML2 OS version** : 1.5.0  
**MLSDK version** : 2.0.0  
**Host OS** : Windows

 ![ObjectsShifting_On_CVCameraImage_Captured_In_MR_Mode](https://us1.discourse-cdn.com/flex019/uploads/magicleap1/original/2X/8/84f491291d8f919fd853de23d64ebd13dc5ac5b7.jpeg)

---

<div class="post-metadata">

### Author: ![kbabilinski](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.magicleap.cloud/kbabilinski/32/19_2.png) [@kbabilinski](https://forum.magicleap.cloud/u/kbabilinski)
#### Post date: [January 31, 2024, 9:39pm UTC](https://forum.magicleap.cloud/t/virtual-objects-shifted-on-cvcamera-image-captured-in-mr-mode/3105/2 "2024-01-31T21:39:37Z")

</div>

The virtual content is offset due to the position of the RGB camera compared to the virtual content. To reduce the performance overhead of recording or streaming mixed reality content, without rendering a third camera pass, the mixed reality stream overlays the virtual content that is rendered by the left eye, then it distorts the image so that it aligns with the RGB camera as closely as possible. However, you will notice that the virtual content becomes offset as it moves away from the camera's focus point. The focus point is where the content will align most accurately.

As mentioned in the previous point, you can reduce the offset of a specific object by setting the `StereoConvergencePoint` inside Unity on the `Magic Leap Camera Component` or the `Focus Point` if you are developing in C++. Here is a guide that explains how to use user's fixation point to control the focus point in Unity: [Unity Stabilization Overview](https://developer-docs.magicleap.cloud/docs/guides/unity/display/unity-stabilization-overview/).

The example script will reduce the apparent offset of the virtual content that the user fixating on. You can also set the `StereoConvergencePoint` to an object's location, this would result in the specified object being aligned in the mixed reality recording regardless of what the user is looking at.

You can also set the point to a specific object if you need that object to be most closely aligned with the real world.

---

<div class="post-metadata">

### Author: ![kbabilinski](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.magicleap.cloud/kbabilinski/32/19_2.png) [@kbabilinski](https://forum.magicleap.cloud/u/kbabilinski)
#### Post date: [January 31, 2024, 9:42pm UTC](https://forum.magicleap.cloud/t/virtual-objects-shifted-on-cvcamera-image-captured-in-mr-mode/3105/3 "2024-01-31T21:42:29Z")

</div>

Let me know if this resolves your issue. If you find that this does not resolve your issue or need the image completely aligned I am happy to share your feedback with our voice of customer team. I would just need a little more information on what you are trying to achieve and why it is important to your project. Additionally, would it be antiquate to render the virtual content aligned with the real world if there was some performance overhead when rendering the content?

---

<div class="post-metadata">

### Author: ![kocsisa](https://avatars.discourse-cdn.com/v4/letter/k/ccd318/32.png) [@kocsisa](https://forum.magicleap.cloud/u/kocsisa)
#### Post date: [February 1, 2024, 9:22am UTC](https://forum.magicleap.cloud/t/virtual-objects-shifted-on-cvcamera-image-captured-in-mr-mode/3105/4 "2024-02-01T09:22:47Z")

</div>

Thank you for quick response.

We recognize objects in the real world and put their 3D model to the same place. (The camera and sensor images and metadata are captured and sent for processing to a server)  
In the test documentation we'd like to include videos/images which show what the user can see meanwhile using the application.  
(For decision makers it would be much more meaningful than numbers or charts which shows positioning errors. 🙂 )

In this special test situation some additional performanc overhead would be OK. E.g. If we had MR and AlignedMR Capture mode for CVCamera, we could use the new AlignedMR for these speacial cases which would align virtual content to the real world on the whole image.

---

<div class="post-metadata">

### Author: ![kbabilinski](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.magicleap.cloud/kbabilinski/32/19_2.png) [@kbabilinski](https://forum.magicleap.cloud/u/kbabilinski)
#### Post date: [February 8, 2024, 4:55pm UTC](https://forum.magicleap.cloud/t/virtual-objects-shifted-on-cvcamera-image-captured-in-mr-mode/3105/5 "2024-02-08T16:55:09Z")

</div>

Did setting the focus distance / Stereo convergence point on the Magic Leap Camera provide a satisfactory result, or was the offset still too large for your documentation ?

---

<div class="post-metadata">

### Author: ![kocsisa](https://avatars.discourse-cdn.com/v4/letter/k/ccd318/32.png) [@kocsisa](https://forum.magicleap.cloud/u/kocsisa)
#### Post date: [February 9, 2024, 12:06pm UTC](https://forum.magicleap.cloud/t/virtual-objects-shifted-on-cvcamera-image-captured-in-mr-mode/3105/6 "2024-02-09T12:06:42Z")

</div>

The priorities has been changed, and now I have to work on another tasks, but I will get back to you with the results ASAP.
